Please help! Matlab slows down when parallel

2 views (last 30 days)
Georg
Georg on 16 Feb 2014
Commented: Georg on 17 Feb 2014
Dear Community,
I am facing a huge problem. I built a complex C application with embedded Matlab functions that I call using the Matlab engine (engOpen and such ...). Now I realize the following:
- I spawn multiple instances of this application on a machine, one for each core
- However! ... The application then slows down to a halt. In fact, on my 16-core machine, the application slows down approximately by factor 16.
- Now I realized this is because there is only a signel matlab engine started per machine and all my 16 instances share the same copy of matlab!
- I tried to replicate this with the matlab GUI and its the same problem. I run a program in the GUI that takes 14 seconds, and THEN I run it in two GUIs at the same time and it takes 28 seconds
This is a huge problem for me, because I will miss my deadline if I have to reprogram my entire c application without matlab and I will lose my job. I know that matlab has commands for parallel programming, but my matlab calls are embedded in the C application and I want to run multiple instances of the C application. Again, I cannot refactor my entire c application because I will miss the deadline.
Can anyone please let me know if there is a solution for this (e.g. really start multiple matlab processes on the same machine). I am willing to pay for extra licenses. I currently have fully lincensed matlab installed on all machines.
Thank you so so much!

Answers (1)

Jan
Jan on 16 Feb 2014
Without seeing any detail of the code, it is impossible to suggest an improvement.
Do the spawned processes run on different cores? How did you spawn which kind of processes?
If the different processes run on different cores, the memory access patterns can prevent an efficient processing: If the different processes access the input data such, that you get a cache miss, the memory interface is the bottleneck and the number of cores does not matter. Anotehr effect appears, when the output is written from different cores to the same cacheline block, e.g. 64 bytes. Then these collisions require to update the caches between the cores. When both problems are combined, multi-threading does not accelerate the program.
These effects have to be considered during the design of the program. There is no easy way to fix such problems afterwards. Try to find the bottlenecks by a profiler and post the correpsonding details of the code.
  1 Comment
Georg
Georg on 17 Feb 2014
Dear Jan Simon,
Thank you for your answer. I use a scheduler called PBS (portable batch system) that assignes each job to a core. The jobs themselves are completely independant, ie they don't share any information. They just C executables spawned with different argv[] settings. They only load data from hard disk once in the beginning (about 50MB). Also, their memory profile does not exceed a few hundred MB. Furthermore, the cluster I run things on is designed for batch processing so it really shouldn't falter in this kind of "easy" environment.
My question really is whether Matlab explicitly disallows parallel execution.
Thanks

Sign in to comment.

Categories

Find more on Manage Products in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!