Codistributed arrays taking too long to run

3 views (last 30 days)
Hello fellow programmers.
I've been exploring the distributed arrays functionality on Matlab. I tried doing a simple matrix multiplication, where both matrices have the same dimension, and comparing the time it takes the serial and parallel execution to run.
Here's the code, not "parallelized" (is that a word?)
N = 5000;
A = eye(N);
B = magic(N);
tic
C = A*B;
toc
I try to run it in parallel by distributing the array of the matrices between the 4 workers I have available.
N = 5000;
A = eye(N);
B = magic(N);
spmd
A = codistributed(A,codistributor1d());
B = codistributed(B,codistributor1d());
end
tic
spmd
C = A*B;
end
toc
However when I run this code it takes a lot of time to distribute the matrices between the workers (with the codistributed() function) and the matrix multiplication takes significantly longer (about 5/6 times longer).
If anyone could tell me what I'm doing wrong I'd appreciate. Am I not understanding how distributed arrays should be used?
Cheers Daniel

Accepted Answer

Jill Reese
Jill Reese on 14 Feb 2013
Here is an example of how to properly benchmark an operation on distributed arrays:
You are comparing the implicit parallelism (multithreading) of core MATLAB to the explicit parallelism (MPI communication between MATLAB workers that are themselves running singlethreaded) of the Parallel Computing Toolbox distributed arrays. If the input matrices A and B fit in memory on a single machine with enough memory left over to perform the operation and store the output, it's often pretty hard to beat the performance of the multithreaded operation. Distributed arrays provide a clear benefit when the problem exceeds the memory available on a single machine.
If you set up a benchmark for mtimes (*) following the example that I've linked, you can investigate the performance you can expect from your own machine. I think that the codistributor you have chosen is not unreasonable for this operation.

More Answers (1)

José-Luis
José-Luis on 13 Feb 2013
Edited: José-Luis on 13 Feb 2013
It's because even though you distribute your array amongst the workers, matrix multiplication might not be efficient, depending on how you distribute the array. In order to compute it you will need to access data in the other arrays, considerably slowing your code due to communication overhead.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!