imread - for vs. parfor - not seeing any gains
4 views (last 30 days)
Show older comments
I am processing a lot of images for work and wanted to take advantage of the parallel architecture to save some time:
With a for loop, my script takes 11.0621 hrs to finish. With a parfor loop, my script takes 10.9742 hrs to finish.
The gain there is so minimal that it could just be fluctuation. The code I'm using calls imread and then regionprops about 800 times per region, and there are 8 regions for 13 samples. Trying to get at what the problem might be, I tried substituting a comparable image that was already in my workspace in lieu of the imread call and in that case the parfor loop went way faster than the for loop (2.692 hrs vs. 4.9351 hrs). This makes me think that my code is fine in terms of the amount of information being passed between workers, that I have a parallel pool, etc.
Anyone have any idea why the gains of using the parallel structure are so minimal with the imread call? Is it my hardware? All code was run on a 64-bit Windows 7 environment with 32 GB of RAM, a 3.4 GHz Intel i7 processor, and two NVIDIA GeForce GTX 570 graphics cards.
Here is a skeleton of my code:
Sample = {'H1', 'H2', 'H3', 'H4', 'H5', 'H6', 'H7', 'H8', 'H9', 'H10', 'H11', 'H12', 'H13'};
Regions = {'R1', 'R2', 'R3', 'R4', 'R5', 'R6', 'R7', 'R8'};
try
parpool('local', 4);
catch
end
AllSamplesTic = tic;
for S = Samples
Sample = char(S);
for R = Regions
Region = char(R);
cd(Sample '\' Region);
load('RegionInfo.mat');
TotalImages = RegionInfo(:,5);
for SliceNum = 1:size(TotalImages,1)
% Gather some info
parfor Ind = 1:TotalImages(SliceNum,1)
I = imread(['pic' num2str(i) '.png']); % or I = dummyImage;
props = regionsprops(I, 'Area', 'Centroid', 'Eccentricity', 'Orientation');
% Do some calculations here
% Store results
end
end
end
end
Thanks!
4 Comments
Thomas Ibbotson
on 13 Jun 2014
Ok, I know this has nothing to do with your original problem, but you can use 'gcp' to do that. See:
help gcp
Answers (2)
Matt J
on 12 Jun 2014
Edited: Matt J
on 13 Jun 2014
My guess would be that the parallel labs are all fighting each other for access to your hard drive and/or you have a slow hard drive. It might be worth a try to read the images in serially into a cell array (or into the slices of a 3D array if all are the same size) and then broadcast them to parfor.
10 Comments
Matt J
on 14 Jun 2014
Edited: Matt J
on 14 Jun 2014
Considering I gain 2 hrs when I use the dummy image I am inclined to disagree that 17 slices is too small to be parallelizing...
The gain of 2 hours represents a factor of 2 speed-up. It's something, but with a parpool of 4 workers, you would hope for something closer to a factor of 4. Clearly the ratio of computation to communication is still not terribly favorable and the size of the parfor loop has a bearing on this.
Is there a way to track the idle time of a worker to see if your guess checks out?
Comment out all the processing steps inside the parfor loop apart from the imread and see how much things speed up. Incidentally, you could then repeat the same, but with 'parfor' replaced with plain 'for'. This would give us an idea how well we do reading from the hard drive in parallel vs. serially.
Rather than waiting 5-10 hours, I'd of course recommend you do these comparisons with the outer loop restricted to a smaller number of Regions and Samples.
Image Analyst
on 12 Jun 2014
Your hardware is pretty impressive. Doesn't seem like it should take 11 hours. What is the value of numSlices? The badly-named I is a binary image, right, not a gray scale image? Are there tons of regions in it? (That's the normal definition of regions, not your custom definition.) Perhaps some noise reduction would speed up the regionprops() if you're spending a lot of time measuring useless little bits of noise.
3 Comments
Image Analyst
on 13 Jun 2014
You didn't answer the question about what numSlices is. Is it 800 slices? So you have 8*13*800 = 83,200 images to do regionprops() on?
See Also
Categories
Find more on Parallel Computing Fundamentals in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!