How can I read an image file from disk asynchronously?
5 views (last 30 days)
Show older comments
I am working on a large number of 12MB image files. The time required to process each image file is split almost equally between Reading the input, Processing, and Writing the output. Reading and Writing speed is limited by disk, whereas Processing speed is limited by CPU/GPU. I am reading and writing to/from different disks.
When using imread and imwrite I have to wait for the function to complete before processing each image, so the disk or CPU/GPU is sitting idle ~2/3 of the time. Is there some way to call imread and imwrite asynchronously in order to accelerate the script?
Output from Example code:
>> readExample
Single worker:
Reading: 0.13
Processing: 1.10
Writing: 6.99
Total: 8.22
Three workers:
Reading: 0.17
Processing: 13.70
Writing: 0.60
Total: 8.23
Example Code:
% Make an example image
imwrite(uint16(rand(1080,1920,3)*65535),'example.tiff');
% Make an example convolution filter
X = gpuArray.colon(-300,300);
S = 100;
H = exp(-(X.*X)/(2*S*S));
H = H'*H;
H = H/sum(H(:));
% Go through the loop with a single worker
N=2;
Tread = zeros(N,1); Tprocess=zeros(N,1); Twrite=zeros(N,1);
Ttotal = tic;
for n=1:N
% Load the file
tic;
Im = single(imread('example.tiff'));
Im = gpuArray(Im);
Tread(n) = toc;
% Apply the convolution
tic;
Im(:,:,1) = conv2(Im(:,:,1), H, 'same');
Tprocess(n) = toc;
% Save the result
tic;
Im = gather(Im);
imwrite(uint16(Im),'output.tiff');
Twrite(n) = toc;
end
fprintf('Single worker:\n');
fprintf('Reading:\t\t%0.02f\n',sum(Tread));
fprintf('Processing:\t\t%0.02f\n',sum(Tprocess));
fprintf('Writing:\t\t%0.02f\n',sum(Twrite));
fprintf('Total:\t\t\t%0.02f\n',toc(Ttotal));
% Go through the loop again with three workers
if matlabpool('size') < 3
matlabpool('open',3);
end
Tread = zeros(N,1); Tprocess=zeros(N,1); Twrite=zeros(N,1);
Ttotal = tic;
parfor n=1:N
% Load the file
tic;
Im = single(imread('example.tiff'));
Im = gpuArray(Im);
Tread(n) = toc;
% Apply the convolution
tic;
Im(:,:,1) = conv2(Im(:,:,1), H, 'same');
Tprocess(n) = toc;
% Save the result
tic;
Im = gather(Im);
imwrite(uint16(Im),'output.tiff');
Twrite(n) = toc;
end
fprintf('\nThree workers:\n');
fprintf('Reading:\t\t%0.02f\n',sum(Tread));
fprintf('Processing:\t\t%0.02f\n',sum(Tprocess));
fprintf('Writing:\t\t%0.02f\n',sum(Twrite));
fprintf('Total:\t\t\t%0.02f\n',toc(Ttotal));
2 Comments
Ashish Uthama
on 14 Oct 2013
Could you update the question with some sample code so we understand the pattern? If you are processing files in a loop, subsequent imread ought to happen while the GPU is processing, most (if not all) GPU calls are async.
Accepted Answer
More Answers (2)
Image Analyst
on 11 Oct 2013
Edited: Image Analyst
on 11 Oct 2013
Perhaps get the Parallel Computing Toolbox?
2 Comments
Image Analyst
on 11 Oct 2013
I think it probably already does things as fast as it can. I doubt they programmed in delays on purpose.
You can set the CPU priority higher. Right click MATLAB in the process list (Windows OS, control-shift-Escape) and do Set Priority->High. DO NOT USE REALTIME or you won't be able to do anything else.
Or try using a solid state drive to speed things up.
Jan
on 11 Oct 2013
Edited: Jan
on 11 Oct 2013
A less compact idea than the Parallel Computing Toolbox is starting three Matlab session and let them communicate through http://www.mathworks.com/matlabcentral/fileexchange/28572-sharedmatrix . But then the control of the inter-process communication is not trivial. But you would not transport the image data through the SharedMatrix, but control, which session processes which file.
Which kind of file type do the images have? Most of the formats are read using standard libraries, such that a C-Mex function could read and write the file asynchronously. You can easily start two threads for this job as long as they do not call any Matlab commands (mx... and mex... functions).
See Also
Categories
Find more on Parallel and Cloud in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!