GPU memory accessed even when code runs on CPU only (See attached code)
1 view (last 30 days)
Show older comments
Geremy Michel
on 29 Sep 2014
Commented: Geremy Michel
on 30 Sep 2014
I am using 10 workers to submit a job. 1 worker is assigned to the GPU while the rest is assigned to the CPU cores. createTask() calls a function that checks for the current worker. Based on the current worker assigned to the task, that function will branch into the GPU or the CPU code.
The problem is that even when it branched into the CPU code, GPU memory is still pre-allocated. I found that initializing the GPU code with gpuArray.zeros() causes MATLAB to pre-allocate GPU memory even if the program never executes that line of code. This is a major issue because while the one GPU worker is running, the other 9 workers are grabbing GPU memory causing it to run out of memrory.
A quick fix was to initialize gpu arrays with gpuArray(zeros()) but this significantly slows down the GPU worker since data has to be pushed to the GPU each time. I have attached some simple code to reproduce the issue. I used nvidia-smi to look for GPU memory usage.
Does anyone know of a workaround the issue that I just described. Thank you. Geremy
1 Comment
Accepted Answer
Edric Ellis
on 30 Sep 2014
I'm still investigating this problem, but as a workaround you can also use the 'like' syntax which avoids the problem:
z = zeros(10000, 'like', gpuArray)
4 Comments
Edric Ellis
on 30 Sep 2014
Sean is correct that both gpuArray.zeros and zeros(..., 'like', gpuArray) are built directly on the GPU - the twist is that zeros(..., 'like', gpuArray) avoids the particular problem that you're seeing which is related to the existence of static gpuArray method calls (i.e. things which look like gpuArray.zeros or gpuArray.rand) inside your function (even if you're not actually calling them).
More Answers (1)
See Also
Categories
Find more on GPU Computing in MATLAB in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!