Nowadays, many instructors are integrating AI to their courses. In a distance learning setting, the hardware students use to train their models vary. Training time of the deep learning models can be shortened with a pool of GPUs, CPUs or a pool of CPUs and GPUs locally or in the cloud. Accuracy of the results can depend on the hyperparameters used to train the models.

In MATLAB, Experiment Manager (introduced in R2020a) makes it easy to train networks with various hyperparameters and compare the results. Different models can be run in parallel by clicking on “Use Parallel” button in Experiment Manager App. But what if your computer has multiple CPU cores and a GPU? Would you be able to use Experiment Manager with a pool of CPUs and a GPU? The answer is “yes”. For example, my computer has 1 NVIDIA GPU and an 8 core CPU. To use all these computational resources, I typed these lines in my command window in MATLAB:

parpool(9);
spmd
if labindex==1
gpuDevice(1); %select GPU on worker 1
else
gpuDevice([]); %deselect GPU on remaining workers
end

Then, I clicked on “Use Parallel” button in Experiment Manager and used a modified setup function in Experiment Manager to change the execution environment from CPU to GPU:

if isempty(parallel.gpu.GPUDeviceManager.instance.SelectedDevice)
options=trainingOptions(~,"ExecutionEnvironment",'cpu');
else
options=trainingOptions(~,"ExecutionEnvironment",'gpu');
end 

Default training options automatically use an NVIDIA GPU if there is one, and specific hardware can be selected using “ExecutionEnvironment” in the trainingOptions.

If you like to learn more about the fundamentals of parallel computing, check out “Parallel Computing Fundamentals” on our documentation and gain some hands-on experience with Parallel Computing through "Parallel Computing Hands-On Workshop" .