Scale MATLAB Online to an AWS Cloud Cluster
Learn how to scale your programs in MATLAB Online™ to Amazon EC2® compute resources using MATLAB Parallel Server™. The in-app interface for MathWorks® Cloud Center allows you to create, manage, and start cloud clusters on Amazon® Web Services, all within MATLAB Online.
Published: 12 Jul 2020
Big data and complex analysis most often require high performance computing resources.
In this video, I’ll show how you can use MATLAB Parallel Server and MATLAB Online, to scale to additional CPU and GPU compute resources, on an Amazon Web Services compute cluster.
MATLAB Online provides access to the latest version of MATLAB through your web browser, without any downloads or installation. When you need to scale up to additional compute resources; you can create, manage and access your cloud clusters on Amazon Web Services, all within MATLAB Online. MATLAB Online allows you to perform these tasks from anywhere and entirely on the cloud.
To get started with cloud computing workflows, you will need
- A License for MATLAB and the Parallel Computing Toolbox
- An Amazon Web Services (or, AWS) Account
and - A MATLAB Parallel Server License configured to use online licensing.
For more information regarding licensing, you can refer to this page on “Online Licensing for MATLAB Parallel Server”.
The Parallel Computing Toolbox allows the use of multiple CPU cores, and GPUs on your local machine, in order to speed up computationally and data intensive problems. With MATLAB Online being on the cloud, we’ll focus on accessing CPU and GPU resources on compute clusters, instead of those on our local machine. We can achieve this via the MATLAB Parallel Server, which allows you to scale your programs to multiple computers in clusters and clouds. These additional resources are especially useful for speeding up any compute intensive programs, which typically cost a lot of time and memory.
Let’s see how we can use these tools together to tackle large data sets and computationally intensive tasks by going through a Deep Learning workflow.
Consider a use-case where we would like to perform transfer learning for image classification. The first step is to access and prepare the data.
Let’s suppose, we have 6000 images with a size of 1.2 GB that will be used to train a neural network. This can be stored in an Amazon S3 bucket to lessen the memory load on our local machine. Then, we can access the data in MATLAB Online via a datastore object. After accessing and preprocessing our data, it will be time to train the deep learning network.
As training a neural network is a computationally intensive task, we can scale up to more compute resources on our AWS cluster, and then execute it as a batch job. This adds a great benefit in our workflow, as with more workers available for use, we can run more tasks in parallel, leading to an overall reduction in time it takes to train the network. Now we can validate and test our model, and then deploy it for others to use.
You can find and try similar examples for cloud workflows on the following documentation page. Now let’s see how we can setup a cloud cluster in MATLAB Online.
First, we’ll add our AWS account information via the in-app interface. A benefit of this in-app GUI is that it automatically checks whether licensing requirements are met. If you don’t have an online license for MATLAB Parallel Server, you can acquire it as part of the billing process right within the interface. If you’re on a Campus-Wide License that has MATLAB Parallel Server, this workflow will automatically associate you to your school’s MATLAB Parallel Server license, providing you the licensing you need, to start a Cloud Center cluster.
At this stage, we’ve ensured licensing and added our AWS credentials, allowing Cloud Center to start clusters our behalf. Now, we can go ahead and create a Cloud Cluster Profile. You can also do this setup through MathWorks Cloud Center. Cloud Center is a web application that allows you to quickly create and manage clusters on AWS. Everything you need to know about setting up your cloud cluster from requirements to details on customizing your profile can be found at the Cloud Center documentation.
Now you can use this cloud cluster in your projects and leverage high performance computing resources to accelerate your tasks. Whether you’re working with Big Data or performing computationally intensive tasks, you can easily get started with cloud computing in MATLAB Online.
You can use these URLs to launch MATLAB Online and Cloud Center to try it out yourself.
Thank you for watching this video.