Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT License.

Train a model using a custom Docker image

In this tutorial, learn how to use a custom Docker image when training models with Azure Machine Learning.

The example scripts in this article are used to classify pet images by creating a convolutional neural network.

Set up the experiment

This section sets up the training experiment by initializing a workspace, creating an experiment, and uploading the training data and training scripts.

Initialize a workspace

The Azure Machine Learning workspace is the top-level resource for the service. It provides you with a centralized place to work with all the artifacts you create. In the Python SDK, you can access the workspace artifacts by creating a workspace object.

Create a workspace object from the config.json file.

Prepare scripts

Create a directory titled fastai-example.

Then run the cell below to create the training script train.py in the directory.

Define your environment

Create an environment object and enable Docker.

This specified base image supports the fast.ai library which allows for distributed deep learning capabilities. For more information, see the fast.ai DockerHub.

When you are using your custom Docker image, you might already have your Python environment properly set up. In that case, set the user_managed_dependencies flag to True in order to leverage your custom image's built-in python environment.

To use an image from a private container registry that is not in your workspace, you must use docker.base_image_registry to specify the address of the repository as well as a username and password.

	fastai_env.docker.base_image_registry.address = "myregistry.azurecr.io"
fastai_env.docker.base_image_registry.username = "username"
fastai_env.docker.base_image_registry.password = "password"

It is also possible to use a custom Dockerfile. Use this approach if you need to install non-Python packages as dependencies and remember to set the base image to None.

Specify docker steps as a string:

	dockerfile = r""" \
FROM mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04
RUN echo "Hello from custom container!" \
"""

Set base image to None, because the image is defined by dockerfile:

	fastai_env.docker.base_image = None \
fastai_env.docker.base_dockerfile = dockerfile

Alternatively, load the string from a file:

	fastai_env.docker.base_image = None \
fastai_env.docker.base_dockerfile = "./Dockerfile"

Create or attach existing AmlCompute

You will need to create a compute target for training your model. In this tutorial, you create AmlCompute as your training compute resource.

Note that if you have an AzureML Data Scientist role, you will not have permission to create compute resources. Talk to your workspace or IT admin to create the compute targets described in this section, if they do not already exist.

Creation of AmlCompute takes approximately 5 minutes. If the AmlCompute with that name is already in your workspace this code will skip the creation process.

As with other Azure services, there are limits on certain resources (e.g. AmlCompute) associated with the Azure Machine Learning service. Please read this article on the default limits and how to request more quota.

Create a ScriptRunConfig

This ScriptRunConfig will configure your job for execution on the desired compute target.

Submit your run

When a training run is submitted using a ScriptRunConfig object, the submit method returns an object of type ScriptRun. The returned ScriptRun object gives you programmatic access to information about the training run.