Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT License.

Tutorial #2: Deploy an image classification model in Azure Container Instance (ACI)

This tutorial is part two of a two-part tutorial series. In the previous tutorial, you trained machine learning models and then registered a model in your workspace on the cloud.

Now, you're ready to deploy the model as a web service in Azure Container Instances (ACI). A web service is an image, in this case a Docker image, that encapsulates the scoring logic and the model itself.

In this part of the tutorial, you use Azure Machine Learning service (Preview) to:

• Set up your testing environment
• Retrieve the model from your workspace
• Test the model locally
• Deploy the model to ACI
• Test the deployed model

ACI is a great solution for testing and understanding the workflow. For scalable production deployments, consider using Azure Kubernetes Service. For more information, see how to deploy and where.

Prerequisites

Complete the model training in the Tutorial #1: Train an image classification model with Azure Machine Learning notebook.

Set up the environment

Start by setting up a testing environment.

Import packages

Import the Python packages needed for this tutorial.

Deploy as web service

Deploy the model as a web service hosted in ACI.

To build the correct environment for ACI, provide the following:

• A scoring script to show how to use the model
• A configuration file to build the ACI
• The model you trained before

Create scoring script

Create the scoring script, called score.py, used by the web service call to show how to use the model.

You must include two required functions into the scoring script:

• The init() function, which typically loads the model into a global object. This function is run only once when the Docker container is started.

• The run(input_data) function uses the model to predict a value based on the input data. Inputs and outputs to the run typically use JSON for serialization and de-serialization, but other formats are supported.

Create configuration file

Create a deployment configuration file and specify the number of CPUs and gigabyte of RAM needed for your ACI container. While it depends on your model, the default of 1 core and 1 gigabyte of RAM is usually sufficient for many models. If you feel you need more later, you would have to recreate the image and redeploy the service.

Deploy in ACI

Estimated time to complete: about 2-5 minutes

Configure the image and deploy. The following code goes through these steps:

1. Create environment object containing dependencies needed by the model using the environment file (myenv.yml)
2. Create inference configuration necessary to deploy the model as a web service using:
   • The scoring file (score.py)
   • envrionment object created in previous step
3. Deploy the model to the ACI container.
4. Get the web service HTTP endpoint.

Get the scoring web service's HTTP endpoint, which accepts REST client calls. This endpoint can be shared with anyone who wants to test the web service or integrate it into an application.

Test the model

Download test data

Download the test data to the ./data/ directory

Load test data

Load the test data from the ./data/ directory created during the training tutorial.

Predict test data

Feed the test dataset to the model to get predictions.

The following code goes through these steps:

1. Send the data as a JSON array to the web service hosted in ACI.

2. Use the SDK's run API to invoke the service. You can also make raw calls using any HTTP tool such as curl.

Examine the confusion matrix

Generate a confusion matrix to see how many samples from the test set are classified correctly. Notice the mis-classified value for the incorrect predictions.

Use matplotlib to display the confusion matrix as a graph. In this graph, the X axis represents the actual values, and the Y axis represents the predicted values. The color in each grid represents the error rate. The lighter the color, the higher the error rate is. For example, many 5's are mis-classified as 3's. Hence you see a bright grid at (5,3).

Show predictions

Test the deployed model with a random sample of 30 images from the test data.

1. Print the returned predictions and plot them along with the input images. Red font and inverse image (white on black) is used to highlight the misclassified samples.

Since the model accuracy is high, you might have to run the following code a few times before you can see a misclassified sample.

You can also send raw HTTP request to test the web service.

Clean up resources

To keep the resource group and workspace for other tutorials and exploration, you can delete only the ACI deployment using this API call:

If you're not going to use what you've created here, delete the resources you just created with this quickstart so you don't incur any charges. In the Azure portal, select and delete your resource group. You can also keep the resource group, but delete a single workspace by displaying the workspace properties and selecting the Delete button.

Next steps

In this Azure Machine Learning tutorial, you used Python to:

• Set up your testing environment
• Retrieve the model from your workspace
• Test the model locally
• Deploy the model to ACI
• Test the deployed model

You can also try out the regression tutorial.