Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT License.

Save and retrieve explanations via Azure Machine Learning Run History

This notebook showcases how to use the Azure Machine Learning Interpretability SDK to save and retrieve classification model explanations to/from Azure Machine Learning Run History.

Table of Contents

1. Introduction
2. Setup
3. Run model explainer locally at training time
   1. Apply feature transformations
   2. Train a binary classification model
   3. Explain the model on raw features
      1. Generate global explanations
      2. Generate local explanations
4. Upload model explanations to Azure Machine Learning Run History
5. Download model explanations from Azure Machine Learning Run History
6. Visualize explanations
7. Next steps

Introduction

This notebook showcases how to explain a classification model predictions locally at training time, upload explanations to the Azure Machine Learning's run history, and download previously-uploaded explanations from the Run History. It demonstrates the API calls that you need to make to upload/download the global and local explanations and a visualization dashboard that provides an interactive way of discovering patterns in data and downloaded explanations.

We will showcase three tabular data explainers: TabularExplainer (SHAP), MimicExplainer (global surrogate), and PFIExplainer.

Problem: IBM employee attrition classification with scikit-learn (run model explainer locally and upload explanation to the Azure Machine Learning Run History)

1. Train a SVM classification model using Scikit-learn
2. Run 'explain-model-sample' with AML Run History, which leverages run history service to store and manage the explanation data

Setup: If you are using Jupyter notebooks, the extensions should be installed automatically with the package. If you are using Jupyter Labs run the following command:

	(myenv) $ jupyter labextension install @jupyter-widgets/jupyterlab-manager

Explain

Run model explainer locally at training time

Load the IBM employee attrition data

Transform raw features

We can explain raw features by either using a sklearn.compose.ColumnTransformer or a list of fitted transformer tuples. The cell below uses sklearn.compose.ColumnTransformer. In case you want to run the example with the list of fitted transformer tuples, comment the cell below and uncomment the cell that follows after.

Train a SVM classification model, which you want to explain

Explain predictions on your local machine

Generate global explanations

Explain overall model predictions (global explanation)

Explain overall model predictions as a collection of local (instance-level) explanations

Generate local explanations

Explain local data points (individual instances)

Upload

Upload explanations to Azure Machine Learning Run History

Download

Download explanations from Azure Machine Learning Run History

Visualize

Load the visualization dashboard

End

Complete the run

Next

Learn about other use cases of the explain package on a:

1. Training time: regression problem
2. Training time: binary classification problem
3. Training time: multiclass classification problem
4. Explain models with engineered features:
   1. Simple feature transformations
   2. Advanced feature transformations
5. Run explainers remotely on Azure Machine Learning Compute (AMLCompute)
6. Inferencing time: deploy a classification model and explainer:
   1. Deploy a locally-trained model and explainer
   2. Deploy a locally-trained keras model and explainer
   3. Deploy a remotely-trained model and explainer