Codegen For Autofeaturization
Copyright (c) Microsoft Corporation. All rights reserved.
Licensed under the MIT License.
Automated Machine Learning - Codegen for AutoFeaturization
Autofeaturization of credit card fraudulent transactions dataset on remote compute and codegen functionality
Contents
Autofeaturization lets you run an AutoML experiment to only featurize the datasets. These datasets along with the transformer are stored in AML Storage and linked to the run which can later be retrieved and used to train models.
To run Autofeaturization, set the number of iterations to zero and featurization as auto.
Please refer to Autofeaturization and custom model training for more details on the same.
Codegen is a feature, which when enabled, provides a user with the script of the underlying functionality and a notebook to tweak inputs or code and rerun the same.
In this example we use the credit card fraudulent transactions dataset to showcase how you can use AutoML for autofeaturization and further how you can enable the Codegen feature.
This notebook is using remote compute to complete the featurization.
If you are using an Azure Machine Learning Compute Instance, you are all set. Otherwise, go through the configuration notebook first if you haven't already, to establish your connection to the AzureML Workspace.
Here you will learn how to create an autofeaturization experiment using an existing workspace with codegen feature enabled.
This sample notebook may use features that are not available in previous versions of the Azure ML SDK.
Create or Attach existing AmlCompute
A compute target is required to execute the Automated ML run. In this tutorial, you create AmlCompute as your training compute resource.
Note that if you have an AzureML Data Scientist role, you will not have permission to create compute resources. Talk to your workspace or IT admin to create the compute targets described in this section, if they do not already exist.
Creation of AmlCompute takes approximately 5 minutes.
If the AmlCompute with that name is already in your workspace this code will skip the creation process. As with other Azure services, there are limits on certain resources (e.g. AmlCompute) associated with the Azure Machine Learning service. Please read this article on the default limits and how to request more quota.
Load Data
Load the credit card fraudulent transactions dataset from a CSV file, containing both training features and labels. The features are inputs to the model, while the training labels represent the expected output of the model.
Here the autofeaturization run will featurize the training data passed in.
Training Dataset
AutoFeaturization
Instantiate an AutoMLConfig object. This defines the settings and data used to run the autofeaturization experiment.
| Property | Description |
|---|---|
| task | classification or regression or forecasting |
| training_data | Input training dataset, containing both features and label column. |
| iterations | For an autofeaturization run, iterations will be 0. |
| featurization | For an autofeaturization run, featurization can be 'auto' or 'custom'. |
| label_column_name | The name of the label column. |
| enable_code_generation | For enabling codegen for the run, value would be True |
You can find more information about primary metrics here
Call the submit method on the experiment object and pass the run configuration. Depending on the data this can run for a while. Validation errors and current status will be shown when setting show_output=True and the execution will be synchronous.
Results
Widget for Monitoring Runs
The widget will first report a "loading" status while running the first iteration. After completing the first iteration, an auto-updating graph and table will be shown. The widget will refresh once per minute, so you should see the graph update as child runs complete.
Note: The widget displays a link at the bottom. Use this link to open a web interface to explore the individual run details
Codegen Script and Notebook
Codegen script and notebook can be found under the Outputs + logs section from the details page of the remote run. Please check for the autofeaturization_notebook.ipynb under /outputs/generated_code. To modify the featurization code, open script.py and make changes. The codegen notebook can be run with the same environment configuration as the above AutoML run.