Utilizing SageMaker Notebook Job Steps To Operationalize ML Workflows

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

In this example we take a look at how we can utilize SageMaker Notebook Job Steps to build a simple end to end ML workflow including Feature Engineering, Training, and Evaluation. We will be utilizing an NLP BERT Model with the SST2 dataset for Text Classification. The SST2 dataset contains two columns including text and a classification column with 2 labels: 0 and 1.

This notebook creates and executes a pipeline with two notebook steps and three notebooks in its workflow:

• Preprocess Step: Pulls down the NLP Dataset and conducts preprocessing.
• Train and Batch Inference Step: Conducts Training and Batch Inference with a Transformers BERT Model.
  • Additional Dependencies: Notebook that prepares a test dataset for Batch Inference.

Setup

For setup we install the SageMaker Python SDK and also need to ensure our role has the proper IAM permissions for the Studio user:

1. Create and execute the SageMaker Pipeline.
2. Notebook Job Execution Role passed to the Notebook Job Step that will execute the underlying Training Job.

Pipeline Setup

Pipeline Execution

This pipeline will take approximately 20 minutes to execute.

Output Notebook Parsing

Cleanup

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.