Amazon Web Services Sm Pipelines Lambda Step

Sm Pipelines Lambda Step

sm-pipelines_lambda_stepdata-scienceinferenceamazon-sagemaker-examplesreinforcement-learningml_opsmachine-learningawsexamplesdeep-learningsagemakerjupyter-notebooktrainingmlops

alph-notebooks/amazon-sagemaker-examples / sm-pipelines_lambda_step.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

[1]

SageMaker Pipelines Lambda Step

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

This notebook illustrates how a Lambda function can be run as a step in a SageMaker Pipeline.

The steps in this pipeline include:

Preprocess the Abalone dataset
Train an XGBoost Model
Evaluate the model performance
Create a model
Deploy the model to a SageMaker Hosted Endpoint using a Lambda Function, through SageMaker Pipelines

A step to register the model into a Model Registry can be added to the pipeline using the RegisterModel step.

Runtime

This notebook takes approximately 15 minutes to run.

Prerequisites
Configuration Setup
Data Preparation
Model Training and Evaluation
Setting up Lambda
Execute the Pipeline
Clean up resources

Prerequisites

The notebook execution role should have policies which enable the notebook to create a Lambda function. The Amazon managed policy AmazonSageMakerPipelinesIntegrations can be added to the notebook execution role to achieve the same effect.

The policy description is as follows:


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction",
                "lambda:DeleteFunction",
                "lambda:InvokeFunction",
                "lambda:UpdateFunctionCode"
            ],
            "Resource": [
                "arn:aws:lambda:*:*:function:*sagemaker*",
                "arn:aws:lambda:*:*:function:*sageMaker*",
                "arn:aws:lambda:*:*:function:*SageMaker*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "sqs:CreateQueue",
                "sqs:SendMessage"
            ],
            "Resource": [
                "arn:aws:sqs:*:*:*sagemaker*",
                "arn:aws:sqs:*:*:*sageMaker*",
                "arn:aws:sqs:*:*:*SageMaker*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": "arn:aws:iam::*:role/*",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": [
                        "lambda.amazonaws.com"
                    ]
                }
            }
        }
    ]
}

Let's start by importing necessary packages and installing the SageMaker Python SDK.

[2]

[3]

/opt/conda/lib/python3.7/site-packages/secretstorage/dhcrypto.py:16: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/opt/conda/lib/python3.7/site-packages/secretstorage/util.py:25: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Requirement already satisfied: sagemaker>=2.99.0 in /opt/conda/lib/python3.7/site-packages (2.99.0)
Requirement already satisfied: protobuf3-to-dict<1.0,>=0.1.5 in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (0.1.5)
Requirement already satisfied: google-pasta in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (0.2.0)
Requirement already satisfied: smdebug-rulesconfig==1.0.1 in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (1.0.1)
Requirement already satisfied: numpy<2.0,>=1.9.0 in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (1.21.1)
Requirement already satisfied: boto3<2.0,>=1.20.21 in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (1.20.47)
Requirement already satisfied: attrs<22,>=20.3.0 in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (21.4.0)
Requirement already satisfied: pandas in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (1.0.1)
Requirement already satisfied: importlib-metadata<5.0,>=1.4.0 in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (1.5.0)
Requirement already satisfied: protobuf<4.0,>=3.1 in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (3.17.3)
Requirement already satisfied: pathos in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (0.2.8)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.7/site-packages (from sagemaker>=2.99.0) (20.1)
Requirement already satisfied: botocore<1.24.0,>=1.23.47 in /opt/conda/lib/python3.7/site-packages (from boto3<2.0,>=1.20.21->sagemaker>=2.99.0) (1.23.47)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /opt/conda/lib/python3.7/site-packages (from boto3<2.0,>=1.20.21->sagemaker>=2.99.0) (0.5.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3<2.0,>=1.20.21->sagemaker>=2.99.0) (0.10.0)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /opt/conda/lib/python3.7/site-packages (from botocore<1.24.0,>=1.23.47->boto3<2.0,>=1.20.21->sagemaker>=2.99.0) (1.26.6)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /opt/conda/lib/python3.7/site-packages (from botocore<1.24.0,>=1.23.47->boto3<2.0,>=1.20.21->sagemaker>=2.99.0) (2.8.1)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata<5.0,>=1.4.0->sagemaker>=2.99.0) (2.2.0)
Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from packaging>=20.0->sagemaker>=2.99.0) (1.14.0)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging>=20.0->sagemaker>=2.99.0) (2.4.6)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.7/site-packages (from pandas->sagemaker>=2.99.0) (2019.3)
Requirement already satisfied: ppft>=1.6.6.4 in /opt/conda/lib/python3.7/site-packages (from pathos->sagemaker>=2.99.0) (1.6.6.4)
Requirement already satisfied: pox>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from pathos->sagemaker>=2.99.0) (0.3.0)
Requirement already satisfied: multiprocess>=0.70.12 in /opt/conda/lib/python3.7/site-packages (from pathos->sagemaker>=2.99.0) (0.70.12.2)
Requirement already satisfied: dill>=0.3.4 in /opt/conda/lib/python3.7/site-packages (from pathos->sagemaker>=2.99.0) (0.3.4)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.1.3; however, version 22.1.2 is available.
You should consider upgrading via the '/opt/conda/bin/python -m pip install --upgrade pip' command.

Configuration Setup

Let's now configure the setup we need, which includes the session object from the SageMaker Python SDK, and neccessary configurations for the pipelines, such as object types, input and output buckets and so on.

[4]

[5]

Data Preparation

An SKLearn processor is used to prepare the dataset for the Hyperparameter Tuning job. Using the script preprocess.py, the dataset is featurized and split into train, test, and validation datasets.

The output of this step is used as the input to the TrainingStep.

[6]

[7]

Writing code/preprocess.py

[8]

/opt/conda/lib/python3.7/site-packages/sagemaker/workflow/pipeline_context.py:197: UserWarning: Running within a PipelineSession, there will be No Wait, No Logs, and No Job being started.
  UserWarning,


Job Name:  lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005
Inputs:  [{'InputName': 'code', 'AppManaged': False, 'S3Input': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005/input/code/preprocess.py', 'LocalPath': '/opt/ml/processing/input/code', 'S3DataType': 'S3Prefix', 'S3InputMode': 'File', 'S3DataDistributionType': 'FullyReplicated', 'S3CompressionType': 'None'}}]
Outputs:  [{'OutputName': 'train', 'AppManaged': False, 'S3Output': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005/output/train', 'LocalPath': '/opt/ml/processing/train', 'S3UploadMode': 'EndOfJob'}}, {'OutputName': 'validation', 'AppManaged': False, 'S3Output': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005/output/validation', 'LocalPath': '/opt/ml/processing/validation', 'S3UploadMode': 'EndOfJob'}}, {'OutputName': 'test', 'AppManaged': False, 'S3Output': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005/output/test', 'LocalPath': '/opt/ml/processing/test', 'S3UploadMode': 'EndOfJob'}}]

Model Training and Evaluation

We will now train an XGBoost model using the SageMaker Python SDK and the output of the ProcessingStep.

Training the Model

[9]

/opt/conda/lib/python3.7/site-packages/sagemaker/workflow/steps.py:391: UserWarning: Profiling is enabled on the provided estimator. The default profiler rule includes a timestamp which will change each time the pipeline is upserted, causing cache misses. If profiling is not needed, set disable_profiler to True on the estimator.
  warnings.warn(msg)

Evaluating the model

Use a processing job to evaluate the model from the TrainingStep. If the output of the evaluation is True, a model is created and a Lambda function is invoked to deploy the model to a SageMaker Endpoint.

[10]

Writing code/evaluate.py

[11]


Job Name:  lambda-step-pipeline/lambda-step-exampl-2022-07-13-15-58-56-545
Inputs:  [{'InputName': 'input-1', 'AppManaged': False, 'S3Input': {'S3Uri': <sagemaker.workflow.properties.Properties object at 0x7f15b0ee9350>, 'LocalPath': '/opt/ml/processing/model', 'S3DataType': 'S3Prefix', 'S3InputMode': 'File', 'S3DataDistributionType': 'FullyReplicated', 'S3CompressionType': 'None'}}, {'InputName': 'input-2', 'AppManaged': False, 'S3Input': {'S3Uri': <sagemaker.workflow.properties.Properties object at 0x7f15b0f199d0>, 'LocalPath': '/opt/ml/processing/test', 'S3DataType': 'S3Prefix', 'S3InputMode': 'File', 'S3DataDistributionType': 'FullyReplicated', 'S3CompressionType': 'None'}}, {'InputName': 'code', 'AppManaged': False, 'S3Input': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-pipeline/lambda-step-exampl-2022-07-13-15-58-56-545/input/code/evaluate.py', 'LocalPath': '/opt/ml/processing/input/code', 'S3DataType': 'S3Prefix', 'S3InputMode': 'File', 'S3DataDistributionType': 'FullyReplicated', 'S3CompressionType': 'None'}}]
Outputs:  [{'OutputName': 'evaluation', 'AppManaged': False, 'S3Output': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-pipeline/evaluation_report', 'LocalPath': '/opt/ml/processing/evaluation', 'S3UploadMode': 'EndOfJob'}}]

Creating the final model object

The model is created and the name of the model is provided to the Lambda function for deployment. The CreateModelStep dynamically assigns a name to the model.

[12]

Setting up Lambda

When defining the LambdaStep, the SageMaker Lambda helper class provides helper functions for creating the Lambda function. Users can either use the lambda_func argument to provide the function ARN to an already deployed Lambda function OR use the Lambda class to create a Lambda function by providing a script, function name and role for the Lambda function.

When passing inputs to the Lambda, the inputs argument can be used and within the Lambda function's handler, the event argument can be used to retrieve the inputs.

The dictionary response from the Lambda function is parsed through the LambdaOutput objects provided to the outputs argument. The output_name in LambdaOutput corresponds to the dictionary key in the Lambda's return dictionary.

Define the Lambda function

Users can choose the leverage the Lambda helper class to create a Lambda function and provide that function object to the LambdaStep. Alternatively, users can use a pre-deployed Lambda function and provide the function ARN to the Lambda helper class in the Lambda step.

[13]

Writing code/lambda_helper.py

Setting up the custom IAM Role

The Lambda function needs an IAM role that allows it to deploy a SageMaker Endpoint. The role ARN must be provided in the LambdaStep.

The Lambda role should at minimum have policies to allow sagemaker:CreateModel, sagemaker:CreateEndpointConfig, sagemaker:CreateEndpoint in addition to the based Lambda execution policies.

A helper function in iam_helper.py is available to create the Lambda function role. Please note that the role uses the Amazon managed policy - SageMakerFullAccess. This should be replaced with an IAM policy with least privileges as per AWS IAM best practices.

[14]

Using ARN from existing role: lambda-deployment-role

[15]

[16]

[17]

Execute the Pipeline

[18]

{'Version': '2020-12-01',
, 'Metadata': {},
, 'Parameters': [{'Name': 'ProcessingInstanceCount',
,   'Type': 'Integer',
,   'DefaultValue': 1},
,  {'Name': 'TrainingInstanceType',
,   'Type': 'String',
,   'DefaultValue': 'ml.m5.xlarge'},
,  {'Name': 'InputDataUrl',
,   'Type': 'String',
,   'DefaultValue': 's3://sagemaker-sample-files/datasets/tabular/uci_abalone/abalone.csv'},
,  {'Name': 'ModelApprovalStatus',
,   'Type': 'String',
,   'DefaultValue': 'PendingManualApproval'}],
, 'PipelineExperimentConfig': {'ExperimentName': {'Get': 'Execution.PipelineName'},
,  'TrialName': {'Get': 'Execution.PipelineExecutionId'}},
, 'Steps': [{'Name': 'PreprocessAbaloneData',
,   'Type': 'Processing',
,   'Arguments': {'ProcessingResources': {'ClusterConfig': {'InstanceType': 'ml.m5.xlarge',
,      'InstanceCount': {'Get': 'Parameters.ProcessingInstanceCount'},
,      'VolumeSizeInGB': 30}},
,    'AppSpecification': {'ImageUri': '246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-scikit-learn:0.23-1-cpu-py3',
,     'ContainerArguments': ['--input-data',
,      {'Get': 'Parameters.InputDataUrl'}],
,     'ContainerEntrypoint': ['python3',
,      '/opt/ml/processing/input/code/preprocess.py']},
,    'RoleArn': 'arn:aws:iam::000000000000:role/SageMakerRole',
,    'ProcessingInputs': [{'InputName': 'code',
,      'AppManaged': False,
,      'S3Input': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005/input/code/preprocess.py',
,       'LocalPath': '/opt/ml/processing/input/code',
,       'S3DataType': 'S3Prefix',
,       'S3InputMode': 'File',
,       'S3DataDistributionType': 'FullyReplicated',
,       'S3CompressionType': 'None'}}],
,    'ProcessingOutputConfig': {'Outputs': [{'OutputName': 'train',
,       'AppManaged': False,
,       'S3Output': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005/output/train',
,        'LocalPath': '/opt/ml/processing/train',
,        'S3UploadMode': 'EndOfJob'}},
,      {'OutputName': 'validation',
,       'AppManaged': False,
,       'S3Output': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005/output/validation',
,        'LocalPath': '/opt/ml/processing/validation',
,        'S3UploadMode': 'EndOfJob'}},
,      {'OutputName': 'test',
,       'AppManaged': False,
,       'S3Output': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/sklearn-abalone-pre-2022-07-13-15-58-56-005/output/test',
,        'LocalPath': '/opt/ml/processing/test',
,        'S3UploadMode': 'EndOfJob'}}]}},
,   'CacheConfig': {'Enabled': True, 'ExpireAfter': '30d'}},
,  {'Name': 'TrainAbaloneModel',
,   'Type': 'Training',
,   'Arguments': {'AlgorithmSpecification': {'TrainingInputMode': 'File',
,     'TrainingImage': '246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-xgboost:1.0-1-cpu-py3'},
,    'OutputDataConfig': {'S3OutputPath': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/AbaloneTrain'},
,    'StoppingCondition': {'MaxRuntimeInSeconds': 86400},
,    'ResourceConfig': {'VolumeSizeInGB': 30,
,     'InstanceCount': 1,
,     'InstanceType': {'Get': 'Parameters.TrainingInstanceType'}},
,    'RoleArn': 'arn:aws:iam::000000000000:role/SageMakerRole',
,    'InputDataConfig': [{'DataSource': {'S3DataSource': {'S3DataType': 'S3Prefix',
,        'S3Uri': {'Get': "Steps.PreprocessAbaloneData.ProcessingOutputConfig.Outputs['train'].S3Output.S3Uri"},
,        'S3DataDistributionType': 'FullyReplicated'}},
,      'ContentType': 'text/csv',
,      'ChannelName': 'train'},
,     {'DataSource': {'S3DataSource': {'S3DataType': 'S3Prefix',
,        'S3Uri': {'Get': "Steps.PreprocessAbaloneData.ProcessingOutputConfig.Outputs['validation'].S3Output.S3Uri"},
,        'S3DataDistributionType': 'FullyReplicated'}},
,      'ContentType': 'text/csv',
,      'ChannelName': 'validation'}],
,    'HyperParameters': {'objective': 'reg:linear',
,     'num_round': '50',
,     'max_depth': '5',
,     'eta': '0.2',
,     'gamma': '4',
,     'min_child_weight': '6',
,     'subsample': '0.7',
,     'silent': '0'},
,    'ProfilerRuleConfigurations': [{'RuleConfigurationName': 'ProfilerReport-1657727936',
,      'RuleEvaluatorImage': '895741380848.dkr.ecr.us-west-2.amazonaws.com/sagemaker-debugger-rules:latest',
,      'RuleParameters': {'rule_to_invoke': 'ProfilerReport'}}],
,    'ProfilerConfig': {'S3OutputPath': 's3://sagemaker-us-west-2-000000000000/lambda-step-example/AbaloneTrain'}},
,   'CacheConfig': {'Enabled': True, 'ExpireAfter': '30d'}},
,  {'Name': 'EvaluateAbaloneModel',
,   'Type': 'Processing',
,   'Arguments': {'ProcessingResources': {'ClusterConfig': {'InstanceType': 'ml.m5.xlarge',
,      'InstanceCount': 1,
,      'VolumeSizeInGB': 30}},
,    'AppSpecification': {'ImageUri': '246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-xgboost:1.0-1-cpu-py3',
,     'ContainerEntrypoint': ['python3',
,      '/opt/ml/processing/input/code/evaluate.py']},
,    'RoleArn': 'arn:aws:iam::000000000000:role/SageMakerRole',
,    'ProcessingInputs': [{'InputName': 'input-1',
,      'AppManaged': False,
,      'S3Input': {'S3Uri': {'Get': 'Steps.TrainAbaloneModel.ModelArtifacts.S3ModelArtifacts'},
,       'LocalPath': '/opt/ml/processing/model',
,       'S3DataType': 'S3Prefix',
,       'S3InputMode': 'File',
,       'S3DataDistributionType': 'FullyReplicated',
,       'S3CompressionType': 'None'}},
,     {'InputName': 'input-2',
,      'AppManaged': False,
,      'S3Input': {'S3Uri': {'Get': "Steps.PreprocessAbaloneData.ProcessingOutputConfig.Outputs['test'].S3Output.S3Uri"},
,       'LocalPath': '/opt/ml/processing/test',
,       'S3DataType': 'S3Prefix',
,       'S3InputMode': 'File',
,       'S3DataDistributionType': 'FullyReplicated',
,       'S3CompressionType': 'None'}},
,     {'InputName': 'code',
,      'AppManaged': False,
,      'S3Input': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-pipeline/lambda-step-exampl-2022-07-13-15-58-56-545/input/code/evaluate.py',
,       'LocalPath': '/opt/ml/processing/input/code',
,       'S3DataType': 'S3Prefix',
,       'S3InputMode': 'File',
,       'S3DataDistributionType': 'FullyReplicated',
,       'S3CompressionType': 'None'}}],
,    'ProcessingOutputConfig': {'Outputs': [{'OutputName': 'evaluation',
,       'AppManaged': False,
,       'S3Output': {'S3Uri': 's3://sagemaker-us-west-2-000000000000/lambda-step-pipeline/evaluation_report',
,        'LocalPath': '/opt/ml/processing/evaluation',
,        'S3UploadMode': 'EndOfJob'}}]}},
,   'CacheConfig': {'Enabled': True, 'ExpireAfter': '30d'},
,   'PropertyFiles': [{'PropertyFileName': 'AbaloneEvaluationReport',
,     'OutputName': 'evaluation',
,     'FilePath': 'evaluation.json'}]},
,  {'Name': 'CheckMSEAbaloneEvaluation',
,   'Type': 'Condition',
,   'Arguments': {'Conditions': [{'Type': 'LessThanOrEqualTo',
,      'LeftValue': {'Std:JsonGet': {'PropertyFile': {'Get': 'Steps.EvaluateAbaloneModel.PropertyFiles.AbaloneEvaluationReport'},
,        'Path': 'regression_metrics.mse.value'}},
,      'RightValue': 6.0}],
,    'IfSteps': [{'Name': 'CreateModel-CreateModel',
,      'Type': 'Model',
,      'Arguments': {'ExecutionRoleArn': 'arn:aws:iam::000000000000:role/SageMakerRole',
,       'PrimaryContainer': {'Image': '246618743249.dkr.ecr.us-west-2.amazonaws.com/sagemaker-xgboost:1.0-1-cpu-py3',
,        'Environment': {},
,        'ModelDataUrl': {'Get': 'Steps.TrainAbaloneModel.ModelArtifacts.S3ModelArtifacts'}}}},
,     {'Name': 'LambdaStep',
,      'Type': 'Lambda',
,      'Arguments': {'model_name': {'Get': 'Steps.CreateModel-CreateModel.ModelName'},
,       'endpoint_config_name': 'demo-lambda-deploy-endpoint-config-07-13-15-58-57',
,       'endpoint_name': 'demo-lambda-deploy-endpoint-07-13-15-58-57'},
,      'FunctionArn': 'arn:aws:lambda:us-west-2:000000000000:function:sagemaker-lambda-step-endpoint-deploy-07-13-15-58-57',
,      'OutputParameters': [{'OutputName': 'statusCode',
,        'OutputType': 'String'},
,       {'OutputName': 'body', 'OutputType': 'String'},
,       {'OutputName': 'other_key', 'OutputType': 'String'}]}],
,    'ElseSteps': []}}]}

[19]

{'PipelineArn': 'arn:aws:sagemaker:us-west-2:000000000000:pipeline/lambda-step-pipeline07-13-15-58-57',
, 'ResponseMetadata': {'RequestId': 'b8946d56-ca49-4275-add7-d48f765b4931',
,  'HTTPStatusCode': 200,
,  'HTTPHeaders': {'x-amzn-requestid': 'b8946d56-ca49-4275-add7-d48f765b4931',
,   'content-type': 'application/x-amz-json-1.1',
,   'content-length': '102',
,   'date': 'Wed, 13 Jul 2022 15:58:59 GMT'},
,  'RetryAttempts': 0}}

[20]

[21]

[22]

Clean up resources

Running the following cell will delete the following resources created in this notebook -

SageMaker Model
SageMaker Endpoint Configuration
SageMaker Endpoint
SageMaker Pipeline
Lambda Function

[23]

{'PipelineArn': 'arn:aws:sagemaker:us-west-2:000000000000:pipeline/lambda-step-pipeline07-13-15-58-57',
, 'ResponseMetadata': {'RequestId': 'f71a3ce5-bac1-4250-9129-4161507f62d4',
,  'HTTPStatusCode': 200,
,  'HTTPHeaders': {'x-amzn-requestid': 'f71a3ce5-bac1-4250-9129-4161507f62d4',
,   'content-type': 'application/x-amz-json-1.1',
,   'content-length': '102',
,   'date': 'Wed, 13 Jul 2022 16:16:08 GMT'},
,  'RetryAttempts': 0}}

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.