Inference Pipeline
data-scienceinferencearchivedamazon-sagemaker-examplesreinforcement-learningmachine-learninginference_pipeline_custom_containersawsexamplesdeep-learningsagemakerjupyter-notebooktrainingmlops
Export
Inference Pipeline with Custom Containers and xgBoost
This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.
Typically a Machine Learning (ML) process consists of few steps: data gathering with various ETL jobs, pre-processing the data, featurizing the dataset by incorporating standard techniques or prior knowledge, and finally training an ML model using an algorithm. In many cases, when the trained model is used for processing real time or batch prediction requests, the model receives data in a format which needs to pre-processed (e.g. featurized) before it can be passed to the algorithm. In the following notebook, we will demonstrate how you can build your ML Pipeline leveraging the ability to create custom Sagemaker algorithms and the out of the box SageMaker xgBoost algorithm. After the model is trained we will deploy the ML Pipeline (data preprocessing, the xgBoost classifier, and data postprocessing) as an Inference Pipeline behind a single SageMaker Endpoint for real time inference. We will also use the preprocessor with batch transformation using Amazon SageMaker Batch Transform to prepare xgBoost training data.
The toy problem that is being solved here is to match a set of keywords to a category of questions. From there we can match that category against a list of available agents who specialize in answering that category of question. The agents and their availability is stored externally in a DynamoDB database. The data transformations, matching against our model, and querying of the database are all done as part of the inference pipeline.
The preprocessing step of the pipeline encodes a comma-separated list of words into a format that xgBoost understands using a CountVectorizer. It also trains a LabelEncoder, which is used to transform from the categories of questions to a set of integers - having the labels encoded as integers is also a requirement of the xgBoost multiclass classifer.
The xgBoost model maps the encoded list of words to an integer, which represents the encoded class of question that best matches those words.
Finally, the postprocessing step of the pipeline uses the LabelEncoding model trained in the preprocessing step to map the number representing the classification of the question back to the text. It then takes the category and queries dynamodb for available agents that matches that category.
Let's first create our Sagemaker session and role, and create a S3 prefix to use for the notebook example.
[1]
[2]
[3]
[4]
[5]
Set up a loader function
The load_data function pulls in the CSV data into two columns: the first column of the CSV is mapped to the label, and every subsequent CSV column is loaded as a dictionary into the second Pandas column
[6]
[7]
[8]
[9]
6
Words in our dataset
Let's take a look at the set of words being used. We use a CountVectorizer with the set analyzer to encode the column.
[10]
['401k', , '403b', , 'capital', , 'charitable', , 'covid', , 'deduction', , 'deferment', , 'delay', , 'donation', , 'estate', , 'expense', , 'gains', , 'inheritance', , 'investment', , 'ira', , 'itemization', , 'late', , 'local', , 'losses', , 'medical', , 'mortgage', , 'payment', , 'properties', , 'rental', , 'state', , 'tax']
[11]
305752278501.dkr.ecr.us-west-2.amazonaws.com/custompipeline/preprocessor:latest
[12]
Parameter image_name will be renamed to image_uri in SageMaker Python SDK v2. 's3_input' class will be renamed to 'TrainingInput' in SageMaker Python SDK v2. 's3_input' class will be renamed to 'TrainingInput' in SageMaker Python SDK v2. 's3_input' class will be renamed to 'TrainingInput' in SageMaker Python SDK v2.
2020-10-02 00:28:04 Starting - Starting the training job... 2020-10-02 00:28:06 Starting - Launching requested ML instances...... 2020-10-02 00:29:07 Starting - Preparing the instances for training... 2020-10-02 00:29:59 Downloading - Downloading input data 2020-10-02 00:29:59 Training - Downloading the training image.....Starting script arguments: ['main.py', 'train'] starting training... Hyperparameters configuration: {} Input data configuration: {'train': {'RecordWrapperType': 'None', 'S3DistributionType': 'FullyReplicated', 'TrainingInputMode': 'File'}} List of files in train channel: /opt/ml/input/data/train/samples.csv Resource configuration: {'current_host': 'algo-1', 'hosts': ['algo-1'], 'network_interface_name': 'eth0'} <class 'pandas.core.frame.DataFrame'> RangeIndex: 100000 entries, 0 to 99999 Data columns (total 2 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 label 100000 non-null object 1 words 100000 non-null object dtypes: object(2) memory usage: 1.5+ MB None fitting... finished fitting... ['401k', '403b', 'capital', 'charitable', 'covid', 'deduction', 'deferment', 'delay', 'donation', 'estate', 'expense', 'gains', 'inheritance', 'investment', 'ira', 'itemization', 'late', 'local', 'losses', 'medical', 'mortgage', 'payment', 'properties', 'rental', 'state', 'tax'] le classes: ['category_deferments' 'category_estate taxes' 'category_investments' 'category_itemization' 'category_medical' 'category_properties'] saved model! 2020-10-02 00:30:56 Uploading - Uploading generated training model 2020-10-02 00:30:56 Completed - Training job completed Training seconds: 69 Billable seconds: 69
[13]
Parameter image will be renamed to image_uri in SageMaker Python SDK v2.
-----------!
[14]
content type csv text/csv b'0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0\n0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0\n'
[ ]
[15]
Parameter image will be renamed to image_uri in SageMaker Python SDK v2. Using already existing model: preprocessor-2020-10-02-00-28-04-621
[16]
Waiting for transform job: preprocessor-2020-10-02-00-36-49-266 ........................... 2020-10-02T00:41:08.139:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD Starting script arguments: ['main.py', 'serve'] Starting the inference server with 4 workers. using port: 8080 nginx.conf: worker_processes 1; daemon off; #Prevent forking pid /tmp/nginx.pid; error_log /var/log/nginx/error.log; events { # defaults } Starting script arguments: ['main.py', 'serve'] Starting the inference server with 4 workers. using port: 8080 nginx.conf: worker_processes 1; daemon off; #Prevent forking pid /tmp/nginx.pid; error_log /var/log/nginx/error.log; events { # defaults } http { include /etc/nginx/mime.types; default_type application/octet-stream; access_log /var/log/nginx/access.log combined; upstream gunicorn { server unix:/tmp/gunicorn.sock; } server { listen 8080 deferred; client_max_body_size 5m; keepalive_timeout 5; location ~ ^/(ping|invocations) { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_redirect off; proxy_pass http://gunicorn; } location / { return 404 "{}"; } } } 2020/10/02 00:41:06 [crit] 18#18: *1 connect() to unix:/tmp/gunicorn.sock failed (2: No such file or directory) while connecting to upstream, client: 169.254.255.130, server: , request: "GET /ping HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock:/ping", host: "169.254.255.131:8080" 169.254.255.130 - - [02/Oct/2020:00:41:06 +0000] "GET /ping HTTP/1.1" 502 182 "-" "Go-http-client/1.1" 2020/10/02 00:41:06 [crit] 18#18: *3 connect() to unix:/tmp/gunicorn.sock failed (2: No such file or directory) while connecting to upstream, client: 169.254.255.130, server: , request: "GET /ping HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock:/ping", host: "169.254.255.131:8080" 169.254.255.130 - - [02/Oct/2020:00:41:06 +0000] "GET /ping HTTP/1.1" 502 182 "-" "Go-http-client/1.1" [2020-10-02 00:41:06 +0000] [17] [INFO] Starting gunicorn 20.0.4 [2020-10-02 00:41:06 +0000] [17] [INFO] Listening at: unix:/tmp/gunicorn.sock (17) [2020-10-02 00:41:06 +0000] [17] [INFO] Using worker: gevent [2020-10-02 00:41:06 +0000] [21] [INFO] Booting worker with pid: 21 [2020-10-02 00:41:06 +0000] [22] [INFO] Booting worker with pid: 22 [2020-10-02 00:41:06 +0000] [23] [INFO] Booting worker with pid: 23 [2020-10-02 00:41:06 +0000] [24] [INFO] Booting worker with pid: 24 169.254.255.130 - - [02/Oct/2020:00:41:08 +0000] "GET /ping HTTP/1.1" 200 1 "-" "Go-http-client/1.1" 169.254.255.130 - - [02/Oct/2020:00:41:08 +0000] "GET /execution-parameters HTTP/1.1" 404 2 "-" "Go-http-client/1.1" data: b'label,words\ncategory_properties,rental,investment,properties\ncategory_medical,medical,covid\ncategory' cookies: ImmutableMultiDict([]) headers: {'X-Forwarded-For': '169.254.255.130', 'Host': '169.254.255.131:8080', 'Connection': 'close', 'Content-Length': '4265739', 'User-Agent': 'Go-http-client/1.1', 'Accept': 'text/csv', 'Content-Type': 'text/csv', 'X-Amzn-Sagemaker-Input-Object': 'sagemaker-us-west-2-305752278501/Custom-Pipeline-Inference-Example/train/samples.csv', 'X-Amzn-Sagemaker-Input-Object-Base64': 'c2FnZW1ha2VyLXVzLXdlc3QtMi0zMDU3NTIyNzg1MDEvQ3VzdG9tLVBpcGVsaW5lLUluZmVyZW5jZS1FeGFtcGxlL3RyYWluL3NhbXBsZXMuY3N2', 'Accept-Encoding': 'gzip'} http { include /etc/nginx/mime.types; default_type application/octet-stream; access_log /var/log/nginx/access.log combined; upstream gunicorn { server unix:/tmp/gunicorn.sock; } server { listen 8080 deferred; client_max_body_size 5m; keepalive_timeout 5; location ~ ^/(ping|invocations) { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_redirect off; proxy_pass http://gunicorn; } location / { return 404 "{}"; } } } 2020/10/02 00:41:06 [crit] 18#18: *1 connect() to unix:/tmp/gunicorn.sock failed (2: No such file or directory) while connecting to upstream, client: 169.254.255.130, server: , request: "GET /ping HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock:/ping", host: "169.254.255.131:8080" 169.254.255.130 - - [02/Oct/2020:00:41:06 +0000] "GET /ping HTTP/1.1" 502 182 "-" "Go-http-client/1.1" 2020/10/02 00:41:06 [crit] 18#18: *3 connect() to unix:/tmp/gunicorn.sock failed (2: No such file or directory) while connecting to upstream, client: 169.254.255.130, server: , request: "GET /ping HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock:/ping", host: "169.254.255.131:8080" 169.254.255.130 - - [02/Oct/2020:00:41:06 +0000] "GET /ping HTTP/1.1" 502 182 "-" "Go-http-client/1.1" [2020-10-02 00:41:06 +0000] [17] [INFO] Starting gunicorn 20.0.4 [2020-10-02 00:41:06 +0000] [17] [INFO] Listening at: unix:/tmp/gunicorn.sock (17) [2020-10-02 00:41:06 +0000] [17] [INFO] Using worker: gevent [2020-10-02 00:41:06 +0000] [21] [INFO] Booting worker with pid: 21 [2020-10-02 00:41:06 +0000] [22] [INFO] Booting worker with pid: 22 [2020-10-02 00:41:06 +0000] [23] [INFO] Booting worker with pid: 23 [2020-10-02 00:41:06 +0000] [24] [INFO] Booting worker with pid: 24 169.254.255.130 - - [02/Oct/2020:00:41:08 +0000] "GET /ping HTTP/1.1" 200 1 "-" "Go-http-client/1.1" 169.254.255.130 - - [02/Oct/2020:00:41:08 +0000] "GET /execution-parameters HTTP/1.1" 404 2 "-" "Go-http-client/1.1" data: b'label,words\ncategory_properties,rental,investment,properties\ncategory_medical,medical,covid\ncategory' cookies: ImmutableMultiDict([]) headers: {'X-Forwarded-For': '169.254.255.130', 'Host': '169.254.255.131:8080', 'Connection': 'close', 'Content-Length': '4265739', 'User-Agent': 'Go-http-client/1.1', 'Accept': 'text/csv', 'Content-Type': 'text/csv', 'X-Amzn-Sagemaker-Input-Object': 'sagemaker-us-west-2-305752278501/Custom-Pipeline-Inference-Example/train/samples.csv', 'X-Amzn-Sagemaker-Input-Object-Base64': 'c2FnZW1ha2VyLXVzLXdlc3QtMi0zMDU3NTIyNzg1MDEvQ3VzdG9tLVBpcGVsaW5lLUluZmVyZW5jZS1FeGFtcGxlL3RyYWluL3NhbXBsZXMuY3N2', 'Accept-Encoding': 'gzip'} args: ImmutableMultiDict([]) args: ImmutableMultiDict([]) Content type text/csv Accept text/csv First entry is: label Length indicates that label is included merged df label words 0 category_properties {rental, investment, properties} 1 category_medical {covid, medical} 2 category_itemization {itemization, donation} 3 category_estate taxes {estate, inheritance, medical} 4 category_estate taxes {estate} label_column in input_data Content type text/csv Accept text/csv First entry is: label Length indicates that label is included merged df label words 0 category_properties {rental, investment, properties} 1 category_medical {covid, medical} 2 category_itemization {itemization, donation} 3 category_estate taxes {estate, inheritance, medical} 4 category_estate taxes {estate} label_column in input_data 169.254.255.130 - - [02/Oct/2020:00:41:11 +0000] "POST /invocations HTTP/1.1" 200 5400000 "-" "Go-http-client/1.1" 169.254.255.130 - - [02/Oct/2020:00:41:11 +0000] "POST /invocations HTTP/1.1" 200 5400000 "-" "Go-http-client/1.1" s3://sagemaker-us-west-2-305752278501/preprocessor-2020-10-02-00-36-49-266
[17]
'get_image_uri' method will be deprecated in favor of 'ImageURIProvider' class in SageMaker Python SDK v2. There is a more up to date SageMaker XGBoost image. To use the newer image, please set 'repo_version'='1.0-1'. For example: get_image_uri(region, 'xgboost', '1.0-1').
'433757028032.dkr.ecr.us-west-2.amazonaws.com/xgboost:1'
[18]
[19]
Parameter image_name will be renamed to image_uri in SageMaker Python SDK v2.
[20]
's3_input' class will be renamed to 'TrainingInput' in SageMaker Python SDK v2.
[21]
2020-10-02 00:41:34 Starting - Starting the training job... 2020-10-02 00:41:36 Starting - Launching requested ML instances...... 2020-10-02 00:42:37 Starting - Preparing the instances for training... 2020-10-02 00:43:31 Downloading - Downloading input data... 2020-10-02 00:44:04 Training - Training image download completed. Training in progress. 2020-10-02 00:44:04 Uploading - Uploading generated training model 2020-10-02 00:44:04 Completed - Training job completed Arguments: train [2020-10-02:00:43:52:INFO] Running standalone xgboost training. [2020-10-02:00:43:52:INFO] Path /opt/ml/input/data/validation does not exist! [2020-10-02:00:43:52:INFO] File size need to be processed in the node: 5.15mb. Available memory size in the node: 54839.87mb [2020-10-02:00:43:52:INFO] Determined delimiter of CSV input is ',' [00:43:52] S3DistributionType set as FullyReplicated [00:43:52] 100000x26 matrix with 2600000 entries loaded from /opt/ml/input/data/train?format=csv&label_column=0&delimiter=, [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 46 extra nodes, 0 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 30 extra nodes, 0 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 56 extra nodes, 0 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 2 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 34 extra nodes, 4 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 40 extra nodes, 4 pruned nodes, max_depth=5 [0]#011train-merror:0.0507 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 48 extra nodes, 10 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 30 extra nodes, 0 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 6 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 2 pruned nodes, max_depth=5 [00:43:52] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 34 extra nodes, 4 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 32 extra nodes, 4 pruned nodes, max_depth=5 [1]#011train-merror:0.03634 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 48 extra nodes, 8 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 30 extra nodes, 2 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 4 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 0 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 36 extra nodes, 4 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 34 extra nodes, 4 pruned nodes, max_depth=5 [2]#011train-merror:0.03495 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 42 extra nodes, 2 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 28 extra nodes, 2 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 0 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 6 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 32 extra nodes, 8 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 36 extra nodes, 0 pruned nodes, max_depth=5 [3]#011train-merror:0.03284 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 40 extra nodes, 6 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 12 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 50 extra nodes, 6 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 50 extra nodes, 6 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 34 extra nodes, 2 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 34 extra nodes, 4 pruned nodes, max_depth=5 [4]#011train-merror:0.03009 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 40 extra nodes, 6 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 26 extra nodes, 10 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 50 extra nodes, 4 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 4 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 30 extra nodes, 6 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 32 extra nodes, 4 pruned nodes, max_depth=5 [5]#011train-merror:0.02876 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 38 extra nodes, 8 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 12 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 8 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 48 extra nodes, 6 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 36 extra nodes, 0 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 30 extra nodes, 6 pruned nodes, max_depth=5 [6]#011train-merror:0.02767 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 34 extra nodes, 12 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 20 extra nodes, 10 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 44 extra nodes, 10 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 48 extra nodes, 4 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 34 extra nodes, 2 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 34 extra nodes, 4 pruned nodes, max_depth=5 [7]#011train-merror:0.02629 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 28 extra nodes, 18 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 18 extra nodes, 12 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 44 extra nodes, 12 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 50 extra nodes, 4 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 28 extra nodes, 12 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 40 extra nodes, 4 pruned nodes, max_depth=5 [8]#011train-merror:0.02592 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 30 extra nodes, 16 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 22 extra nodes, 14 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 40 extra nodes, 12 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 52 extra nodes, 4 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 32 extra nodes, 12 pruned nodes, max_depth=5 [00:43:53] src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 38 extra nodes, 4 pruned nodes, max_depth=5 [9]#011train-merror:0.02487 Training seconds: 33 Billable seconds: 33
Set up the inference pipeline
Setting up a Machine Learning pipeline can be done with the Pipeline Model. This sets up a list of models in a single endpoint; in this example, we configure our pipeline model with the fitted preprocessor model, the fitted xgBoost model, and the postprocessor (which uses the preprocessor model data). Deploying the model follows the same deploy pattern in the SDK.
Notice that we pass in the trained model from the proprocessor into the postpostprocessor. The reason we do this is so that the postprocesser can access the LabelEncoder that was trained in the preprocessor to invert that operation. This allows the inference pipeline to return the actual name of the category instead of the category label (e.g. "medical" instead of 7).
[22]
'305752278501.dkr.ecr.us-west-2.amazonaws.com/custompipeline/postprocessor:latest'
[23]
Parameter image will be renamed to image_uri in SageMaker Python SDK v2.
[24]
Parameter image will be renamed to image_uri in SageMaker Python SDK v2. Parameter image will be renamed to image_uri in SageMaker Python SDK v2.
-------------!
Make a request to our pipeline endpoint
Here we just grab the first line from the test data (you'll notice that the inference python script is very particular about the ordering of the inference request data). The ContentType field configures the first container, while the Accept field configures the last container. You can also specify each container's Accept and ContentType values using environment variables.
[25]
b'{"response": [{"category": "properties", "agent": {"ID": "2345", "FirstName": "Megan", "LastName": "Duvernoy"}}, {"category": "medical", "agent": {"ID": "5678", "FirstName": "Mohammad", "LastName": "Asif"}}]}'
[26]
{'ResponseMetadata': {'RequestId': '9eda73a2-7319-4e8a-ae5e-a0edf85d86d2',
, 'HTTPStatusCode': 200,
, 'HTTPHeaders': {'x-amzn-requestid': '9eda73a2-7319-4e8a-ae5e-a0edf85d86d2',
, 'content-type': 'application/x-amz-json-1.1',
, 'content-length': '0',
, 'date': 'Fri, 02 Oct 2020 00:50:50 GMT'},
, 'RetryAttempts': 0}} Notebook CI Test Results
This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.