Copyright (c) Microsoft Corporation. All rights reserved.
Licensed under the MIT License.

How to Setup a PipelineEndpoint and Submit a Pipeline Using the PipelineEndpoint.

In this notebook, we will see how to setup a PipelineEndpoint and run a specific pipeline version.

PipelineEndpoint can be used to update a published pipeline while maintaining the same endpoint. PipelineEndpoint provides a way to keep track of PublishedPipelines using versions. PipelineEndpoint uses endpoint with version information to trigger an underlying published pipeline. Pipeline endpoints are uniquely named within a workspace.

Prerequisites and AML Basics

If you are using an Azure Machine Learning Notebook VM, you are all set. Otherwise, make sure you go through the configuration Notebook first if you haven't. This sets you up with a working config file that has information on your workspace, subscription id, etc.

Notebook Overview

In this notebook, we provide an introduction to Azure machine learning PipelineEndpoints. It covers:

• Create PipelineEndpoint, How to create PipelineEndpoint.
• Retrieving PipelineEndpoint, How to get specific PipelineEndpoint from worskpace by name/Id and get all PipelineEndpoints within workspace.
• PipelineEndpoint Properties. How to get and set PipelineEndpoint properties, such as default version of PipelineEndpoint.
• PipelineEndpoint Submission. How to run a Pipeline using PipelineEndpoint.

Create PipelineEndpoint

Following are required input parameters to create PipelineEndpoint:

• workspace: AML workspace.
• name: name of PipelineEndpoint, it is unique within workspace.
• description: description details for PipelineEndpoint.
• pipeline: A Pipeline or PublishedPipeline, to set default version of PipelineEndpoint.

Initialization, Steps to create a Pipeline

The best practice is to use separate folders for scripts and its dependent files for each step and specify that folder as the source_directory for the step. This helps reduce the size of the snapshot created for the step (only the specific folder is snapshotted). Since changes in any files in the source_directory would trigger a re-upload of the snapshot, this helps keep the reuse of the step when there are no changes in the source_directory of the step.

Note that if you have an AzureML Data Scientist role, you will not have permission to create compute resources. Talk to your workspace or IT admin to create the compute targets described in this section, if they do not already exist.

Publish Pipeline

Publishing PipelineEndpoint

Create PipelineEndpoint with required parameters: workspace, name, description and pipeline

Retrieving PipelineEndpoint

PipelineEndpoint is uniquely defined by name and id within workspace. PipelineEndpoint in workspace can be retrived by Id or by name.

Get PipelineEndpoint by Name

Get PipelineEndpoint by Id

Get all PipelineEndpoints in workspace

Returns all PipelineEndpoints within workspace

PipelineEndpoint properties

Default Version of PipelineEndpoint

Default version of PipelineEndpoint starts from "0" and increments on addition of pipelines.

Get the Default Version

Set default version

Get the Published Pipeline corresponds to specific version of PipelineEndpoint

Get default version Published Pipeline

Add Published Pipeline to PipelineEndpoint,

Adds a published pipeline (if its not present) using add() and if you want to add and set to default use add_default()

Add Published pipeline to PipelineEndpoint and set it to default version

Adding published pipeline to PipelineEndpoint if not present and set it to default

Get all Versions in PipelineEndpoint

Returns list of published pipelines and its versions

Get all Published Pipelines in PipelineEndpoint

Returns all active pipelines in PipelineEnpoint, if active_only flag is set to True.

Name property of PipelineEndpoint

PipelineEndpoint is uniquely identified by name

Set Name PipelineEndpoint

PipelineEndpoint Submission

PipelineEndpoint triggers specific versioned pipeline or default pipeline by:

• Rest Endpoint
• Submit call.

Run Pipeline by endpoint property of PipelineEndpoint

Run specific pipeline using endpoint property of PipelineEndpoint and executing http post.

This notebook shows how to authenticate to AML workspace.

Run Pipeline by Submit call of PipelineEndpoint

Run specific pipeline using Submit api of PipelineEndpoint

Use Experiment.Submit() to Submit Pipeline

Run specific pipeline using Experiment submit api