E2E Phi 3 Vision Image Text To Text Online Endpoint

(Visual) Chat Completion inference using Online Endpoints

This sample shows how to deploy `Phi-3-vision-128k-instruct' to an online endpoint for inference.

Outline

• Set up pre-requisites
• Pick a model to deploy
• Download and prepare data for inference
• Deploy the model for real time inference
• Test the endpoint
• Test the endpoint using Azure OpenAI style payload
• Clean up resources

1. Set up pre-requisites

• Install dependencies
• Connect to AzureML Workspace. Learn more at set up SDK authentication. Replace <WORKSPACE_NAME>, <RESOURCE_GROUP> and <SUBSCRIPTION_ID> below.
• Connect to azureml system registry

2. Deploy the model to an online endpoint

Online endpoints give a durable REST API that can be used to integrate with applications that need to use the model.

3. Test the endpoint with sample data

We will send a sample request to the model, using the json that we create below.

4. Test the endpoint using Azure OpenAI style payload

We will send a sample request with Azure OpenAI Style payload to the model.

5. Delete the online endpoint

Don't forget to delete the online endpoint, else you will leave the billing meter running for the compute used by the endpoint.