Azure AI Search as a vector database + Azure Functions for GPT integration in ChatGPT

This notebook provides step by step instuctions on using Azure AI Search (f.k.a Azure Cognitive Search) as a vector database with OpenAI embeddings, then creating an Azure Function on top to plug into a Custom GPT in ChatGPT.

This can be a solution for customers looking to set up RAG infrastructure contained within Azure, and exposing it as an endpoint to integrate that with other platforms such as ChatGPT.

Azure AI Search is a cloud search service that gives developers infrastructure, APIs, and tools for building a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications.

Azure Functions is a serverless compute service that runs event-driven code, automatically managing infrastructure, scaling, and integrating with other Azure services.

Prerequisites:

For the purposes of this exercise you must have the following:

• Azure user with permission to create Azure AI Search Service and Azure Function Apps
• Azure subscription ID and a resource group.
• OpenAI Key

Architecture

Below is a diagram of the architecture of this solution, which we'll walk through step-by-step.

Note: This architecture pattern of vector data store + serverless functions can be extrapolated to other vector data stores. For example, if you would want to use something like Postgres within Azure, you'd change the Configure Azure AI Search Settings step to set-up the requirements for Postgres, you'd modify the Create Azure AI Vector Search to create the database and table in Postgres instead, and you'd update the function_app.py code in this repository to query Postgres instead of Azure AI Search. The data preparation and creation of the Azure Function would stay consistent.

Table of Contents:

1. Setup of Environment Setup environment by installing and importing the required libraries and configuring our Azure settings. Includes:

   • Install and Import Required Libraries
   • Configure OpenAI Settings
   • Configure Azure AI Search Settings

2. Prepare Data Prepare the data for uploading by embedding the documents, as well as capturing additional metadata. We will use a subset of OpenAI's docs as example data for this.

3. Create Azure AI Vector Search Create an Azure AI Vector Search and upload the data we've prepared. Includes:

   • Create Index: Steps to create an index in Azure AI Search.
   • Upload Data: Instructions to upload data to Azure AI Search.
   • Test Search: Steps to test the search functionality.

4. Create Azure Function Create an Azure Function to interact with the Azure AI Vector Search. Includes:

   • Create Storage Account: Steps to create a storage account for the Azure Function.
   • Create Function App: Instructions to create a function app in Azure.

5. Input in a Custom GPT in ChatGPT Integrate the Azure Function with a Custom GPT in ChatGPT. Includes:

   • Create OpenAPI Spec: Steps to create an OpenAPI specification for the Azure Function.
   • Create GPT Instructions: Instructions to create GPT-specific instructions for the integration.

Set up environment

We'll set up our environment by importing the required libraries and configuring our Azure settings.

Install and import required libraries

We categorize these libraries into standard Python libraries, third-party libraries, and Azure-related libraries for readability.

Configure OpenAI settings

Before going through this section, make sure you have your OpenAI API key.

Configure Azure AI Search Settings

You can locate your Azure AI Search service details in the Azure Portal or programmatically via the Search Management SDK.

Prerequisites:

• Subscription ID from Azure
• Resource Group name from Azure
• Region in Azure

Create and Configure Azure AI Search Service

Below we'll generate a unique name for the search service, set up the service properties, and create the search service.

Get the Search Service API Key

Now that we have the search service up and running, we need the Search Service API Key, which we'll use to initiate the index creation, and later to execute the search.

Prepare data

We're going to embed and store a few pages of the OpenAI docs in the oai_docs folder. We'll first embed each, add it to a CSV, and then use that CSV to upload to the index.

In order to handle longer text files beyond the context of 8191 tokens, we can either use the chunk embeddings separately, or combine them in some way, such as averaging (weighted by the size of each chunk).

We will take a function from Python's own cookbook that breaks up a sequence into chunks.

Now we define a function that encodes a string into tokens and then breaks it up into chunks. We'll use tiktoken, a fast open-source tokenizer by OpenAI.

To read more about counting tokens with Tiktoken, check out this cookbook.

Finally, we can write a function that safely handles embedding requests, even when the input text is longer than the maximum context length, by chunking the input tokens and embedding each chunk individually. The average flag can be set to True to return the weighted average of the chunk embeddings, or False to simply return the unmodified list of chunk embeddings.

Note: there are other, more sophisticated techniques you can take here, including:

• using GPT-4o to capture images/chart descriptions for embedding.
• keeping text overlap between the chunks to minimize cutting off important context.
• chunking based on paragraphs or sections.
• adding more descriptive metadata about each article.

Next, we can define a helper function that will capture additional metadata about the documents. This is useful to use as a metadata filter for search queries, and capturing richer data for search.

In this example, I'll choose from a list of categories to use later on in a metadata filter.

Now, we can define some helper functions to process the .txt files in the oai_docs folder within the data folder. You can use this with your own data as well and supports both .txt and .pdf files.

We'll now use this helper function to process our OpenAI documentation. Feel free to update this to use your own data by changing the folder in process_files below.

Note that this will process the documents in chosen folder concurrently, so this should take <30 seconds if using txt files, and slightly longer if using PDFs.

We now have an embedded_data.csv file with six columns that we can upload to our vector database!

Create Azure AI Vector Search

Create index

We'll define and create a search index using the SearchIndexClient from the Azure AI Search Python SDK. The index incorporates both vector search and hybrid search capabilities. For more details, visit Microsoft's documentation on how to Create a Vector Index

Upload Data

Now we'll upload the articles from above that we've stored in embedded_data.csv from a pandas DataFrame to an Azure AI Search index. For a detailed guide on data import strategies and best practices, refer to Data Import in Azure AI Search.

Test search

Now that the data is uploaded, we'll test both vector similarity search and hybrid search locally below to make sure it is working as expected.

You can test both a pure vector search and hybrid search. Pure vector search passes in None to the search_text below and will only search on vector similarity. Hybrid search will combines the capabilities of traditional keyword-based search by passing in the query text query to the search_text with vector-based similarity search to provide more relevant and contextual results.

Create Azure Function

Azure Functions are an easy way to build an API on top of our new AI search. Our code (see the function_app.py file in this folder, or linked here) does the following:

1. Takes in an input of the user's query, search index endpoint, the index name, the k_nearest_neighbors*, the search column to use (either content_vector or title_vector), and whether it should use a hybrid query
2. Takes the user's query and embeds it.
3. Conducts a vector search and retrieves relevant text chunks.
4. Returns those relevant text chunks as the response body.

*In the context of vector search, k_nearest_neighbors specifies the number of "closest" vectors (in terms of cosine similarity) that the search should return. For example, if k_nearest_neighbors is set to 3, the search will return the 3 vectors in the index that are most similar to the query vector.

Note that this Azure Function does not have any authentication. However, you can set authentication on it following docs here

Create storage account

We can create a new storage account using the code below, but feel free to skip that block and modify the subsequent steps to use an existing storage account. This may take up to 30 seconds.

Create Function App

This Function App is where the python code will execute once it is triggered via a GPT Action. To read more about Function Apps, see the docs here.

To deploy Function Apps, we'll need to use the Azure CLI and Azure Functions Core Tools.

The below will attempt to install it and run it based on your platform type in your virtual environment, but if that does not work, read the Azure documentation to figure out how to install Azure Function Core Tools and Azure CLI. After doing that, run the below subprocess.run commands in your terminal after navigating to this folder.

First we'll make sure we have the relevant tools in the environment in order to run the Azure commands necessary. This may take a few minutes to install.

Now, we need to create a local.settings.json file with our key environment variables for Azure

Check the local.settings.json file and make sure that the environment variables match what you expect.

Now, give your app a name below, and you are ready to create your Function App and then publish your function.

Once we've created the Function App, we now want to add the configuration variables to the function app to use in the function. Specifically, we need the OPENAI_API_KEY, the SEARCH_SERVICE_API_KEY, and the EMBEDDINGS_MODEL as these are all used in the function_app.py code.

We are now ready to publish your function code function_app.py to the Azure Function. This may take up to 10 minutes to deploy. Once this is finished, we now have an API endpoint using an Azure Function on top of Azure AI Search.

Input in a Custom GPT in ChatGPT

Now that we have an Azure Function that queries this Vector Search Index, let's put it as a GPT Action!

See documentation here on GPTs and here on GPT Actions. Use the below as the instructions for the GPT and as the OpenAPI spec for the GPT Action.

Create OpenAPI Spec

Below is a sample OpenAPI spec. When we run the block below, a functional spec should be copied to the clipboard to paste in the GPT Action.

Note that this does not have any authentication by default, but you can set up Azure Functions with OAuth by following the pattern in this cookbook in the Authentication section or looking at the documentation here.

Create GPT Instructions

Feel free to modify instructions as you see fit. Check out our docs here for some tips on prompt engineering.

We now have a GPT that queries a vector database!

Recap

We've now successfully integrated Azure AI Search with GPT Actions in ChatGPT by doing the following:

1. embedded them using OpenAI's embeddings, while adding some additional metadata using gpt-4o.
2. uploaded that data to Azure AI Search.
3. created an endpoint to query it using Azure Functions.
4. incorporated it into a Custom GPT.

Our GPT can now retrieve information to help answer user queries, making it much more accurate and customized to our data. Here's the GPT in action: