Notebooks
A
Azure Semantic Text Search Using Embeddings

Semantic Text Search Using Embeddings

azure-openai-samplesBasic_Samplesdotnetembeddingscsharp
alph-notebooks/azure-openai-samples / Semantic_text_search_using_embeddings.ipynb

Semantic text search using embeddings

We can search through all our reviews semantically in a very efficient manner and at very low cost, by embedding our search query, and then finding the most similar reviews. The dataset is created in the Get_embeddings_from_dataset Notebook.

Installation

Install the Azure Open AI SDK using the below command.

[1]
[ ]

Run this cell, it will prompt you for the apiKey, endPoint, and embedding deployment

[3]

Import namesapaces and create an instance of OpenAiClient using the azureOpenAIEndpoint and the azureOpenAIKey

[4]
[5]
[6]
[7]
[8]

Let's define the function SearchReviews that is used to search a dataset of reviews using embeddings. The function takes an array of data rows, a query string, and an optional result count (defaulting to 5), and returns an array of the top matching reviews.

The code starts by making an asynchronous request to an AI service (likely OpenAI) to generate an embedding for the query. The GetEmbeddingsAsync method of the client object is used to make this request. The method takes an instance of EmbeddingsOptions as a parameter, which specifies the deployment of the embedding model and the text to be embedded (in this case, the query). The response from the AI service is then processed to extract the query's embedding.

Next, it calculates the similarity between the query's embedding and the embeddings of all data rows using the ScoreBySimilarityTo method. This calculates a measure of similarity between two non-zero vectors, between the query's embedding and each row's embedding. The CosineSimilarityComparer<float[]>(t => t) is used to specify how to calculate the similarity.

The resulting scores are then ordered in descending order and the top resultCount scores are selected. This means that the method is returning the top resultCount rows that have the highest similarity scores with the query's embedding.

Finally, the it extracts the text of each selected row using the Select(r => r.Key.Text) expression and converts the resulting collection to an array. This array of review texts is returned as the search results.

[9]
[10]
Barilla Whole Grain Fusilli with Vegetable Marinara is tasty and has an excellent chunky vegetable marinara.  I just wish there was more of it.  If you aren't starving or on a diet, the 9oz serving is enough for lunch although you might want to add a piece of fruit to feel full.  The whole grain fusilli cooked to al dente tenderness following the instructions and the chunky marinara sauce is so good that I wished there was more of it.  Rarely do I eat sauce alone but this sauce is good enough to.
tastes so good. Worth the money. My boyfriend hates wheat pasta and LOVES this. cooks fast tastes great.I love this brand and started buying more of their pastas. Bulk is best.
Anything this company makes is worthwhile eating! My favorite is their Trenne.<br />Their whole wheat pasta is the best I have ever had.
[11]
The bag came broken. Product was leaking out of the box, due to poor packing standards.<br />Hope next items arrive unscathed. Quinoa tasted good.
[12]
The bag came broken. Product was leaking out of the box, due to poor packing standards.<br />Hope next items arrive unscathed. Quinoa tasted good.
[13]
The only dry food my queen cat will eat. Helps prevent hair balls. Good packaging. Arrives promptly. Recommended by a friend who sells pet food.