Notebooks
W
Weaviate Hybrid Search Azure Openai

Hybrid Search Azure Openai

vector-searchazure-openaivector-databaseretrieval-augmented-generationllm-frameworksweaviate-featuresfunction-callingweaviate-recipesmodel-providersPythongenerative-ai
alph-notebooks/weaviate-recipes / hybrid_search_azure_openai.ipynb

Open In Colab

Hybrid Search with Azure OpenAI

This recipe will show you how to run hybrid search with embeddings from Azure OpenAI.

Requirements

  1. Weaviate cluster

    1. You can create a 14-day free sandbox on WCD
    2. Embedded Weaviate
    3. Local deployment
    4. Other options

  2. Azure API key. Grab one here.

Import Dependencies, Libraries, and Keys

[ ]
[1]

Connect to Weaviate

Only choose one option from the below.

Weaviate Cloud Deployment

[13]
True

Embedded Weaviate

[ ]

Local Deployment

[ ]

Create a collection

Collection stores your data and vector embeddings.

[18]
Successfully created collection: JeopardyQuestion.

Import the Data

[19]
Insert complete.

Hybrid Search

The alpha parameter determines the weight given to the sparse and dense search methods. alpha = 0 is pure sparse (bm25) search, whereas alpha = 1 is pure dense (vector) search.

Alpha is an optional parameter. The default is set to 0.75.

Hybrid Search only

The below query is finding Jeopardy questions about animals and is limiting the output to only two results. Notice alpha is set to 0.80, which means it is weighing the vector search results more than bm25. If you were to set alpha = 0.25, you would get different results.

[20]
ID: f6849411-1553-43fa-a240-13d69114945d
Data: {
  "answer": "species",
  "question": "2000 news: the Gunnison sage grouse isn't just another northern sage grouse, but a new one of this classification",
  "category": "SCIENCE"
} 

ID: 0de6ee13-b260-4cf9-b368-e7ffeddb14a4
Data: {
  "answer": "the diamondback rattler",
  "question": "Heaviest of all poisonous snakes is this North American rattlesnake",
  "category": "ANIMALS"
} 

ID: 5e3c2cf9-4633-400a-8c97-78ab4b183b8a
Data: {
  "answer": "Antelope",
  "question": "Weighing around a ton, the eland is the largest species of this animal in Africa",
  "category": "ANIMALS"
}

Hybrid Search with a where filter

Find Jeopardy questions about elephants, where the category is set to Animals.

[21]
ID: 0de6ee13-b260-4cf9-b368-e7ffeddb14a4
Data: {
  "answer": "the diamondback rattler",
  "question": "Heaviest of all poisonous snakes is this North American rattlesnake",
  "category": "ANIMALS"
} 

ID: 5e3c2cf9-4633-400a-8c97-78ab4b183b8a
Data: {
  "answer": "Antelope",
  "question": "Weighing around a ton, the eland is the largest species of this animal in Africa",
  "category": "ANIMALS"
} 

ID: 2b4424c2-65f2-4295-974d-906ad74e0e55
Data: {
  "answer": "Elephant",
  "question": "It's the only living mammal in the order Proboseidea",
  "category": "ANIMALS"
}