Notebooks
W
Weaviate Similarity Search All MiniLM L6 V2

Similarity Search All MiniLM L6 V2

vector-searchvector-databaseretrieval-augmented-generationllm-frameworksweaviate-featuresfunction-callingweaviate-recipesmodel-providersPythongenerative-aihuggingface
alph-notebooks/weaviate-recipes / similarity_search_all_MiniLM_L6_v2.ipynb

Open In Colab

Similarity search with all_MiniLM_L6_v2 via Hugging Face and Weaviate

Dependencies

[1]

Connect to Weaviate

Now, you will need to connect to a running Weaviate vector database cluster.

You can choose one of the following options:

  1. Option 1: You can create a 14-day free sandbox on the managed service Weaviate Cloud (WCD)
  2. Option 2: Embedded Weaviate
  3. Option 3: Local deployment
  4. Other options
[2]
[ ]
[4]
{'generative-openai': {'documentationHref': 'https://platform.openai.com/docs/api-reference/completions',
,  'name': 'Generative Search - OpenAI'},
, 'qna-openai': {'documentationHref': 'https://platform.openai.com/docs/api-reference/completions',
,  'name': 'OpenAI Question & Answering Module'},
, 'ref2vec-centroid': {},
, 'reranker-cohere': {'documentationHref': 'https://txt.cohere.com/rerank/',
,  'name': 'Reranker - Cohere'},
, 'text2vec-cohere': {'documentationHref': 'https://docs.cohere.ai/embedding-wiki/',
,  'name': 'Cohere Module'},
, 'text2vec-huggingface': {'documentationHref': 'https://huggingface.co/docs/api-inference/detailed_parameters#feature-extraction-task',
,  'name': 'Hugging Face Module'},
, 'text2vec-openai': {'documentationHref': 'https://platform.openai.com/docs/guides/embeddings/what-are-embeddings',
,  'name': 'OpenAI Module'}}

Create a collection

Collection stores your data and vector embeddings.

[5]
Successfully created collection: MyCollection.

Import the Data

[6]
Insert complete.

Quick check to see if all objects are in.

[7]
10

Query Weaviate: Similarity Search (Text objects)

Similarity search options for text objects in Weaviate:

  1. near_text

  2. near_object

  3. near_vector

nearText Example

Find a object in MyCollection closest to the query "What's the capital of Germany?". Limit it to only 4 responses.

[9]
ID: adcdd665-87ad-45f1-85c2-dd7d6beac702
Data: {
  "text": "The capital of Germany is Berlin."
}
Vector: [0.04136650264263153, 0.08663720637559891, -0.007826615124940872] ...
Distance: 0.39156365394592285 

ID: 58a67982-224c-46eb-8f34-68c73d33d65d
Data: {
  "text": "Die Hauptstadt Deutschlands ist Berlin."
}
Vector: [-0.018270188942551613, 0.10900147259235382, -0.06833386421203613] ...
Distance: 0.5286470651626587 

ID: 74a85b70-f6ca-457f-b40d-29206f852663
Data: {
  "text": "Der schnelle braune Fuchs springt \u00fcber den faulen Hund."
}
Vector: [-0.0851321667432785, 0.06763748824596405, -0.02962086722254753] ...
Distance: 0.7293647527694702
[ ]

nearObject Example

Search through the JeopardyQuestion class to find the top 4 objects closest to id (The id was taken from the query above)

[ ]

nearVector Example

Search through the MyCollection class to find the top 2 objects closest to the query vector.

[ ]