Notebooks
W
Weaviate Multi Modal Search Nova Embeddings

Multi Modal Search Nova Embeddings

vector-searchvector-databaseretrieval-augmented-generationllm-frameworksweaviate-featuresfunction-callingawsweaviate-recipesmodel-providersPythongenerative-ai
alph-notebooks/weaviate-recipes / multi-modal-search-nova-embeddings.ipynb

Multimodal Search with Nova Embeddings

This notebook demonstrates how to build a multimodal search system using AWS Nova embeddings in Weaviate, enabling image search, hybrid search, and the Weaviate Query Agent.

At the time of writing this notebook, we're using:

  • weaviate-client == 4.18.0
  • weaviate database == 1.34.0
  • weaviate-agents == 1.1.0

Install Dependencies and Libraries

[ ]
[ ]
[ ]
[1]

Connect to Weaviate

You can create a free 14-day sandbox on Weaviate Cloud!

[ ]
True

Create Collection

[ ]
<weaviate.collections.collection.sync.Collection at 0x111df3230>

Grab Data

Convert Images into Base64

[19]

Ingest Data into Weaviate

[36]

Query Time

Image Search

We can use the near_image operator to search for images in our local path.

[5]
Title: monkey-side-eye
Output

Hybrid Search on the Title Property

The hybrid operator uses vector and keyword search together and is weighted depending on the alpha parameter.

  • alpha = 0 -> pure keyword search
  • alpha = 1 -> pure vector search
  • alpha = 0.5 -> equal weighting of the two search algorithms
[4]
Title: spiderman
Output

Query Agent

We can pass in images to the Query Agent to answer natural language queries based on our images.

For example, below we're asking, "Which meme should I use to explain that two things are similar?", and it is searching through our database to find relevant images.

[ ]
[13]