Video RAG with Weaviate & Google Gemini

This notebook demonstrates semantic search and RAG over video content with:

• A local MP4 video file (video.mp4).
• ffmpeg-python to split the video into overlapping chunks.
• Weaviate with multi2vec_google_gemini vectorizer to embed and index each clip using Gemini Embedding 2, Google’s first fully multimodal embedding model.
• Gemini 3 Flash to generate a response based on the relevant video segments.

Note: Place your video.mp4 in the same directory as this notebook. You can use any MP4 file - we used the Weaviate Agent Skills introduction video.

Step 1: Connect to Weaviate

First, set up a Weaviate instance and connect to it. If you don’t already have one, sign up for a free Weaviate Cloud sandbox cluster here.

Once your cluster is ready:

1. Copy your cluster URL and generate an API key.
2. Add them as environment variables named WEAVIATE_URL and WEAVIATE_API_KEY.
3. Go to Google AI Studio, generate a Google API key, and save it as GOOGLE_API_KEY environment variable.

Step 2: Create Weaviate Collection

Step 3: Load Local Video

Place your video.mp4 file in the same directory as this notebook before running this cell.
You can use any MP4 file of your choice.

Step 4: Split Video into Overlapping Chunks

Each chunk is written to video_chunks/ directory.

Step 5: Ingest Video Chunks into Weaviate

We base64-encode each MP4 clip and store it in the video_clip BLOB field. Weaviate forwards the raw clip bytes to Gemini's embedding API and stores the resulting vectors.

Step 6: Semantic Search over Video Chunks

Step 7: RAG - Answer a Question from Video Context