MultiQueryRetriever with elasticsearch and langchain

This workbook demonstrates example of Elasticsearch's MultiQuery Retriever to generate multiple queries for a given user input query and apply all queries to retrieve a larger set of relevant documents from a vectorstore.

Before we begin, we first split the documents into chunks with langchain and then using ElasticsearchStore.from_documents, we create a vectorstore and index data to elasticsearch.

We will then see few examples query demonstrating full power of elasticsearch powered multiquery retriever.

Install packages and import modules

Create documents

Next, we will create list of documents with summary of movies using langchain Schema Document, containing each document's page_content and metadata .

Connect to Elasticsearch

ℹ️ We're using an Elastic Cloud deployment of Elasticsearch for this notebook. If you don't have an Elastic Cloud deployment, sign up here for a free trial.

We'll use the Cloud ID to identify our deployment, because we are using Elastic Cloud deployment. To find the Cloud ID for your deployment, go to https://cloud.elastic.co/deployments and select your deployment.

We will use ElasticsearchStore to connect to our elastic cloud deployment, This would help create and index data easily. We would also send list of documents that we created in the previous step

Setup query retriever

Next we will instantiate MultiQuery retriever by providing a bit information about our document attributes and a short description about the document.

We will then instantiate retriever with MultiQueryRetriever.from_llm

Test retriever with simple query

We will test the retriever with a simple query: What are some movies about dream.

The output shows all the relevant documents to the query.