Hyde Rag
llmsathina-rag-cookbooksadvanced_rag_techniquesopenaiAItutorialsChromaDBcookbooksfaissqdrantLLMPythonpineconeraglangchainweaviate
Export
Hypothetical Document Embeddings (HyDE) RAG
HyDE operates by creating hypothetical document embeddings that represent ideal documents relevant to a given query. This method contrasts with conventional RAG systems, which typically rely on the similarity between a user's query and existing document embeddings. By generating these hypothetical embeddings, HyDE effectively guides the retrieval process towards documents that are more likely to contain pertinent information.
Research Paper: HyDE
Initial Setup
[2]
[3]
Indexing
[4]
[5]
[6]
Weaviate Vector Database
[7]
[8]
[9]
[Document(metadata={'source': './context.csv', 'row': 36.0}, page_content='context: ["The European theatre of World War II was one of the two main theatres of combat during World War II. It saw heavy fighting across Europe for almost six years, starting with Germany\'s invasion of Poland on 1 September 1939 and ending with the Western Allies conquering most of Western Europe, the Soviet Union conquering most of Eastern Europe and Germany\'s unconditional surrender on 8 May 1945 although fighting continued elsewhere in Europe until 25 May. On 5 June 1945, the Berlin'),
, Document(metadata={'source': './context.csv', 'row': 59.0}, page_content='war to end all wars" due to their perception of its then-unparalleled scale, devastation, and loss of life. After World War II\''),
, Document(metadata={'source': './context.csv', 'row': 59.0}, page_content='similarly wrote, "Some wars name themselves. This is the Great War." Contemporary Europeans also referred to it as "the war to end war" and it was also described as "the war to end all wars" due to their perception of its then-unparalleled scale, devastation, and loss of life. After World War II began in 1939, the terms (often abbreviated as WWI or WW1) became more standard, with British Empire historians, including Canadians, favouring "The First World War" and Americans "World War I".\\n\\n\\n==')] [ ]
Chromadb (Optional)
[ ]
Retriever
[ ]
Hypothetical Answer Chain
[ ]
[ ]
[ ]
"Interlibrary loan is a service offered by libraries that allows patrons to borrow materials from other libraries. Here is how it typically works:\n\n1. **Request:** If a patron needs a book, article, or other material that their local library does not have, they can request it through interlibrary loan.\n \n2. **Search:** The library staff will search for a library that owns the requested material and is willing to lend it.\n \n3. **Request Submission:** Once a lending library is found, the request is submitted, and the material is shipped to the patron's local library.\n \n4. **Pickup:** The patron can then pick up the material from their library, usually for a limited loan period.\n \n5. **Return:** After using the material, the patron returns it to their library, which then returns it to the lending library.\n\nThere may be fees associated with interlibrary loan, depending on the policies of the lending library or the patron's local library. Interlibrary loan is a valuable service that allows patrons to access a wide range of materials beyond what their local library may have in its collection."
Combined RAG Chain
[ ]
[ ]
[ ]
'Interlibrary loan works by allowing patrons of one library to borrow physical materials or receive electronic documents that are held by another library. The borrowing library identifies potential lending libraries with the desired item, and the lending library delivers the item either physically or electronically. The borrowing library then receives the item, delivers it to their patron, and arranges for its return if necessary. In some cases, fees may accompany interlibrary loan services. Libraries negotiate for interlibrary loan eligibility, especially for digital materials like ebooks, through legal, technical, and licensing aspects.'
Preparing Data for Evaluation
[ ]
[ ]
[ ]
[ ]
[ ]
Evaluation in Athina AI
We will use Context Relevancy eval here. It Measures the relevancy of the retrieved context, calculated based on both the query and contexts. Please refer to our documentation for further details
[ ]
[ ]
[ ]
evaluating with [context_relevancy]
100%|██████████| 1/1 [00:01<00:00, 1.82s/it]
You can view your dataset at: https://app.athina.ai/develop/1db6cbc3-1096-4b1d-881e-db538858691d