Adaptive Rag
Adaptive RAG
Adaptive Retrieval-Augmented Generation (RAG) is a smart approach that adjusts how information is retrieved and used based on the complexity of a query. Instead of treating all queries the same way, Adaptive RAG chooses the best strategy for each query type.
In our case, system analyzes the query and decides the best way to handle it:
Self-corrective RAG: For questions that can be answered using the indexed data.
Web Search: For questions that require information not present in the index.
Research Paper: Adaptive RAG
Initial Setup
Indexing
Retriever
Question Router
The question router determines whether a query should be answered using our internal index or through a web search.
Please change the prompt based on your topics
RouteQuery(datasource='vectorstore')
RouteQuery(datasource='web_search')
Document Grader
The document grader evaluates whether a document is relevant to the given query.
/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `BaseRetriever.get_relevant_documents` was deprecated in langchain-core 0.1.46 and will be removed in 0.3.0. Use invoke instead. warn_deprecated(
binary_score='yes'
RAG Chain
'Interlibrary loan works by allowing patrons of one library to borrow physical materials or receive electronic documents from another library that holds the desired item. The borrowing library identifies potential lending libraries, which then deliver the item either physically or electronically. The borrowing library receives the item, delivers it to their patron, and arranges for its return if necessary. Fees may accompany interlibrary loan services, and requests can be managed through semi-automated or manual systems. Libraries negotiate for interlibrary loan eligibility, and with the increasing demand for digital materials, libraries are exploring the legal, technical, and licensing aspects of lending and borrowing ebooks through interlibrary loan.'
Hallucination Grader
The hallucination grader checks whether the answer is grounded in or supported by the given set of facts
GradeHallucinations(binary_score='yes')
Answer Grader
The answer grader evaluates whether an answer effectively addresses the given question.
GradeAnswer(binary_score='yes')
Web Search
Create Graph
Define Graph State
Build Graph
---ROUTE QUESTION---
---ROUTE QUESTION TO RAG---
---RETRIEVE---
"Node 'retrieve':"
'\n---\n'
---CHECK DOCUMENT RELEVANCE TO QUESTION---
---GRADE: DOCUMENT RELEVANT---
---GRADE: DOCUMENT RELEVANT---
---GRADE: DOCUMENT RELEVANT---
---GRADE: DOCUMENT RELEVANT---
---ASSESS GRADED DOCUMENTS---
---DECISION: GENERATE---
"Node 'grade_documents':"
'\n---\n'
---GENERATE---
---CHECK HALLUCINATIONS---
---DECISION: GENERATION IS GROUNDED IN DOCUMENTS---
---GRADE GENERATION vs QUESTION---
---DECISION: GENERATION ADDRESSES QUESTION---
"Node 'generate':"
'\n---\n'
('Interlibrary loan works by allowing patrons of one library to borrow '
'physical materials or receive electronic documents from another library that '
'holds the desired item. The borrowing library identifies potential lending '
'libraries, and the lending library delivers the item either physically or '
'electronically. The borrowing library then receives the item, delivers it to '
'their patron, and arranges for its return if necessary. Fees may accompany '
'interlibrary loan services, and requests can be managed through '
'semi-automated or manual systems.')
---ROUTE QUESTION---
---ROUTE QUESTION TO WEB SEARCH---
---WEB SEARCH---
"Node 'web_search':"
'\n---\n'
---GENERATE---
---CHECK HALLUCINATIONS---
---DECISION: GENERATION IS GROUNDED IN DOCUMENTS---
---GRADE GENERATION vs QUESTION---
---DECISION: GENERATION ADDRESSES QUESTION---
"Node 'generate':"
'\n---\n'
('RAG, or retrieval-augmented generation, is a method that combines retrieval '
'models and generative models in natural language processing (NLP) to produce '
'coherent, contextually relevant text. It allows AI models to access external '
'knowledge bases and incorporate up-to-date information into generated '
'responses. RAG is significant in NLP as it bridges the gap between retrieval '
'and generative models, enhancing the accuracy and relevance of AI-generated '
'content.')
Preparing Data for Evaluation
---ROUTE QUESTION--- ---ROUTE QUESTION TO RAG--- ---RETRIEVE--- ---CHECK DOCUMENT RELEVANCE TO QUESTION--- ---GRADE: DOCUMENT RELEVANT--- ---GRADE: DOCUMENT RELEVANT--- ---GRADE: DOCUMENT RELEVANT--- ---GRADE: DOCUMENT RELEVANT--- ---ASSESS GRADED DOCUMENTS--- ---DECISION: GENERATE--- ---GENERATE--- ---CHECK HALLUCINATIONS--- ---DECISION: GENERATION IS GROUNDED IN DOCUMENTS--- ---GRADE GENERATION vs QUESTION--- ---DECISION: GENERATION ADDRESSES QUESTION---
Evaluation in Athina AI
We will use Context Relevancy eval here. It Measures the relevancy of the retrieved context, calculated based on both the query and contexts. To learn more about this. Please refer to our documentation for further details
evaluating with [context_relevancy]
100%|██████████| 1/1 [00:01<00:00, 1.07s/it]
You can view your dataset at: https://app.athina.ai/develop/e6867bf1-6320-474b-8dca-3284b41df567
