Corrective Rag
agentic_rag_techniquesllmsathina-rag-cookbooksopenaiAItutorialsChromaDBcookbooksfaissqdrantLLMPythonpineconeraglangchainweaviate
Export
Corrective RAG
Corrective Retrieval-Augmented Generation (Corrective RAG or CRAG) is a method that improves the accuracy of generated responses by evaluating and correcting retrieved documents before using them. It works as follows:
Correct: If documents are relevant, they are refined to remove unnecessary parts and used for generation.
Incorrect: If documents are irrelevant, they are discarded, and the system retrieves additional information, like using web search. (Note: In our case, we are not performing this step.)
Ambiguous: If it's unclear, the system combines retrieved and web-searched information for a balanced response.
Research Paper: Corrective RAG
Initial Setup
[ ]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.5/83.5 kB 2.7 MB/s eta 0:00:00
[ ]
[ ]
Indexing
[ ]
[ ]
[ ]
[ ]
Retriever
[ ]
Document Grader
The document grader evaluates whether a document is relevant to the given query.
[ ]
/usr/local/lib/python3.10/dist-packages/langchain_core/_api/beta_decorator.py:87: LangChainBetaWarning: The method `ChatOpenAI.with_structured_output` is in beta. It is actively being worked on, so the API may change. warn_beta(
[ ]
/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `BaseRetriever.get_relevant_documents` was deprecated in langchain-core 0.1.46 and will be removed in 0.3.0. Use invoke instead. warn_deprecated(
binary_score='yes'
RAG Chain
[ ]
[ ]
'Interlibrary loan (ILL) is a service that allows patrons of one library to borrow physical materials or receive electronic documents from another library. After receiving a request from a patron, the borrowing library identifies potential lending libraries with the desired item. The lending library then delivers the item physically or electronically to the borrowing library, who then delivers it to their patron. If necessary, arrangements are made for the return of the item. Fees may accompany interlibrary loan services, and the majority of requests are now managed through semi-automated systems. Libraries negotiate for ILL eligibility, and with the increasing demand for digital materials, they are exploring the legal, technical, and licensing aspects of lending and borrowing ebooks through interlibrary loan.'
Web Search
[ ]
[ ]
Create Graph
Define Graph State
[ ]
Build Graph
[ ]
[ ]
[ ]
---RETRIEVE---
"Node 'retrieve':"
'\n---\n'
---CHECK DOCUMENT RELEVANCE TO QUESTION---
---GRADE: DOCUMENT RELEVANT---
---GRADE: DOCUMENT RELEVANT---
---GRADE: DOCUMENT RELEVANT---
---GRADE: DOCUMENT RELEVANT---
---ASSESS GRADED DOCUMENTS---
---DECISION: GENERATE---
"Node 'grade_documents':"
'\n---\n'
---GENERATE---
"Node 'generate':"
'\n---\n'
('Interlibrary loan works by allowing patrons of one library to borrow '
'physical materials or receive electronic documents from another library that '
'holds the desired item. The borrowing library identifies potential lending '
'libraries, which then deliver the item either physically or electronically. '
'The borrowing library receives the item, delivers it to their patron, and '
'arranges for its return if necessary. Fees may accompany interlibrary loan '
'services, and the majority of requests are now managed through '
'semi-automated systems.')
[ ]
---RETRIEVE---
"Node 'retrieve':"
'\n---\n'
---CHECK DOCUMENT RELEVANCE TO QUESTION---
---GRADE: DOCUMENT NOT RELEVANT---
---GRADE: DOCUMENT NOT RELEVANT---
---GRADE: DOCUMENT NOT RELEVANT---
---GRADE: DOCUMENT NOT RELEVANT---
---ASSESS GRADED DOCUMENTS---
---DECISION: WEB SEARCH---
"Node 'grade_documents':"
'\n---\n'
---WEB SEARCH---
"Node 'web_search_node':"
'\n---\n'
---GENERATE---
"Node 'generate':"
'\n---\n'
('Retrieval-Augmented Generation (RAG) is a technique that enhances generative '
'artificial intelligence models by allowing them to access and reference '
'external knowledge bases, such as specific documents or databases, before '
'generating a response. This process helps improve the relevance and accuracy '
"of the generated text by incorporating information beyond the model's own "
'training data.')
Preparing Data for Evaluation
[ ]
---RETRIEVE--- ---CHECK DOCUMENT RELEVANCE TO QUESTION--- ---GRADE: DOCUMENT RELEVANT--- ---GRADE: DOCUMENT RELEVANT--- ---GRADE: DOCUMENT RELEVANT--- ---GRADE: DOCUMENT RELEVANT--- ---ASSESS GRADED DOCUMENTS--- ---DECISION: GENERATE--- ---GENERATE---
[ ]
[ ]
[ ]
Evaluation in Athina AI
We will use Context Precision eval here. It Evaluates whether all of the ground-truth relevant items present in the contexts are ranked higher or not. Please refer to our documentation for further details.
[ ]
[ ]
[ ]
evaluating with [context_precision]
100%|██████████| 1/1 [00:01<00:00, 1.28s/it]
You can view your dataset at: https://app.athina.ai/develop/2c0c4475-a30e-4d2c-92f9-c57f494ba947
