RAG With LangChain LCEL And DSPy
vector-searchvector-databaseretrieval-augmented-generationllm-frameworksfunction-callingweaviate-recipesintegrationsPythonLCELgenerative-aillm-agent-frameworkslangchain
Export
[2]
/Users/cshorten/Desktop/DSPy-local/cohere_fix/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
['Hello! How can I help you today? \n']
Load Dataset (Questions derived from Weaviate's Blog Posts)
[3]
Metric to Assess Response Quality
[4]
LCEL RAG Program
[5]
[notice] A new release of pip available: 22.3.1 -> 24.0 [notice] To update, run: pip install --upgrade pip
[6]
AIMessage(content='Hello! \n\nHow can I help you today? \n', response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'safety_ratings': [{'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability': 'NEGLIGIBLE', 'blocked': False}]}, id='run-9ff74818-3165-4276-9a9c-a3cc33802c81-0') [8]
[16]
langchain_core.runnables.base.RunnableSequence
[9]
[17]
dspy.predict.langchain.LangChainModule
[10]
You are a processor for prompts. I will give you a prompt template (Python f-string) for an arbitrary task for other LMs.
Your job is to prepare three modular pieces: (i) any essential task instructions or guidelines, (ii) a list of variable names for inputs, (iv) the variable name for output.
---
Follow the following format.
Template:
```
${template}
```
Let's now prepare three modular pieces.
Essential Instructions: ${essential_instructions}
Input Keys: comma-separated list of valid variable names
Output Key: a valid variable name
---
Template:
```
Given the context: {context}. What is the answer to the question: `{question}`? IMPORTANT! ONLY OUTPUT THE ANSWER!
```
Let's now prepare three modular pieces.
Essential Instructions:Essential Instructions: The context should be a string containing relevant information. The question should be a string asking a question about the context. The output should be a string containing only the answer to the question.
Input Keys: context, question
Output Key: answer
"\n\n\nYou are a processor for prompts. I will give you a prompt template (Python f-string) for an arbitrary task for other LMs.\nYour job is to prepare three modular pieces: (i) any essential task instructions or guidelines, (ii) a list of variable names for inputs, (iv) the variable name for output.\n\n---\n\nFollow the following format.\n\nTemplate:\n```\n\n${template}\n\n```\n\nLet's now prepare three modular pieces.\n\nEssential Instructions: ${essential_instructions}\n\nInput Keys: comma-separated list of valid variable names\n\nOutput Key: a valid variable name\n\n---\n\nTemplate:\n```\n\nGiven the context: {context}. What is the answer to the question: `{question}`? IMPORTANT! ONLY OUTPUT THE ANSWER!\n\n```\n\nLet's now prepare three modular pieces.\n\nEssential Instructions:\x1b[32mEssential Instructions: The context should be a string containing relevant information. The question should be a string asking a question about the context. The output should be a string containing only the answer to the question.\n\nInput Keys: context, question\n\nOutput Key: answer \n\x1b[0m\n\n\n" [11]
'Answer: Lock Striping is a pattern that solves race conditions, that can occur during parallel data imports, by using a fixed number of locks. It assigns objects to a specific lock based on their UUID, ensuring that objects with the same UUID are not processed concurrently while allowing objects with different UUIDs to be processed in parallel. \n'
[12]
12%|█████▎ | 3/25 [00:09<01:12, 3.29s/it]
[13]
'Answer: Lock Striping is a solution for race conditions in database design, particularly when importing data in parallel streams. It uses a fixed number of locks to ensure objects with the same UUID are never processed concurrently, preventing data duplication without sacrificing import performance. \n'
[15]
dspy.predict.langchain.LangChainModule
[19]
langchain_core.runnables.base.RunnableSequence
Understanding LCEL in the LangChain Ecosystem
