Notebooks
M
Meta Llama Langgraph Rag Agent Local

Langgraph Rag Agent Local

llamaAIvllmmachine-learning3p-integrationsllama2LLMllama-cookbookPythonfinetuningpytorchlangchain
alph-notebooks/llama-cookbook / langgraph_rag_agent_local.ipynb

Open In Colab

[ ]

Local LangGraph RAG agent with Llama 3

Previously, we showed how to build simple agents with LangGraph and Llama 3.

Now, we'll pick a more advanced use-case: advanced RAG, with the requirement that it runs locally.

Ideas

We'll combine ideas from three RAG papers into a RAG agent:

  • Routing: Adaptive RAG (paper). Route questions to different retrieval approaches
  • Fallback: Corrective RAG (paper). Fallback to web search if docs are not relevant to query
  • Self-correction: Self-RAG (paper). Fix answers w/ hallucinations or don’t address question

langgraph_adaptive_rag.png

Note that this will incorporate a few general ideas for agents:

  • Reflection: The self-correction mechanism is a form of reflection, where the LangGraph agent reflects on its retrieval and generations
  • Planning: The control flow laid out in the graph is a form of planning
  • Tool use: Specific nodes in the control flow (e.g., web search) will use tools

Local models

Embedding

GPT4All Embeddings:

pip install langchain-nomic

LLM

Use Ollama and llama3:

ollama pull llama3

Prompt -

https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/

Tracing

### Tracing (optional)
os.environ['LANGCHAIN_TRACING_V2'] = 'true'
os.environ['LANGCHAIN_ENDPOINT'] = 'https://api.smith.langchain.com'
os.environ['LANGCHAIN_API_KEY'] = 'LANGCHAIN_API_KEY'

Search

Uses Tavily

[ ]
[ ]
[ ]
[ ]
[ ]
[ ]
[ ]
[ ]
[ ]

We'll implement these as a control flow in LangGraph.

[ ]

Graph Build

[ ]
[ ]
[ ]
[ ]