LangGraph Handling LangChain Agent Intermediate_Steps

In this notebook we will learn how to build a basic agent executor leveraging langGraph.

We demonstrate how to handle the logic of the intermediate steps from the agent leveraging different provided tools within langGraph.

• We will be leveraging LLM mixtral-8x7b-instruct-v0.1 from NVIDIA API Catalog.

• Simple Faiss Retriever as one of the tools with the NV-Embed-QA from NVIDIA API Catalog.

• Wikipedia (the pip installable package) as one of the tools.

Then we will utilize with LangGraph to control and intervene intermediate steps as well as the outputs from the agent.

Prerequisites

To run this notebook, you need to complete the setup and generate an API key.

Install additional Python packages

Install the additional packages that required for this example

Step 1 - Export the NVIDIA_API_KEY

Optionally, we can set API key for LangSmith tracing, which will give us best-in-class observability.

Step 2 - Initialize the LLM and embedding models

The following code sets ai-mixtral-8x7b-instruct as the main LLM and ai-embed-qa-4 as the embedding model.

Step 3 - Retriever from FAISS vector store

We need to process a toy example, here we use Sweden.txt from the data folder.

Step 4 - Construct a Retriever for Sweden data

The following code creates a SwedenRetriever class that inherits from LangChain's BaseTool class.

We'll use the class as a tool for retrieving data about Sweden to augment responses.

Step 5 - Construct wikipedia as the second tool

Step 6 - Give your tools a good name and populate the description

Step 7 - Wrap tools into ToolExecutor

We will use these ToolExecutor to invoke tool in LangGraph nodes later on.

Step 8 - Create the prompt template and conversation memory

The following code creates a memory buffer for storing queries and responses. It also demonstrates how to write a prompt template for a Mistral mode that uses conversation memory and the Wiki and retriever tools.

Step 9 - Establish agent executor using LangChain

Step 10 - Define the graph state

We now define the graph state. The state for the traditional LangChain agent has a few attributes:

1. input: This is the input string representing the main ask from the user, passed in as input.
2. chat_history: This is any previous conversation messages, also passed in as input.
3. intermediate_steps: This is list of actions and corresponding observations that the agent takes over time. This is updated each iteration of the agent.
4. agent_outcome: This is the response from the agent, either an AgentAction or AgentFinish. The AgentExecutor should finish when this is an AgentFinish, otherwise it should call the requested tools.

Step 11 - Define the nodes

We now need to define a few different nodes in our graph. In LangGraph, a node can be either a function or a runnable. There are two main nodes we need for this:

1. The agent (run_agent): responsible for deciding what (if any) actions to take.
2. A function to invoke tools (execute_tools): if the agent decides to take an action, this node will then execute that action.

We will also need to define some edges. Some of these edges may be conditional. The reason they are conditional is that based on the output of a node, one of several paths may be taken. The path that is taken is not known until that node is run (the LLM decides).

1. Conditional Edge (should_continue): after the agent is called, we should either:

   • If the agent said to take an action, then the function to invoke tools is called.
   • If the agent said that it was finished, then it finishes.

2. Normal Edge: after the tools are invoked, it should always go back to the agent to decide what to do next.

Let's define the nodes, as well as a function to decide how what conditional edge to take.

Step 12 - Connect the nodes with edges to form the graph, let's call it app

Step 13 - Time to test it out

Let's start by seeing if we can trigger the retriever tool (tool name: AboutSweden).

Then, we will try to call the Wikipedia tool (tool name : Wiki).