Notebooks
d
deepset Llama Stack With Agent

Llama Stack With Agent

agentic-aiagenticagentsgenaiAIhaystack-cookbookgenai-usecaseshaystack-ainotebooksPythonragai-tools
alph-notebooks/haystack-cookbook / llama_stack_with_agent.ipynb

🛠️🦙 Build with Llama Stack and Haystack Agent

This notebook demonstrates how to use the LlamaStackChatGenerator component with Haystack Agent to enable function calling capabilities. We'll create a simple weather tool that the Agent can call to provide dynamic, up-to-date information.

We start with installing integration package.

[ ]

Setup

Before running this example, you need to:

  1. Set up Llama Stack Server through an inference provider
  2. Have a model available (e.g., llama3.2:3b)

For a quick start on how to setup server with Ollama, see the Llama Stack documentation.

Once you have the server running, it will typically be available at http://localhost:8321/v1/openai/v1.

Defining a Tool

Tool in Haystack allow models to call functions to get real-time information or perform actions. Let's create a simple weather tool that the model can use to provide weather information.

[1]

Setting Up Agent

Now, let's create a LlamaStackChatGenerator and pass it to the Agent. The Agent component will use the model running with LlamaStackChatGenerator to reason and make decisions.

[ ]

Using Tools with the Agent

Now, when we ask questions, the Agent will utilize both the provided tool and the LlamaStackChatGenerator to generate answers. We enable the streaming in Agent through streaming_callback, so you can observe the tool calls and results in real time.

[7]
[TOOL CALL]
Tool: weather 
Arguments: {"city":"Tokyo"}

[TOOL RESULT]
The weather in Tokyo is sunny and 32°C

In[ASSISTANT]
 Tokyo, the current weather conditions are mostly sunny with a temperature of 32°C. Would you like to know more about Tokyo's climate or weather forecast for a specific date?

Simple Chat with ChatGenerator

For a simpler use case, you can also create a lightweight mechanism to chat directly with LlamaStackChatGenerator.

[15]
🤖 The main character in The Witcher series, also known as the eponymous figure, is Geralt of Rivia, a monster hunter with supernatural abilities and mutations that allow him to control the elements. He was created by Polish author_and_polish_video_game_development_company_(CD Projekt).
🤖 One of the most fascinating aspects of dolphin behavior is their ability to produce complex, context-dependent vocalizations that are unique to each individual, similar to human language. They also exhibit advanced social behaviors, such as cooperation, empathy, and self-awareness.

If you want to switch your model provider, you can reuse the same LlamaStackChatGenerator code with different providers. Simply run the desired inference provider on the Llama Stack Server and update the model name during the initialization of LlamaStackChatGenerator.

For more details on available inference providers, see Llama Stack docs.