Function Calling with OpenAIChatGenerator 📞

⚠️ As of Haystack 2.9.0, this recipe has been deprecated. For the same example, follow Tutorial: Building a Chat Agent with Function Calling

Notebook by Bilge Yucel (LI & X (Twitter))

A guide to understand function calling and how to use OpenAI function calling feature with Haystack.

📚 Useful Sources:

• OpenAIChatGenerator Docs
• OpenAIChatGenerator API Reference

Overview

Here are some use cases of function calling from OpenAI Docs:

• Create assistants that answer questions by calling external APIs (e.g. like ChatGPT Plugins) e.g. define functions like send_email(to: string, body: string), or get_current_weather(location: string, unit: 'celsius' | 'fahrenheit')
• Convert natural language into API calls e.g. convert "Who are my top customers?" to get_customers(min_revenue: int, created_before: string, limit: int) and call your internal API
• Extract structured data from text e.g. define a function called extract_data(name: string, birthday: string), or sql_query(query: string)

Set up the Development Environment

Learn about the OpenAIChatGenerator

OpenAIChatGenerator is a component that supports the function calling feature of OpenAI.

The way to communicate with OpenAIChatGenerator is through ChatMessage list. Therefore, create a ChatMessage with "USER" role using ChatMessage.from_user() and send it to OpenAIChatGenerator:

Basic Streaming

OpenAIChatGenerator supports streaming, provide a streaming_callback function and run the client again to see the difference.

Function Calling with OpenAIChatGenerator

We'll try to recreate the example on OpenAI docs.

Define a Function

We'll define a get_current_weather function that mocks a Weather API call in the response:

Create the tools

We'll then add information about this function to our tools list by following OpenAI's tool schema

Run OpenAIChatGenerator with tools

We'll pass the list of tools in the run() method as generation_kwargs.

Let's define messages and run the generator:

It's a function call! 📞 The response gives us information about the function name and arguments to use to call that function:

Optionally, add the message with function information to the message list

See how we can extract the function_name and function_args from the message

Make a Tool Call

Let's locate the corresponding function for function_name in our available_functions dictionary and use function_args when calling it. Once we receive the response from the tool, we'll append it to our messages for later sending to OpenAI.

Make the last call to OpenAI with response coming from the function and see how OpenAI uses the provided information

Improve the Example

Let's add more tool to our example and improve the user experience 👇

We'll add one more tool use_haystack_pipeline for OpenAI to use when there's a question about countries and capitals:

Start the Application

Have fun having a chat with OpenAI 🎉

Example queries you can try:

• "What's the capital of Utopia", "Is it sunny there?": To test the messages are being recorded and sent
• "What's the weather like in the capital of Utopia?": To force two function calls
• "What's the weather like today?": To force OpenAI to ask more clarification

Print the summary of the conversation

This part can help you understand the message order