Notebooks
M
Meta Llama Parallel Tool Use

Parallel Tool Use

llamagroqAIvllmmachine-learning3p-integrationsllama2LLMparallel-tool-usellama-cookbookPythonfinetuningpytorchgroq-api-cookbooklangchain
alph-notebooks/llama-cookbook / parallel-tool-use.ipynb

Parallel Tool use

Setup

Make sure you have ipykernel and pip pre-installed

[ ]
[1]
'Groq API key configured: gsk_7FdrzM...'

We will use the llama3-70b-8192 model in this demo. Note that you will need a Groq API Key to proceed and can create an account here to generate one for free. Only Llama 3 models support parallel tool use at this time (05/07/2024).

We recommend using the 70B Llama 3 model, 8B has subpar consistency.

[2]

Let's define a dummy function we can invoke in our tool use loop

[3]

Now we define our messages and tools and run the completion request.

[4]

Processing the tool calls

Now we process the assistant message and construct the required messages to continue the conversation.

Including invoking each tool_call against our actual function.

[5]
[
  {
    "role": "system",
    "content": "You are a helpful assistant."
  },
  {
    "role": "user",
    "content": "What is the weather in Paris, Tokyo and Madrid?"
  },
  {
    "role": "assistant",
    "tool_calls": [
      {
        "id": "call_5ak8",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\":\"Paris\"}"
        },
        "type": "function"
      },
      {
        "id": "call_zq26",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\":\"Tokyo\"}"
        },
        "type": "function"
      },
      {
        "id": "call_znf3",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\":\"Madrid\"}"
        },
        "type": "function"
      }
    ]
  },
  {
    "role": "tool",
    "content": "20",
    "tool_call_id": "call_5ak8"
  },
  {
    "role": "tool",
    "content": "15",
    "tool_call_id": "call_zq26"
  },
  {
    "role": "tool",
    "content": "35",
    "tool_call_id": "call_znf3"
  }
]

Now we run our final completion with multiple tool call results included in the messages array.

Note

We pass the tool definitions again to help the model understand:

  1. The assistant message with the tool call
  2. Interpret the tool results.
[6]
The weather in Paris is 20°C, in Tokyo is 15°C, and in Madrid is 35°C.