Tracing and Evaluating a Groq Agent Application

Observability is a critical part of building and maintaining a robust application. Arize Phoenix allows you to easily trace and evaluate your application. A Phoenix instance can be self-hosted or run in the cloud alongside your application, and Arize's Groq instrumentor lets you automatically capture traces, latency data, and token usage data.

This guide will walk you through the process of tracing and evaluating a basic Groq function calling agent application.

Install dependencies & Set environment variables

Connect to Phoenix

In this example, we'll connect to a cloud instance of Phoenix. If you'd rather self-host Phoenix, follow the instructions here.

To get an API key, sign up for a free account on Arize Phoenix.

Now that we have Phoenix configured, any calls to Groq in our application will be traced and logged to Phoenix. If you're incorporating Phoenix into an existing application, the code above is all you need to add to start tracing.

Read on for an example, and for details on running evaluations.

Set up your Groq Agent

Here we'll set up a basic Groq agent that can use tools to generate jokes, look up the weather, and calculate age.

View Traces in Phoenix

You should now see traces in Phoenix!

Download trace dataset from Phoenix

Generate evaluations

Now that we have our trace dataset, we can generate evaluations for each trace. Evaluations can be generated in many different ways. Ultimately, we want to end up with a set of labels and/or scores for our traces.

You can generate evaluations using:

• Plain code
• Phoenix's built-in LLM as a Judge evaluators
• Your own custom LLM as a Judge evaluator
• Other evaluation packages

As long as you format your evaluation results properly, you can upload them to Phoenix and visualize them in the UI.

For this example, we'll use an LLM as a Judge evaluator to determine whether the output of our agent matches the user's input.

We now have a DataFrame with a column for whether each joke is a repeat of a previous joke. Let's upload this to Phoenix.

Upload evaluations to Phoenix

Our evals_df has a column for the span_id and a column for the evaluation result. The span_id is what allows us to connect the evaluation to the correct trace in Phoenix. Phoenix will also automatically look for columns named "label" and "score" to display in the UI.

You should now see evaluations in the Phoenix UI!

From here you can continue collecting and evaluating traces, or move on to one of these other guides:

• If you're interested in more complex evaluation and evaluators, start with how to use LLM as a Judge evaluators
• If you're ready to start testing your application in a more rigorous manner, check out how to run structured experiments