---

title: Open Source LLM Observability for Gradio sidebarTitle: Gradio logo: /images/integrations/gradio_icon.svg description: Build an LLM Chat UI with 🤗 Gradio and trace it with 🪢 Langfuse. category: Integrations

Build an LLM Chat UI with 🤗 Gradio and trace it with 🪢 Langfuse

This is a simple end-to-end example notebook which showcases how to integrate a Gradio application with Langfuse for LLM Observability and Evaluation.

Note: We recommend to run this notebook in Google Colab (see link above). This notebook is also available as a Hugging Face Space template here.

Thank you to @tkmamidi for the original implementation and contributions to this notebook.

Introduction

What is Gradio?

Gradio is an open-source Python library that enables quick creation of web interfaces for machine learning models, APIs, and Python functions. It allows developers to wrap any Python function with an interactive UI that can be easily shared or embedded, making it ideal for demos, prototypes, and ML model deployment. See docs for more details.

What is Langfuse?

Langfuse is an open-source LLM engineering platform that helps build reliable LLM applications via LLM Application Observability, Evaluation, Experiments, and Prompt Management. See docs for more details.

Walkthrough

We've recorded a walkthrough of the implementation below. You can follow along with the video or the notebook.

Outline

This notebook will show you how to

1. Build a simple chat interface in Python and rendering it in a Notebook using Gradio Chatbot
2. Add Langfuse Tracing to the chatbot
3. Implement additional Langfuse tracing features used frequently in chat applications: chat sessions, user feedback

Setup

Install requirements. We use OpenAI for this simple example. We could use any model here.

Note: This guide uses our Python SDK v2. We have a new, improved SDK available based on OpenTelemetry. Please check out the SDK v3 for a more powerful and simpler to use SDK.

Set credentials and initialize Langfuse SDK Client used to add user feedback later on.

You can either create a free Langfuse Cloud account or self-host Langfuse in a couple of minutes.

Implementation of Chat functions

Sessions/Threads

Each chat message belongs to a thread in the Gradio Chatbot which can be reset using clear (reference).

We implement the following method that creates a session_id that is used globally and can be reset via the set_new_session_id method. This session_id will be used for Langfuse Sessions.

Response handler

When implementing the respond method, we use the Langfuse @observe() decorator to automatically log each response to Langfuse Tracing.

In addition we use the openai integration as it simplifies instrumenting the LLM call to capture model parameters, token counts, and other metadata. Alternatively, we could use the integrations with LangChain, LlamaIndex, other frameworks, or instrument the call itself with the decorator (example).

User feedback handler

We implement user feedback tracking in Langfuse via the like event for the Gradio chatbot (reference). This methdod reuses the current trace id available in the global state of this application.

Retries

Allow you to retry a completion via the Gradio Chatbot retry event (docs). This is not specific to the integration with Langfuse.

Run Gradio Chatbot

After implementing all methods above, we can now put together the Gradio Chatbot and launch it. If run within Colab, you should see an embedded Chatbot interface.

Explore data in Langfuse

When interacting with the Chatbot, you should see traces, sessions, and feedback scores in your Langfuse project. See video above for a walkthrough.

Example trace, session, and user feedback in Langfuse (public link):

If you have any questions or feedback, please join the Langfuse Discord or create a new thread on GitHub Discussions.