Creating Demos with Spaces and Gradio

Authored by: Diego Maniloff

Introduction

In this notebook we will demonstrate how to bring any machine learning model to life using Gradio, a library that allows you to create a web demo from any Python function and share it with the world 🌎!

📚 This notebook covers:

• Building a Hello, World! demo: The basics of Gradio
• Moving your demo to Hugging Face Spaces
• Making it interesting: a real-world example that leverages the 🤗 Hub
• Some of the cool "batteries included" features that come with Gradio

⏭️ At the end of this notebook you will find a Further Reading list with links to keep going on your own.

Setup

To get started install the gradio library along with transformers.

Your first demo: the basics of Gradio

At its core, Gradio turns any Python function into a web interface.

Say we have a simple function that takes name and intensity as parameters, and returns a string like so:

If you run this function for the name 'Diego' you will get an output string that looks like this:

With Gradio, we can build an interface for this function via the gr.Interface class. All we need to do is pass in the greet function we created above, and the kinds of inputs and outputs that our function expects:

Notice how we passed in ["text", "slider"] as inputs and ["text"] as outputs -- these are called Components in Gradio.

That's all we need for our first demo. Go ahead and try it out 👇🏼! Type your name into the name textbox, slide the intensity that you want, and click Submit.

Let's make it interesting: a meeting transcription tool

At this point you understand how to take a basic Python function and turn it into a web-ready demo. However, we only did this for a function that is very simple, a bit boring even!

Let's consider a more interesting example that highlights the very thing that Gradio was built for: demoing cutting-edge machine learning models. A good friend of mine recently asked me for help with an audio recording of an interview she had done. She needed to convert the audio file into a well-organized text summary. How did I help her? I built a Gradio app!

Let's walk through the steps to build the meeting transcription tool. We can think of the process as two parts:

1. Transcribe the audio file into text
2. Organize the text into sections, paragraphs, lists, etc. We could include summarization here too.

Audio-to-text

In this part we will build a demo that handles the first step of the meeting transcription tool: converting audio into text.

As we learned, the key ingredient to building a Gradio demo is to have a Python function that executes the logic we are trying to showcase. For the audio-to-text conversion, we will build our function using the awesome transformers library and its pipeline utility to use a popular audio-to-text model called distil-whisper/distil-large-v3.

The result is the following transcribe function, which takes as input the audio that we want to convert:

Now that we have our Python function, we can demo that by passing it into gr.Interface. Notice how in this case the input that the function expects is the audio that we want to convert. Gradio includes a ton useful components, one of which is Audio, exactly what we need for our demo 🎶 😎.

Go ahead and try it out 👆! You can upload an .mp3 file or hit the 🎤 button to record your own voice.

For a sample file with an actual meeting recording, you can check out the MeetingBank_Audio dataset which is a dataset of meetings from city councils of 6 major U.S. cities. For my own testing, I tried out a couple of the Denver meetings.

[!TIP] Also check out Interface's from_pipeline constructor which will directly build the Interface from a pipeline.

Organize & summarize text

For part 2 of the meeting transcription tool, we need to organize the transcribed text from the previous step.

Once again, to build a Gradio demo we need the Python function with the logic that we care about. For text organization and summarization, we will use an "instruction-tuned" model that is trained to follow a broad range of tasks. There are many options to pick from such as meta-llama/Meta-Llama-3-8B-Instruct or mistralai/Mistral-7B-Instruct-v0.3. For our example we are going to use microsoft/Phi-3-mini-4k-instruct.

Just like for part 1, we could leverage the pipeline utility within transformers to do this, but instead we will take this opportunity to showcase the Serverless Inference API, which is an API within the Hugging Face Hub that allows us to use thousands of publicly accessible (or your own privately permissioned) machine learning models for free! Check out the cookbook section of the Serverless Inferfence API here.

Using the Serverless Inferfence API means that instead of calling a model via a pipeline (like we did for the audio conversion part), we will call it from the InferenceClient, which is part of the huggingface_hub library (Hub Python Library). And in turn, to use the InferenceClient, we need to log into the 🤗 Hub using notebook_login(), which will produce a dialog box asking for your User Access Token to authenticate with the Hub.

You can manage your tokens from your personal settings page, and please remember to use fine-grained tokens as much as possible for enhanced security.

Now that we are logged into the Hub, we can write our text processing function using the Serverless Inference API via InferenceClient.

The code for this part will be structured into two functions:

• build_messages, to format the message prompt into the LLM;
• organize_text, to actually pass the raw meeting text into the LLM for organization (and summarization, depending on the prompt we provide).

And now that we have our text organization function organize_text, we can build a demo for it as well:

Go ahead and try it out 👆! If you hit "Submit" in the demo above, you will see that the output text is a much clearer and organized version of the transcript, with a title and sections for the different parts of the meeting.

See if you can get a summary by playing around with the user_input variable that controls the LLM prompt.

Putting it all together

At this point we have a function for each of the two steps we want out meeting transcription tool to do:

1. convert the audio into a text file, and
2. organize that text file into a nicely-formatted meeting document.

All we have to do next is stitch these two functions together and build a demo for the combined steps. In other words, our complete meeting transcription tool is just a new function (which we'll creatively call meeting_transcript_tool 😀) that takes the output of transcribe and passes it into organize_text:

Go ahead and try it out 👆! This is now the full demo of our transcript tool. If you give it an audio file, the output will be the already-organized (and potentially summarized) version of the meeting. Super cool 😎.

Move your demo into 🤗 Spaces

If you made it this far, now you know the basics of how to create a demo of your machine learning model using Gradio 👏!

Up next we are going to show you how to take your brand new demo to Hugging Face Spaces. On top of the ease of use and powerful features of Gradio, moving your demo to 🤗 Spaces gives you the benefit of permanent hosting, ease of deployment each time you update your app, and the ability to share your work with anyone! Do keep in mind that your Space will go to sleep after a while unless you are using it or making changes to it.

The first step is to head over to https://huggingface.co/new-space, select "Gradio" from the templates, and leave the rest of the options as default for now (you can change these later):

This will result in a newly created Space that you can populate with your demo code. As an example for you to follow, I created the 🤗 Space dmaniloff/meeting-transcript-tool, which you can access here.

There are two files we need to edit:

• app.py -- This is where the demo code lives. It should look something like this:

  # outline of app.py:
  
  def meeting_transcript_tool(...):
     ...
  
  def transcribe(...):
     ...
  
  def organize_text(...):
     ...
  
  

• requirements.txt -- This is where we tell our Space about the libraries it will need. It should look something like this:

  # contents of requirements.txt:
  torch
  transformers
  

Gradio comes with batteries included 🔋

Gradio comes with lots of cool functionality right out of the box. We won't be able to cover all of it in this notebook, but here's 3 that we will check out:

• Access as an API
• Sharing via public URL
• Flagging

Access as an API

One of the benefits of building your web demos with Gradio is that you automatically get an API 🙌! This means that you can access the functionality of your Python function using a standard HTTP client like curl or the Python requests library.

If you look closely at the demos we created above, you will see at the bottom there is a link that says "Use via API". If you click on it in the Space I created (dmaniloff/meeting-transcript-tool), you will see the following:

Let's go ahead and copy-paste that code below to use our Space as an API:

Wow! What happened there? Let's break it down:

• We installed the gradio_client, which is a package that is specifically designed to interact with APIs built with Gradio.
• We instantiated the client by providing the name of the 🤗 Space that we want to query.
• We called the predict method of the client and passed in a sample audio file to it.

The Gradio client takes care of making the HTTP POST for us, and it also provides functionality like reading the input audio file that our meeting transcript tool will process (via the function handle_file).

Again, using this client is a choice, and you can just as well run a curl -X POST https://dmaniloff-meeting-transcript-tool.hf.space/call/predict [...] and pass in all the parameters needed in the request.

[!TIP] The output that we get from the call above is a made-up meeting that was generated by the LLM that we are using for text organization. This is because the sample input file isn't an actual meeting recording. You can tweak the LLM's prompt to handle this case.

Share via public URL

Another cool feature built into Gradio is that even if you build your demo on your local computer (before you move it into a 🤗 Space) you can still share this with anyone in the world by passing in share=True into launch like so:

 demo.launch(share=True)

You might have noticed that in this Google Colab environment that behaviour is enabled by default, and so the previous demos that we created already had a public URL that you can share 🌎. Go back ⬆ and look at the logs for Running on public URL: to find it 🔎!

Flagging

Flagging is a feature built into Gradio that allows the users of your demo to provide feedback. You might have noticed that the first demo we created had a Flag button at the bottom.

Under the default options, if a user clicks that button then the input and output samples are saved into a CSV log file that you can review later. If the demo involves audio (like in our case), these are saved separately in a parallel directory and the paths to these files are saved in the CSV file.

Go back and play with our first demo once more, and then click the Flag button. You will see that a new log file is created in the flagged directory:

In this case I set inputs to name=diego and intensity=29, which I then flagged. You can see that the log file includes the inputs to the function, the output "Hello, diego!!!!!!!!!!!!!!!!!!!!!!!!!!!!!", and also a timestamp.

While a list of inputs and outputs that your users found problematic is better than nothing, Gradio's flagging feature allows you to do much more. For example, you can provide a flagging_options parameter that lets you customize the kind of feedback or errors that you can receive, such as ["Incorrect", "Ambiguous"]. Note that this requires that allow_flagging is set to "manual":

Go ahead and try it out 👆! You can see that the flagging buttons now are Flag as Incorrect and Flag as Ambiguous, and the new log file will reflect those options:

Wrap up & Next Steps

In this notebook we learned how to demo any machine learning model using Gradio.

First, we learned the basics of setting up an interface for a simple Python function; and second, we covered Gradio's true strength: building demos for machine learning models.

For this, we learned how easy it is to leverage models in the 🤗 Hub via the transformers library and its pipeline function, and how to use multimedia inputs like gr.Audio.

Third, we covered how to host your Gradio demo on 🤗 Spaces, which lets you keep your demo running in the cloud and gives you flexibility in terms of the compute requirements for your demo.

Finally, we showcased a few of the super cool batteries included that come with Gradio such as API access, public URLs, and Flagging.

For next steps, check out the Further Reading links at the end of each section.

⏭️ Further reading

• Your first demo with gradio
• Gradio Components
• The transformers library
• The pipeline function
• Hub Python Library
• Serverless Inference API
• 🤗 Spaces
• Spaces documentation