Whisper Podcast Rag
Using Whisper and RAG to Build a Podcast Chat App
Whisper, an Automatic Speech Recognition (ASR) model, is a powerful tool that converts spoken language into written text. This capability is crucial in numerous applications, including transcription services, voice assistants, and more. It allows us to transform audio content into a format that can be processed and understood by text-based models.
In this notebook, we demonstrate how to use the Whisper model on Groq API, along with Retrieval-Augmented Generation (RAG) to build an interactive application. The application converts podcast audio into text transcriptions, stores these transcriptions in a Pinecone vector database, and then use LangChain to perform RAG on those transcriptions to answer user queries based on the podcast content. By leveraging Groq's powerful inference capabilities, we can efficiently process and understand the podcast content, providing users with accurate and contextually relevant responses to their queries.
Setup
A Groq API Key is required for this demo - you can generate one for free here. We will be using Pinecone as our vector database, which also requires an API key (you can create one index for a small project there for free on their Starter plan) Finally, we will be using the whisper-large-v3 model for audio transcription.
Using Whisper for Audio Transcription
First, let's test out using Whisper to transcribe a single, smaller audio file. We will start with transcribing an .mp3 conversion of our CEO Jonathan Ross' talk on CNN: Groq's AI Chip Breaks Speed Records:
Welcome back. You're with Connect the World. I'm Becky Anderson. We're at the World Government Summit in Dubai. And one thing I've noticed here is that whenever a discussion about artificial intelligence takes place, the rooms here, the huge halls get packed. That is because some of the leading minds behind the technological revolution have been gathered here at the conference over the past couple of days and in ai's race to the top my next guest is sprinting at speeds never seen before jonathan ross is the brain behind groq the world's first language processing unit now before i lose you in the technological jargon of ai let me put it this way what Ross created is a chip that can run programs like Meta's Lama 2 model for example faster than anything else in the world ten to 100 times faster in fact and he's here with me now to explain how that is possible before I ask you that groq why groq Thank You Becky it's groq and we We spell it with a Q, and it's because it comes from a science fiction novel, and it means to understand something deeply and with empathy. Of course it does. Tell us about your chip and what makes Groq Chip LPU different from other AI chips and accelerators. I have to tell our viewers that the NVIDIA CEO was here, of course, this week at the beginning of the week. So we've had all the greatest minds in here. What's your story? Well, asking me how the chip works before I show you what it does is a bit like asking how a magic trick works before showing you the magic trick, you're going to get bored. But I'll give it a shot. Cool. So most chips, they they don't have enough memory inside of them. Sort of like if you were building cars and you use a giant factory, you need about a million square feet of assembly line space. Well, if you don't have a building that's large enough to fit that, then you need to set up part of the assembly line, tear it down over and over again. Right. And that's slow and it takes a lot of time. And that's what happens w
With the audio from the interview successfully converted to text, we can now feed it into the context of a simple chat completion request to ask questions about the contents of the interview (using Meta's smaller 8b Llama-3 model) for text completion:
According to the transcript, Jonathan Ross explained the importance of fast language models as follows: "The reason you care about the speed is it's about engagement. Imagine if I spoke that slowly, you'd just drift off, you'd go away. Most certainly. So the statistic is if you improve the speed by 100 milliseconds on a website on desktop, you will get about an 8% increase in user engagement. On mobile, it's 34%. People have no patience on mobile." In this quote, Ross emphasizes that speed is crucial for keeping users engaged. He explains that even a small improvement in speed can lead to a significant increase in user engagement, with a 8% increase on desktop and a 34% increase on mobile for every 100 millisecond improvement in speed. This highlights the importance of fast language models in maintaining user attention and keeping them engaged with the application or website.
Building a Podcast Chat App
I've stored .mp3 files of the 10 most recent episodes of the This Week in Startups podcast in the mp3-files/ folder that we will transcribe and use for RAG. Please note that when using podcast RSS feeds or any digital content in your application, it's crucial to respect copyright permissions to avoid potential legal issues.
First, we will have to split these files into smaller chunks so that they can be fully transcribed in a Whisper request:
Splitting Episode ID: 215f919b-845b-4d5a-9593-860c29dbd7fa Exporting mp3-chunks/215f919b-845b-4d5a-9593-860c29dbd7fa_chunk1.mp3 Exporting mp3-chunks/215f919b-845b-4d5a-9593-860c29dbd7fa_chunk2.mp3 Exporting mp3-chunks/215f919b-845b-4d5a-9593-860c29dbd7fa_chunk3.mp3 Exporting mp3-chunks/215f919b-845b-4d5a-9593-860c29dbd7fa_chunk4.mp3 Splitting Episode ID: ca8faab3-b039-47c6-85a9-3e430528c5b0 Splitting Episode ID: 4753dd85-7cf2-4f20-a919-1bdf54e194f5 Splitting Episode ID: e00c7094-c70b-4173-b9d3-0cfa6c0ea7d7 Splitting Episode ID: 615b84d2-44d5-4c76-8c03-3f7312640ccf Splitting Episode ID: 0f7b6ff1-62e9-4319-8dc2-286fd909cbd7 Splitting Episode ID: 9b0c5bd4-9ec6-4ced-82af-d40206fc75ca Splitting Episode ID: e85e445a-85fc-48ab-8507-eaf0ced99072 Splitting Episode ID: 96a380e9-a7d1-428c-9347-7e27fcb152a0 Splitting Episode ID: 897994d8-b0a0-4ca4-b9c5-2e74e1c4a5f4
Podcast metadata is stored in episode_metadata.csv, which we can use to attach relevant info like Date and Title to our transcriptions:
Next, we'll loop through each episode chunk, extract their transcriptions, split those transcriptions further so that they fit in the context window, and store them in to Document object so that it integrates with the vector database. We'll use LangChain's TokenTextSplitter to split the text into 500 token chunks:
20.0 % of transcripts processed... 41.0 % of transcripts processed... 61.0 % of transcripts processed... 82.0 % of transcripts processed... # Transcription Chunks: 329
Finally, we'll define our embedding function and store our transcription chunks to a Pinecone index called twist-transcripts. For more info on embeddings and building RAG solutions, see this cookbook post.
Now, we can search our vector database to find relevant podcast transcripts that answer questions about recent episodes:
According to the transcript, Vibecheck is a machine learning model that produces a special website, vibecheck.market, which helps to find or recommend the best products based on user input. In the specific example given, the host asks the AI to find the best headphones, and Vibecheck provides a list of recommendations. The system uses a complex algorithm that takes into account various factors, including user preferences, reviews, and ratings from various online sources. The goal of Vibecheck is to provide personalized recommendations to users, making it easier for them to find the best products that match their needs and preferences.
According to the transcript from Episode 1937, Jason mentions that he likes the Golden State Warriors, referencing their winning championships with the help of multiple All-Star players.
Conclusion
In conclusion, this notebook showcases the transformative potential of converting audio content into text using the Whisper model with Groq. This process unlocks a wealth of information previously trapped in audio format, making it accessible and searchable. By storing these text transcriptions in a Pinecone vector database and using Retrieval-Augmented Generation (RAG) for query handling, we can efficiently retrieve and present relevant information. While we demonstrated this capability through the creation of a Podcast Chat App, the implications extend far beyond this specific use case. The ability to accurately transcribe and query audio content opens up vast opportunities for information retrieval, content analysis, and user interaction across a wide range of applications.