OpenAI Speech Transcription Methods

Speech Transcription Methods

chatgptopenaigpt-4examplesopenai-apiopenai-cookbook

alph-notebooks/openai-cookbook / Speech_transcription_methods.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

🗣️ Comparing Speech-to-Text Methods with the OpenAI API

Overview

This notebook provides a clear, hands-on guide for beginners to quickly get started with Speech-to-Text (STT) using the OpenAI API. You'll explore multiple practical methods, their use cases, and considerations.

By the end you will be able to select and use the appropriate transcription method for your use use cases.

Note:

This notebook uses WAV audio files for simplicity. It does not demonstrate real-time microphone streaming (such as from a web app or direct mic input).
This notebook uses WebSockets to connect to the Realtime API. Alternatively, you can use WebRTC, see the OpenAI docs for details.

📊 Quick-look

Mode	Latency to first token	Best for (real examples)	Advantages	Key limitations
File upload + `stream=False` (blocking)	seconds	Voicemail, meeting recordings	Simple to set up	• No partial results, users see nothing until file finishes • Max 25 MB per request (you must chunk long audio)
File upload + `stream=True`	subseconds	Voice memos in mobile apps	Simple to set up & provides a “live” feel via token streaming	• Still requires a completed file • You implement progress bars / chunked uploads
Realtime WebSocket	subseconds	Live captions in webinars	True real-time; accepts a continuous audio stream	• Audio must be pcm16, g711_ulaw, or g711_alaw • Session ≤ 30 min, reconnect & stitch • You handle speaker-turn formatting to build the full transcript
Agents SDK VoicePipeline	subseconds	Internal help-desk assistant	Real-time streaming and easy to build agentic workflows	• Python-only beta • API surface may change

Installation (one‑time)

To set up your environment, uncomment and run the following cell in a new Python environment:

[1]

This installs the necessary packages required to follow along with the notebook.

Authentication

Before proceeding, ensure you have set your OpenAI API key as an environment variable named OPENAI_API_KEY. You can typically set this in your terminal or notebook environment: export OPENAI_API_KEY="your-api-key-here"

Verify that your API key is set correctly by running the next cell.

[2]

✅ OpenAI client ready

1 · Speech-to-Text with Audio File

model = gpt-4o-transcribe

When to use

You have a completed audio file (up to 25 MB).The following input file types are supported: mp3, mp4, mpeg, mpga, m4a, wav, and webm.
Suitable for batch processing tasks like podcasts, call-center recordings, or voice memos.
Real-time feedback or partial results are not required.

How it works

STT Not Streaming Transcription flow

Benefits

Ease of use: Single HTTP request – perfect for automation or backend scripts.
Accuracy: Processes the entire audio in one go, improving context and transcription quality.
File support: Handles WAV, MP3, MP4, M4A, FLAC, Ogg, and more.

Limitations

No partial results: You must wait until processing finishes before seeing any transcript.
Latency scales with duration: Longer recordings mean longer wait times.
File-size cap: Up to 25 MB (≈ 30 min at 16-kHz mono WAV).
Offline use only: Not intended for real-time scenarios such as live captioning or conversational AI.

Let's first preview the audio file. I've downloaded the audio file from here.

[3]

Now, we can call the STT endpoint to transcribe the audio.

[4]


--- TRANSCRIPT ---

And lots of times you need to give people more than one link at a time. A band could give their fans a couple new videos from a live concert, a behind-the-scenes photo gallery, an album to purchase, like these next few links.

2 · Speech-to-Text with Audio File: Streaming

model = gpt-4o-transcribe

When to use

You already have a fully recorded audio file.
You need immediate transcription results (partial or final) as they arrive.
Scenarios where partial feedback improves UX, e.g., uploading a long voice memo.

STT Streaming Transcription flow

Benefits

Real-time feel: Users see transcription updates almost immediately.
Progress visibility: Intermediate transcripts show ongoing progress.
Improved UX: Instant feedback keeps users engaged.

Limitations

Requires full audio file upfront: Not suitable for live audio feeds.
Implementation overhead: You must handle streaming logic and progress updates yourself.

[5]

And lots of times you need to give people more than one link at a time. A band could give their fans a couple new videos from a live concert, a behind-the-scenes photo gallery, an album to purchase, like these next few links.

And lots of times you need to give people more than one link at a time. A band could give their fans a couple new videos from a live concert, a behind-the-scenes photo gallery, an album to purchase, like these next few links.

3 · Realtime Transcription API

model = gpt-4o-transcribe

When to use

Live captioning for real-time scenarios (e.g., meetings, demos).
Need built-in voice-activity detection, noise suppression, or token-level log probabilities.
Comfortable handling WebSockets and real-time event streams.

How it works

Realtime Transcription flow

Benefits

Ultra-low latency: Typically 300–800 ms, enabling near-instant transcription.
Dynamic updates: Supports partial and final transcripts, enhancing the user experience.
Advanced features: Built-in turn detection, noise reduction, and optional detailed log-probabilities.

Limitations

Complex integration: Requires managing WebSockets, Base64 encoding, and robust error handling.
Session constraints: Limited to 30-minute sessions.
Restricted formats: Accepts only raw PCM (no MP3 or Opus); For pcm16, input audio must be 16-bit PCM at a 24kHz sample rate, single channel (mono), and little-endian byte order.

[8]

[9]

And lots of times you need to give people more than one link at a time.A band could give their fans a couple new videos from a live concert, a behind-the-scenes photo galleryLike these next few linksAn album to purchase.

'And lots of times you need to give people more than one link at a time. A band could give their fans a couple new videos from a live concert, a behind-the-scenes photo gallery Like these next few linksAn album to purchase. '

4 · Agents SDK Realtime Transcription

models = gpt-4o-transcribe, gpt-4o-mini

When to use

Leveraging the OpenAI Agents SDK for real-time transcription and synthesis with minimal setup.
You want to integrate transcription directly into agent-driven workflows.
Prefer high-level management of audio input/output, WebSockets, and buffering.

How it works

Agents Transcription flow

Benefits

Minimal boilerplate: VoicePipeline handles resampling, VAD, buffering, token auth, and reconnects.
Seamless agent integration: Enables direct interaction with GPT agents using real-time audio transcription.

Limitations

Python-only beta: not yet available in other languages; APIs may change.
Less control: fine-tuning VAD thresholds or packet scheduling requires digging into SDK internals.

[10]

[11]


[User]: And lots of times you need to give people more than one link at a time.
[Assistant]: Et souvent, vous devez donner aux gens plusieurs liens à la fois.
[User]: A band could give their fans a couple new videos from a live concert, a behind-the-scenes photo gallery.
[Assistant]: Un groupe pourrait donner à ses fans quelques nouvelles vidéos d'un concert live, ainsi qu'une galerie de photos des coulisses.
[User]: An album to purchase.
[Assistant]: Un album à acheter.
[User]: like these next few links.
[Assistant]: comme ces quelques liens suivants.

Conclusion

In this notebook you explored multiple ways to convert speech to text with the OpenAI API and the Agents SDK, ranging from simple file uploads to fully-interactive, real-time streaming. Each workflow shines in a different scenario, so pick the one that best matches your product’s needs.

Key takeaways

Match the method to the use-case:
• Offline batch jobs → file-based transcription.
• Near-real-time updates → HTTP-streaming.
• Conversational, low-latency experiences → WebSocket or Agents SDK.
Weigh trade-offs: latency, implementation effort, supported formats, and session limits all differ by approach.
Stay current: the models and SDK continue to improve; new features ship regularly.

Next steps

Try out the notebook!
Integrate your chosen workflow into your application.
Send us feedback! Community insights help drive the next round of model upgrades.

References

Explore the Transcriptions API docs.
Read the Realtime guide.
Explore the Agents SDK reference.
Explore the Agents SDK Voice Pipeline reference