Google releases Gemini 3.5 Transcribe with streaming speech-to-text across 85 languages
Google's new transcription model is pitched on formatting, speaker attribution and context, a sign that speech-to-text is now judged on more than word accuracy.
Google introduced Gemini 3.5 Transcribe on 26 August, a speech-to-text model it describes as producing polished, formatted text rather than a literal record of every sound. According to the announcement, the model handles mid-sentence self-corrections, drops filler words and applies formatting on its own.
It is offered in two forms. A streaming variant, gemini-3.5-transcribe-live, runs through the Live API with sub-second latency for real-time use. A non-streaming variant, gemini-3.5-transcribe, is served through the Interactions API for pre-recorded audio and can attribute speech to up to three speakers with timestamps. Both are available in Google AI Studio, and in public preview on the Gemini Enterprise Agent Platform.
Google reported a word error rate of 4.0 percent in streaming mode and 2.6 percent in non-streaming mode on its own tests, and 5.50 percent and 5.04 percent respectively on the multilingual FLEURS benchmark. It also said time to final transcript is 70 percent faster than its earlier Chirp 3 model. The model detects and transcribes more than 85 languages, follows speakers who switch language mid-session, accepts custom vocabulary for jargon and unusual spellings, and can draw on screen context and chat history to resolve ambiguous words. Through function calling it can hand tasks to other Gemini models.
The same model already sits behind the Rambler dictation feature in Gboard on Android, Google Antigravity and the Gemini app on macOS, with Chrome support described as coming soon. Early integrators named by Google include vivo, Intellitek Health, Lingopal, LiveKit, Pipecat, Agora, Stream, Fishjam, LangChain, Vercel and Vision Agents. The post did not include pricing.
Why it matters
Transcription has long been judged almost entirely on word error rate, yet this launch is pitched on what happens after recognition: formatting, speaker attribution and context. That framing suggests speech is becoming an input layer for agents rather than a standalone product. Exposing it through the same APIs as other Gemini models signals that audio is now a first-class modality in developer tooling.
Transcription has long been judged almost entirely on word error rate, yet this launch is pitched on what happens after recognition: formatting, speaker attribution and context.
- Non-streaming, Google internal tests2.6%
- Streaming, Google internal tests4.0%
- Non-streaming, FLEURS benchmark5.04%
- Streaming, FLEURS benchmark5.50%
Figures: Google, 26 August 2026 announcement
What you can learn from this
- Word error rate is a blunt instrument. WER compares a transcript with a human reference and counts substitutions, deletions and insertions, divided by the number of reference words. It is the standard metric because it is simple and comparable across systems. But it weights a dropped filler word the same as a dropped medication name, so a model that deliberately removes fillers can score worse on strict WER while producing more useful text. That is why vendors quote several figures, and why you should score on your own audio before trusting any of them.
- Streaming and offline decoding are different problems. A streaming recogniser must commit to words with only a small window of audio ahead of the current position, while an offline model can examine the whole recording before deciding. That extra right-hand context is why non-streaming numbers are almost always lower. Streaming systems emit partial hypotheses that are revised as more audio arrives, which is why time to final transcript is a separate metric from raw latency. Providers tune this window to balance responsiveness against accuracy, and the right setting differs for live captions and for dictation.
- Diarisation is separate from recognition. Working out who spoke when involves embedding short windows of audio into a voice-signature space, clustering them, and aligning clusters with word timestamps. It degrades with overlapping speech, similar voices and short utterances, which is why products state a supported speaker count rather than promising unlimited attribution. Treat diarisation output as a strong hint, not ground truth. Timestamps are what make attribution useful downstream, because they let you join the transcript to other events in a meeting or call.
- Context biasing fixes the words acoustics cannot. Names, acronyms and product terms are acoustically ambiguous, so recognisers rely on a language model to pick between similar-sounding candidates. A custom vocabulary or screen context shifts that choice toward terms you expect. This is the cheapest lever for domain accuracy because it needs no retraining, only a list of strings passed with the request. The same mechanism explains how a model can use chat history: earlier turns become extra evidence for which words are likely next.
- Voice becomes an agent's input, and therefore an attack surface. When a transcription model can emit a structured function call rather than plain text, spoken audio flows straight into tool execution. Anything that can inject text into an agent can now do so through a microphone or an uploaded recording. The same defences apply as for any untrusted input: validate arguments, constrain which tools are callable, and keep a human in the loop for consequential actions.
We teach this
How to use this in practice
- Measure WER on your own audio. Record or pick five minutes of speech you own, write a careful human transcript, run the audio through whichever speech-to-text API you can access, then install the
jiwerpackage (pip install jiwer) and calljiwer.wer(reference, hypothesis). Done looks like: a WER figure and a list of the ten most common substitutions, which tells you whether errors are acoustic or vocabulary problems. Keep the audio and reference transcript in one folder so you can rerun the identical test whenever a provider ships a new model. - Compare streaming against batch on the same file. If your provider offers both modes, run the identical recording through each and log two numbers: WER and the time until the transcript stops changing. Done looks like: a two-row table showing the accuracy you give up for real-time output, which is the trade-off you will need to explain to whoever asks for live captions. Note also whether partial results ever flip a word that had already appeared, since that flicker is what users notice most.
- Build and test a vocabulary list. Take the thirty domain terms that were wrong in the first exercise, pass them through the provider's vocabulary or phrase-biasing option, and re-run the same audio. Done looks like: a before-and-after count of how many of those thirty terms are now correct. If the option is not available, add a post-processing dictionary and measure the same thing. Record which terms still fail; those usually need a spelling normaliser rather than better acoustics.
- Draw the transcription pipeline as an agent input. Sketch the boxes: microphone or file, voice activity detection, streaming recogniser, partial hypotheses, finalisation, post-processing for formatting and diarisation, then the downstream consumer such as a chat model or tool call. Mark where latency accumulates and where untrusted text first enters something that can take actions. Done looks like: a one-page diagram on which a colleague can point to the exact box where an injected instruction in a recording would land. Keep the drawing next to your code as the reference for where input validation belongs.
Sources
- Intelligent transcription with Gemini 3.5 Transcribe — Google DeepMind
Our reporting is an original summary; full coverage is at the links above.
Don't just read about it — build it.
Square 1 teaches the skills behind the headlines, with every line of your work graded by AI. Find your starting point in 3 minutes.
Get your free skill report