Glossary
Speaker diarization is the process of segmenting a multi-speaker recording into utterances per speaker — usually labeled Speaker 1, Speaker 2, Speaker 3, etc. It's a separate model from base ASR; Whisper alone gives you text without speaker labels. Diarization is what turns "...we should ship Friday. I disagree." into "Speaker 1: we should ship Friday. Speaker 2: I disagree."
How diarization works
A diarization pipeline does three things: (1) Voice activity detection — find segments where someone is speaking. (2) Speaker embeddings — extract a fixed-length vector representing the voice characteristics of each segment. (3) Clustering — group segments by similar embeddings, each cluster = one speaker. Modern systems (pyannote, NeMo, WhisperX) achieve 5-15% Diarization Error Rate (DER) on conversational benchmarks.
Why it matters for transcription
Without diarization, a 60-minute interview transcript reads like a wall of text — you can't tell who's speaking. With diarization, each utterance is attributed, and you can do speaker-level analysis: pull every quote from one person, calculate talk-time ratios, separate the journalist's questions from the interviewee's answers. We add Speaker IDs as a one-click feature on transcripts where it matters.
Limitations
Diarization fails on: voices that sound alike (siblings, same gender + age), heavy overlap (two people talking simultaneously), very short utterances (a single "yeah"), and recordings with strong reverb. Expect 90%+ accuracy on a clean podcast interview, dropping to 70-85% on a 5-person conference call recorded on one mic.
In practice
Drop a 2-host podcast episode into Transcript.you and click Speaker ID. We label every paragraph with Speaker 1 / Speaker 2. You then find-and-replace "Speaker 1" with "Sarah" and "Speaker 2" with "James" in the exported DOCX. Total time: 3 minutes for an hour of content.
Related terms
Further reading
Try interview transcription
Get started — freeLast updated: September 20, 2026
Frequently Asked Questions
What does Diarization mean in transcription?
The process of identifying "who said what" in a multi-speaker recording. Often labeled as Speaker 1, Speaker 2, etc. Requires a separate model from base transcription.
Why does Diarization matter when choosing a transcription workflow?
Diarization affects how you understand transcript quality, timing, compatibility, or the technology behind speech-to-text results.
Where can I apply Diarization on Transcript.you?
Use the related workflow at /interview-transcription when you want to see how this glossary concept connects to an actual transcription task.
Browse the full transcription glossary or read the complete guide to AI transcription.