ASR
Automatic Speech Recognition. The umbrella technical term for any system that converts spoken audio into written text.
Reference
Transcription Glossary
Plain-English definitions of the technical terms used in speech-to-text and AI transcription. Top entries link out to dedicated pages with deeper explanations and examples.
Core ASR & models
Whisper
An open-source ASR model released by OpenAI in 2022. Trained on 680,000 hours of multilingual audio. Now standard infrastructure for most AI transcription tools, including Transcript.you.
Whisper versions (whisper-1 vs Whisper Large v3)
OpenAI's API-served Whisper variant (whisper-1) is older and supports verbose JSON with segment timestamps. Whisper Large v3 is the latest open-source release with better accuracy on noisy and accented audio.
Acoustic model
The component of an ASR system that maps raw audio waveforms to phoneme probabilities.
Language model
The component that turns phoneme guesses into actual words by predicting likely word sequences. Modern Whisper-style transformers integrate both into one model.
Beam search
A decoding strategy used by ASR engines: instead of picking the single most-likely next word at each step (greedy), the system tracks the top-k candidate sequences and picks the highest-scoring complete path. Trades compute for accuracy.
CTC loss
Connectionist Temporal Classification loss — a training objective that lets ASR networks learn alignments between audio frames and characters without per-frame labels. Pioneered modern end-to-end speech recognition.
Mel spectrogram
A visual representation of audio frequencies plotted on the Mel scale (perceptually uniform). The standard input format for modern ASR neural networks, including Whisper.
Chunking
Splitting a long audio file into shorter segments before sending to the ASR engine. Whisper has a 30-second context window internally; tools chunk at 10-minute boundaries to stay within model limits.
Forced alignment
Matching a known transcript to its audio to produce per-word timing. Used to improve subtitle precision after the initial ASR pass.
VAD (Voice Activity Detection)
A pre-processing step that detects which parts of an audio stream contain speech vs silence vs noise. Saves processing time by skipping non-speech segments.
End-pointing
Real-time detection of when a user has stopped speaking — the trigger for finalizing live captions or sending dictation results to downstream systems.
Streaming ASR
Speech recognition that emits partial results as audio arrives, instead of waiting for the recording to complete. Required for live captions and conversational AI.
Real-time vs batch transcription
Real-time transcription processes a live audio stream as it's spoken (live captions). Batch transcription processes a complete recorded file. Batch is typically more accurate; real-time has a 1-3 second latency.
Punctuation prediction
A post-processing step that adds commas, periods, and capitalization to raw ASR output, which originally produces unpunctuated streams of words.
Punctuation casing models
A specialized post-processing model that turns lowercase unpunctuated ASR output into properly cased and punctuated text. Whisper integrates this directly; older pipelines use a separate model.
Audio formats & properties
Codec
The algorithm used to compress and decompress audio data. MP3, AAC, FLAC, Opus, and PCM are all codecs. Different codecs trade off file size, quality, and computational cost.
Sample rate
How many times per second an audio signal is measured, in Hz. CD quality is 44,100 Hz; modern recordings often use 48,000 Hz. Higher sample rates capture more detail but produce larger files.
Bitrate
How many bits of data are used per second of audio, in kbps. Streaming MP3 is 128 kbps; podcast-grade is 192-256 kbps; archival lossless is 1,000+ kbps.
Hertz (Hz)
The unit of frequency: cycles per second. Audio signals are described by their sample rate (e.g. 44,100 Hz) and frequency content. Human speech ranges roughly 80 Hz to 8,000 Hz.
Lossy vs lossless audio
Lossy compression (MP3, AAC, OGG) discards audio data to reduce file size. Lossless (FLAC, ALAC, WAV) preserves every bit. For transcription, lossy at 128 kbps+ is indistinguishable from lossless.
ID3 tags
Metadata fields embedded inside MP3 files: title, artist, album, year, genre, etc. Don't affect transcription but are useful when batch-organizing podcast files before transcription.
Speakers & diarization
Diarization
The process of identifying "who said what" in a multi-speaker recording. Often labeled as Speaker 1, Speaker 2, etc. Requires a separate model from base transcription.
Speaker embedding
A fixed-length vector that captures the voice characteristics of a speaker. Used in diarization to cluster segments by speaker identity without knowing names in advance.
RTTM
Rich Transcription Time Marked format. Standard line-based format for storing diarization output: when each speaker starts and stops talking. Used in research benchmarks like DIHARD.
DER (Diarization Error Rate)
The standard accuracy metric for speaker diarization: percent of audio time where the system attributes the wrong speaker (or misses speech). State-of-the-art systems achieve 5-15% DER on conversational benchmarks.
Subtitles & captions
SRT
SubRip Subtitle file format. The most widely supported subtitle format, compatible with YouTube, VLC, Premiere, Final Cut, and DaVinci Resolve.
VTT (WebVTT)
Web Video Text Tracks. The subtitle format native to HTML5 video. Used by Vimeo, browser-based players, and modern web video CMS systems.
Timestamp
A time marker on each line of a transcript, typically in MM:SS or HH:MM:SS format. Required for subtitles; optional for plain text.
Transcription vs subtitling
Transcription produces a plain text file with everything spoken. Subtitling produces time-coded short lines optimized for on-screen reading (typically 32-40 chars per line, max 2 lines on screen).
Closed captions vs subtitles
Closed captions include non-speech audio (music cues, sound effects, speaker labels) for the deaf and hard-of-hearing. Subtitles assume the viewer can hear and only translate spoken dialogue.
Burn-in vs sidecar captions
Burn-in captions are baked into the video pixels (cannot be turned off). Sidecar captions are stored separately as SRT/VTT files and toggled by the player. Social-media captions are typically burn-in; web video uses sidecar.
WCAG audio captions
Web Content Accessibility Guidelines requirements for captioned audio content: synchronized captions for prerecorded media (Level A), live captions for live media (Level AA), and audio descriptions for video (Level AA).
Accuracy metrics
WER
Word Error Rate. The percentage of words a transcription engine gets wrong, including substitutions, insertions, and deletions. Lower is better. Modern engines achieve 2-5% WER on clean audio.
RTF (Real-Time Factor)
How fast an ASR system processes audio relative to its duration. RTF = 0.5 means processing 1 hour of audio takes 30 minutes. Whisper-1 on a fast GPU runs at RTF ~0.1-0.3.
Linguistics
Phoneme
The smallest unit of sound in a language. English has roughly 44 phonemes. ASR systems internally map audio to phoneme sequences before assembling words.
Prosody
The patterns of stress, intonation, and rhythm in spoken language. Modern ASR mostly ignores prosody for word-level transcription, but it's the next frontier for emotion and intent detection.
IPA (International Phonetic Alphabet)
A standardized notation system for representing the sounds of any spoken language. Used in linguistics and pronunciation guides.
Code-switching
Mixing two or more languages within a single utterance — common in bilingual speakers. A challenging case for ASR; Whisper-3 handles common bilingual pairs (Spanish-English, Mandarin-English) but accuracy drops on rare combinations.
Lombard effect
The involuntary tendency of speakers to raise their voice and articulate more clearly in noisy environments. Recordings made under high noise sound louder than normal conversation, which complicates ASR if the model wasn't trained on Lombard speech.
ELAN
A widely-used tool from the Max Planck Institute for annotating audio and video recordings, common in linguistics and qualitative research workflows. Reads transcripts produced elsewhere.
Looking for more depth? Read the Complete Guide to AI Transcription for a full walk-through of formats, accuracy, and tools.
Last updated: September 20, 2026