Speaker Recognition and Diarization

Speaker recognition produces identity-related similarity scores from speech samples. Identification, verification and diarization are different decision problems.

Identification and Verification

Speaker recognition contains two different decision problems.

Speaker verification tests a claimed identity. A test recording is typically compared with one enrolled speaker model, asking whether the recording belongs to the claimed person.

Speaker identification searches for which known speaker best matches the recording. The test sample is compared with multiple candidates, and an open-set system can return “unknown speaker” if no candidate reaches a sufficient score.

Both tasks can share the same embedding infrastructure, but candidate population, threshold calibration, and error costs differ. A verification threshold should not automatically be assumed to be suitable for multi-candidate identification.

Text-Dependent and Text-Independent Recognition

A text-dependent system expects a particular word, passphrase, or utterance during enrollment and testing. The system can use both speaker characteristics and consistency with the expected content, which is useful in controlled access scenarios.

A text-independent system does not know in advance which words the speaker will say. The model attempts to extract speaker-related characteristics that remain useful across changing phonetic content. This is more appropriate for free speech, telephone conversations, or archival recordings where the linguistic content is not controlled.

Text dependence is not the same problem as speech recognition. Automatic speech recognition can verify whether the expected phrase was spoken, while the speaker model separately scores how well the voice characteristics match the enrolled person.

Classical approaches

GMM-UBM systems compare a speaker distribution with a universal background model. The i-vector approach represents speech in a compact total-variability space. Methods such as PLDA score similarity between these representations.

Learned representations

Modern systems use x-vectors and related speaker embeddings. Training encourages samples from the same speaker to be close while separating different speakers.

Speaker recognition converts a continuous score into accept or reject decisions using a threshold. Channel mismatch, codec, utterance duration and background noise can shift score distributions.

Speaker diarization

Speaker diarization answers “who spoke when?” by combining VAD, segment embeddings and clustering. Unknown speaker count and overlapping speech remain difficult conditions.

A speaker score is not a deterministic identity marker; it should be interpreted together with threshold calibration, recording conditions and measured error rates.

Parent topic: Image and Audio Processing

References

• David Snyder; Daniel Garcia-Romero; Gregory Sell; Daniel Povey; Sanjeev Khudanpur. X-Vectors: Robust DNN Embeddings for Speaker Recognition. IEEE ICASSP 2018, 2018.

• Lawrence R. Rabiner; Ronald W. Schafer. Theory and Applications of Digital Speech Processing. Pearson, 2011.