Repository navigation
docs(asr/vocab): document the rescue floors as an opt-in, with the measured trade-off - #1101
Merged
Merged
Conversation
…asured trade-off (#899) Streaming and batch keep the floors off by default: on Earnings-22 they cost 138 brand and personal names, while they cut false insertions on unrelated speech from 332 to 40.
Speaker Diarization Benchmark ResultsSpeaker Diarization PerformanceEvaluating "who spoke when" detection accuracy
Diarization Pipeline Timing BreakdownTime spent in each stage of speaker diarization
Speaker Diarization Research ComparisonResearch baselines typically achieve 18-30% DER on standard datasets
Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:
🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 61.1s diarization time • Test runtime: 4m 22s • 11/01/2026, 05:14 PM EST |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #899.
#900 fixed #899's bug. The issue stayed open for one question: should #702's similarity floors (0.30 / 0.50) become the sliding-window streaming default? Measured on main b3eda84, streaming
SlidingWindowAsrManagerwith the CLI's default--streamingconfig:Floors on lose 138 real brand and personal names (recall 95.8% → 84.9%) and gain none, but they stop the rescue pass firing on unrelated speech. Batch shows the same trade-off. The default stays off. This PR documents the floors in
CustomVocabulary.mdas an opt-in, with these numbers and theVocabularyRescorer.Configto enable them. Since #1100, floors-off false insertions on the FLEURS set are 220.Docs only.
🤖 Generated with Claude Code