canary: add long-form chunking

[brauliobo]

Dependency

This is a stacked PR on top of #111 (canary: add v2 word timestamps). The branch intentionally includes that commit; please review this PR after #111, then rebase it onto main once #111 lands.

Summary

• add offline long-form transcription for Canary 1B v2 using 30-40 second windows, low-energy acoustic boundaries, and 1 second of overlap
• rebase timestamped chunk output onto the source timeline and assign overlap words by midpoint ownership
• reconcile text-only overlap against the previous hypothesis suffix so repeated phrases are not mistaken for duplicated context
• report Canary 1B v2 audio capacity as unbounded while leaving older Canary variants at their existing positional-table limit
• document that long-form input remains memory-resident and batch rows are processed serially

Motivation

Canary 1B v2 has a 40 second decoder window. Running longer audio as one window can silently omit substantial interior speech even when inference reaches the final phrase. On the 84.381 second samples/whole-earth.wav, the pre-long-form path returned 141 normalized words with a 35.040 second internal transcript gap. The chunked path returned 217 normalized words with a maximum 1.840 second inter-word gap and continuous coverage through the final phrase.

The first fixed-duration prototype also lost repeated words at chunk seams. Selecting low-energy boundaries and restricting text deduplication to a true suffix fixed that failure rather than masking it with a broader overlap heuristic.

Correctness

Adversarial repeated-speech tests passed without missing or duplicated repetitions:

Audio	Words	Result
44 s	88	exact 4x repetition
55 s	110	exact 5x repetition
88 s	176	exact 8x repetition
121 s	242	exact 11x repetition

The 39 s, 40 s, 40.001 s, and 41 s threshold cases also passed. Every benchmark returned monotonic positive word intervals within the audio duration, with normalized full text matching the word sequence.

Performance

Warm Q8_0 measurements on a shared RTX 4060 Ti:

Mode	55 s	121 s
timestamps	2,784.51 ms (19.8x realtime)	6,366.06 ms (19.0x)
text only	2,042.28 ms (26.9x realtime)	4,376.20 ms (27.6x)

F32 timestamp validation at 55 seconds passed on an RTX 3090 in 899.36 ms (61.2x realtime). The shared RTX 4060 Ti did not have enough free memory for the F32 model's 6.1 GiB CUDA allocation, so that cell was moved to the 3090 rather than changing running services.

Verification

• 52/52 configured CTest tests passed; 18 unrelated fixture-gated tests remained skipped by default
• gated timestamped and text-only Canary Q8_0 real-model smoke passed manually
• pinned clang-format check passed
• git diff --check passed

[ctolkmit]

Independent reproduction of the problem this PR fixes, and a test of the branch
itself, on a corpus you don't have: 22 real German voicemail recordings from a
production telephony system (8 kHz mono, resampled to 16 kHz with
ffmpeg -ac 1 -ar 16000 -c:a pcm_s16le), 18.4 minutes total, 12 of them under
40 s and 3 over 70 s.

The reference for every comparison is NeMo's own output for the same model
(nvidia/canary-1b-v2, fp32, ASRModel.transcribe()), i.e. the path that
enables dynamic chunking with 1 s overlap automatically above 40 s. Not ground
truth — but it is the behaviour a port is expected to match.

Builds: main at v0.2.3 and pull/112/head (e9a7b1e), both CPU backend,
x86_64, no BLAS, canary-1b-v2 Q8_0.

The failure on main

WER is word-level, case-folded, punctuation-stripped, with spelled-out German
digits normalised to digits (otherwise pure spelling style dominates).

	main	#112
mean WER vs NeMo	0.055	0.022
median WER	0.013	0.000
recordings with WER >= 0.15	2 / 22	0 / 22
73.2 s recording	0.200	0.048
102.4 s recording	0.473	0.012
300.0 s recording	0.057	0.014

The two main outliers are exactly the silent interior omission described in
the PR body. A halving experiment isolates it to the single-window path rather
than the weights or the quantisation:

recording	NeMo	main, whole file	main, two halves
102.4 s	166 words	87	81 + 85 = 166
73.2 s	125 words	103	77 + 46 = 123

F16 and Q8_0 produce the same word counts to the word, so quantisation is not
involved. Under 40 s the two paths agree exactly — as expected, since NeMo does
not chunk there either.

With this branch

Eleven of the 22 recordings now match NeMo word for word, and no recording is
worse in any way I can demonstrate.

One recording diverges (WER 0.079, 35.07 s, three differing words). It is the
worst audio in the set — a mobile call with severe dropouts. Two of the three
differences were checked by ear against the recording:

• one phrase: this branch is correct, the NeMo reference is wrong
• one short numeric ID: both are wrong, each one edit away from what is spoken
• one phrase in the badly corrupted opening seconds: unintelligible either way

So it is a divergence from an imperfect reference, not a regression. Note the
timestamps land at 5-7 s and ~22 s, i.e. nowhere near a 30-40 s window
boundary, so this is not a seam artefact.

max audio correctly reports unbounded (long audio chunked internally) on
the branch.

Two questions

1. Anything blocking this leaving draft state besides canary: add v2 word timestamps #111? The branch is based
   on b6a6aca, i.e. before 0.2.1, so anyone using it today gives up six weeks of
   fixes.
2. qwen3_asr shows the same single-pass loss — 143 words against NeMo's 168 on
   the 102.4 s recording, while the two halves reconstruct it. It is in the same
   "hard context cap" bucket. Is comparable long-form handling planned for it,
   or should that be a separate issue?

The audio and the transcripts cannot be shared — they are real customer calls.

[ctolkmit]

Follow-up: I rebased both commits onto main after v0.2.3 (92fc36d). Three
conflicts, all in plumbing, each resolved by keeping both sides:

• tests/CMakeLists.txt — whisper_bin_suppress_unit (main) next to
  canary_timestamp_unit (this PR)
• scripts/validate.py — main's sortformer stage list plus this PR's canary
  decode arguments inside the same loop
• examples/cli/main.cpp — main's parse_device_index plus this PR's
  UTF-8-validating json_escape; the final exit check needs both flags,
  output_ok (main, --output writes) and json_write_ok (this PR)

ctest is 38/38 green afterwards, including both of the test targets that
collided in the CMake conflict.

On the same 22 German voicemail recordings the rebased version is slightly
better than the draft branch: mean WER against NeMo 0.017 vs 0.022, median
0.000, none at or above 0.15. The one recording that diverged in my previous
comment — the badly corrupted mobile call — is back at 0.000, so those six
weeks of main carry something too.

[brauliobo]

Thanks for the independent German voicemail check, and for writing down the rebase conflicts.

Draft / rebase

Still stacked on #111 — that remains the thing blocking this leaving draft. Both commits are now rebased onto current main (585b98f, after v0.2.3 / Granite 5.0), so the branch is no longer stuck on b6a6aca.

Conflicts were the three you hit, keeping both sides:

• tests/CMakeLists.txt — whisper_bin_suppress_unit next to canary_timestamp_unit
• scripts/validate.py — sortformer stages plus canary decode arguments
• examples/cli/main.cpp — parse_device_index plus UTF-8 json_escape, with the exit path checking both output_ok and json_write_ok

Later main also moved canary onto opaque device handles and per-run compute scratch (release_scratch between long-form windows so the scheduler does not pin peak memory across a long clip).

canary-1b-v2 now advertises TRANSCRIBE_FEATURE_LONG_FORM. After the rebase the CLI was printing max audio: ~0 s because the decoder still has a finite n_ctx (per-window output budget). That check now comes after the long-form feature bit, so it prints unbounded (long audio chunked internally) again.

qwen3_asr

Separate issue. Same hard context-cap bucket, but a different decoder and no Canary-style window/overlap stitcher. I would not fold it into this PR.

WER on the rebased branch

Q8_0 timestamp GGUF, CUDA RTX 3090, transcribe-cli from this HEAD. Short-form German is a regression check against the existing FLEURS-de sample100 table. Long-form is a 109 s concatenation of 7 of those clips (0.5 s silence between them) plus the continuous files from samples/.

Set	Duration	Words	WER vs FLEURS ref	Notes
FLEURS-de sample100 (short-form)	100 clips, mean 13.2 s	—	3.60% (95% CI 2.58–4.77)	previous table 3.55%; 63 sub / 12 del / 3 ins vs 63 / 12 / 2
FLEURS-de concat (long-form)	106 s speech / 109 s wall	185 ref	2.15% (3 sub, 0 del, 1 ins)	the original failure mode was interior deletion

Continuous coverage (word timestamps, no FLEURS ref):

Audio	Duration	Words	Max inter-word gap
samples/whole-earth.wav	84.381 s	216	1.840 s (age. → On at 47.24–49.08)
samples/cj-swimming-drop.wav	70.110 s	172	1.440 s
samples/product-names.wav	56.104 s	115	1.120 s

whole-earth.wav is the clip from the PR body: pre-long-form was 141 words with a 35.040 s interior gap. Warm Q8_0 on the 3090: 1916 ms (44×) for 84.4 s.

I do not have your 22 voicemails, so I cannot rerun that NeMo comparison here. The FLEURS-de concat is the closest public stand-in: German, >40 s, scored against ground truth rather than NeMo, and it does not drop interior words.