drama3: Fish Audio Drama 3 text to speech on one CUDA GPU (new family, plain decoding; proposal)

[drowzeys]

Thank you, Ash, for the invitation to send this for review, and for the CUDA runtime it builds on.

This is the first speech model on TensorFold's Zig runtime. Fish Audio's Drama 3 is a 35B-A3B MoE backbone with a codebook tower and a neural codec. It now runs end to end on TensorFold's CUDA runtime, with no Python and no PyTorch at run time. Text, and optionally a voice to clone, go in; streamed audio comes out. This is the case for TensorFold beyond text: the same runtime, kernels contract and lean code can carry an audio model whose step is a frame rather than a token.

On one DGX Spark (GB10):

• 34 frames/s, about 5.4× the PyTorch reference's 6.1–6.4 on the same GPU.
• First audio in 0.24 s, and 0.44–0.46 s with a voice clone, the reference's encoding included.
• 37 GiB peak process memory.
• Repeated runs are byte-identical. Three requests in one process equal fresh processes.
• 98–99 % teacher-forced agreement with the oracle, with every differing pick listed: 93 of 97 are within 2 bf16 ulps.
• Codec decode SNR 115 dB.
• Kernel contract: 13 of 13 kernels pass gates 1–4.
• Lean check: 0 new findings.

Precision is the checkpoint's own:

• bf16 activations.
• The checkpoint's FP8 experts, with the dynamic per-row activations the reference serves them with.
• fp32 for norms, softmax, rotary, DeltaNet state, attention probabilities and the codec, with no TF32.

What comes next, for the gate we agree on:

• Putting Drama 3 on the lane core. Today the frame loop runs beside the lane engine, because a Drama 3 position is an eleven-column frame and the core carries one u32 token. How it fits the engine below lists the core changes. Plain decoding already satisfies "draft equals plain".
• Chunked prompts. Making prompt chunking invariant: prompts up to 1024 rows run as one chunk today.
• The kernel contract. Taking the other 44 kernels through gates 1–4.
• Metal. A Zig + Metal backend for Apple Silicon is in progress, as a follow-up pull request stacked on this one.

I did not open an issue first, since this came by invitation. Happy to move the gate discussion into one if you prefer.

What it adds

These are new files, plus the minimal wiring (13 lines in zig/build/cuda.zig, 3 in zig/src/cuda/kernels.zig, 7 in the contract runner, and 45 records in registry.zig):

• zig/src/families/drama3/ (Zig over the CUDA runtime; no Python and no PyTorch at run time):
  • config.zig checks every number the kernels assume and refuses any other checkpoint shape.
  • weights.zig stacks the bf16 projections and repacks the FP8 experts.
  • engine.zig and forward.zig hold the prompt pass and the frame step. The step samples, runs the tower, records the frame and runs the next backbone row, all as one CUDA graph. The tower's codes 2-9 run on a second stream.
  • prompt.zig builds prompt rows in SGLang-Omni's reference-bracket-v1 layout.
  • codec.zig and codec_encode.zig are the Fish S2 codec: a streamed causal decode with carried state, and the encoder for voice references.
  • wav.zig holds the WAV I/O and torchaudio's resampler.
  • tts.zig decodes each finished burst of frames while the next burst generates.
  • blas.zig opens cuBLAS's strided batched GEMM at run time.
• zig/kernels/cuda/drama3*.cu / .cuh:
  • drama3.cu is one fatbin: the backbone and tower glue, decode and prompt attention, the router, and the samplers. The samplers implement the Philox exponential race of SGLang-Omni's Triton kernels, plus RAS.
  • drama3_fp8.cu is fp8_experts.cu on FP8 rows with per-row scales.
  • drama3_codec.cu holds the codec's row kernels between cuBLAS SGEMMs.
• zig build drama3-generate:
  • text (with an optional voice reference given as audio or as codes) to a WAV, or raw PCM as it streams;
  • --encode for audio to codes;
  • --check DUMP against the oracle's dumps: every layer probed, the free-running greedy divergence, and the oracle's frames teacher-forced, with every differing pick and its margin;
  • --codec-check.
• tools/drama3/ (development tools):
  • oracle.py is a standalone PyTorch reference following SGLang-Omni 416a70c0's fishaudio_drama3.
  • oracle_prompt.py is a copy of its prompt.py.
  • codec_convert.py is the one-time conversion of codec.pth into the layout codec.zig reads.
  • The Fish Speech codec modules the oracle imports are not included: FISHCODEC_DIR points at them.
• THIRD_PARTY_NOTICES.md: a Drama 3 section.

How it fits the engine, and how it does not

• What a step decodes. A step decodes one frame: the main token, then codes 0-9.

  • The backbone, a Qwen3.5-35B-A3B MoE with Gated DeltaNet and gated full attention, scores 4,097 candidates: end-of-speech and 4,096 semantic tokens. The pick fixes the main token and code 0.
  • The 13-layer codebook tower then draws codes 1-9 one after another. It has its own KV cache over the backbone's positions.
  • The next backbone row embeds the frame: the token plus the code-1 embedding, times 1/sqrt(2).

• Lanes. Drafted frames could be verified the way lanes verify tokens. The backbone would take k drafted rows in one forward, the tower would take each row, and the engine would accept the longest prefix whose whole frame equals plain decoding. With no drafter, "drafted equals plain" holds trivially. This branch decodes plain only, one request at a time.

• What it would take to put Drama 3 on the lane core:

  • a position type wider than one u32 (an eleven-column frame), or a family hook that maps a frame to and from the core's tokens;
  • prompt rows that carry codes, since reference audio is part of the prompt;
  • per-codebook sampling with the 16-frame RAS window;
  • an output that is audio, through the codec, rather than detokenized text.

  Each of these is a core change, which is why I am asking before writing it. If you see a way to express a frame as lane tokens that I have missed, I will port to that instead.

• Precision.

  • bf16 activations and residual stream, as the checkpoint ships.
  • The FP8 experts round their inputs to e4m3 per row and per 128 inputs (amax / 448). This is SGLang's per_token_group_quant_fp8 for this checkpoint's activation_scheme: dynamic. It keeps the reference's arithmetic: skipping it moves each MoE block's output by about 1e-2 relative.
  • fp32 everywhere the reference uses fp32. cuBLAS runs with CUBLAS_COMPUTE_32F, default math, and no TF32.

Receipts

Machine and settings for every run:

GPU	NVIDIA GB10 (DGX Spark), sm_121, 128 GB unified memory, driver 580.173.02
OS	Ubuntu 24.04.4 LTS, kernel 6.17
CUDA	nvcc 13.0.88, cuBLAS 13.1.1.3, Zig 0.17.0, -Doptimize=ReleaseFast -Dsm=121
Checkpoint	fishaudio/drama-3 at revision e0a9218aa48888270f06ef1ed9ea67622c0e17d2 (including its codec.pth)
Process	one model process, alone on the GPU, under a MemAvailable kill-guard

Prompts (public):

• "plain": Harvard sentences list 1, sentences 1-2: "The birch canoe slid on the smooth planks. Glue the sheet to the dark blue background."
• "clone": "[sighs] Fine. If we miss it, it's on you. [laughs] Well, maybe that's fair." The voice reference is Common Voice en clip 10119832 (CC0), "We asked over twenty different people, and they all said it was his." That reference is 85 frames, giving 168 prompt rows.

Sampling: Fish's defaults (temperature 1.0, top-p 0.9, top-k 30), seed 1, at most 1,500 frames. Greedy runs are for parity only, because pure greedy loops.

Fidelity against the PyTorch oracle (drama3-generate --check)

The oracle is tools/drama3/oracle.py: bf16 PyTorch, FP8 experts with dynamic activations, and fp32 where SGLang-Omni is fp32. It dumped two prompts greedily for 300 frames: the clone prompt, and a no-reference sentence ("Hello! This is the Wasserman Slate AI Studio, powered by TensorFold, built with integrated Fish Audio Drama 3.", 46 rows).

Prompt pass, clone (168 rows):

• Backbone layers 0-29: cosine ≥ 0.99988 overall and ≥ 0.9991 on the worst row.
• After layer 30 one row (153) drifts: at the final norm it is 0.711 on that row and 0.99709 overall. On that row, the router's 8th and 9th experts are 6e-5 apart at layer 34 and exactly tied, on bf16 logits, at layer 38.
• The tower prefill re-run from the oracle's own final-norm rows is ≥ 0.99897 overall. Its worst row is 0.833, at tower layer 12, row 149.
• No-reference prompt: backbone layers 0-29 are ≥ 0.99995 overall (worst row 0.99949). The tower re-run from the oracle's rows is ≥ 0.99988 overall (worst row 0.99917).
• An fp32 run of the oracle (dumps/fp32ref, same rows) shows these are ill-conditioned rows, not a port error. The oracle's own bf16 run is 0.959 on row 153 and 0.840 on row 149 against fp32. This port is 0.861 on row 153 against fp32: the expert tie above goes the other way.

Teacher-forced, 300 frames each (my greedy pick against the oracle's at every column):

prompt	picks equal	differences	my largest margin over the oracle's pick
clone	98.80 % of 3,000	36	0.25
no-reference	97.97 % of 3,000	61	0.25

• Every difference is a near tie on my side. Measured in bf16 ulps at the logit's magnitude, my pick's logit leads the oracle's pick by:

  prompt	0 ulps (tie)	1	2	3	4	7
  clone	13	19	4	0	0	0
  no-reference	19	35	4	1	1	1

  The 7-ulp case is a weak codebook head: 7.97 against 7.75. The run prints every difference.

• The oracle dumps its logits only for frame 0's codebook heads and the candidate logits of its first 20 frames, so the two-sided tie policy can be checked only there. The first free-running divergence is frame 0, codebook head 7: my margin there is 0.0625 and the oracle's is 0.1875, between the same two codes (604 and 255).

• Free-running greedy frames therefore differ from frame 0. Token-for-token greedy parity with the oracle does not hold. The teacher-forced numbers above are the fidelity claim.

Sampling: seeded runs cannot be compared with the oracle. The oracle draws with torch.exponential_; this port implements the Philox stream of SGLang-Omni's Triton sampler, which I could not run here.

End of speech, plain prompt, seeds 1-3:

	ends before the cap	frames when it ends
this port	1 of 3	213
oracle	2 of 3	232 and 150

Each side runs one seed to its cap with RAS repeatedly firing. This looks like the model's behaviour on this text, not the port's.

Codec (--codec-check):

check	result
Streamed decode of 300 frames (a first chunk of 4, then 32) against the PyTorch codec in fp32	SNR 115.22 dB
Resampler, 24 kHz to 44.1 kHz, against torchaudio	144.0 dB
Encoder blocks	99.4-138.4 dB
Reference codes	850 of 850 equal (100 %)

Generation from the WAV reference gives the same frames as generation from the oracle's prompt rows: MD5 160c87ce….

Determinism and request state

• Repeats: three runs of each receipt below give byte-identical frames and WAVs.
• Reset: three requests in one process give the frames of a fresh process.
• Prompt chunk seams: FAIL. The clone prompt prefilled in 64-row chunks gives different frames from frame 0 than one 168-row chunk. cuBLAS's GEMMs and the attention chunks change the bf16 rounding of the prompt rows. With the default --max-rows 1024, every prompt up to 1,024 rows (about 44 s of reference audio plus the text) is one chunk. Longer prompts are not chunk-invariant. I have not fixed this.

Performance: before and after on the same machine

• Before is the fork's drama3-cuda at 81173e6. That is the code this branch was ported from: built on an older base, with the experiments switched off by default.
• After is this branch.
• Runs alternated, three each, with the same checkpoint, flags and seeds.
• Frames and WAVs are byte-identical before and after.

run	frames	frames/s (before → after)	first audio ms	prefill ms
plain, seed 1 (no end of speech before the 1,500-frame cap)	1,500	34.39 / 34.43 / 34.35 → 34.37 / 34.31 / 34.22	236-239 → 236-251	115-116 → 115-124
clone from WAV, seed 1 (ends)	146	34.02 / 33.80 / 33.75 → 33.94 / 33.81 / 33.75	442-462 → 442-445	217-220 → 217-218

• Audio runs 21.53 frames a second, so about 34 frames/s is 1.6x real time.
• First audio for the clone run includes encoding the reference WAV (100-120 ms).
• Load takes 14.5 s from a local NVMe copy.
• Reference implementation: the PyTorch oracle decodes 6.1-6.4 frames/s on the same GPU.
• Standard server not run: SGLang-Omni was not run here, so this table has no comparison with the applicable standard server.

Memory:

• Peak process RSS: 37.1 GiB.
• Device peak reported by the runtime: 41.6 GB with the codec, 39.3 GB without.
• Both measured on the 128 GB unified-memory class.

Kernel contract

SUMMARY 13 kernels: 13 pass gates 1-4, 0 fail, 0 known findings; 5 controls: 5 red as planted, 0 missed; 0 limit problems; gates 43862 ms, limits 4 ms
PASS limits: 71 registered CUDA launches against sm_80/86/89/120/121, 0 problems
PASS kernel-contract-limits: every registered launch fits sm_80, sm_86, sm_89, sm_120 and sm_121, every control red

• Gates 1-4 (GB10): d3_rmsnorm, plain-gamma form, against the rows family's float64 RMS norm. It is the 13th kernel in the SUMMARY.
  • nonfinite: 2,052 non-finite outputs match their classes.
  • repeat: byte-identical.
  • shapes: worst 0.996 of the bound, with 4 edges refused, as for tf_rms_rows_bf16.
  • sums: within the bound at k = 2,048.
• Gate 5: all 45 Drama 3 launches, with records from fatbins built for sm_80, 86, 89, 120 and 121.
• Not registered for gates 1-4 (44 kernels): the GEMV, the DeltaNet recurrence, attention, the router, the samplers, the FP8 experts and the codec kernels. They need test types of their own, and I would rather agree the gate with you first.
• No FP64 references exist per projection class. The fidelity numbers above are against the bf16 oracle and an fp32 run of it.
• Other SMs: sm_80 and sm_86 needed a __CUDA_ARCH__ >= 900 guard around griddepcontrol.wait, and PDL is switched off below sm_90. Those images compile and fit, but have not run.

Lean check and host tests

• python3 tools/zig/lean_check.py: 517 problems on the base (46a08c1), 517 on this branch. tools/zig/lean_new.py reports 0 new. All 517 are inherited.

run	steps	tests passed	skipped	failed
Linux (GB10 host), zig build test, fresh cache	86/86	1,382 of 1,389	7 (libjpeg-turbo 3.1.4.1 absent, JPEG cases)	0
macOS (M5 Ultra), zig build test -Dcpu=apple_m1	134/206	1,338 of 1,346	8	0 tests
macOS (M5 Ultra), zig build test-cuda-host -Dcpu=apple_m1	41/41	856 of 858	2	0

• Drama 3 adds 5 host tests: config parse, config refusal, speaker tags, WAV round trip and resampler length/level.
• macOS: the 39 failed steps are all xcrun not finding metal, because that machine has no Metal toolchain. This branch touches no Metal code.

Not tested, or not in this pull request

• Metal. Drama 3 on Macs exists on the fork only as an MLX/Python family on the frozen 0.6 tree, so it cannot land here. A Zig and Metal port would be its own pull request.
• Other GPUs: sm_80, sm_86, sm_89 and sm_120 are compiled and gate-5 checked, never run.
• Multi-GPU.
• Concurrency: there is no concurrent request path; one request at a time.
• Serving: no server route, so no /v1/audio/speech, and no prompt cache or resume.
• Long contexts: prompts over 1,024 rows (the seam failure above), and contexts near --max-ctx 4096.
• The Philox draws have not been checked against SGLang-Omni's Triton sampler itself.
• Kernel contract: gates 1-4 cover one of the 45 kernels.

Licenses

• Code: Apache-2.0, as TensorFold.
• Weights: none are included.
  • fishaudio/drama-3 and its codec.pth are under the Fish Audio Community License Agreement v1.0. That covers research and non-commercial use, and commercial use below a USD 500,000 annual revenue threshold. It grants no Hosted Service without a separate license from Fish Audio. Users download the checkpoint and accept that license themselves.
  • The file codec_convert.py writes is a derivative of codec.pth under the same license.
• Sources followed:
  • The family follows SGLang-Omni (Apache-2.0) at 416a70c0. tools/drama3/oracle_prompt.py is a copy of its prompt.py.
  • The codec arithmetic follows Fish Speech's modded DAC, which is "Software" under the same Fish Audio license, and descript-audio-codec (MIT). codec.zig and drama3_codec.cu are separate implementations and include none of that source. The oracle imports those modules from a directory the user supplies.
• A question for you: whether a separate implementation of Fish Speech's codec arithmetic is acceptable in this repository, given that license. I have kept every Fish file out of the tree.

Reproducing

zig build drama3-generate -Dnvcc=/usr/local/cuda/bin/nvcc -Dsm=121 -Doptimize=ReleaseFast
# once: codec.pth -> the native codec's file (PyTorch; FISHCODEC_DIR holds the Fish Speech DAC modules)
python3 tools/drama3/codec_convert.py /path/to/drama-3 codec-v1.safetensors
zig-out/bin/drama3-generate --model /path/to/drama-3 --codec codec-v1.safetensors \
  --text "The birch canoe slid on the smooth planks. Glue the sheet to the dark blue background." --seed 1 --wav out.wav
# parity: an oracle dump, then the check
python3 tools/drama3/oracle.py --model /path/to/drama-3 --text "..." --greedy --max-frames 300 --dump DUMP
zig-out/bin/drama3-generate --model /path/to/drama-3 --check DUMP --check-frames 300
# kernel contract (GB10) and gate 5 for every SM
zig build kernel-contract -Dnvcc=/usr/local/cuda/bin/nvcc
zig build fatbins kernel-contract-fatbins -Dnvcc=/usr/local/cuda/bin/nvcc -Dsm=80,86,89,120,121
zig-out/bin/tf-cuda-test kernel-contract-limits zig-out/fatbin

🤖 Generated with Claude Code

https://claude.ai/code/session_01PfjmufXm4L7qsVNFyRw9ZK

[BobClawblaw]

@drowzeys For the "Other GPUs: … sm_89 … never run" line: #939's kernels on a real sm_89 card. RTX 4090 (24 GB), Windows 11, driver 616.56, nvcc 13.3, -Dsm=89.

How it was built. #939 targets Linux, and this machine has none, so I merged #836 (the Windows port) into 2cfd812b locally. Merging onto 1.0.12 needed a few local stand-ins, none of them in Drama 3's kernels or launches:

• __restrict__ on three prefill_attention.cu instances, and _USE_MATH_DEFINES for vit.cu.
• sys.DynLib in blas.zig, jpeg.zig and tfvideo.zig.
• tp_watch and the JPEG version check stubbed out.

The kernel contract on sm_89 (tf-cuda-test kernel-contract):

Check	Result
d3_rmsnorm nonfinite	PASS: 2,052 non-finite and 8,188 finite outputs match the float64 classes
d3_rmsnorm repeat	PASS: five runs byte-identical
d3_rmsnorm shapes	PASS: worst 0.996 of the bound, 4 edges refused
d3_rmsnorm sums	PASS: within the bound at k = 2,048
gate 5, live on the sm_89 driver	45 of 45 Drama 3 launches pass: 28 drama3, 13 drama3_codec and 4 drama3_fp8

On the live launches:

• Every Drama 3 launch's register and shared-byte counts under nvcc 13.3 equal the recorded ones.
• d3_sample_main has 65,536 dynamic shared bytes, under the 101,376 that sm_89 allows a block.
• d3_gdn uses 255 registers at 32 threads.

Not from #939. The run's summary is 10 of 13 kernels passing gates 1–4, with 1 limit problem. All of those are kernels already on main:

• qmm_group (cluster tile) fails all four gates with CudaFailed. On this card that's expected: a cluster tile needs sm_90. The contract still runs it on sm_89 rather than skipping it.
• The shapes gate fails with CudaFailed for lane_gemv and lane_gemv split-K.
• The limit problem is lane_gemv at 58 registers against the recorded 57, which may be nvcc 13.3 against 13.0.

Not verifiable on a 24 GB card: frames/s, first audio, the oracle check and memory. The checkpoint is 38.7 GiB, so the model doesn't fit. --codec-check needs the PyTorch codec's reference waveform, which I didn't produce.

[cwschroeder]

Host-side review of 2cfd812b14f9b9f28b76ffa0d8018f7806cce33d, based on main 46a08c18. The five family tests are wired into test-cuda-host; I ran them and the module wrapper. Three reachable issues are below.

Small warmup fix, separate formatter patch and CPU reproducers. The warmup fix is e17e03ac5e7e20f22538e288d42fad0229483571; it bounds the actual decode to its output allocation. The harness calls the checkout's exact Tts.create with inert transports: four small-chunk cases fail before the fix, and all six cases pass in both Debug and Fast afterward. It does not perform CUDA or codec arithmetic.

The unchanged PR head also fails the pinned zig fmt --check build.zig zig on config.zig, kernels.zig and codec.zig. Separate commit b34800ef47ec24debafb61f5a1a5de756f256484 applies that formatter. On both fix commits together, all four exact No GPU test commands, the family tests, full-tree format, whitespace and lean comparison pass locally with Zig 0.17.0. Lean is 517 versus 517, zero new. The intentionally failing output-error diagnostic is separate from those passing CI tests.

Output-error propagation and codec-only admission remain unfixed in the offered patches. Upstream workflow runs currently conclude action_required; the No GPU run 38901868963 has zero jobs, so there is no executed remote test failure to diagnose. No GPU, model, audio quality or performance qualification is claimed here.

[cwschroeder]

[P1] Keep the codec warmup inside the allocated audio buffer

--chunk 1 --first-chunk 1 reaches Tts.create with a one-frame maximum. The audio allocation is then 1 × 2048 × 4 = 8192 bytes, but this unconditional four-frame decode requires 32768 bytes under Codec.decode's output contract. Maxima 2 and 3 are also too small. The CPU transport fixture calls the actual create function and records this mismatch without running a GPU. The linked patch uses min(t.max_chunk, 4) for both the silent input allocation and decode call; all six bounds cases pass, and defaults at four or more retain their existing warmup.

[cwschroeder]

[P2] Propagate audio write and finalization failures before reporting success

These fwrite and fflush results are discarded, so a full or broken output can still increase samples, let speak succeed and reach RESULT drama3 OK. A CPU test of the exact AudioOut block confirms both an immediate write failure and a buffered flush failure: ferror is set, yet put returns success and counts the two samples. The valid-sink control writes all four PCM bytes. Please check writes/flushes, and make final header seeking/writing/close failures observable before the success line too; current close silently discards them. The linked output diagnostic remains red and is not fixed by the warmup patch.

[cwschroeder]

[P2] Allow the documented codec-only encode mode past model admission

The usage advertises --codec CODEC --encode OUT --ref-audio REF.wav without --model, but this unconditional model-directory check exits with usage before the independent encode branch below. That branch only uses the codec and reference audio, not a.model. The actual CLI diagnostic reproduces exit 2 before any CUDA/codec access; its macOS link-only posix_fadvise trap shim is never called. Route encode before the model-only check, or exempt that mode from the model-directory requirement. This remains unfixed in the offered patches. They do not claim successful encoding.

[drowzeys]

Thank you, @cwschroeder: all three findings were right, and they're fixed in 3515c27f (on top of 36368296, the formatter).

• [P1] Warmup: your bound, min(max_chunk, 4) for both the silent input and the decode, credited to you as co-author.
• [P2] Write failures: every WAV/PCM fwrite and fflush, the header seek and rewrite, and fclose are now checked. A failure ends the request before its RESULT line, and the error path closes the file without reporting.
• [P2] Encode without --model: --codec CODEC --encode OUT --ref-audio REF.wav now gets past admission without a model directory, as the usage says. --check still requires one.

The CI commands on the new head, Zig 0.17.0, run on a DGX Spark with the GPU hidden:

Command	Result
zig fmt --check build.zig zig	clean
zig build test	86/86 steps, 222/223 tests passed, 1 skipped
zig build test-kernel-contract	18/18
zig build test-golden	pass
test-cuda-host	41/41 steps, 48/48 tests
lean_check	517, the same as main

Thanks also to @BobClawblaw for the sm_89 contract run: all 45 Drama 3 launches pass gate 5 on the 4090.

🤖 Generated with Claude Code

[cwschroeder]

The three earlier findings are addressed at 3515c27f. I verified that the adopted warmup and formatting files match the offered patch. The six warmup cases pass in Debug and Fast. The actual CLI argument/admission prefix now admits codec-only encode without a model, with five other controls still passing. The unbuffered/buffered stdio-error cases now return AudioWrite, and successful PCM output still counts its samples.

There are two small cleanup notes at the newly fallible boundaries, below. An applicable two-line patch and CPU reproductions register error cleanup for the opened WAV and the returned host frames. Both cleanup cases fail on this head; all five output/ownership cases pass with the patch in Debug and Fast. These currently abort the CLI, so I am treating them as cleanup nits rather than a new generation or GPU failure.

At candidate 7b0f897c, fork validation #14 has all six current-head checks green, including the complete No-GPU commands, formatter and app/mobile workflows. Local formatting, whitespace, lean (517/517, zero new) and a compile-only Linux arm64 CLI build also pass; source review found no issue in the two-line delta. Your upstream workflows remain action_required, with no executed job results there. These CPU/CI receipts do not qualify model fidelity, codec output quality or GPU performance.

[cwschroeder]

[P3] Release the opened WAV if the initial header write fails

The new try writeAll can return before AudioOut.open transfers ownership to the caller. There is no cleanup registered here yet. A CPU fixture returns a real read-only FILE from the opening transport; the actual write returns AudioWrite, but F_GETFD shows its descriptor remains open. Add errdefer o.abandon() immediately after constructing o. This also covers the older later-open failure path.

[cwschroeder]

[P3] Own the returned frames before fallible output finalization

speak has returned caller-owned r.frames, but the outer frame cleanup is registered only after this branch. If out.close() fails, those frames are lost. A CPU fixture closes the WAV descriptor after a stub speaker returns its host array; the actual finalization returns AudioWrite and DebugAllocator reports that array leaked. errdefer gpa.free(r.frames) immediately after speak covers the failure. It ends on successful branch exit, leaving the existing outer defer to free successful results.

[drowzeys]

@cwschroeder: both cleanup notes are in 75eeed78, credited to you.

• AudioOut.open now has an errdefer o.abandon() right after construction.
• errdefer gpa.free(r.frames) follows speak.

At the new head, with the GPU hidden:

Command	Result
drama3-generate	builds
test-cuda-host	41/41 steps
zig build test	86/86 steps
lean	517, the same as main

The Metal branch (#956) merges this commit.

🤖 Generated with Claude Code

[ashhart]

Thank you, @drowzeys, we will carry the CUDA model and FP32 codec work with your credit into the combined CUDA/Metal Drama family; the landing order and remaining checks are in our reply on #956.

Posted by Ash's AI Assistant Synthena

[drowzeys]

Rebased on main 6ee828c2; head 75eeed78 is now 048140e1. The pull request's diff against main is patch-identical to before (main's new commits only touch docs); zig fmt --check clean on the changed Zig files; python3 tools/zig/lean_check.py reports no new findings against main.

[drowzeys]

Following up on the two points in the reply on #956 that apply to this branch. Both are fixed here; the head is now 582e83a4. These are appended commits only (no rebase), and #956 merges them.

• EOS-shortened slice: fixed in f2b9d945. Engine.generate and Tts.speak returned host[0..made] of the allocation sized to the frame limit, so after an early end of speech a caller freed the wrong length.
  • A new frames.zig has owned(), which shrinks the allocation to exactly the frames kept (realloc). If that fails, the original buffer stays the caller's to free, and the existing errdefer frees it.
  • Both paths share endOfSpeech() for the scan.
  • Tests: an early end of speech returns exactly the frames before it, freed at that length, and no end of speech keeps every frame; one at the first frame keeps none. Both free through the testing allocator, which fails a free of the wrong length.
• Wrong JSON kinds in the shared config: fixed in 582e83a4. Config.parse no longer reads .object, .array, .string or .bool straight off the JSON.
  • Each field is matched on its kind and refused by name: error.ConfigNotObject, ConfigNotInteger, ConfigNotArray, ConfigNotString or ConfigNotBool. A warning names the field.
  • A token id that doesn't fit int32 is error.BadConfig instead of a cast that traps.
  • The semantic band's width is computed in i64.
  • Test: a field of the wrong JSON kind is refused by name, not trapped on. It takes the shipped config with one field changed per case: a string, a float or null for an integer, an id past int32, a negative id, arrays and strings where objects belong, an object for layer_types, a number in layer_types, a number for rotary_interleaved, and a non-object root.

Checks on a DGX Spark with the GPU hidden:

Check	Result
zig fmt --check build.zig zig	clean
lean_check	517, the same as main
zig build test	86/86 steps
test-cuda-host	41/41 steps, including the new tests
drama3-generate	builds

Kernels and launches are unchanged, so I didn't rerun the kernel contract. The prompt-chunk failure cases, reference margins and long/reset/concurrent audio cases are all kept.

[cwschroeder]

Rechecked the shared output cleanup at 582e83a4. The exact AudioOut/frame-ownership blocks, WAV helper, frames helper and config parser are byte-identical to #956 at ec6f16dc, where I independently ran the five actual stdio/frame-cleanup cases and five frames/config cases in Debug and Fast. All pass. The two cleanup notes from my earlier review are closed.

The early-EOS regression also fails at the wrong-length free when only the old shortened-view return is restored in a temporary copy. Astra checked the owned-frame handoff, realloc failure cleanup and named config refusals. This confirms the shared host delta; it adds no CUDA device, codec/model fidelity or performance qualification. Current-head remote checks are absent.

[ashhart]

Thanks @drowzeys for the exact-length EOS ownership fix and named config-kind refusals, with allocator and malformed-config controls. Those address the two requested boundaries and belong in the common CUDA/Metal code.

We are carrying one Drama family with both contributors' work, the stronger codec, shared frame serving and completion-aware cleanup. The host tests and generator build do not settle the retained prompt-chunk failures, reference margins or long/reset/concurrent audio cases. Those remain gates on the combined source. The updated branch's unchanged kernels are not a reason to omit the final integrated numerical and resource checks.

Posted by Ash's AI Assistant Synthena

[ashhart]

Thank you, @plotarmordev, for the speech protocol and infrastructure, and @drowzeys for the family, backends and codec. The selected work from #939 is included through #927 (5dcf77d7c), the consolidated Drama 3 landing in 1.1.0. One CUDA/Metal family uses bounded shared frame rounds, request-owned state, native checkpoint/codec loading and streamed PCM speech. HTTP retains its checked default-voice subset. Full scope and audio, interleaving and completion results are on #927: Apple M5 Ultra norm checks passed 1/1 and CUDA resources passed; checkpoint audio fidelity, seeded concurrency and cancellation remain unrun.

Posted by Ash's AI Assistant Synthena

[ashhart]

The consolidated Drama contribution shipped in 1.1.0 as 5dcf77d7c; thanks @plotarmordev and @drowzeys. Experimental attention variants remain separate. We're closing it as landed.

Posted by Ash's AI Assistant Synthena