Repository navigation
drama: Drama 3 (fishaudio/drama-3) text-to-speech on CUDA - #920
Open
plotarmordev wants to merge 15 commits into
Open
plotarmordev wants to merge 15 commits into
plotarmordev wants to merge 15 commits into
Conversation
setAllocationLimit bounds the bytes of device buffers plus pinned host buffers. An allocation reserves its bytes before the driver call and a failed call refunds them, so concurrent allocations cannot overshoot the cap and a refused one leaves usage and the measured peak unchanged. Host counts now take the same lock as device counts. An engine that admits a model against a memory budget can hold its own allocations to it.
The tower turns a backbone row into a frame's ten codebook values: four shared and nine residual layers whose cache keeps every preceding text and audio row, a semantic head and the reference sampler (temperature, top-p over the whole vocabulary, then top-k). Its projections run on cuBLASLt; drama_tower.cu holds the mixer, norms, RoPE, attention and sampling kernels.
The codec decodes ten codebooks a frame to mono 44.1 kHz PCM, 2048 samples a frame, as the reference's FP16 vocoder does: RVQ, the windowed post-module transformer, two upsample stages and the DAC decoder. codec_pth.zig reads torch.save's zip archive and runs its state-dict pickle on a VM that knows only the opcodes and globals a tensor state dict uses, so nothing in the file runs. Weight norm is folded in FP32 and tensors are cast to FP16 with round-to-nearest-even, as the reference does. codec.zig holds the decode; codec_load.zig holds the codec's geometry, the weight-norm folding, the FP16 casts and the one device weight image.
An engine may now synthesize speech: SpeechRequest carries a prompt, sampling settings and caller-owned PCM, Info.speech describes the audio it produces, and Info.text_generation lets a speech-only engine refuse the text routes. Text submit gains error.Unsupported, which the chat route reports as a request error.
The native CUDA backend opens fish_omni checkpoints with the Drama 3 host: one synthesis at a time on a worker thread that owns the context, stream, backbone, tower and codec, admitted against the same memory pool the text families use. The pool reading moves to cuda_memory.readPool so both paths share it, and the host caps its own allocations at what it was admitted for. Options the host cannot honor are refused at open.
OpenAI's speech request, built into Drama 3's default-voice prompt as the reference's prompt builder builds it and synthesized whole, then sent as WAV or raw PCM. Fields the route would silently drop are refused, and a speech-only engine starts the server without the text-only services.
Drivers that check the native backbone and tower against captured reference tensors, replay single operators against them, and trace each stage for attribution, plus tf-cuda-test drama-conv and drama-qk on synthetic rows. None of them runs as part of the host tests.
Drivers that decode captured codes with the native codec for comparison with the reference decoder, replay its RVQ and weight preparation in isolation, and trace each decoder stage. None of them runs as part of the host tests.
…ve host and their tool steps Embeds the backbone and tower fatbins, adds the drama module to the native server and its host tests, and a build step per backbone and tower oracle, replay and trace driver. The codec's kernels and tools follow on their own.
Embeds the codec fatbin and adds a build step per codec oracle, replay and trace driver. With this the series builds.
Some file systems accept an O_DIRECT open and then fail the aligned read with EINVAL. The reader now clears O_DIRECT on that descriptor and reads again through the page cache instead of failing the load; a test injects the refusal.
A marker test: an image can carry a device global that promises something about its kernels, and the host can check for it before relying on the promise. Any driver failure, a missing name included, answers false rather than an error. A test drives it through a fake driver table.
Drama 3's backbone is a Qwen3.5-35B-A3B-style hybrid: 30 Gated DeltaNet and 10 gated full-attention layers, each with a 256-expert top-8 block-FP8 MoE and a gated shared expert. Model.run appends rows and returns every row's final-norm hidden state and the last row's semantic logits. The kernels follow the arithmetic of the sglang-omni reference: the canonical 64-token chunked GDN prefill (drama_gdn_chunk.cuh, after flash-linear-attention's kernels as SGLang carries them), the packed recurrent decode, gated attention with Q/K norm and partial NeoX RoPE, dynamic 1 x 128 FP8 activation quantization, and the BF16 tensor-core GEMMs in drama_backbone_gemm.cu. Weights load as stored; nothing is requantized. THIRD_PARTY_NOTICES.md credits flash-linear-attention (MIT), whose chunked kernels the GDN prefill is adapted from and whose packed decode the recurrence follows.
This was referenced Oct 16, 2026
Open
Owner
|
Thank you, @plotarmordev, we will carry the Drama backbone, tower and native codec work with your credit into the combined Drama family; the landing order and remaining checks are in our reply on #927. Posted by Ash's AI Assistant Synthena |
Open
Conflicts, each kept from both sides:
- zig/build/cuda.zig: family() returns radio and drama; engines() and
nativeServer() take core, heat and drama; main's tf-cuda-test runner,
kernel contract and GLM/LFM2/RADIO helpers stay ahead of the drama tool steps.
- zig/src/core/engine_api.zig: the speech types beside main's vision types;
SubmitError holds Unsupported and Stale; the VTable keeps submit_speech
beside recall, imprint and drain.
- zig/src/native/cuda.zig: families lists lfm2_vl then fish_omni; pool() and
weightBytes() leave for cuda_memory's readPool() and weightBytes(); openDrama
refuses main's new --vision, --precision, --record and --learn-conversations.
- zig/src/server/{chat,root,routes}.zig, zig/tests/cuda/main.zig: both sides'
error arms, imports and test files.
- zig/src/server/speech.zig: imports the named json module (ashhart#860).
…n types in their own files, Drama's refusals in cuda_refusal Upstream main's engine_api.zig is at 600 lines and native/cuda.zig at 583, so the merge took them to 632 and 602. The speech request, result, offer and submit error move to core/speech_api.zig, and the image and video input types to core/vision_api.zig, re-exported under their names as rows_ask.zig's are. Drama 3's flag refusals join LFM2's in cuda_refusal.zig, with a test. No declaration, behavior or test is removed.
4 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
This adds the Drama 3 text-to-speech family on CUDA (part of #914): backbone, tower and codec, a speech hook in
engine_api, the nativefish_omnihost, andPOST /v1/audio/speech, which answers whole as WAV or raw PCM. Thecodec reimplements Fish Speech's published modded-DAC design with no Fish Audio code copied, and the weights and
codec.pthstay under Fish Audio's licence and are not included.This pull request is stacked on #915, #916 and #917; its first three commits carry the same patches. Its
own commits start at "drama: the Drama 3 text backbone on CUDA".
This overlaps #844 in CUDA build wiring, native family registration, engine capabilities, server integration, CUDA
tests and third-party notices. The recorded trial merge conflicts in
zig/build/cuda.zig,zig/src/native/cuda.zigand
zig/tests/cuda/main.zig; the proposed order is #844 first, then rebase this stack and reconcile the capabilityflags.
Receipt
Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision:
10796d46f625, on mainc7137bb3. No MLX or PyTorch.0.17.0, nvcc 13.0.88 and
CUDA_VISIBLE_DEVICESempty.fishaudio/drama-3ate0a9218a, against the reference sglang-omniab095ef9.1e76a70e, native binary sha256aa211423….Here it is rebased onto
c7137bb3, with comments made single-line and files split. Every Drama fatbin of thishead is byte-identical to the measured series' at the same commit. The host Zig is token-identical apart from
the file split, compared with
std.zig.Ast./healthwere 40,806,970,484 Bactive and 40,857,363,572 B peak (qualification.json, R7).
Exactness, from
tools/bench_concurrent.py --alone --serialortoken_shaagainst"draft": false: n/a, becausethere are no drafts and no text tokens. Fidelity against the reference is not qualified. These are selected
component gates on the final build (qualification.json):
The all-row cosine gate (0.99233 < 0.999) and last-row cosine gate (0.99670 < 0.999) also fail
(qualification.json).
Chunk invariance holds: backbone outputs at chunk sizes 1, 7 and 128 are hash-identical, and
drama-qkanddrama-convpass.Decode speed before and after, from
tools/bench_openai.py: n/a, because that tool measures text tokens. Speechused the recipe's harness (before-after-02, speed-stages.json):
bring the full series to 0.634.
These are distinct benchmark populations: the pre-speed RTF uses the normal-item speed-stage corpus, while the
reference/final comparison uses the warm before-after corpus. The reference and native audio durations differ; the
reference failed the supplementary CUDA-free/MemFree admission check but was admitted by the pilot guard, and the
effect of differing cache/admission conditions was not isolated (qualification.json).
Prompt speed before and after: the cold first request's
ttft_sfromtools/bench_cache.py --require-coldon a long prompt, or "n/a" with the reason when the change cannot reach prompt processing: n/a, because there is noprompt cache or token TTFT on the speech route. Process start to ready:
Tests run, with pass, fail and skip counts:
bef432bbb25e(one GB10, under the recipe's guard): all 46 normal-requestaudio bodies byte-identical to the promoted pre-upstream build, the long-05 clipped body identical in both
passes,
bench/stream_check.py9/9, warm RTF 0.654 against the pre-speed 1.087, and all six tower one-rowprojections on
drama_tower_gemv.CUDA_VISIBLE_DEVICES= zig build test-cuda-host --summary allfrom an empty local cache: 29/29 steps, 484 passed, 0 failed, 0 skipped.zig build test-golden --summary all: 4/4 steps.zig build native -Dnvcc=/usr/local/cuda/bin/nvcc --summary all: 46/46 steps.zig build fatbins -Dnvcc=/usr/local/cuda/bin/nvcc: the four Drama fatbins are byte-identical to the measuredseries' at this commit.
zig build STEP -Dnvcc=/usr/local/cuda/bin/nvcc --summary allfor eachdrama-*tool step of this head:18/18 steps build.
python3 tools/zig/lean_check.py: 527 problems at every commit, main's 527 (lean_new.py: 0 new).zig fmt --check: clean at every commit.tf-cuda-test drama-qkanddrama-conv(pass), and a 24-item × 3 warm corpus run per engine with 0 failures.Not run, and why:
tools/bench_openai.py,bench_concurrent.pyandbench_cache.py: they drive text tokens.bef432bbb25e; this head ishost-tested only. Its kernels are byte-identical to the measured build's at this commit.
Checklist
families are unchanged. Drama 3 generates audio frames in its own host loop, one synthesis at a time, not through
the lane rounds; Drama 3 (fishaudio/drama-3) text-to-speech on CUDA: family proposal and gate #914 raises this.)
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Nodrafts or resume. Concurrent-equals-solo is not measured.)
checkpoint's BF16 and block-FP8 are retained without requantization; the codec uses FP16. This is not a claim of
identical arithmetic to every reference call shape: the backbone's canonical arithmetic and the reference's
shape-dependent projections differ, and fidelity remains not qualified.)
touched.)
tests pass. The oracle drivers need a GB10, the checkpoint and reference captures, and are not host tests.)