[drowzeys]

This puts MiniMax Music 3 on TensorFold's own Metal runtime. A caption and lyrics go in; a 44.1 kHz stereo song with vocals comes out, from Zig and Metal on Apple Silicon, with no MLX, Python or PyTorch at run time. It is the Metal half of #982 and shares that pull request's config, prompt, checkpoint loaders, request settings, WAV writer, --serve protocol and parity tools. All three stages run here: the 8B AR LM with its 4-layer depth decoder (int8 by default, --weights bf16 for the checkpoint's own), the 2.4B flow-matching DiT and the DAV decoder.

On one Mac Studio (M5 Ultra, 256 GB), 60 s songs:

• The AR makes 68.9–69.1 frames/s in int8. The MLX engine on the same machine, prompt and seed makes 64.7–64.8 (runs alternated, three each). In bf16: 41.4.
• A whole 60 s song takes 59.5 / 62.8 / 63.0 s, against MLX's 62.3 / 63.9 / 63.8 s. The DiT is the long pole in both and runs on the same GEMM blocks, so the two are close; the AR is 6% faster here.
• 17.7–17.9 GB peak process footprint, against MLX's 28.1 GB. A 120 s song peaks at 18.4 GB and takes 123.7 s.
• Repeats are byte-identical. Three processes, and three requests in one --serve process, write the same WAV bytes.
• Parity with ComfyUI at every stage, teacher-forced on ComfyUI's own dumps from the same Mac: code-0 logits cos 0.999975 (bf16) and 0.999906 (int8), DiT final latent 38.97 dB, DAV 103.9 dB.
• Lyrics: Whisper scores 99 / 100 / 97 (int8) and 98 / 100 / 100 (bf16) on music3: MiniMax Music 3 lyrics to song on one CUDA GPU, from Zig (new family; proposal) #982's three songs.
• Kernel contract: all 57 new Metal launches pass gate 5, and m3_rmsnorm passes gates 1–4.
• Lean check: 0 new findings. zig build test passes every step on macOS.

Precision is the reference's, as on CUDA:

• bf16 activations in the AR, with bf16 roundings where torch rounds; fp32 norms, softmax, rope and attention sums.
• int8 weight-only, as music3: MiniMax Music 3 lyrics to song on one CUDA GPU, from Zig (new family; proposal) #982: a scale a row in the body and lm_head, a scale a group of 128 in the depth decoder and its heads, quantized on the GPU at load.
• fp16 in the DiT with the reference's fp16 roundings and fp32 sums.
• fp32 in the DAV. Its GEMMs run on the tensor units from bf16 halves of both operands (x = xh + xl: xh wh + xh wl + xl wh, fp32 sums). The tensor units' own fp32 path gave 42.3 dB here; plain fp32 simdgroup matrices gave 118.5 dB at 8 times the time; the halves give 103.9 dB.
• music3.metallib is built with safe math and precise fp32 functions. One exception: the fused GLU's sigmoid uses fast::exp, since precise::exp in that epilogue made the ff_in GEMM 2.6 times slower (final latent 38.92 dB with it, 38.97 dB without).

Next steps:

• The DiT. It is about 60% of a song. Its GEMMs run at 79–86 TFLOP/s on the same blocks as MLX's steel_gemm_fused_nax; int8 or fp8 products would be the next lever, a quality risk against the fp16 reference, not tried.
• The AR's frame overhead. The GEMVs alone read at 1,040 GB/s (body) and 990 GB/s (depth decoder); the other 3.6 ms of a 14.5 ms frame is attention, norms, the 8 draws and the dispatches between them. Folding the RMS norms into the GEMVs that read them would take part of it.
• The DAV. 1.4 s for 30 s, against MLX's 1.0 s at 50 dB. The halves could be written by the kernels that produce each GEMM's input instead of a pass of their own.
• GPUs before M5. The DiT, the prompt rows and the DAV need the tensor units, so the engine stops at load on M1–M4 with a message. nax.h's simdgroup-matrix form (TF_SIMD_FRAGS) compiles for them under gate 5; a second metallib for them was not built or run.
• Streaming. The MLX engine streams audio while the AR draws; this port answers when the song is done.
• The CUDA engine after the first commit: its host code builds for aarch64 Linux, but I did not re-run it on a GB10. The commit moves buffer handling only; the kernels, buffers and their order are unchanged.

What it adds

On top of #982, in two commits:

1. music3: share the backend-independent parts before a Metal backend. No change to what the CUDA engine computes.
   • common.zig holds the AR's sampling, result, trace and progress types, the song's timings, the weight bytes a frame reads and the clock.
   • weights.zig's loaders take any store with alloc / upload / stage / unstage; the CUDA store moves to store.zig, so weights.zig no longer touches the CUDA runtime.
   • generate.zig names its backend from the family module and asks the engine for its weight bytes; parity.zig takes either engine. The same two files now drive both backends.
2. music3: MiniMax Music 3 lyrics to song on Metal:
   • zig/kernels/metal/music3/ (one metallib, built ahead of time):
     • ar.metal holds the embeddings, the RMS norm, the int8 quantizer and the decode GEMVs (bf16, int8 a scale a row, int8 a scale a group of 128; a simdgroup two weight rows, or a gate and up row whose SwiGLU it writes).
     • attn.metal holds the body's q / k norm and rope, split-key decode attention (four keys a step under one simd_sum) and its merge, and the depth decoder's attention.
     • sample.metal holds music3: MiniMax Music 3 lyrics to song on one CUDA GPU, from Zig (new family; proposal) #982's CFG top-k draw (radix select, inverse CDF on the same Philox counters) and the frame's bookkeeping.
     • gemm.metal holds the tensor-unit GEMMs through tfp::gemm_block (bf16 prompt rows; fp16 for the DiT with the q / k rope and the GLU folded into their GEMMs' epilogues) and the DAV's fp32 GEMM from bf16 halves.
     • dit.metal holds the DiT's condition, windows, LayerNorm, timestep MLP, guided velocity, Euler step and noise; dit_attn.metal its attention on the tensor units (heads of 64, fp16 probabilities for P V as on CUDA).
     • dav.metal holds snake, the transposed convs' overlap-add, the stereo halves and tanh.
   • zig/src/families/music3/metal/: gpu.zig (the queue, the pipelines, and buffers addressed as u64, so the shared loaders and tools work unchanged), kernels.zig, ar.zig, dit.zig, dav.zig and engine.zig. music3_metal.zig is the family's Metal root.
   • zig/build/music3_metal.zig, plus 4 lines in build.zig: the metallib, the family module, and zig build music3-generate on macOS.
   • The kernel contract: 57 records in registry.zig; zig/tests/contract/metal_music3.zig (gates 1–4 for m3_rmsnorm, and the music3 sources for gate 5); a few lines in the runner; the sources in zig/kernels/metal/sources.zig.
   • docs/recipes/minimax-music3-metal.md and its JSON (parity_metal_int8.json, parity_metal_bf16.json, lyrics.json).

Each AR frame is encoded straight into a serial compute encoder and committed as soon as it is encoded, so the next frame encodes while the GPU runs the last one. Position, step and count advance on the GPU, as on CUDA, and the host reads the drawn tokens every 16 frames for the stop code.

Receipts

Machine and settings for every run:

Machine	Mac Studio, Apple M5 Ultra, 256 GB unified memory
OS	macOS 27.0.1, Xcode 27.1, Metal toolchain 27.1 (-std=metal4.1)
Build	Zig 0.17.0, zig build music3-generate -Doptimize=ReleaseFast
Checkpoint	Comfy-Org's MiniMax-Music3 repack, as #982 (minimax_music3_text_encoder_pruned_bf16, minimax_music3_dit_fp16, minimax_music3_dav)
Process	one model process at a time, under a lock shared with the other GPU jobs on the machine and a memory guard

Other processes on the machine: an Ollama server and another app were resident during the timed runs. I did not measure whether they used the GPU.

Parity against ComfyUI (tools/music3/parity.py)

ComfyUI's native nodes on the same Mac (MPS) recorded the dumps #982 describes: hiphop bench prompt, 30 s, seed 7. The MLX engine is against the same dumps; #982's CUDA numbers are against ComfyUI on its own GB10.

Check	Metal bf16	Metal int8 (default)	MLX bf16	MLX int8	CUDA bf16 / int8 (#982)
prompt ids equal	yes	yes	yes	yes	yes / yes
frame states cos, mean / min	0.999955 / 0.999748	0.999838 / 0.997195	0.999951 / 0.999821	0.999889 / 0.998982	0.999956 / 0.999827 (means)
code-0 logits cos (cond), mean / min	0.999975 / 0.999839	0.999906 / 0.99749	0.999973 / 0.999724	0.999938 / 0.99862	0.999977 / 0.999897 (means)
code-0 greedy agreement, cond / guided	0.9814 / 0.968	0.968 / 0.9694	0.9774 / 0.964	0.98 / 0.9507	0.9747 / 0.9667 (cond)
ComfyUI's drawn code, median rank in ours	1	1	1	1	1 / 1
depth logits cos, codebooks 1–7	0.99994–0.999955	0.999807–0.999859	0.99993–0.99996	0.99986–0.99991	0.99996 / 0.9998
depth greedy agreement, codebooks 1–7	0.951–0.976	0.933–0.977	0.951–0.983	0.933–0.969	0.964–0.981 / 0.945–0.967
DiT probe cos (cond / uncond), sigma 1.0	0.999995 / 0.999996	(the DiT is fp16 in both)	0.99999 / 0.999998		0.999991 / 0.999994
DiT final latent (ComfyUI's noise and states)	cos 0.999939, 38.97 dB		cos 0.999838, 34.9 dB		cos 0.999914, 37.6 dB
DAV of ComfyUI's latent, chunked as served	cos 1.0000000, 103.9 dB		cos 0.9999994, 50.5 dB		cos 1.0000000, 71.0 dB

The draws cannot match ComfyUI's (torch's multinomial against the Philox inverse-CDF draw) or CUDA's (the logits differ in the last bits), so a song is the same model's, not the same song.

Speed and memory: this port against the MLX engine on the same machine

Hiphop bench prompt, 60 s, seed 7. The MLX engine runs with --quant int8 (8 bits in groups of 64 everywhere). The runs alternated (Metal, MLX, Metal, MLX, ...), three each. Peak is the process's own peak footprint (/usr/bin/time -l).

Run	Engine	AR frames/s	AR s	DiT s	DAV s	Total s	Peak footprint
1	this port, int8	69.09	22.0	34.7	2.80	59.5	17.9 GB
1	MLX, int8	64.66	23.4	35.3	2.06	62.3	28.1 GB
2	this port, int8	68.87	22.1	37.9	2.81	62.8	17.9 GB
2	MLX, int8	64.66	23.4	37.2	2.02	63.9	28.1 GB
3	this port, int8	68.90	22.1	38.1	2.80	63.0	17.7 GB
3	MLX, int8	64.76	23.3	37.3	2.02	63.8	28.1 GB
one	this port, bf16	41.36	36.6	37.9	2.80	77.2	25.3 GB
one, long-pop 120 s, seed 5	this port, int8	66.95	45.3	72.5	5.90	123.7	18.4 GB

• The AR reads 11.15 GB a frame in int8, so 69 frames/s is 770 GB/s.
• Load takes 2.8–3.4 s from a warm page cache: 14.2 GB of weights are copied into shared Metal buffers, the int8 rows quantized on the GPU as they load. It is outside "total".
• Prompt: 300 tokens a CFG row in 0.27 s (MLX 0.18 s).

Lyric accuracy (tools/music3/lyrics_check.py)

#982's three 45 s songs in each format through --serve, transcribed by faster-whisper large-v3-turbo on the CPU and scored by the Studio's music.heard.

Song	int8 score	words heard	bf16 score	words heard
hiphop, seed 7	99	143 of 145	98	121 of 124
pop, seed 11	100	78 of 78	100	70 of 70
long-pop, seed 5	97	56 of 58	100	46 of 46

Determinism and request state

• Repeats: three processes (pop prompt, 30 s, seed 11) give byte-identical WAVs, MD5 e81021f8….
• Reset: three requests in one --serve process give the bytes of a fresh process.

Kernel contract (M5 Ultra)

SUMMARY 27 kernels: 27 pass gates 1-4, 0 fail, 2 known findings; 5 controls: 5 red as planted, 0 missed; 0 limit problems; gates 107510 ms, limits 254 ms
PASS limits: 134 registered Metal launches against M1-M5, 0 problems

• Gates 1–4: m3_rmsnorm, plain-gamma form, against the rows family's float64 RMS norm at the served shapes (2, 4 and 600 rows of 4096).
  • nonfinite: 4,100 non-finite outputs match their classes.
  • repeat: byte-identical.
  • shapes: worst 0.996 of the bound, with 4 edges refused.
  • sums: within the bound at k = 4,096.
• Gate 5: all 57 music3 Metal launches.
  • Each kernel reserves its threads in the source (max_total_threads_per_threadgroup), so none needs an M1/M2 compile record.
  • All 57 pass the on-GPU check of their pipelines and threadgroup bytes.
• The two known findings are affine_mm's, from before this pull request.
• Not run: M1–M4. Gate 5 checks the declarations against them, but the kernels have not been run there.

Lean check and tests

• python3 tools/zig/lean_check.py: 517 problems on music3: MiniMax Music 3 lyrics to song on one CUDA GPU, from Zig (new family; proposal) #982's head and 517 on this branch, so there are no new ones. Every new file is at or below 600 lines.

run	steps	tests passed	skipped	failed
macOS (M5 Ultra), zig build test -Dcpu=apple_m1, fresh cache, on #982	206/206	2,993 of 3,003	10	0
macOS (M5 Ultra), zig build test -Dcpu=apple_m1, fresh cache, this branch	217/217	2,998 of 3,008	10	0

[ashhart]

Thank you, @drowzeys, for the Metal Music 3 port and repeated WAV measurements.

Music 3 moves to 1.1.1, the next patch release. The reviewed integration uses the shared frame queue and prompt cache, independent session state and completion-aware cleanup; the prompt rewrite retains the credited model and design references.

Please provide the ten frozen prompt-ID cases with the matching tokenizer. The combined build still needs audio fidelity, resumed/fresh and concurrent/solo WAV comparisons, cancellation/reopen checks, resources, memory and matched speed.

No Metal Music 3 audio run is recorded; frozen prompt/tokenizer fixtures and fidelity, resume, concurrency and cancellation checks remain owed.

Posted by Ash's AI Assistant Synthena