[drowzeys]

Stacked on #949 (upstream/h3-metal); review that one first.

Image to video now runs on TensorFold's Metal runtime. A FastH3 clip can start from a given first frame, not only from a prompt. The encoded image's rows sit in the packed sequence between the text and the audio. On every pass they enter like video rows, held at the keyframe noise level, and they are never stepped. The model generates the remaining 124 frames and the soundtrack around them. This is what the Studio's image-to-video shots run on.

On the same Mac Studio (M5 Ultra), with FastH3 8-Step V2 at 864x480 and 124 frames, a case with a first frame has 16,253 packed rows: 449 text (the prompt plus the image's vision tokens), 405 keyframe, 414 audio and 14,985 video, in 302 tiles (22 prefix). Results:

• Parity at step 1 against the MLX bf16 reference: video cosine 0.99711 (relative error 0.076, worst row 0.560), audio cosine 0.99986 (relative error 0.017). With a real keyframe in the prefix, the routing at step 1 is less noise-driven than in text to video, where it is 0.98981.
• Text to video is bit-identical with and without this change: the final video and audio rows after 8 steps, compared with cmp.
• No new kernels. The rows reuse h3_rows_in. Lean check: 517 before and after, 0 added.

What it adds

zig/src/families/h3/h3.zig (+16 lines):

• the case's seventh geometry count;
• the held condition rows;
• their pass through the video input projection, ahead of the audio rows.

Also: the tool's row summary in zig/tests/h3_dit.zig, and a section in the recipe.

Receipt

The case was exported by the fork's h3_case.py --first-frame fox-first.png, with seed 7 and the same public prompt as the base pull request. The image is the first frame of the base pull request's own text-to-video render, so it carries no third-party rights. Run:

zig-out/bin/tf-h3-dit <FastH3>/transformer 14 i2v/case.safetensors out

• Peak memory footprint 4.50 GB (/usr/bin/time -l), plus the checkpoint mapped read-only.
• Timing is not quoted. This run shared the GPU with another job before the machine's GPU lock was in place, so its 12.6–17.9 s a step is not a clean receipt.

Next steps

• Timed receipts under the GPU lock.
• Last-frame and multi-keyframe conditioning, which use the same held-row mechanism with more blocks of rows.
• The image encoder (VAE encoder and vision tower) in Zig. Today the fork's Python exporter encodes the image.

Licenses

The code is Apache-2.0. MiniMax H3 and FastH3 weights are under the MiniMax H3 Community License. No weights are included.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PfjmufXm4L7qsVNFyRw9ZK

[ashhart]

Thank you, @drowzeys, we will carry the first-frame conditioning work with your credit into the gated H3 reference-input path; the landing order and remaining checks are in our reply on #961.

Posted by Ash's AI Assistant Synthena

[ashhart]

Thank you, @drowzeys, for H3/FastH3 on Metal. The selected work from #950 is included through #961 (013b75368) in 1.1.0, within one family and the bf16 tf-h3-dit test tool. Geometry/media checks and transactional schedule replacement accompany the integration. Full scope and numerical, resource and reference-media results are on #961: Apple M5 Ultra H3 kernels passed 15/15 with controls; full FastH3/H3 checkpoint/media-tool cells remain unrun.

Posted by Ash's AI Assistant Synthena

[ashhart]

The bf16 H3 Metal component shipped in 1.1.0 as 013b75368; thanks @drowzeys. We're closing it as landed.

Posted by Ash's AI Assistant Synthena