[iamwavecut]

Changes

• Fibo 2.0 fused family (orbitquant.fused.families.fibo2) for Diffusers BriaFibo2Transformer2DModel:
  • Z-Image blocks: RMSNorm and per-token modulation in the activation prologue, with row tables for the noisy image tokens and the clean context (gist, images to edit) tokens;
  • Q|K|V is one GEMM, and Q/K RMSNorm with interleaved RoPE is one kernel;
  • sandwich post-norms and tanh gates run as row kernels on the residual;
  • the five injection blocks run their gated cross-attention to the text bundle (Q, K|V and output groups);
  • the two Perceiver layers run cross-attention, self-attention and SwiGLU groups with plain residuals;
  • one sample at a time; 8-bit activations for every group.
• SwiGLU padding: a SwiGLU group whose halves are not a whole number of 128-row interleave chunks (Fibo 2.0: 9536 rows) gets zero rows up to the next chunk (9600). The down projection reads only the real width.
• Tests: a tiny Fibo 2.0 transformer (896 wide, 7 heads, SwiGLU width 2368, one injection block, one Perceiver layer): fused BF16 groups against the eager model for text to image and edit, and a quantized W4A8 model.
• README: Fibo 2.0 in the fused-family table. Version: 0.15.0.

Verification

• macOS: ruff check and the full pytest suite pass (630 passed, 137 skipped: CUDA tests).
• RTX 5090 (Torch 2.12.1, CUDA 13.0): tests/test_fused_fibo2.py (6 passed) and the other fused-family tests pass.
• Fibo 2.0 Dev, one transformer forward on real prompt embeddings: fused BF16 groups against the eager model give cosine 0.99995 (text to image and edit). The 4-bit transformer with 8-bit activations gives 0.992 (text to image) and 0.994 (edit); INT8 attention changes the fourth digit.
• Fibo 2.0 Dev, 1024×1024, 50 steps, guidance 5, full pipeline on the RTX 5090: 20.3 s per image with a 12.3 GiB device peak, against 45 s and 27.9 GiB for the BF16 original.