Repository navigation
Add fixed diffusion tensor execution for SDXL and PixArt - #314
Draft
schopde-nvidia wants to merge 1 commit into
Draft
schopde-nvidia wants to merge 1 commit into
schopde-nvidia wants to merge 1 commit into
Conversation
Flare's token-serving contract does not describe complete diffusion tensor requests. Comparing diffusion against native Flare requires loading exported computation independently of Python. Keep model composition and solver math in Ignition's prepared graphs, and let Helios validate tensor storage and replay the declared graph sequence. Before this change, signed cores and the CLI required token-serving metadata and offered no offline tensor-request command. Add physical graph-only core publication/loading, a generic TensorRunner, and flare run tensor input, output, repetition, and timing options. Validate tensor bounds and input overlap, retain GPU resource owners through queued work, and preserve the paired metadata requirements for token-serving cores. Add fixed SDXL and PixArt preparation recipes, geometry and solver helpers, intermediate-output comparison, and reproducible setup instructions. A separate iterative-serving contract is documented as proposed and is not implemented. The resulting executor runs both diffusion architectures without model-specific native computation. The measured result is byte-identical native replay of captured compiled outputs for the tested profiles. Canonical eager PyTorch numerical equivalence remains unresolved: exact comparisons fail, and the comparator withholds performance ratios. This does not establish concurrent diffusion serving, broader model support, or a speedup. On an RTX PRO 6000 Blackwell Server Edition (SM120), tested SDXL at 1024x1024 with 4 and 30 updates and PixArt at 512x512 with 20 updates, each with two prompts and seeds 42/43. Composed eager recipes match canonical Diffusers; independently loaded native outputs and repetitions match captured outputs. The native process loads neither Python nor Torch. Focused validation passed 114 Rust tests, 28 SDXL Python checks, 41 PixArt Python checks, and 38 geometry and schedule checks. After tightening native acceptance to compare bytes including signed zero, 15 comparator tests pass and all six retained GPU comparisons retain their native-pass/canonical-fail result. These are focused checks, not the full repository suite. GPU execution used source based on a3e8778; integration with subsequent upstream changes has not been built or GPU-tested. Reproduction details and open qualification are in docs/diffusion-design.md and docs/TODO.txt. Design-Signed-Off-By: Codex (GPT-6); design-review=ed22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Flare's token-serving contract does not describe complete diffusion tensor requests. A meaningful comparison with PyTorch needs a separate native process that loads the exported text encoder, denoiser, solver updates, and image decoder. This change keeps those computations in Ignition's prepared graphs and adds shared native tensor transport and graph replay.
Before this change, signed cores required token-serving metadata and the CLI had no offline tensor-request execution path. This PR adds:
TensorRunnerandflare run --tensor-request --tensor-output. They validate tensor extents, input overlap, and graph order, retain resource owners until queued work drains, and write intermediate tensors, repeated outputs, and timing samples.The demonstrated result is complete native execution of two diffusion architectures and byte-identical export/replay for the tested profiles. Canonical eager PyTorch numerical equivalence remains unresolved. Compiled outputs differ from the canonical references, so this is a draft PR with no speedup claim. The command handles fixed offline tensor requests; it does not provide an HTTP image endpoint, concurrent diffusion scheduling, native prompt processing, or general diffusion-model support.
Validation used one NVIDIA RTX PRO 6000 Blackwell Server Edition (SM120), PyTorch 2.14.1+cu132, Triton 3.8.0, and Diffusers 0.43.0. Every profile used two prompts with seeds 42 and 43, three warmups, and five measured native repetitions.
SDXL uses FP16 text encoders/UNet, the canonical FP32 VAE upcast, guidance 5, and negative prompt
blurry, low quality. PixArt uses canonical mixed FP16/FP32 T5 parameters, an FP16 transformer/VAE, guidance 4.5, 120 text positions, and a second-order midpoint solver with an empty negative prompt. Checkpoint revisions, prompts, and executable reproduction commands are in the diffusion design document linked below.Independent native processes loaded neither Python nor Torch. Isolated compiled update and decoder comparisons also differ when supplied canonical intermediates; PixArt's second update differs with canonical solver history. Native replay fidelity therefore does not establish canonical numerical correctness. Raw decoded-image RMSE for the two SDXL 30-step cases is 0.009159 and 0.018393; for PixArt it is 0.008709 and 0.085828. No performance ratio is claimed.
Focused validation passed 114 Rust tests covering schema, protection, export, conversion, tensor execution, and CLI behavior; 28 SDXL Python checks; 41 PixArt Python checks; and 38 geometry/schedule checks. The PixArt count includes the original 13 comparator tests. The byte-sensitive comparator now passes 15 tests, and rechecking all six retained GPU comparisons preserves the native-pass/canonical-fail result. The lightweight local Python run also reported four dependency-related skips. These are focused checks, not a full repository suite.
GPU validation used the implementation based on
a3e8778. The combined tree with subsequent upstream changes has not been built or GPU-tested. Broader checkpoints, shapes, solvers, tensor parallelism, concurrent serving, and numerical acceptance bounds beyond exact comparison remain unqualified. Open qualification work is recorded indocs/TODO.txt.Reproduction: pinned checkpoints, environment setup, preparation, and native comparison commands. Qualification requirements: diffusion TODOs.