[VPTO] Represent MAD CTRL requirements as ctrl_state_guard and materialize minimal hardware accesses - #1504
Draft
jimmychou0 wants to merge 1 commit into
Draft
Conversation
jimmychou0
force-pushed
the
zjm/ctrl_opt_issue1279
branch
4 times, most recently
from
September 14, 2026 06:27
ea4e3f8 to
70a2459
Compare
…mal accesses Issue hw-native-sys#1279: every semantic MAD expansion emitted a fixed get_ctrl -> 4x sbitset -> set_ctrl -> mad_raw -> set_ctrl restore chain, adding scalar-issue pressure that cost 1-8% on GEMM kernels. CSE cannot remove the sequence because the ops carry ordered implicit-state effects. Represent the temporary requirement structurally instead: - Add StateAccessOpInterface (Query/Write/Consume/Clobber) and StateGuardOpInterface with StateResource/StateAccessKind enums; get_ctrl/set_ctrl declare Query/Write and the four raw MAD ops declare Consume. - Add internal ctrl_state_guard{controlled_bits, required_bits} wrapping the raw MAD (verifier: R subset of C, C nonzero, exactly one CTRL Consume implementing MadRawOpInterface, no nesting, func-local). VPTOExpandWrapperOps now computes (C,R) statically (HiF8 bit45, TF32 bit46/47, explicit sat bit48, n_dir bit51) and emits the guard instead of the hardware access sequence. - Add VPTOOptimizeCtrlStatePass after the scheduler: bottom-up scf.for summaries (CTRL cleanliness, unique requirement, statically proven at-least-once trip count) hoist one shared configuration to the outermost legal loop entry, unwrap covered guards, and restore once after the loop. Remaining guards use a per-block forward scan sharing one logical entry read, reusing an installed active state for an equal requirement, and deferring restores to the first real observation point (explicit get/set_ctrl, outside Consume, unknown op, block end). - Emission-stage VPTO validation now rejects leftover ctrl_state_guard. Validation (144, LLVM 19.1.7 assert build): full lit suite 1876 tests, 1875 passed, 0 failed; new tests cover shared configuration, explicit observation points, requirement switching, loop hoisting, nested M/K loops, dynamic zero-trip no-hoist, and in-loop boundary no-hoist. Validation (238, Ascend950PR, TileLang 8192 bf16 manual GEMM): numerics identical to baseline (max_diff 6.409e-04); emitted VPTO IR drops from 4x get_ctrl + 8x set_ctrl + 16x sbitset inside the loop nest to a single configuration hoisted outside three nesting levels with zero CTRL accesses in any loop body.
jimmychou0
force-pushed
the
zjm/ctrl_opt_issue1279
branch
from
September 14, 2026 06:57
70a2459 to
6dcd2d5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the ctrl_state_guard design for issue #1279: MAD semantic-to-raw lowering no longer materializes the fixed 7-instruction CTRL round-trip per MAD (
get_ctrl -> 4x sbitset -> set_ctrl -> mad_raw -> set_ctrl). Instead the temporary CTRL requirement is represented structurally and a newVPTOOptimizeCtrlStatePassmaterializes the minimal set of hardware CTRL accesses.Changes
StateAccessOpInterfacewith Query/Write/Consume/Clobber,StateGuardOpInterface,StateResource/StateAccessKindenums):get_ctrl=Query,set_ctrl=Write, the four raw MAD ops=Consume.pto.ctrl_state_guard{controlled_bits, required_bits}wrapping the raw MAD consumer; verifier enforcesR ⊆ C,C ≠ 0, exactly one CTRL-Consume (must implementMadRawOpInterface), no nesting, func-local. Eliminated before emission validation.VPTOExpandWrapperOpscomputes(C,R)statically (HiF8 bit45, TF32 bit46/47, explicit sat bit48, n_dir bit51; unspecified sat inherits the entry state) and emits the guard instead of the hardware sequence.VPTOOptimizeCtrlStatePass(after the scheduler, before emission validation):scf.forsummaries: CTRL cleanliness, unique requirement, statically proven ≥1 trip count;get_ctrl/set_ctrl, a Consume outside a guard, an unknown op, or the block end);scf.ifregions materialize branch-locally.ctrl_state_guard.Validation
test/lit/vpto/ctrl/: shared configuration, explicit observation point forcing a restore, requirement switching without intermediate restores, loop hoisting, nested M/K loops (single config at the outermost entry), dynamic zero-trip no-hoist, in-loop boundary no-hoist.6.409e-04PASS on both); emitted VPTO IR for the real kernel drops from 4× get_ctrl + 8× set_ctrl + 16× sbitset inside the innermost loop (28 scalar CTRL ops per K iteration) to 1× get_ctrl + 2× set_ctrl + 4× sbitset hoisted outside three nesting levels with zero CTRL accesses in any loop body.Closes #1279 (IR-level scalar elimination; issue-case performance regression numbers to be quantified on the 12 reported GEMM shapes as follow-up — the validated manual GEMM is deeply software-pipelined and shows no end-to-end delta, as expected for that kernel class).
Design doc
https://github.com/KurrinQu/PTOAS/blob/docs/vpto-ctrl-state-optimization-design/docs/designs/vpto-ctrl-state-optimization-design.md (implementation phase 1 + phase 2 loop hoisting; scf.if meet-based sharing and cross-block propagation left as future refinement per §6.8 conservative-exit rules)