[Tutorial] Add Hopper Tutorials + Fixed Benchmarking - #161
Open
WilliamZhang20 wants to merge 1 commit into
Open
Conversation
Adds the V0-V6 Hopper matmul tutorial (docs/source/tutorials/matmul-hopper/)
and corrects the benchmark methodology the numbers rest on.
benchmark.py, aligned with examples/blackwell_matmul/benchmark.py:
- cuBLAS is timed last, under the same 3 s cooldown and iteration count as
every version. Previously it was timed first, on the coldest GPU of the
run, which flattered whichever entry followed it. That bias is why V0 was
recorded at 3.35 ms: its best schedule measures 3.51 ms in isolation.
- Both operands are scaled by k**0.25 so C has unit variance, replacing the
asymmetric one-operand scaling. atol and rtol are now both meaningful.
- assert_close argument order corrected to (actual, expected).
- repeat stays at 30 rather than Blackwell's 100, deliberately. An 8192^3
fp16 GEMM is ~1.5 ms on an H100, so 100 iterations is ~150 ms of
power-capped work and the board throttles inside the timed window: three
fresh processes give v6/cuBLAS = 108.1/106.5/107.8 % at repeat=30 but
99.0/98.9/110.4 % at repeat=100. Added --repeat to compare.
- New "accumulate" column, because v6 accumulates in fp16 while every other
version and cuBLAS accumulate in fp32 -- which is what its looser atol was
silently encoding.
Numbers re-measured on an idle H100 SXM, median of three fresh processes, and
cross-checked on a second GPU. All prose, tables, and plot data updated:
v0 3.60 ms / 305 TF v4 1.71 ms / 642 TF
v1 2.04 ms / 540 TF v5 1.62 ms / 678 TF
v2 2.12 ms / 518 TF v6 1.37 ms / 800 TF (fp16 accumulate)
v3 1.91 ms / 575 TF cuBLAS 1.48 ms / 742 TF
v6 leads cuBLAS by 6-8% on wall clock and 2% under NCU replay, now stated with
the fp16-vs-fp32 accumulation caveat in the summary table, the V6 warning box,
and the closing caution.
Tutorial corrections found while verifying against the source and the hardware:
- v0: autotuner picks block_k=64, not 32 (confirmed fastest of the five
schedules that compile).
- v3: the TMA calls are inside single_thread, not issued at warp scope --
that is V4's form.
- v4: tile rasterization is used by V4 itself, not "turned on by V5".
- v5: a 2-stage ring buffer does not deadlock. It runs correctly at 2.24 ms.
The 4th stage is a tie with 3 (1.62 vs 1.60 ms), not a requirement.
- v1/v2/v3: plot captions still claimed NCU-derived TFLOPS and a
cuBLAS-estimated peak; both plots use event timing and the published peak.
- v6: 4x128x128 fp32 accumulator registers is the whole 64K SM register
file, so fp32 at 256x256 does not fit at all.
.gitignore: ignore .venv.
Docs build clean with -W. Cold-cache tests/examples/test_examples.py -k
hopper_matmul: 7 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
WilliamZhang20
force-pushed
the
hopper-tutorials
branch
from
August 15, 2026 02:08
8e006d3 to
b33a69e
Compare
Member
|
/ok to test b33a69e |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.