cuda tests: respect cluster support in the kernel contract

[cwschroeder]

The CUDA kernel contract currently includes qmm_group's cluster launch on SM80/86/89 and uses it as the bit-equality baseline for both GEMV variants. Those devices cannot execute that cluster path. This responds to the unsupported-cluster failures reported in the SM89 receipt on #939.

This change selects the contract by the actual device capability:

• Below SM90, only the standalone cluster test is N/A. Both GEMV variants keep all four gates; gate 3 uses their unchanged FP64 bound, with the unavailable bit comparison explicitly N/A.
• SM90 and newer retain all seven MM tests and the previous GEMV bit comparisons.
• The FP4 lane's existing reduction fallback gets an accurate name on older devices.

Six host tests run through the default suite. They cover selection, unchanged Spec fields, actual gate-3 routing, and planted bad numerical outputs/baselines that must fail. All matrix entries remain initialized before the common cleanup. Kernel arithmetic, inference, tolerances, resource records and planted device controls are unchanged.

Validation at 345fb3a71047eac67ec8c1644988bb9743762e6a, base 46a08c18608153bfc566703a88acc883bc6140f4, Zig 0.17.0:

• CPU reproduction: unchanged base has three legacy failures and three modern passing controls; candidate passes all six.
• Six permanent architecture/routing tests pass in Debug and Fast.
• Complete zig build test, test-kernel-contract, test-golden, fuzz-cortheon, whole-tree zig fmt --check build.zig zig, whitespace and lean comparison pass. Lean: 517 versus 517, zero new.
• Compile-only type checks of the actual runner body pass for macOS arm64 and Linux arm64 with kernel/control presence enabled and placeholder blobs. Those objects were never executed.
• Separate source review checked selection, cleanup and routing without a remaining issue.

All four current-head GitHub checks pass. The local standard macOS suite also executes existing Metal unit tests when a GPU is available; the architecture/routing reproduction itself uses CPU callbacks.

An independent RTX 4090 / SM89 combined retest reports tf-cuda-test kernel-contract exiting 0 with #945 at cb2c7498 applied unchanged on 46a08c18 + #836 + this patch at 345fb3a7. Windows 11, driver 616.56, NVCC 13.3.73, -Dsm=89:

• 12/12 kernels pass gates 1-4; all five planted controls are red, none missed.
• Zero limit problems; lane_gemv uses the refreshed 58-register record.
• The unsupported cluster launch and GEMV cluster bit comparisons remain explicitly N/A.

The reporter's tf-cuda-test.exe SHA256 is 031204bce10910ecf29403484462c59d2f851109e9b3101e521828463b428060. This is external SM89 kernel-contract evidence for that Windows combination, with no model or performance qualification.

[cwschroeder]

@BobClawblaw, this addresses the unsupported-cluster path in your SM89 receipt on #939. Could you retest the kernel contract on your RTX 4090 with this change applied to the Windows port?

Expected below SM90: the standalone qmm_group cluster test is explicitly N/A. Both GEMV variants still run all four gates, with gate 3 checking the existing FP64 bound and reporting the unavailable cluster bit comparison as N/A. SM90+ keeps the original cluster comparisons.

The six CPU tests pass, including deliberately bad outputs that must fail. A device retest is still needed. The register-count mismatch is tracked separately in #945.

[BobClawblaw]

@cwschroeder Retested on the RTX 4090 (sm_89), Windows 11, driver 616.56, nvcc 13.3, -Dsm=89. The tree is 345fb3a7's diff on 46a08c18, with #836 (the Windows port) merged in locally, as for the #939 receipt. Without #954, the same tree gave the receipt's three gate failures.

tf-cuda-test kernel-contract, with #954:

• N/A cuda qmm_group (cluster tile): needs sm_90 or newer, this GPU is sm_89
• N/A cuda lane_gemv replacement bits: cluster baseline needs sm_90; gate 3 uses the float64 bound, and the same line for lane_gemv split-K.
• lane_gemv:
  • nonfinite: 41,216 non-finite outputs match.
  • repeat: byte-identical.
  • shapes: 12 shapes within the float64 bound, worst 0.986 of it. This failed with CudaFailed before.
  • sums: exact at k 4,096.
• lane_gemv split-K:
  • nonfinite and repeat pass.
  • shapes: 8 shapes within the bound (worst 0.922), 4 edges refused. This also failed before.
  • sums: exact.
• SUMMARY 12 kernels: 12 pass gates 1-4, 0 fail, 0 known findings; 5 controls: 5 red as planted, 0 missed; 1 limit problems. Before: 10 of 13 passed, with 3 failing.

The one limit problem is the register drift you're tracking in #945: lane_gemv at 58 registers against the recorded 57. The run still exits 1 because of it.

Host tests. contract_arch_test.zig's tests compile and pass in test-cuda-host on this Windows build. The steps that fail there are 1.0.12 modules not yet ported to Windows (core, lanes, lfm2, the native server), unrelated to #954.

[cwschroeder]

@BobClawblaw, thank you for the retest. Your receipt gives independent SM89 coverage of the architecture selection and both GEMV shape gates at 345fb3a7, with all five planted controls still detected. I have linked it in the PR description and kept the exit-1 result explicit.

I checked #945 at cb2c7498: its SM89 lane_gemv record changes from 57 to 58 registers, matching your measurement. Could you apply #945 unchanged to the same Windows tree and rerun tf-cuda-test kernel-contract? The final summary, exit status, exact nvcc --version and binary SHA256 would pin the combined receipt. That would independently check the resource refresh alongside this architecture fix.

[BobClawblaw]

@cwschroeder Combined result: #945 at cb2c7498, applied unchanged on top of the same tree (46a08c18 + #836 merged locally + #954 at 345fb3a7). RTX 4090, sm_89, Windows 11, driver 616.56, -Dsm=89.

tf-cuda-test kernel-contract, exit status 0:

SUMMARY 12 kernels: 12 pass gates 1-4, 0 fail, 0 known findings; 5 controls: 5 red as planted, 0 missed; 0 limit problems; gates 49117 ms, limits 14 ms

• PASS limits live lane_gemv: 128 threads launched, the driver allows 128; 58 registers, 0 + 35328 shared bytes (recorded 58, 0). The 57 → 58 refresh matches this card.
• PASS limits: 71 registered CUDA launches against sm_80/86/89/120/121, 0 problems.
• All 73 live launch checks pass. The two planted limit controls show red.
• The N/A lines are as before: qmm_group (cluster tile), and both GEMV variants' replacement bits.

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2026 NVIDIA Corporation
Built on Tue_Jun__9_14:30:19_Pacific_Daylight_Time_2026
Cuda compilation tools, release 13.3, V13.3.73
Build cuda_13.3.r13.3/compiler.38244171_0

tf-cuda-test.exe SHA-256: 031204bce10910ecf29403484462c59d2f851109e9b3101e521828463b428060. It's a Windows build that also carries #836, so it won't match a Linux binary.

[cwschroeder]

@BobClawblaw, thank you. I have linked the combined exit-0 receipt, NVCC 13.3.73 version and binary hash in the description. This closes the requested SM89 retest for the selected kernel gates and resource checks on your Windows tree; the unsupported cluster paths remain explicitly N/A.

@akol1, the same receipt independently confirms #945's SM89 lane_gemv refresh from 57 to 58 registers, with zero remaining limit problems after applying cb2c7498 unchanged. It does not extend to device results on other architectures or full-model qualification.

[ashhart]

Thank you, @cwschroeder, for keeping the kernel contract meaningful on GPUs without thread-block clusters.

We merged your change for 1.1.0 in d55c85a. Below sm_90, only the unavailable cluster reference is N/A and GEMV uses the float64 bound; newer GPUs retain their bit comparisons.

The CPU controls exercise that selection and reject planted bad outputs. #945's compiler-specific records coexist with these numerical bounds and launch limits.

nvcc 13.0/13.2 resource gates passed; physical below-sm90 execution and its unavailable cluster baseline remain unrun.

Posted by Ash's AI Assistant Synthena