Skip to content

Add new application: sbd - #73

Draft
william-dawson wants to merge 7 commits into
RIKEN-RCCS:developfrom
william-dawson:add-sbd-benchkit
Draft

Add new application: sbd#73
william-dawson wants to merge 7 commits into
RIKEN-RCCS:developfrom
william-dawson:add-sbd-benchkit

Conversation

@william-dawson

@william-dawson william-dawson commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Depends on #74.

The branch intentionally contains #74 so the complete SBD pipeline can be
tested before the shared site fix merges. The SBD-only changes can be reviewed
at:
william-dawson/benchkit@fix-rikyu-cloud-site...add-sbd-benchkit

Summary

Add SBD selected-basis diagonalization as a BenchKit application for Rikyu
B200 GPUs and R-CCS Cloud DGX Spark.

Changes

  • Add programs/sbd/build.sh with NVHPC 26.3 Thrust builds:
    • Rikyu: cc100, rank distribution, block distribution, index reordering,
      safe MPI allreduce, and NCCL.
    • DGX Spark: cc120, rank distribution, block distribution, index
      reordering, and safe MPI allreduce.
  • Add programs/sbd/run.sh with one MPI rank per GPU and explicit local GPU
    binding.
  • Use SBD's internal Davidson timer as the FOM instead of scheduler or shell
    wall time.
  • Require SBD completion markers and validate the converged energy before
    emitting a result.
  • Add Rikyu strong-scaling rows at 4, 8, and 16 B200 GPUs.
  • Add a one-GPU DGX Spark row using the memory-safe H2O 1em5 input.
  • Include the upstream H2O FCIDUMP/determinant inputs with revision,
    checksum, and Apache-2.0 provenance.

Validation

DGX Spark

  • NVHPC 26.3, one GB10 GPU.
  • Exact energy: -76.24373504205295 Ha.
  • Internal Davidson FOM: 228.928853 s.
  • Complete scheduler elapsed time: 4 minutes 9 seconds.
  • The larger 1em7 input was rejected for this target after reaching about
    98 GB before Davidson; 1em5 completes with adequate memory headroom.

Rikyu

  • NVHPC 26.3, project rkp00012, one MPI rank per B200.
  • All full scaling runs converged to -76.243776776861 Ha.
GPUs Nodes Davidson FOM (s) 4-GPU-relative speedup
4 1 565.510 1.00x
8 2 290.222 1.95x
16 4 152.144 3.72x

A fresh validation of this BenchKit branch:

  • generated the 4/8/16-GPU scheduler matrix with partition gpu and account
    rkp00012;
  • rebuilt upstream SBD commit
    9481f290c2f49d4f8e5df9b0c9c87ea0f7937c2c through build.sh;
  • confirmed the effective cc100 rank-distributed/index-reordered/NCCL
    CMake configuration;
  • submitted the two-node/eight-GPU row through BenchKit's own
    scripts/test_submit.sh;
  • completed at -76.2437767768609 Ha with a 290.281226 s internal Davidson
    FOM; and
  • produced valid normalized result JSON with source provenance and the mult
    timing section.

Additional checks

  • Shell syntax checks pass for build.sh and run.sh.
  • git diff --check passes.
  • All included input SHA-256 checksums match their recorded upstream values.

@william-dawson
william-dawson force-pushed the add-sbd-benchkit branch 2 times, most recently from a53bf73 to 86370ce Compare July 31, 2026 06:24
@william-dawson william-dawson changed the title Add SBD selected-basis diagonalization benchmark Add new application: sbd Jul 31, 2026
@william-dawson
william-dawson marked this pull request as ready for review July 31, 2026 06:26
@william-dawson
william-dawson marked this pull request as draft July 31, 2026 06:59
Signed-off-by: William Dawson <william.dawson@riken.jp>
Signed-off-by: William Dawson <william.dawson@riken.jp>
Signed-off-by: William Dawson <william.dawson@riken.jp>
Signed-off-by: William Dawson <william.dawson@riken.jp>
Signed-off-by: William Dawson <william.dawson@riken.jp>
Signed-off-by: William Dawson <william.dawson@riken.jp>
list.csv set nthreads=1 for all three RIKYU rows, which queue.csv wires
straight into --cpus-per-task and run.sh's OMP_NUM_THREADS. That starves
RemakeHelpers/TaskCostSize, a genuine host-side OpenMP pass that builds
the rank-distribution cost estimate before Davidson starts -- confirmed
via gdb backtrace on 2026-08-06 while investigating a run that looked
hung for 18+ minutes on the 1em7 input at 4 GPUs. Raising threads to 32
(matching Rikyu's per-GPU CPU cap on shared nodes) fixed it; the same
degenerate layout completes in about two minutes on a comparable input.

module purge already precedes module load in both build.sh and run.sh
for RIKYU, so no fix was needed there -- checked before touching it.

Only the 8-GPU row in README.md's validated table was independently
re-run before now (per the paragraph above it); flag the 4- and 16-GPU
rows as unverified under the old nthreads=1 configuration rather than
silently changing the numbers.

Signed-off-by: William Dawson <william.dawson@riken.jp>
@yoshifuminakamura

Copy link
Copy Markdown
Collaborator

Could you please remove the large SBD input data files from this PR?

Keeping large application-specific input files in the BenchKit repository increases the clone/fetch cost for every pipeline, including runs for unrelated applications. I think the better direction is to keep only the input metadata, provenance, URLs, and checksums in the repository, and have the SBD build/run path fetch the required data when SBD is executed.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants