Repository navigation
bench: reproducible Rust-vs-C sweep (sweep.py) - #44
Merged
Merged
Conversation
bench/sweep.py builds Rust's align_benchmark and C's at the pinned commit (generic as the harness builds it, and with -march=native), generates deterministic datasets, and runs every metric x memory mode, ends-free/ extension (incl. match < 0) and heuristic configuration with each tool. It records best-of-N alignment time, max RSS (wait4), instructions (perf stat on Linux, time -l on macOS) and whether the output matches generic C, and writes results.tsv and summary.md under target/bench-sweep/<host>/. Rust and C get their own argument forms (clap needs -g=-1,..., C's getopt the separate value). build_c.sh: C_NATIVE=1 keeps -march=native and builds into c-<commit>-native/. For #31 (x86/AVX2 measurements). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For compute nodes without network: --offline runs cargo with --offline (after `cargo fetch` elsewhere) and requires a local ./WFA2-lib checkout containing the pinned commit (or WFA2_LIB_REPO), instead of cloning. Rust (target-cpu=native) and the native C build target the CPU they are built on, and cargo doesn't track the CPU, so a binary built on a login node would be silently reused on the node. Both now build into per-host directories (target/bench-sweep/build-<host>, c-<commit>-native-<host>). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
bench/sweep.py, a reproducible Rust-vs-C benchmark sweep, for #31 and in particular its x86/AVX2 measurements.Builds Rust's
align_benchmarkand C's at the pinned commit, both generic (as the differential harness builds it) and with-march=native. On x86 the native build runs C's AVX2 and AVX-512 kernels.build_c.shgainsC_NATIVE=1for this; the harness is unchanged.Data: deterministic datasets with a fixed seed, so every machine aligns the same pairs. C's
generate_datasetseeds from the clock. Sets:Configs: every metric × memory mode, ends-free/extension (including
match < 0) and all five heuristics, about 300 in total.Records:
wait4;perf staton Linux when permitted,time -lon macOS;Writes
target/bench-sweep/<host>/results.tsvandsummary.md, with machine info, compiler versions and load.Arguments: Rust and C get their own argument forms. clap needs
-g=-1,4,6,2, while C's getopt reads the=into a short option's value. The ad hoc sweep in Performance: high-memory mode slower and larger than C on long pairs; port-quirk optimizations #31 hit exactly this.Offline nodes:
--offlineruns cargo with--offline(runcargo fetchwith network first) and requires a local./WFA2-libcheckout that contains the pinned commit. The native builds (Rusttarget-cpu=native, C-march=native) go into per-host directories. Cargo doesn't track the CPU, so without this a binary built on a login node would be silently reused on a compute node.Without
perf, instruction counts are left blank; times, memory and output checks still work.Tested on macOS (M2 Pro) with
--quick, including--offlineand--no-build. (M2 Pro) The results agree with the #31 sweep. The Linux paths (perf,taskset,/proc/cpuinfo) are untested here; they fall back gracefully, and the summary says which instruction source was used.🤖 Generated with Claude Code