Skip to content

bench: reproducible Rust-vs-C sweep (sweep.py) - #44

Merged
cjfields merged 2 commits into
mainfrom
bench/sweep
Oct 15, 2026
Merged

cjfields merged 2 commits into
mainfrom
bench/sweep

Conversation

@cjfields

@cjfields cjfields commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Adds bench/sweep.py, a reproducible Rust-vs-C benchmark sweep, for #31 and in particular its x86/AVX2 measurements.

  • Builds Rust's align_benchmark and C's at the pinned commit, both generic (as the differential harness builds it) and with -march=native. On x86 the native build runs C's AVX2 and AVX-512 kernels. build_c.sh gains C_NATIVE=1 for this; the harness is unchanged.

  • Data: deterministic datasets with a fixed seed, so every machine aligns the same pairs. C's generate_dataset seeds from the clock. Sets:

    • 150 bp at 2%;
    • 1 kb at 5% and 20%;
    • 1 kb with long indels;
    • 10 kb at 5%;
    • 2 kb with random flanks.
  • Configs: every metric × memory mode, ends-free/extension (including match < 0) and all five heuristics, about 300 in total.

  • Records:

    • best-of-N alignment time;
    • max RSS, via wait4;
    • instructions: perf stat on Linux when permitted, time -l on macOS;
    • whether Rust's output matches generic C's.
  • Writes target/bench-sweep/<host>/results.tsv and summary.md, with machine info, compiler versions and load.

  • Arguments: Rust and C get their own argument forms. clap needs -g=-1,4,6,2, while C's getopt reads the = into a short option's value. The ad hoc sweep in Performance: high-memory mode slower and larger than C on long pairs; port-quirk optimizations #31 hit exactly this.

  • Offline nodes: --offline runs cargo with --offline (run cargo fetch with network first) and requires a local ./WFA2-lib checkout that contains the pinned commit. The native builds (Rust target-cpu=native, C -march=native) go into per-host directories. Cargo doesn't track the CPU, so without this a binary built on a login node would be silently reused on a compute node.

  • Without perf, instruction counts are left blank; times, memory and output checks still work.

Tested on macOS (M2 Pro) with --quick, including --offline and --no-build. (M2 Pro) The results agree with the #31 sweep. The Linux paths (perf, taskset, /proc/cpuinfo) are untested here; they fall back gracefully, and the summary says which instruction source was used.

🤖 Generated with Claude Code

cjfields and others added 2 commits October 15, 2026 08:22
bench/sweep.py builds Rust's align_benchmark and C's at the pinned commit
(generic as the harness builds it, and with -march=native), generates
deterministic datasets, and runs every metric x memory mode, ends-free/
extension (incl. match < 0) and heuristic configuration with each tool.
It records best-of-N alignment time, max RSS (wait4), instructions
(perf stat on Linux, time -l on macOS) and whether the output matches
generic C, and writes results.tsv and summary.md under
target/bench-sweep/<host>/. Rust and C get their own argument forms
(clap needs -g=-1,..., C's getopt the separate value).

build_c.sh: C_NATIVE=1 keeps -march=native and builds into
c-<commit>-native/.

For #31 (x86/AVX2 measurements).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For compute nodes without network: --offline runs cargo with --offline
(after `cargo fetch` elsewhere) and requires a local ./WFA2-lib checkout
containing the pinned commit (or WFA2_LIB_REPO), instead of cloning.

Rust (target-cpu=native) and the native C build target the CPU they are
built on, and cargo doesn't track the CPU, so a binary built on a login
node would be silently reused on the node. Both now build into per-host
directories (target/bench-sweep/build-<host>, c-<commit>-native-<host>).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cjfields
cjfields merged commit 99943f4 into main Oct 15, 2026
17 checks passed
@cjfields
cjfields deleted the bench/sweep branch October 17, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant