Skip to content

Speed up Compat::Corrected ends-free (fast kernel, exact pruning); fix #41 - #42

Merged
cjfields merged 2 commits into
mainfrom
perf/corrected-endsfree-pruning
Oct 15, 2026
Merged

cjfields merged 2 commits into
mainfrom
perf/corrected-endsfree-pruning

Conversation

@cjfields

@cjfields cjfields commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #40 (part of #31 / #32). Makes Compat::Corrected ends-free with match < 0 nearly as fast as C mode, so dada2-rs (HPCBio/dada2-rs#51) can re-measure it against NW.

Changes

  • Faster extend kernel. The corrected-mode overhead (+26–28% instructions, +26–51% time over C mode) wasn't from the extra steps: those were only +0.8% on long.seq. It came from the _best extend kernel, which called endsfree_left (two Options, merged) on every diagonal. Samply showed extend at +74%. The kernel now walks the offsets like the C-mode kernel and runs the cheap termination_endsfree check first, counting the bases left only at an end.

  • Exact pruning. From diagonal k at score s, any completion totals at least s + M·|k − k_end| (k_end = |T| − |P|). The bases left differ by k − k_end all along a diagonal, and each of them costs at least M. Once an end candidate exists, every component at each score is trimmed to the diagonals that could still beat it (ties kept), and the dropped cells are set to OFFSET_NULL, so later steps can't read them (Wavefront::trim_nulled). This runs only with no heuristic or BandedStatic, because the other heuristics read the wavefronts.

  • Bug fix (FIXED_BUGS 14c). Null steps no longer end a corrected search while a candidate is pending. With BandedStatic, the band (or the pruning) can empty every wavefront before begin-free seeds outside it are reached, and the search then returned a worse end. Example: SW 1 instead of 2 on tests/data/corrected_banded_seeds.seq, band -10,200.

  • Fixes Debug panic (subtract overflow) in backtrace when both sequences are empty #41: a fresh aligner given an empty pattern and empty text panicked in debug builds (usize underflow of the backward CIGAR write offset; release builds wrapped and happened to be right). cigar_buffer_len adds one slot to the C-sized 2·(|P|+|T|) buffer at all three sizing sites (FIXED_BUGS 15). test_empty_sequences_cigar covers edit/affine, high/med/low, end-to-end/ends-free and BiWFA; it fails without the fix.

Results

Instructions retired on long.seq (300 × 2 kb), CIGAR / score-only. I used instruction counts because the machine was loaded for part of this. Wall time is best of 5, interleaved, at load ~3.

Config main C main corr. branch C branch corr.
affine -5,4,0,8, ends-free 0.1 ×4 14.54 / 14.25 G 18.30 / 18.01 G 14.54 / 14.25 G 14.87 / 14.58 G
same, ms 1092 / 715 1451 / 1054 1103 / 717 1181 / 796
ends-free 1 ×4, ms 1713 / 1230 2344 / 1856 1710 / 1237 1854 / 1382
extension, ms 145 / 119 198 / 169 145 / 120 159 / 130
affine2p -1,6,2,4,12,1, ends-free 0.1 ×4, ms 1444 / 1008 1814 / 1372 1451 / 1008 1533 / 1087
ends-free 0.1 ×4, banded-static ±200, ms 904 / 560 1160 / 816 880 / 561 956 / 613
global, ms 128 / 105 127 / 106 128 / 105 129 / 105
  • The corrected-mode overhead over C mode is now +2–2.5% instructions (+7–12% time), down from +26–28% (+26–51%). C mode is unchanged.
  • Pruning on its own is modest: 0.3–2%. On long.seq the first end found is usually near-optimal already. On pairs with random flanks and whole-length free ends, corrected mode is still +15–55% (up to +160% on short pairs) over C mode. In those cases C stops at a wrong, Known issue (C parity): ends-free with whole-length free ends and match < 0 returns the no-overlap alignment #38-style first hit, so the extra search is what makes the result optimal.

Verification

  • Pruned vs unpruned output: byte-identical (both builds include the 14c fix):
    • 360 configs over 506 generated pairs (#37/#38 cases included);
    • 240 configs over long.seq;
    • configs cover linear/affine/affine2p, five ends-free/extension spans, high/med/low, CIGAR and score-only, and no heuristic plus two banded-static bands.
  • New tests:
    • random_long_against_oracle: 300 pairs up to about 200 bp with flanks, checked against the DP oracle in all memory modes. It fails if the pruning band is one diagonal too narrow.
    • banded_static_pending_seeds: fails without the 14c fix.
    • A trim_nulled unit test.
  • cargo test --release and clippy are clean. The new code is rustfmt-formatted.
  • With the Debug panic (subtract overflow) in backtrace when both sequences are empty #41 fix, the full suite also passes in a debug build (on main, three corrected_endsfree tests hit Debug panic (subtract overflow) in backtrace when both sequences are empty #41 in debug). wfa_utest.sh 60/60.

Fixes #41

🤖 Generated with Claude Code

cjfields and others added 2 commits October 14, 2026 21:36
Corrected mode cost +26-28% instructions (+26-51% time) over C mode on
ends-free with match < 0. Profiling showed it was not the extra steps
(+0.8%) but the `_best` extend kernel, which called `endsfree_left` on
every diagonal. It now walks the offsets like the C-mode kernel and
checks `termination_endsfree` first. Overhead on long.seq is now +2-2.5%
instructions (+7-12% time); C mode unchanged.

Also prune, exactly: from diagonal k at score s any completion totals at
least s + M*|k - k_end|, so once there is an end candidate the
wavefronts are trimmed to the diagonals that could still beat it, with
the dropped cells nulled. Output is byte-identical to unpruned (600
configs over generated pairs and long.seq). Only with no heuristic or
BandedStatic, since the other heuristics read the wavefronts.

Fix (FIXED_BUGS 14c): null steps no longer end a corrected search while
a candidate is pending. With BandedStatic the band (or pruning) can
empty every wavefront before begin-free seeds outside it are reached,
returning a worse end.

Tests: random_long_against_oracle (pruning trims most cells; a band one
diagonal narrower fails), banded_static_pending_seeds (fails without the
fix), Wavefront::trim_nulled unit test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CIGAR buffer is 2 * (|P| + |T|) ops, as in C, and the backtraces
write backwards, leaving begin_offset one below the first op. With usize
offsets, `max_ops - 1` underflowed when a fresh aligner got an empty
pattern and text: a panic in debug builds (release wrapped and happened
to return the right result). `cigar_buffer_len` now adds one slot at all
three sizing sites, which also covers a completely full buffer.

Test: aligner::tests::test_empty_sequences_cigar (edit and affine,
high/med/low, end-to-end and ends-free, BiWFA); fails without the fix.
The full test suite now passes in debug as well as release.

Fixes #41

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cjfields cjfields changed the title Speed up Compat::Corrected ends-free: fast extend kernel, exact pruning Speed up Compat::Corrected ends-free (fast kernel, exact pruning); fix #41 Oct 15, 2026
@cjfields
cjfields merged commit 001a820 into main Oct 15, 2026
15 checks passed
@cjfields
cjfields deleted the perf/corrected-endsfree-pruning branch October 17, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Debug panic (subtract overflow) in backtrace when both sequences are empty

1 participant