ci(fuzz): stop the scheduler_solver nightly OOMing on ASan bookkeeping (#361)

[avrabe]

Closes #361.

What

fuzz_scheduler_solver gets ASAN_OPTIONS=malloc_context_size=10:quarantine_size_mb=64
and a per-leg -max_len=128. No Rust changes.

The evidence, not the hypothesis

It has ended in libFuzzer: out-of-memory on every nightly run the API still
returns — 97 of 97, 2026-04-25 → 2026-07-30. There is no passing run in the
window at all.

It is not a leak, and ASan's own accounting at the moment of death rules the
code under test out (run 30516991027):

used: 2056Mb; limit: 2048Mb
Live Heap:    27.7 MB in     8,340 chunks
quarantined: 155.7 MB in 2,104,213 chunks
total chunks:              4,345,508

Live plus quarantined contents are 183 MB of 2056 MB. The other ~1.87 GB is
ASan's per-chunk bookkeeping across 4.3M chunks — a product of two terms, so both
get addressed:

knob	default	here	term it bounds
quarantine_size_mb	256	64	chunk count — 2.1M chunks retained only to catch use-after-free
malloc_context_size	30	10	per-chunk cost — each stores alloc/free traces of that depth

Use-after-free is not the bug class this harness hunts; its contract is that
solve_milp never panics. Ten frames still names the allocation site, so crash
triage survives — deliberately not cut to 2, which saves little and turns a
report into an unactionable address.

-rss_limit_mb=4096 is headroom, not the mechanism. If it turns out to be
load-bearing, the diagnosis above was wrong.

Why -max_len is per-leg — the part worth reviewing

The obvious patch is to append -max_len=128 to the shared run line. That would
have been a silent regression: the line is shared across a 3-target matrix, and
across all 97 runs fuzz_aadl_parse and fuzz_codegen_roundtrip concluded
success every time. Only fuzz_scheduler_solver fails.

Those two parse arbitrarily long AADL/WIT source text, so a shared cap shrinks two
healthy fuzzers' reachable input space to fix a third's bug. Entropy budget is a
property of the harness, so the flag moved into matrix.include.

For this leg the cap is lossless rather than a coverage trade. The harness opens:

let n_tasks = input.tasks.len().min(8);
let n_procs = input.processors.len().min(4);

while Arbitrary grows those Vecs to consume whatever buffer it is handed. Every
byte past the ~120 the capped domain can encode yields a Task that is allocated
and then discarded unread. Capping removes allocation churn, not reachable states.

The saved crash artifact is NOT a reproducer

fuzz/artifacts/fuzz_scheduler_solver/oom-c5aafda340317c72f5ba9eb2431f0c189d159c9d
is 267 bytes and is simply whichever input happened to be executing when the
process crossed the RSS limit. Replaying it proves nothing. Recording this so
nobody loses a day treating it as the trigger.

Verification

This is the honest part: I cannot verify this locally. No nightly Rust or
cargo-fuzz in this environment, and the failure needs a ~10-minute ASan fuzzing
run to reproduce. The numbers above are read from the CI log, and the -max_len
argument is read off the harness source; the fix itself is so-far unproven.

The oracle is the nightly, and it is two-sided:

1. fuzz_scheduler_solver must flip failure → success, and
2. fuzz_aadl_parse and fuzz_codegen_roundtrip must stay success.

A green run where a parser leg regressed is not a fix. YAML was parsed locally to
confirm the matrix expands to the three intended (target, extra_args) pairs.

I will dispatch the workflow on this branch and report the result before merging,
rather than merging on the strength of the reasoning.

No rivet artifact, deliberately

Matching #353 (the CI path-filter PR), which was also workflow-only and carried
none. There are no REQ-CI-* artifacts in this repo — rivet tracks what spar shall
do, not CI plumbing. Flagging the omission explicitly so it reads as a choice
rather than an oversight.

Meta

Neither this workflow nor trace-fixtures (#362, 60/60 red) is a required
context, which is the whole reason a job that has never once passed sat red for
three months. A gate nobody reads is not a gate. Making them required is a separate
decision — it needs them green first, which is what these two PRs are for.

🤖 Generated with Claude Code

[github-actions]

Rivet verification gate

✅ 20/20 passed

	count
Passed	20
Failed	0
Skipped (no steps)	0

Filter: (and (= type "feature") (or (has-tag "v093") (has-tag "v0100")))

Failed artifacts

(none)

_{Updated automatically by tools/post_verification_comment.py. Source of truth: artifacts/verification.yaml.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[avrabe]

Oracle satisfied — both sides, full 1h budget

Run 30560598236 — conclusion: success.

target	before	this run
fuzz_scheduler_solver	failure (OOM at 2058Mb/2048Mb, 97/97 red since 2026-04-25)	success
fuzz_aadl_parse	success	success
fuzz_codegen_roundtrip	success	success

Both sides of the oracle stated in this PR hold: the OOM leg flipped, and neither parser leg regressed. All three consumed the full 1h budget rather than exiting early, so the green is not a fail-fast artifact.

Rebased dc54b6f → 5199cec onto main (protection is strict: true and #367 had landed). The evidence carries across the rebase: git diff dc54b6f HEAD -- .github/workflows/fuzz-nightly.yml is empty, and the only commit main gained was artifacts-only, so nothing under fuzz/ or the workspace changed between the tree that produced the green run and the tree being merged.

Grounding note: this is a dispatch result. The scheduled nightly on main remains the real gate and will not have run until this lands.