From f4773400355b91db76801fab14b94d2fe936865d Mon Sep 17 00:00:00 2001 From: Tim Date: Wed, 7 Oct 2026 20:37:34 -0700 Subject: [PATCH 1/4] docs(plans): msvc-funclet-asm-goto -- the reproduction, with the lock update it needs A run that skipped 'cargo update' compiled the locked 2.2.0 and tested nothing. The steps are re-run from a clean rusty_sloth against the published 2.2.4 (it fails as reported), with how to check the version before building and how to point the consumer at a rusty_alloc tree. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/plans/msvc-funclet-asm-goto.md | 33 +++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/plans/msvc-funclet-asm-goto.md b/docs/plans/msvc-funclet-asm-goto.md index 944e0bb..f8fe410 100644 --- a/docs/plans/msvc-funclet-asm-goto.md +++ b/docs/plans/msvc-funclet-asm-goto.md @@ -30,6 +30,39 @@ error: could not compile `rusty_sloth-cli` (bin "sloth") The consumer's release profile is `opt-level = 3`, `lto = "thin"`, `codegen-units = 1`, and the default `panic = "unwind"`. 2.2.0 builds in the same profile. +### Reproduce (re-run 2026-10-07 from a clean rusty_sloth `main`, no patch) + +The consumer's `Cargo.lock` pins **2.2.0**. Cargo keeps a locked version until it is told to +move. Without step 2 the build compiles 2.2.0 from crates.io and tests nothing; a `[patch]` +entry is likewise left unused (cargo only warns "Patch … was not used in the crate graph"). + +```sh +cd rusty_sloth # F:/coding/rusty_sloth +cargo add rusty_alloc-api@2.2.4 -p rusty_sloth-alloc +cargo update -p rusty_alloc-api -p rusty_alloc # REQUIRED: moves the lock off 2.2.0 +cargo tree -i rusty_alloc-api -e normal # must print "rusty_alloc-api v2.2.4" +cargo build --release -p rusty_sloth-cli --features cuda +``` + +**Expected:** `Compiling rusty_alloc v2.2.4`, then four "Bogus funclet pad use" and `rustc-LLVM +ERROR: Broken module found`. Reproduced with the published 2.2.4 (`source = +"registry+…crates.io-index"` in the lock). The LTO link of the `sloth` binary is where it fails; +building only the library crates passes. + +**To test a rusty_alloc tree instead** (the fix, or `main`), put the two entries in rusty_sloth's +**existing** `[patch.crates-io]` table. It already holds `serde_json` and `tokenizers`, and a +second `[patch.crates-io]` header is a duplicate-key error: + +```toml +rusty_alloc = { path = "/crates/rusty_alloc" } +rusty_alloc-api = { path = "/crates/rusty_alloc_api" } +``` + +Then the same `cargo add` (the requirement must admit the tree's version), `cargo update -p +rusty_alloc-api -p rusty_alloc`, and check that the lock's two entries have **no `source =` +line** (a path dependency) before building. Restore rusty_sloth afterwards (`git checkout +Cargo.toml Cargo.lock crates/rusty_sloth-alloc/Cargo.toml`). + ## 2. Mechanism - `free_inline` (`crates/rusty_alloc/src/alloc.rs`, `#[inline(always)]`) has the free fast path From 54eb6af9a7d854a06a48b6b7ed4efc6e988141b4 Mon Sep 17 00:00:00 2001 From: Tim Date: Wed, 7 Oct 2026 20:42:38 -0700 Subject: [PATCH 2/4] fix(alloc): keep free_inline's asm goto out of MSVC cleanup funclets 2.2.1-2.2.4 failed to build in an MSVC consumer that unwinds, with LTO (rusty_sloth: thin LTO, one codegen unit, panic = unwind). free_inline's fast path is an asm! goto (LLVM callbr); since 2.2.1 GlobalAlloc::dealloc inlines it into every Rust drop, including the drops a panic runs, which on MSVC are EH cleanup funclets. LLVM cannot place a callbr in a funclet: "Bogus funclet pad use ... Broken module found". On all(target_env = "msvc", panic = "unwind") free_inline now takes the plain-Rust decrement aarch64 and Miri already use. The gate is in free_inline itself, so it covers every caller (rusty_alloc::alloc::free and rusty_alloc_api::Heap::dealloc as well as dealloc), unlike outlining dealloc alone (7037d8d, not merged). MSVC builds with panic = abort and all other targets are unchanged. Cost on affected builds only, callgrind with the same code forced on Linux: +3 Ir per local free (small 52.26 -> 55.26); Rust GlobalAlloc workloads +0.30 % to +3.56 %, checksums identical. Regression gate: rusty_sloth joins the downstream corpus as a `build = "release"` row (run.sh builds such rows in the consumer's own release profile, and blames an LLVM verifier failure on us even though it names no crate). No small consumer reproduced the defect: rustc marks cleanup calls cold, so the inliner keeps drops out of funclets in every shape tried (docs/plans/msvc-funclet-asm-goto.md section 8). Co-Authored-By: Claude Opus 5.5 --- crates/rusty_alloc/CHANGELOG.md | 18 ++++++++++++++++ crates/rusty_alloc/src/alloc.rs | 26 +++++++++++++++++++++-- crates/rusty_alloc_api/CHANGELOG.md | 7 +++++++ docs/plans/msvc-funclet-asm-goto.md | 32 ++++++++++++++++++++++++++++- tools/corpus/README.md | 9 ++++++++ tools/corpus/corpus.toml | 16 +++++++++++++++ tools/corpus/run.sh | 30 ++++++++++++++++++--------- 7 files changed, 125 insertions(+), 13 deletions(-) diff --git a/crates/rusty_alloc/CHANGELOG.md b/crates/rusty_alloc/CHANGELOG.md index e02f0d1..189a757 100644 --- a/crates/rusty_alloc/CHANGELOG.md +++ b/crates/rusty_alloc/CHANGELOG.md @@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed + +- **2.2.1–2.2.4 failed to build in an MSVC consumer that unwinds, with LTO.** + `free_inline`'s fast path is an `asm!` goto (LLVM `callbr`), and since + 2.2.1 `GlobalAlloc::dealloc` inlines it into every Rust deallocation, + including the drops a panic runs. On MSVC targets those are EH cleanup + funclets, which cannot hold a `callbr`: LLVM rejected the module ("Bogus + funclet pad use ... Broken module found"). Reported by rusty_sloth (thin + LTO, one codegen unit, `panic = "unwind"`). On `target_env = "msvc"` with + `panic = "unwind"`, `free_inline` now uses the plain-Rust decrement that + aarch64 and Miri already use. The fix sits in `free_inline` itself, so it + covers every caller, `rusty_alloc::alloc::free` and + `rusty_alloc_api::Heap::dealloc` included, not just `dealloc`. MSVC builds + with `panic = "abort"` and every other target are unchanged. + Cost, on affected builds only: +3 instructions per local free (callgrind, + the same code forced on Linux: small 52.26 -> 55.26 Ir/op); Rust + `GlobalAlloc` workloads +0.30 % to +3.56 % whole-program, output identical. + ## [2.2.4](https://github.com/Remade-With-Rust/rusty_alloc/compare/rusty_alloc-v2.2.3...rusty_alloc-v2.2.4) - 2026-10-08 ### Semver note (read before upgrading) diff --git a/crates/rusty_alloc/src/alloc.rs b/crates/rusty_alloc/src/alloc.rs index ccabd53..ca61d00 100644 --- a/crates/rusty_alloc/src/alloc.rs +++ b/crates/rusty_alloc/src/alloc.rs @@ -1135,7 +1135,25 @@ pub unsafe fn free_inline(p: *mut u8) { { (*owner_heap(pg)).stats.frees += 1; } - #[cfg(all(target_arch = "x86_64", not(miri)))] + // NOT on an MSVC target that unwinds. This function inlines + // into `GlobalAlloc::dealloc`, so into every Rust drop, + // including the drops a panic runs; on MSVC those are EH + // cleanup FUNCLETS, and LLVM cannot place an `asm!` goto + // (`callbr`) in a funclet: its successors lose the funclet + // colouring and the verifier rejects the module ("Bogus + // funclet pad use"). 2.2.1-2.2.4 failed to build in an + // unwinding MSVC consumer with LTO that way (rusty_sloth; + // docs/plans/msvc-funclet-asm-goto.md). Those builds take the + // plain-Rust decrement below (+3 Ir per local free, opps #6), + // which reaches every caller, not just `dealloc`. An MSVC + // build with `panic = "abort"` has no funclets and keeps the + // asm. Regression gate: the corpus's rusty_sloth row, built + // `--release` (no small consumer reproduces it). + #[cfg(all( + target_arch = "x86_64", + not(miri), + not(all(target_env = "msvc", panic = "unwind")) + ))] { // SAFETY: `pg` is a live page of this thread; `USED_OFFSET` // is `offset_of!(Page, used)`, checked against the real @@ -1180,7 +1198,11 @@ pub unsafe fn free_inline(p: *mut u8) { options(nostack), ); } - #[cfg(not(all(target_arch = "x86_64", not(miri))))] + #[cfg(not(all( + target_arch = "x86_64", + not(miri), + not(all(target_env = "msvc", panic = "unwind")) + )))] { let u = (*pg).used.wrapping_sub(1); (*pg).used = u; diff --git a/crates/rusty_alloc_api/CHANGELOG.md b/crates/rusty_alloc_api/CHANGELOG.md index ebd539e..fdda64a 100644 --- a/crates/rusty_alloc_api/CHANGELOG.md +++ b/crates/rusty_alloc_api/CHANGELOG.md @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed + +- **2.2.1–2.2.4 failed to build in an MSVC consumer that unwinds, with LTO** + ("Bogus funclet pad use"): `GlobalAlloc::dealloc` inlined the core's + `asm!` goto into EH cleanup funclets. Fixed in `rusty_alloc` (see its + changelog); this crate's code is unchanged. + ## [2.2.1](https://github.com/Remade-With-Rust/rusty_alloc/compare/rusty_alloc-api-v2.2.0...rusty_alloc-api-v2.2.1) - 2026-09-25 ### Performance diff --git a/docs/plans/msvc-funclet-asm-goto.md b/docs/plans/msvc-funclet-asm-goto.md index f8fe410..f7daf2f 100644 --- a/docs/plans/msvc-funclet-asm-goto.md +++ b/docs/plans/msvc-funclet-asm-goto.md @@ -1,6 +1,6 @@ # 2.2.1–2.2.4 do not build in an unwinding MSVC consumer with LTO -**Status:** fix written and verified in a consumer; **not merged, not released**. To do: §6. +**Status:** fixed differently from §3, on branch `fix/msvc-funclet-asm-goto`, for 2.2.5. See §8. **Date:** 2026-10-07 · **Affects:** rusty_alloc-api 2.2.1, 2.2.2, 2.2.3, 2.2.4 · **Platform:** `x86_64-pc-windows-msvc`, consumer built with `panic = "unwind"` (the default), `lto = "thin"`, `codegen-units = 1` · @@ -198,3 +198,33 @@ workspace (`tests/msvc-unwind-consumer/`, with `path` dependencies) built with - Would an LLVM / rustc issue be worth filing? A `callbr` inlined into a funclet should either be handled or refused at inline time, not left for the verifier. A minimal reproducer is an `asm!` goto in an `#[inline(always)]` function called from a `Drop` on MSVC with LTO. + +## 8. Resolution (2026-10-07, rusty_alloc session) + +**The fix is in `free_inline`, not in `dealloc`.** `7037d8d` outlined only +`GlobalAlloc::dealloc`, so the other callers were still exposed: +`rusty_alloc_api::Heap::dealloc` calls `rusty_alloc::alloc::free`, whose body +is `free_inline`, and LTO is free to inline it into a consumer's drop. Instead, +the asm goto's cfg in `free_inline` now excludes +`all(target_env = "msvc", panic = "unwind")`. Those builds take the +plain-Rust decrement that aarch64 and Miri already use. MSVC builds with +`panic = "abort"`, which have no funclets, keep the asm. `7037d8d` is not +merged; its branch is left in place. + +**Cost** (§6 step 3). Callgrind cannot run an MSVC build, so the same code was +forced on Linux and counted there: +3 Ir per local free, exactly opps #6's +figure. Opscan small 52.26 -> 55.26, big 94 -> 97, mixed 94.26 -> 97.26. On the +Rust `GlobalAlloc` workloads it cost maps +0.30 %, threads +0.40 %, +overaligned +0.62 % and trees +3.56 % (free-heavy), with identical checksums. +Outlining would have cost about +2 (call and return) and covered one caller. + +**Regression gate** (§5). A synthetic consumer did NOT reproduce. Four +shapes, thin and fat LTO, all built clean on the unfixed code: a `Vec` owned +across a panicking call, a `Heap::dealloc` in a `Drop`, a direct +`GlobalAlloc::dealloc` in a `Drop`, and an unwind-only guard with an +`#[inline(always)]` drop. The IR shows why: rustc marks calls in cleanup blocks +cold, so the inliner leaves the drop out of line and the `callbr` stays on the +normal path. Whatever rusty_sloth does to defeat that is not small. So the gate +is **rusty_sloth itself**, as a corpus row built `--release` +(`tools/corpus/corpus.toml`, `build = "release"`). It is a local gate, not a CI +job, because rusty_sloth is not on CI's machines. diff --git a/tools/corpus/README.md b/tools/corpus/README.md index f4889bd..f8236b7 100644 --- a/tools/corpus/README.md +++ b/tools/corpus/README.md @@ -17,6 +17,15 @@ bash tools/corpus/run.sh --test # + each consumer's test suite Consumers are registered in `corpus.toml`. Add one the moment it takes a dependency on this crate. +A consumer with `build = "release"` is BUILT in its own release profile +instead, in both modes. `check` never runs codegen and `test` never uses LTO, +and some defects exist only in optimised codegen. 2.2.1–2.2.4 failed LLVM's +verifier in rusty_sloth, an unwinding MSVC consumer with thin LTO +(`docs/plans/msvc-funclet-asm-goto.md`). That row is slow: an LTO build of an +ML stack, twice. A candidate failing LLVM's verifier ("Broken module found") +is blamed on us even though the error names no crate, because the baseline +built and only the allocator differs between the two builds. + ## What it does Two arms per consumer. **BASELINE** is the consumer as it sits on disk, with its diff --git a/tools/corpus/corpus.toml b/tools/corpus/corpus.toml index 785ed11..fcde118 100644 --- a/tools/corpus/corpus.toml +++ b/tools/corpus/corpus.toml @@ -59,3 +59,19 @@ features = "" # packages — the tree crates.io receives. Registered separately because it is # what ships, and it carries its own workspace root (no synth needed). note = "generated mirror of packages/spacedb; rusty_alloc-api =1.1.6, default-on" + +[[consumer]] +name = "rusty_sloth" +path = "F:/coding/rusty_sloth" +pkg = "rusty_sloth-cli" +features = "" +build = "release" +# The shape no other consumer has: it UNWINDS (default panic strategy) and +# ships thin LTO with one codegen unit, so the allocator's inlined free lands +# in EH cleanup funclets on MSVC. 2.2.1-2.2.4 failed LLVM's verifier there +# ("Bogus funclet pad use"); docs/plans/msvc-funclet-asm-goto.md. Also the +# consumer that exercised 2.2.4's stranded-span fix: it quantises on worker +# threads that exit and frees on main. Built with `cargo build --release`, +# not `check`: the defect exists only in optimised codegen. Slow (an LTO build +# of an ML stack, twice), and wants ~10 GB free in TMPDIR. +note = "via crates/rusty_sloth-alloc; unwinding + thin LTO + cgu=1; release BUILD" diff --git a/tools/corpus/run.sh b/tools/corpus/run.sh index 109ac98..d77dd94 100644 --- a/tools/corpus/run.sh +++ b/tools/corpus/run.sh @@ -155,7 +155,7 @@ PY } run_one() { - local name="$1" path="$2" feats="$3" pkg="$4" synth="$5" note="$6" + local name="$1" path="$2" feats="$3" pkg="$4" synth="$5" note="$6" build="${7:-}" if [ ! -d "$path" ]; then echo " SKIP $name -- $path not on this machine" ROWS+=("SKIP|$name|not on this machine") @@ -169,6 +169,13 @@ run_one() { # bins and `rusty_alloc_default` both claim it), which is this harness picking # the wrong target rather than anything being wrong downstream. [ -n "${pkg:-}" ] && fargs+=(-p "$pkg") + # `build = "release"`: this consumer is built, not checked or tested, in ITS + # OWN release profile. Some defects exist only in optimised codegen: 2.2.1- + # 2.2.4 failed LLVM's verifier in an unwinding MSVC build with LTO (an asm + # goto inlined into an EH funclet), which `check` never codegens and `test` + # (no LTO) never inlines (docs/plans/msvc-funclet-asm-goto.md). + local cmd=("$mode") + [ "$build" = "release" ] && cmd=(build --release) echo "== $name" echo " $note" @@ -189,7 +196,7 @@ run_one() { synth_workspace "$base_dir" fi local base_out base_rc - base_out="$(cd "$base_dir" && cargo "$mode" --quiet "${fargs[@]}" 2>&1)" + base_out="$(cd "$base_dir" && cargo "${cmd[@]}" --quiet "${fargs[@]}" 2>&1)" base_rc=$? # CANDIDATE: a copy, repointed at this tree. @@ -198,7 +205,7 @@ run_one() { repoint "$dst" patch_shim "$dst" local cand_out cand_rc - cand_out="$(cd "$dst" && cargo "$mode" --quiet "${fargs[@]}" 2>&1)" + cand_out="$(cd "$dst" && cargo "${cmd[@]}" --quiet "${fargs[@]}" 2>&1)" cand_rc=$? if resolved_from_registry "$dst"; then @@ -226,7 +233,10 @@ run_one() { echo " PASS both arms green" ROWS+=("PASS|$name|both arms green") pass=$((pass + 1)) - elif ! grep -qiE "rusty[_-]alloc" <<<"$cand_out"; then + # An LLVM verifier failure names no crate ("Bogus funclet pad use ... + # Broken module found"), yet the baseline built: with only the allocator + # changed between the arms, it is ours. + elif ! grep -qiE "rusty[_-]alloc|Broken module found" <<<"$cand_out"; then # The candidate is red but nothing in the error mentions us. A consumer can # be broken for its own reasons -- a missing dependency, a resolver shift # from the rewritten manifest -- and blaming the upgrade for it would make @@ -236,10 +246,10 @@ run_one() { ROWS+=("UNRELATED|$name|$(echo "$cand_out" | grep -E '^error' | head -1 | cut -c1-70)") skip=$((skip + 1)) else - echo " FAIL 2.0.0 BREAKS THIS CONSUMER" - echo "$cand_out" | grep -E "^error" | head -3 | sed 's/^/ /' + echo " FAIL THIS TREE BREAKS THIS CONSUMER" + echo "$cand_out" | grep -E "^error|Bogus|Broken module" | head -3 | sed 's/^/ /' local first - first="$(echo "$cand_out" | grep -E "^error" | head -1 | cut -c1-110)" + first="$(echo "$cand_out" | grep -E "Bogus|Broken module|^error" | head -1 | cut -c1-110)" ROWS+=("FAIL|$name|$first") fail=$((fail + 1)) fi @@ -252,15 +262,15 @@ import re, sys s = open(sys.argv[1], encoding='utf-8').read() for blk in s.split('[[consumer]]')[1:]: g = lambda k: (re.search(rf'^{k}\s*=\s*"(.*)"', blk, re.M) or [None, ''])[1] - print('|'.join([g('name'), g('path'), g('features'), g('pkg'), g('synth'), g('note')])) + print('|'.join([g('name'), g('path'), g('features'), g('pkg'), g('synth'), g('note'), g('build')])) PY mkdir -p "$work" # `|`, not tab: tab is IFS WHITESPACE, so bash collapses runs of it and an empty # `features` field silently shifts every column after it. That fed each # consumer's NOTE to `--features` and turned all five baselines red -- the # harness reporting on itself, again. -while IFS='|' read -r n p f pk sy note; do - [ -n "$n" ] && run_one "$n" "$p" "$f" "$pk" "$sy" "$note" +while IFS='|' read -r n p f pk sy note bld; do + [ -n "$n" ] && run_one "$n" "$p" "$f" "$pk" "$sy" "$note" "$bld" done < "$work.list" echo From 236dcc336b039edc1d95378827ce7f9d078cf51e Mon Sep 17 00:00:00 2001 From: Tim Date: Wed, 7 Oct 2026 20:44:49 -0700 Subject: [PATCH 3/4] corpus: build the rusty_sloth row with --features cuda Without it the release build is green on unfixed 2.2.4: the drops that land in an MSVC funclet are in the CUDA paths. cuda is also the shipped shape. Co-Authored-By: Claude Opus 5.5 --- docs/plans/msvc-funclet-asm-goto.md | 6 ++ tools/corpus/corpus.toml | 157 ++++++++++++++-------------- 2 files changed, 86 insertions(+), 77 deletions(-) diff --git a/docs/plans/msvc-funclet-asm-goto.md b/docs/plans/msvc-funclet-asm-goto.md index f7daf2f..6621ce1 100644 --- a/docs/plans/msvc-funclet-asm-goto.md +++ b/docs/plans/msvc-funclet-asm-goto.md @@ -228,3 +228,9 @@ normal path. Whatever rusty_sloth does to defeat that is not small. So the gate is **rusty_sloth itself**, as a corpus row built `--release` (`tools/corpus/corpus.toml`, `build = "release"`). It is a local gate, not a CI job, because rusty_sloth is not on CI's machines. + +**The row needs `--features cuda`.** Without it, rusty_sloth's release build +is GREEN on unfixed 2.2.4 (corpus run, 2026-10-07): the drops that land in a +funclet are in its CUDA paths. `cuda` is also the shape it ships, so the row +builds that, and a machine without the CUDA toolkit reports BASELINE ALREADY +RED rather than a pass. diff --git a/tools/corpus/corpus.toml b/tools/corpus/corpus.toml index fcde118..8981824 100644 --- a/tools/corpus/corpus.toml +++ b/tools/corpus/corpus.toml @@ -1,77 +1,80 @@ -# The downstream corpus: crates that ship rusty_alloc to production. -# -# A major version is a promise that consumers can keep compiling. 2.0.0 makes -# two breaking changes -- `default-features = false` now selects `no_std`, and -# `heap::Heap` gained a field -- and neither is visible from inside this repo. -# The only way to know what they cost is to build the things that depend on us. -# -# `path` is absolute because these are sibling checkouts, not submodules; a -# consumer whose path is missing is SKIPPED and reported as such, never silently -# passed. Add a consumer here the moment it takes a dependency on this crate. -# -# `features`/`no_default` describe the configuration that actually SHIPS. There -# is no value in testing a shape nobody deploys. - -[[consumer]] -name = "spacedb-sdk" -path = "F:/coding/mata-master/packages/spacedb" -pkg = "spacedb-sdk" -synth = "yes" -# `rusty-alloc` is default-ON here on purpose: a SpaceDB replica often runs on a -# machine its operator does not control, so the unconfigured node is the -# hardened one. That makes this the shape in production. -features = "" -note = "rusty_alloc-api pinned =1.1.6, optional, default-on" - -[[consumer]] -name = "spacedb-sdk (secure)" -path = "F:/coding/mata-master/packages/spacedb" -pkg = "spacedb-sdk" -synth = "yes" -features = "secure" -note = "forwards rusty_alloc-api/secure -- guard pages + encrypted free lists" - -[[consumer]] -name = "rusty_alloc_default" -path = "F:/coding/rusty_alloc_default" -features = "" -note = "the #[global_allocator] shim every Remade UI crate installs; rusty_zstd reaches us THROUGH it" - -[[consumer]] -name = "rusty_zstd" -path = "F:/coding/rusty_zstd" -features = "rusty-alloc" -pkg = "rusty_zstd" -note = "depends via rusty_alloc_default, not directly" - -[[consumer]] -name = "rusty_maplibre" -path = "F:/coding/rusty_maplibre" -features = "" -note = "KNOWN RISK: takes rusty_alloc AND rusty_alloc-api with default-features = false, which is exactly what 2.0.0 redefines" - -[[consumer]] -name = "spacedb (published mirror)" -path = "F:/coding/spacedb" -pkg = "spacedb-sdk" -features = "" -# The standalone checkout `release/release.sh` generates from mata-master's -# packages — the tree crates.io receives. Registered separately because it is -# what ships, and it carries its own workspace root (no synth needed). -note = "generated mirror of packages/spacedb; rusty_alloc-api =1.1.6, default-on" - -[[consumer]] -name = "rusty_sloth" -path = "F:/coding/rusty_sloth" -pkg = "rusty_sloth-cli" -features = "" -build = "release" -# The shape no other consumer has: it UNWINDS (default panic strategy) and -# ships thin LTO with one codegen unit, so the allocator's inlined free lands -# in EH cleanup funclets on MSVC. 2.2.1-2.2.4 failed LLVM's verifier there -# ("Bogus funclet pad use"); docs/plans/msvc-funclet-asm-goto.md. Also the -# consumer that exercised 2.2.4's stranded-span fix: it quantises on worker -# threads that exit and frees on main. Built with `cargo build --release`, -# not `check`: the defect exists only in optimised codegen. Slow (an LTO build -# of an ML stack, twice), and wants ~10 GB free in TMPDIR. -note = "via crates/rusty_sloth-alloc; unwinding + thin LTO + cgu=1; release BUILD" +# The downstream corpus: crates that ship rusty_alloc to production. +# +# A major version is a promise that consumers can keep compiling. 2.0.0 makes +# two breaking changes -- `default-features = false` now selects `no_std`, and +# `heap::Heap` gained a field -- and neither is visible from inside this repo. +# The only way to know what they cost is to build the things that depend on us. +# +# `path` is absolute because these are sibling checkouts, not submodules; a +# consumer whose path is missing is SKIPPED and reported as such, never silently +# passed. Add a consumer here the moment it takes a dependency on this crate. +# +# `features`/`no_default` describe the configuration that actually SHIPS. There +# is no value in testing a shape nobody deploys. + +[[consumer]] +name = "spacedb-sdk" +path = "F:/coding/mata-master/packages/spacedb" +pkg = "spacedb-sdk" +synth = "yes" +# `rusty-alloc` is default-ON here on purpose: a SpaceDB replica often runs on a +# machine its operator does not control, so the unconfigured node is the +# hardened one. That makes this the shape in production. +features = "" +note = "rusty_alloc-api pinned =1.1.6, optional, default-on" + +[[consumer]] +name = "spacedb-sdk (secure)" +path = "F:/coding/mata-master/packages/spacedb" +pkg = "spacedb-sdk" +synth = "yes" +features = "secure" +note = "forwards rusty_alloc-api/secure -- guard pages + encrypted free lists" + +[[consumer]] +name = "rusty_alloc_default" +path = "F:/coding/rusty_alloc_default" +features = "" +note = "the #[global_allocator] shim every Remade UI crate installs; rusty_zstd reaches us THROUGH it" + +[[consumer]] +name = "rusty_zstd" +path = "F:/coding/rusty_zstd" +features = "rusty-alloc" +pkg = "rusty_zstd" +note = "depends via rusty_alloc_default, not directly" + +[[consumer]] +name = "rusty_maplibre" +path = "F:/coding/rusty_maplibre" +features = "" +note = "KNOWN RISK: takes rusty_alloc AND rusty_alloc-api with default-features = false, which is exactly what 2.0.0 redefines" + +[[consumer]] +name = "spacedb (published mirror)" +path = "F:/coding/spacedb" +pkg = "spacedb-sdk" +features = "" +# The standalone checkout `release/release.sh` generates from mata-master's +# packages — the tree crates.io receives. Registered separately because it is +# what ships, and it carries its own workspace root (no synth needed). +note = "generated mirror of packages/spacedb; rusty_alloc-api =1.1.6, default-on" + +[[consumer]] +name = "rusty_sloth" +path = "F:/coding/rusty_sloth" +pkg = "rusty_sloth-cli" +# `cuda` is the shipped shape AND the trigger: without it the release build +# is green on 2.2.4 (checked 2026-10-07). Needs the CUDA toolkit; a machine +# without it reports BASELINE ALREADY RED, not a pass. +features = "cuda" +build = "release" +# The shape no other consumer has: it UNWINDS (default panic strategy) and +# ships thin LTO with one codegen unit, so the allocator's inlined free lands +# in EH cleanup funclets on MSVC. 2.2.1-2.2.4 failed LLVM's verifier there +# ("Bogus funclet pad use"); docs/plans/msvc-funclet-asm-goto.md. Also the +# consumer that exercised 2.2.4's stranded-span fix: it quantises on worker +# threads that exit and frees on main. Built with `cargo build --release`, +# not `check`: the defect exists only in optimised codegen. Slow (an LTO build +# of an ML stack, twice), and wants ~10 GB free in TMPDIR. +note = "via crates/rusty_sloth-alloc; --features cuda; unwinding + thin LTO + cgu=1; release BUILD" From 8ca6eb8fdf77fda8cea20abc0f68303a832d34bd Mon Sep 17 00:00:00 2001 From: Tim Date: Wed, 7 Oct 2026 21:10:45 -0700 Subject: [PATCH 4/4] corpus: registry-identity candidates; the rusty_sloth row uses one A path-repointed candidate cannot reproduce the MSVC funclet defect: unfixed 2.2.4 as a path dependency built rusty_sloth green, the same source from crates.io failed (package id -> symbol hashes -> inlining). identity = "registry" vendors the consumer's dependencies and swaps only the vendored rusty_alloc src, so the candidate keeps the crates.io identity. Verified by hand on rusty_sloth --release --features cuda: published 2.2.4 fails (4 x Bogus funclet pad use), 2.2.4 + the fix builds and runs. Corrects the previous commit's note: cuda is the shipped shape, not the trigger. Co-Authored-By: Claude Opus 5.5 --- docs/plans/msvc-funclet-asm-goto.md | 52 +++++++++++++++++-------- tools/corpus/README.md | 8 ++++ tools/corpus/corpus.toml | 11 ++++-- tools/corpus/run.sh | 60 ++++++++++++++++++++++++----- 4 files changed, 103 insertions(+), 28 deletions(-) diff --git a/docs/plans/msvc-funclet-asm-goto.md b/docs/plans/msvc-funclet-asm-goto.md index 6621ce1..0ffc487 100644 --- a/docs/plans/msvc-funclet-asm-goto.md +++ b/docs/plans/msvc-funclet-asm-goto.md @@ -218,19 +218,39 @@ Rust `GlobalAlloc` workloads it cost maps +0.30 %, threads +0.40 %, overaligned +0.62 % and trees +3.56 % (free-heavy), with identical checksums. Outlining would have cost about +2 (call and return) and covered one caller. -**Regression gate** (§5). A synthetic consumer did NOT reproduce. Four -shapes, thin and fat LTO, all built clean on the unfixed code: a `Vec` owned -across a panicking call, a `Heap::dealloc` in a `Drop`, a direct -`GlobalAlloc::dealloc` in a `Drop`, and an unwind-only guard with an -`#[inline(always)]` drop. The IR shows why: rustc marks calls in cleanup blocks -cold, so the inliner leaves the drop out of line and the `callbr` stays on the -normal path. Whatever rusty_sloth does to defeat that is not small. So the gate -is **rusty_sloth itself**, as a corpus row built `--release` -(`tools/corpus/corpus.toml`, `build = "release"`). It is a local gate, not a CI -job, because rusty_sloth is not on CI's machines. - -**The row needs `--features cuda`.** Without it, rusty_sloth's release build -is GREEN on unfixed 2.2.4 (corpus run, 2026-10-07): the drops that land in a -funclet are in its CUDA paths. `cuda` is also the shape it ships, so the row -builds that, and a machine without the CUDA toolkit reports BASELINE ALREADY -RED rather than a pass. +**Regression gate** (§5). The defect depends on LLVM's inlining choices, +and those depend on more than the code. + +- **No small consumer reproduced it.** Four shapes, thin and fat LTO, all + built clean on the unfixed code: a `Vec` owned across a panicking call, a + `Heap::dealloc` in a `Drop`, a direct `GlobalAlloc::dealloc` in a `Drop`, an + unwind-only guard with an `#[inline(always)]` drop. The IR shows why: rustc + marks calls in cleanup blocks cold, so the inliner leaves the drop out of + line and the `callbr` stays on the normal path. +- **rusty_sloth repointed to a PATH did not reproduce it either.** Unfixed + 2.2.4 as a path dependency built green, with and without `--features cuda`. + The same source from crates.io failed. The source is byte-identical + (`diff -r` of the published crate against the tree). What differs is the + package id, which goes into every symbol hash and so into LLVM's inlining + order. +- **What does reproduce it: crates.io identity, source swapped.** + `cargo vendor` plus source replacement keeps rusty_alloc a crates.io + package, so only its `src/` changes. rusty_sloth `--release --features cuda`: + + | allocator source | result | + |---|---| + | 2.2.4 as published | **fails**, 4 × "Bogus funclet pad use" | + | 2.2.4 + this fix | **builds**; `sloth --help` runs | + + A vendored package is immutable to Cargo: swapping its source does NOT + rebuild it. `cargo clean -p rusty_alloc -p rusty_alloc-api` first, or the + "fixed" build reuses the broken rlib (it did once here: 4 errors, nothing + recompiled). +- **Deterministic check.** In `rusty_alloc`'s own IR for + `x86_64-pc-windows-msvc`, `callbr` count: unfixed 3 (unwind) / 3 (abort); + fixed **0** (unwind) / 3 (abort). With no `callbr` there is nothing for any + inlining order to misplace. + +The gate is the corpus row for rusty_sloth: `build = "release"` and +`identity = "registry"` (`run.sh`'s `registry_swap`). It runs locally only, +because CI has neither rusty_sloth nor CUDA. diff --git a/tools/corpus/README.md b/tools/corpus/README.md index f8236b7..5d3dba9 100644 --- a/tools/corpus/README.md +++ b/tools/corpus/README.md @@ -26,6 +26,14 @@ ML stack, twice. A candidate failing LLVM's verifier ("Broken module found") is blamed on us even though the error names no crate, because the baseline built and only the allocator differs between the two builds. +`identity = "registry"` keeps the candidate's rusty_alloc a crates.io +package, vendored with only its `src/` swapped for this tree's +(`registry_swap`), instead of rewriting it to a path. Use it when a defect +depends on codegen: a path dependency has a different package id, so +different symbol hashes and different LLVM inlining. The funclet defect +fails rusty_sloth from crates.io and builds clean as a path dependency of the +very same source. + ## What it does Two arms per consumer. **BASELINE** is the consumer as it sits on disk, with its diff --git a/tools/corpus/corpus.toml b/tools/corpus/corpus.toml index 8981824..c666a8e 100644 --- a/tools/corpus/corpus.toml +++ b/tools/corpus/corpus.toml @@ -64,11 +64,16 @@ note = "generated mirror of packages/spacedb; rusty_alloc-api =1.1.6, default-on name = "rusty_sloth" path = "F:/coding/rusty_sloth" pkg = "rusty_sloth-cli" -# `cuda` is the shipped shape AND the trigger: without it the release build -# is green on 2.2.4 (checked 2026-10-07). Needs the CUDA toolkit; a machine -# without it reports BASELINE ALREADY RED, not a pass. +# `cuda` is the shipped shape, and the one the defect was reproduced in. Needs +# the CUDA toolkit; a machine without it reports BASELINE ALREADY RED, not a +# pass. features = "cuda" build = "release" +# AND with rusty_alloc kept a crates.io package (vendored, source swapped). +# Repointed to a path, unfixed 2.2.4 built GREEN here, with and without cuda, +# while the same source from crates.io failed: the package id changes the +# symbol hashes, and with them LLVM's inlining (checked 2026-10-07). +identity = "registry" # The shape no other consumer has: it UNWINDS (default panic strategy) and # ships thin LTO with one codegen unit, so the allocator's inlined free lands # in EH cleanup funclets on MSVC. 2.2.1-2.2.4 failed LLVM's verifier there diff --git a/tools/corpus/run.sh b/tools/corpus/run.sh index d77dd94..18651e3 100644 --- a/tools/corpus/run.sh +++ b/tools/corpus/run.sh @@ -108,6 +108,37 @@ patch_shim() { # Did the candidate actually build THIS tree? A registry `source` on any # rusty_alloc crate in its lock means no — and such a row used to PASS. +# `identity = "registry"`: build the candidate with rusty_alloc STILL a +# crates.io package and only its SOURCE swapped for this tree's. A path +# dependency gets a different package id, so different symbol hashes, and LLVM +# makes different inlining decisions. 2.2.4's MSVC funclet defect fails +# rusty_sloth from crates.io and builds clean as a path dependency +# (docs/plans/msvc-funclet-asm-goto.md section 8). Moves the lock to this tree's +# version when crates.io has it, vendors every dependency, overwrites the two +# vendored `src/` trees, and prints the cargo flags that select the vendor. +registry_swap() { + local tree="$1" vend="$1.vendor" ver + ver="$(grep -m1 '^version' "$root/Cargo.toml" | cut -d'"' -f2)" + (cd "$tree" && cargo update -q -p rusty_alloc-api --precise "$ver") >/dev/null 2>&1 || true + rm -rf "$vend" + # The config `cargo vendor` prints covers git sources too, not just crates.io. + (cd "$tree" && cargo vendor --versioned-dirs "$vend") >"$vend.toml" 2>/dev/null || return 1 + local n=0 d + for d in "$vend"/rusty_alloc-[0-9]* "$vend"/rusty_alloc-api-[0-9]*; do + [ -d "$d" ] || continue + case "$d" in *rusty_alloc-api-*) src="$root/crates/rusty_alloc_api/src" ;; *) src="$root/crates/rusty_alloc/src" ;; esac + rm -rf "$d/src" && cp -r "$src" "$d/src" || return 1 + python3 - "$d/.cargo-checksum.json" <<'PY2' || return 1 +import json, sys +p = sys.argv[1]; j = json.load(open(p)); j['files'] = {}; json.dump(j, open(p, 'w')) +PY2 + n=$((n + 1)) + done + [ "$n" -ge 1 ] || return 1 + grep -q 'replace-with' "$vend.toml" || return 1 + echo "--offline --config $vend.toml" +} + resolved_from_registry() { local lock="$1/Cargo.lock" [ -f "$lock" ] || return 1 @@ -155,7 +186,7 @@ PY } run_one() { - local name="$1" path="$2" feats="$3" pkg="$4" synth="$5" note="$6" build="${7:-}" + local name="$1" path="$2" feats="$3" pkg="$4" synth="$5" note="$6" build="${7:-}" ident="${8:-}" if [ ! -d "$path" ]; then echo " SKIP $name -- $path not on this machine" ROWS+=("SKIP|$name|not on this machine") @@ -202,13 +233,24 @@ run_one() { # CANDIDATE: a copy, repointed at this tree. copy_tree "$dst" [ "${synth:-}" = "yes" ] && synth_workspace "$dst" - repoint "$dst" - patch_shim "$dst" - local cand_out cand_rc - cand_out="$(cd "$dst" && cargo "${cmd[@]}" --quiet "${fargs[@]}" 2>&1)" + local cand_out cand_rc vargs=() + if [ "$ident" = "registry" ]; then + local va + if ! va="$(registry_swap "$dst")"; then + echo " NOT-SWAPPED could not vendor or swap the crates.io source -- this row tests nothing" + ROWS+=("NOT-SWAPPED|$name|registry_swap failed") + fail=$((fail + 1)) + return + fi + eval "vargs=($va)" + else + repoint "$dst" + patch_shim "$dst" + fi + cand_out="$(cd "$dst" && cargo "${cmd[@]}" --quiet "${fargs[@]}" "${vargs[@]}" 2>&1)" cand_rc=$? - if resolved_from_registry "$dst"; then + if [ "$ident" != "registry" ] && resolved_from_registry "$dst"; then echo " NOT-REPOINTED the candidate resolved rusty_alloc from crates.io -- this row tests nothing" ROWS+=("NOT-REPOINTED|$name|candidate still built the crates.io allocator") fail=$((fail + 1)) @@ -262,15 +304,15 @@ import re, sys s = open(sys.argv[1], encoding='utf-8').read() for blk in s.split('[[consumer]]')[1:]: g = lambda k: (re.search(rf'^{k}\s*=\s*"(.*)"', blk, re.M) or [None, ''])[1] - print('|'.join([g('name'), g('path'), g('features'), g('pkg'), g('synth'), g('note'), g('build')])) + print('|'.join([g('name'), g('path'), g('features'), g('pkg'), g('synth'), g('note'), g('build'), g('identity')])) PY mkdir -p "$work" # `|`, not tab: tab is IFS WHITESPACE, so bash collapses runs of it and an empty # `features` field silently shifts every column after it. That fed each # consumer's NOTE to `--features` and turned all five baselines red -- the # harness reporting on itself, again. -while IFS='|' read -r n p f pk sy note bld; do - [ -n "$n" ] && run_one "$n" "$p" "$f" "$pk" "$sy" "$note" "$bld" +while IFS='|' read -r n p f pk sy note bld idn; do + [ -n "$n" ] && run_one "$n" "$p" "$f" "$pk" "$sy" "$note" "$bld" "$idn" done < "$work.list" echo