diff --git a/Cargo.lock b/Cargo.lock index 99702dd..fd54b3e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -275,7 +275,7 @@ checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" [[package]] name = "rusty_alloc" -version = "2.2.5" +version = "2.2.6" dependencies = [ "libc", "loom", @@ -286,14 +286,14 @@ dependencies = [ [[package]] name = "rusty_alloc-api" -version = "2.2.5" +version = "2.2.6" dependencies = [ "rusty_alloc", ] [[package]] name = "rusty_alloc-bench" -version = "2.2.5" +version = "2.2.6" dependencies = [ "libloading", "rusty_alloc", @@ -302,14 +302,14 @@ dependencies = [ [[package]] name = "rusty_alloc-ffi" -version = "2.2.5" +version = "2.2.6" dependencies = [ "rusty_alloc", ] [[package]] name = "rusty_alloc-override" -version = "2.2.5" +version = "2.2.6" dependencies = [ "rusty_alloc", "rusty_alloc-ffi", @@ -317,7 +317,7 @@ dependencies = [ [[package]] name = "rusty_alloc-wasm" -version = "2.2.5" +version = "2.2.6" dependencies = [ "rusty_alloc", ] diff --git a/Cargo.toml b/Cargo.toml index 1060815..eeb9fb6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -132,7 +132,7 @@ members = [ # different files that never got the section. Fixed here, in both, and # verified by extracting the packaged `.crate` before publishing rather than # checking the live page afterwards. -version = "2.2.5" +version = "2.2.6" edition = "2024" # MSRV. Declared for the first time in 1.0.1 because this release genuinely # needs it: `asm!` label blocks (`asm_goto`, stable 1.87) carry the free path's diff --git a/crates/rusty_alloc/CHANGELOG.md b/crates/rusty_alloc/CHANGELOG.md index 9c6ff40..cd29b30 100644 --- a/crates/rusty_alloc/CHANGELOG.md +++ b/crates/rusty_alloc/CHANGELOG.md @@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [2.2.6](https://github.com/Remade-With-Rust/rusty_alloc/compare/rusty_alloc-v2.2.5...rusty_alloc-v2.2.6) - 2026-10-09 + +### Fixed + +- **A merged free span no longer hides a decommitted neighbour** (Windows, + purging on). When a freed span merged with an already-purged neighbour and + the merged span was not re-purged as a whole (the purge failed, or purging + had been switched off at runtime), the merged span kept a clear `purged` + flag over the neighbour's decommitted memory, and the next allocation wrote + to it: an access violation, reproduced as `STATUS_ACCESS_VIOLATION`. The + merged span now keeps the flag if any part of it was purged, so it is + re-committed before reuse. Cost: about 2 instructions per span free. + Mechanism B of `docs/plans/recommit-failure-ignored.md`. + ## [2.2.5](https://github.com/Remade-With-Rust/rusty_alloc/compare/rusty_alloc-v2.2.4...rusty_alloc-v2.2.5) - 2026-10-08 ### Fixed diff --git a/crates/rusty_alloc/UNSAFE.md b/crates/rusty_alloc/UNSAFE.md index 89c7134..e51ff5a 100644 --- a/crates/rusty_alloc/UNSAFE.md +++ b/crates/rusty_alloc/UNSAFE.md @@ -23,7 +23,7 @@ unsafe in DEPENDENCIES, and ours are `libc` plus bindings-only `windows-sys`. | `alloc.rs` | 110 | The public entry points: raw-pointer reads on the malloc fast path (must never form `&mut` on the shared empty-heap sentinel), pointer-derived metadata on the free path (`segment_of`/`page_of`), block-content copies in the realloc family. **+15 on 2026-08-22**: the free path's two `asm!` sites — the memory-destination `used--` whose flags drive the retire branch (its `label` block is a separate item and carries its own `unsafe`), and the fused `cmp {tid}, fs:0` in both `free_inline` and `free_general` — plus `malloc_or`/`malloc_or_slow`, which give `operator new` a fast path whose miss is a tail call. Each asm reads or writes exactly one field it already had a valid pointer to; none widens what the surrounding code could already touch. **+2 on 2026-09-24 (instruction-count campaign, LEDGER):** `realloc` split into an inline null test and `unsafe fn realloc_live(p: NonNull, ..)` — one signature, and the `NonNull::new_unchecked` sits inside the block that already called it, on a pointer the line above proved non-null. The body is the old one unchanged; nothing new is dereferenced. **+10 on 2026-09-24 (round two, LEDGER):** (a) `realloc_move`, the moving arm shared by the growth and the shrink — one `unsafe fn`, its body block, and the two call-site blocks: the old inline move code, now reached with the copy length each arm already holds; (b) `zalloc` in `malloc`'s raw-read shape — its fast-path block reads through `hb`, which may be the sentinel, exactly as `malloc` does (no `&mut`, no write unless a block was popped, which proves the heap is real), and the cold `unsafe fn zalloc_slow` that performs the sentinel test the fast path dropped (signature + block); (c) `free_local_owned` / `free_local_no_xheap`, `owner_heap` split so its heap-creating fallback is a tail call — two signatures and their two blocks, the same loads and the same `free_local_at` as before; (d) `malloc_aligned_pow2`, an `unsafe fn` only because its caller must have proven the alignment a power of two (a violation is not memory-unsafe — the bound test still refuses a mask at or above half a segment). Nothing new is dereferenced anywhere; each carries its SAFETY line. The baseline had drifted to 106 without a row edit; recorded now. **+4 on 2026-10-07 (the Miri Stacked Borrows failure on `main`):** the deferred-free hook used to fire inside `Heap::malloc_generic`, where `&mut self` is protected, so a hook that allocates re-entered the heap through a raw pointer under that borrow. It now fires at the allocation entries before any `&mut Heap` exists: `fire_deferred_if_due` (an `unsafe fn` and its block: a raw read and write of the heap's own `deferred_due` flag, then the hook) and `malloc_slow_deferred` (an `unsafe fn` and its block: the same, then the `malloc_generic` call `malloc_slow` already made). The nine entry blocks that call it existed before; nothing new is dereferenced | 2026-10-07 | | `heap.rs` | 80 | Owner-thread page/queue manipulation under raw pointers (no two `&mut Page` may coexist), the aligned-allocation peek, span carving. **+3 on 2026-08-19**: `try_unlink_huge_segment` split out of `remove_huge_segment`. **+15 on 2026-08-22**: `malloc_generic` split into a small entry plus `malloc_generic_walk`, with `grow_front`, `try_guarded` and `drain_delayed` as cold arms — each split adds an `unsafe fn` signature and its block while dereferencing nothing the single function did not — and the immortal `EMPTY_DELAYED` sentinel that lets the heartbeat read its list without a null test. **+2 on 2026-09-07 (P4e, §2.15 of `docs/plans/small-metal.md`)**: the reclamation fixes — one `unsafe` for the periodic `generic_collect` sweep and one for the reclaim-and-retry that runs before `malloc_generic` returns null. Both call `collect_inner`, which allocates nothing and touches only this heap’s own pages on the owner thread; each carries its SAFETY line. `malloc_generic` itself became a safe wrapper over the renamed `malloc_generic_once`, so the split added no signature. **+2 on 2026-09-08:** the medium collect-and-retry ahead of the heartbeat -- one `unsafe` around `page_collect` + `page_pop` on the bin queue front, one reading `free_is_zero` off the page just popped. Both are the SAME operations `malloc_generic_walk` performs on the same page a few lines later, on the owner thread under the heap lock: the block moves earlier, nothing new is dereferenced. Each carries its SAFETY line. **+2 on 2026-09-16 (Openheimer):** the two refuse arms in `malloc_aligned_at_slow` when `bins::aligned_at_from` reports that `block + offset + align` would wrap for a caller-chosen offset — each is one `free_local` of the block just allocated on this thread, which has not escaped. **+1 on 2026-09-24 (instruction-count campaign, LEDGER):** the medium collect-and-retry's `page_collect` + `page_pop` block became two blocks so the `saw_remote_free` latch is stored between them — same two calls on the same page, owner thread, in the same order; nothing new is dereferenced. **+1 on 2026-09-24 (round two):** a medium miss grows the front page directly — one block around the `capacity < reserved` test and the `grow_front` call, on the same page the block above just collected, which is `grow_front`'s whole contract **+2 on 2026-09-25 (CURIOSITY ROUND FOUR):** the medium arm pops the front page BEFORE collecting it — one `page_pop` block and the `free_is_zero` read beside it, the same two operations the collect-then-pop below already performs on the same owner-thread page. **+1 on 2026-10-07 (`docs/opps.md` vein census, vein 2):** `collect_inner` hands the per-heap cached large span (`Heap::large_cache`) back with `retire_span` — one block around `segment_of` on the cached page's own address and the same `retire_span` a large free performed before the cache existed, on the owner thread. **+2 the same day:** `free_large_span` — an `unsafe fn` and its block — is the large-span arm of `free_local_at` moved into its own function (double-free check, cache, free-run flag); the same operations on the same page, owner thread | 2026-10-07 | | `init.rs` | 42 | Thread/heap lifecycle: the initial-exec TLS slot (`global_asm!` + fs-relative asm reads), thread-pointer register reads (`fs:0`/`gs:0x30`/`tpidrro_el0`), heap-box creation/teardown, the abandonment path run inside platform TLS destructors. **+1 on 2026-09-09 (`firmware-what-is-left.md` §7.3): an ATTRIBUTE, not an operation** — `unsafe(link_section = ".rodata.…")` on `EMPTY_HEAP_BOX`, bare metal only, so the never-written sentinel lives in flash instead of costing 1,752 bytes of RAM. The token is counted because the census is a text search; the contract it rests on (raw reads only, no page ever stores its address) is the one the `Sync` impl below already carries, and a write would now fault against the flash cache instead of silently landing. `create_heap`'s template copy from it is a `ptr::read` inside the block that already existed. **+5 on 2026-09-16 (Openheimer, OH-rusty_alloc-13):** thread exit now abandons EVERY heap the dying thread still owns, not only the one in `done_slot` — a first-class heap that was never `heap_delete`d, or one installed with `set_default_heap`, used to stay DELAYED under a dead owner forever. `take_heap_owned_by` unlinks one such box under `HEAPS_LOCK` (one block: the registry walk); `thread_done` reads `owner_tid` off its own box and calls the split-out `unsafe fn thread_done_one` for the primary box and for each extra one (three sites plus the signature). Nothing new is dereferenced that the single function did not; each carries its SAFETY line | 2026-09-16 | -| `segment.rs` | 45 | Segment/page metadata addressing: the mask trick (`segment_of`), `page_of`'s contract-based indexing — **the bound is now PROVED for every in-segment offset by `proofs.rs` (Kani), not merely asserted** — span tiling, purge/recommit. The baseline had drifted to 36 without a row edit; recorded now. **+2 on 2026-10-04 (`docs/plans/recommit-failure-ignored.md`):** `restore_for_reuse` — an `unsafe fn` and its block — is the guard-lift and re-commit `segment_free` and `huge_free` each did inline, moved into one function so both can act on its answer: a segment whose memory the OS will not make accessible and committed again is released whole or retired in place, never recycled. It touches the same range the inline code did; `segment_free`'s own block became the expression that calls it (no net change). `span_recommit` now returns whether the span is backed (same block, same call). **+5 the same day, all `#[cfg(test)]`:** `a_failed_recommit_is_a_failed_span_not_an_unbacked_one` writes and frees the blocks it allocated (four blocks) and frees them at the end (one); each carries its SAFETY line. **+2 on 2026-10-07 (`docs/opps.md` vein census, vein 1):** `scrub_recycled` — an `unsafe fn` and its block — replaces the whole-header `write_bytes` that `segment_alloc` and `huge_alloc` did on every recycled chunk: it zeroes the 2 KB owner table (all of it for a Normal segment, entry 0 for a Huge one, which overwrites the rest) and the first `slots` page slots, inside the same header the old call covered. The other slots of a Normal segment are zeroed where the bump cursor first carves them (inside `span_alloc`'s existing block); a Huge segment reads only slots 0 and 1, and `wait_no_remote_in_flight` now scans slot 1 alone there. Nothing new is dereferenced | 2026-10-07 | +| `segment.rs` | 55 | Segment/page metadata addressing: the mask trick (`segment_of`), `page_of`'s contract-based indexing — **the bound is now PROVED for every in-segment offset by `proofs.rs` (Kani), not merely asserted** — span tiling, purge/recommit. The baseline had drifted to 36 without a row edit; recorded now. **+2 on 2026-10-04 (`docs/plans/recommit-failure-ignored.md`):** `restore_for_reuse` — an `unsafe fn` and its block — is the guard-lift and re-commit `segment_free` and `huge_free` each did inline, moved into one function so both can act on its answer: a segment whose memory the OS will not make accessible and committed again is released whole or retired in place, never recycled. It touches the same range the inline code did; `segment_free`'s own block became the expression that calls it (no net change). `span_recommit` now returns whether the span is backed (same block, same call). **+5 the same day, all `#[cfg(test)]`:** `a_failed_recommit_is_a_failed_span_not_an_unbacked_one` writes and frees the blocks it allocated (four blocks) and frees them at the end (one); each carries its SAFETY line. **+2 on 2026-10-07 (`docs/opps.md` vein census, vein 1):** `scrub_recycled` — an `unsafe fn` and its block — replaces the whole-header `write_bytes` that `segment_alloc` and `huge_alloc` did on every recycled chunk: it zeroes the 2 KB owner table (all of it for a Normal segment, entry 0 for a Huge one, which overwrites the rest) and the first `slots` page slots, inside the same header the old call covered. The other slots of a Normal segment are zeroed where the bump cursor first carves them (inside `span_alloc`'s existing block); a Huge segment reads only slots 0 and 1, and `wait_no_remote_in_flight` now scans slot 1 alone there. Nothing new is dereferenced. **+10 on 2026-10-09, all test-only:** `recommit_tests` gains the two merge tests for mechanism B of `docs/plans/recommit-failure-ignored.md` and their helpers (block writes and slot reads on the test's own live spans). The fix itself, carrying a merged neighbour's `purged` flag in `span_free`, adds none: it sits in the existing block | 2026-10-09 | | `prim/windows.rs` | 34 | OS FFI: VirtualAlloc family, FLS destructors, QPC, BCryptGenRandom. **+3 on 2026-09-16 (Openheimer):** `range_is_reserved` (`zeroed` `MEMORY_BASIC_INFORMATION` + `VirtualQuery`, the same OS answer `unix.rs` gets from `mincore`), and `alloc_aligned` now refuses a garbage alignment and a `size + align` that wraps BEFORE it reserves anything, and releases a re-reservation that the OS placed anywhere but the aligned address it asked for instead of returning it misaligned (`VirtualFree`, one block). **+1 on 2026-09-24 (`docs/plans/youslowbro.md` §4):** `getenv` — one `GetEnvironmentVariableA` into a caller buffer of exactly the length passed, the allocation-free environment read that replaced `std::env::var`'s owned strings in the options pass (251 allocations through the global allocator on every process's first allocation, now none) | 2026-09-24 | | `page.rs` | 38 | Free-list links written into dead blocks, the lock-free `xthread_free` four-state protocol (loom-modeled in `tests/loom_xthread.rs`), the immortal `EMPTY_PAGE` sentinel — **+1 on 2026-09-09: the same `unsafe(link_section)` attribute as `init.rs`, placing it in flash on bare metal; its free list is permanently null, so nothing writes it**. **+8 on 2026-08-22**: `page_link_local` split out of `page_push_local` so the caller can decrement `used` in one memory-destination RMW, `page_collect_impl` const-generic over whether it also writes the protocol flag, and `USED_OFFSET` — whose value is asserted against `offset_of!(Page, used)` by a unit test, because an asm operand is not type-checked and a field reordering would silently decrement the wrong bytes | 2026-08-22 (free campaign) | | `prim/unix.rs` | 33 | OS FFI: mmap family, madvise/decommit, pthread keys, /dev/urandom. **+1 on 2026-09-16 (Openheimer, OH-201):** `range_is_reserved` asks `mincore` whether a caller-supplied `manage_os_memory` range is mapped at all before it becomes an arena — one FFI call on a page-aligned probe. **+1 on 2026-09-24 (`docs/plans/youslowbro.md` §4):** `getenv` — `libc::getenv` and a byte-by-byte read of the returned C string up to its NUL or the caller buffer's end, nothing written through it; the same call upstream's prim makes, with the same standing caveat about a concurrent `setenv` **+2 on 2026-09-25:** `env_for_each` — an `unsafe extern "C"` declaration of `environ` and one block that walks it to its NULL terminator, handing each NUL-terminated entry on as a pointer; read only, with the same concurrent-`setenv` caveat as `getenv`. **+2 on 2026-10-01 (macOS):** the Apple arm of `range_is_reserved` — an `unsafe extern "C"` declaration of `mach_vm_region` and the `mach_task_self_` global, and one call per region: a read-only query of our own task into a `packed(4)` `vm_region_basic_info_64` whose 36-byte size is asserted at compile time and is exactly the word count passed, because XNU's `mincore` succeeds on unmapped ranges | 2026-10-01 | diff --git a/crates/rusty_alloc/src/arena.rs b/crates/rusty_alloc/src/arena.rs index 87d2a46..682a8ca 100644 --- a/crates/rusty_alloc/src/arena.rs +++ b/crates/rusty_alloc/src/arena.rs @@ -781,7 +781,7 @@ fn heapless_fmt( } #[cfg(test)] -mod adopt_tests { +pub(crate) mod adopt_tests { use super::*; /// One lock for all adoption tests. They are the only adopters on native @@ -795,7 +795,11 @@ mod adopt_tests { /// written to survive LANDING in a shared or pre-extended arena, because /// arenas are never unregistered and an earlier test's arena can adopt a /// later test's adjacent block. - fn lock() -> std::sync::MutexGuard<'static, ()> { + /// + /// Visible to the crate: tests elsewhere that allocate whole segments (and so may + /// take a chunk from an arena one of these tests just adopted, before it + /// drains it) take this lock too. `segment::recommit_tests` does. + pub fn lock() -> std::sync::MutexGuard<'static, ()> { static LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(()); LOCK.lock().unwrap_or_else(|e| e.into_inner()) } diff --git a/crates/rusty_alloc/src/os.rs b/crates/rusty_alloc/src/os.rs index 9b6b430..fcdf822 100644 --- a/crates/rusty_alloc/src/os.rs +++ b/crates/rusty_alloc/src/os.rs @@ -217,6 +217,22 @@ pub(crate) mod test_hooks { std::thread_local! { static FAIL_COMMITS: Cell = const { Cell::new(0) }; + static FAIL_PURGES: Cell = const { Cell::new(0) }; + } + + /// Fail this thread's next `n` purges. + pub fn fail_next_purges(n: usize) { + FAIL_PURGES.with(|c| c.set(n)); + } + + pub(super) fn take_purge_failure() -> bool { + FAIL_PURGES.with(|c| { + let n = c.get(); + if n > 0 { + c.set(n - 1); + } + n > 0 + }) } /// Fail this thread's next `n` commits. @@ -250,6 +266,10 @@ pub unsafe fn decommit(ptr: *mut u8, size: usize) -> Result { /// # Safety /// As [`commit`]; contents are lost either way. pub unsafe fn purge(ptr: *mut u8, size: usize, purge_decommits: bool) -> Result { + #[cfg(all(test, feature = "std"))] + if test_hooks::take_purge_failure() { + return Err(test_hooks::INJECTED); + } if purge_decommits { // SAFETY: forwarded contract. unsafe { decommit(ptr, size) } diff --git a/crates/rusty_alloc/src/segment.rs b/crates/rusty_alloc/src/segment.rs index fecdd5b..a854a79 100644 --- a/crates/rusty_alloc/src/segment.rs +++ b/crates/rusty_alloc/src/segment.rs @@ -822,6 +822,14 @@ pub unsafe fn span_free(seg: *mut Segment, page: *mut Page) -> bool { (*page).free_is_zero = false; // Merge right: the slot at idx+len (if carved) is a span START. + // Whether any free neighbour merged in below is still purged. Only a + // span's FIRST slot carries the flag (`span_recommit` reads nothing + // else), so a merge that is not re-purged as a whole must carry its + // parts' flags onto the merged start, or a decommitted neighbour ends + // up behind a clear flag and is reused unbacked: the write fault of + // `docs/plans/recommit-failure-ignored.md`, mechanism B. A spurious + // flag costs only a re-commit of committed memory. + let mut merged_purged = false; let right = idx + len; if right < (*seg).next_free_slice as usize { let rslot: *mut Page = &raw mut (*seg).pages[right]; @@ -829,6 +837,7 @@ pub unsafe fn span_free(seg: *mut Segment, page: *mut Page) -> bool { if (*rslot).block_size == 0 { span_list_remove(seg, rslot); len += (*rslot).slice_count as usize; + merged_purged |= (*rslot).purged; } } // Merge left: follow the left slot back to its span start. @@ -843,6 +852,7 @@ pub unsafe fn span_free(seg: *mut Segment, page: *mut Page) -> bool { span_list_remove(seg, lstart); len += idx - lstart_idx; idx = lstart_idx; + merged_purged |= (*lstart).purged; } } // Re-mark only what does not already point to `idx`: with no left @@ -874,6 +884,12 @@ pub unsafe fn span_free(seg: *mut Segment, page: *mut Page) -> bool { return true; } } + // Not re-purged as a whole (purging off by now, the span too small, + // or the purge failed): the merged span is backed only if every part + // was. Never cleared here: the start may already be flagged. + if merged_purged { + (*seg).pages[idx].purged = true; + } false } } @@ -1219,6 +1235,10 @@ mod recommit_tests { /// purged, so a later re-commit that succeeds makes it usable again. #[test] fn a_failed_recommit_is_a_failed_span_not_an_unbacked_one() { + // Serialised with each other (they set the PROCESS-WIDE `purge_delay`) + // and with the arena adoption tests (they allocate segments, which + // can take a chunk from an arena one of those tests just adopted). + let _g = crate::arena::adopt_tests::lock(); // A span `span_free` will purge: at least MEDIUM_PAGE_SLICES slices. let span = SEGMENT_SLICE_SIZE * MEDIUM_PAGE_SLICES + SEGMENT_SLICE_SIZE; crate::options::set(15, 0); // purge_delay: purge at once @@ -1274,4 +1294,122 @@ mod recommit_tests { } crate::options::set(15, -1); // restore the shipped default } + + /// Two adjacent purge-eligible spans in one segment, pinned live on the + /// left so nothing merges past them. Returns `(pin, a, b, span)` with `b` + /// immediately right of `a`. + fn adjacent_pair() -> (*mut u8, *mut u8, *mut u8, usize) { + let span = SEGMENT_SLICE_SIZE * MEDIUM_PAGE_SLICES + SEGMENT_SLICE_SIZE; + let pin = malloc(span); + let a = malloc(span); + let b = malloc(span); + assert!(!pin.is_null() && !a.is_null() && !b.is_null()); + let seg = segment_of(a); + assert!( + segment_of(pin) == seg && segment_of(b) == seg, + "setup: one segment" + ); + // SAFETY: live blocks of this segment. + unsafe { + let pa = page_of(seg, a); + let pb = page_of(seg, b); + assert_eq!( + page_index(seg, pa) + (*pa).slice_count as usize, + page_index(seg, pb), + "setup: b must be a's right neighbour" + ); + core::ptr::write_bytes(a, 1, span); + core::ptr::write_bytes(b, 1, span); + } + (pin, a, b, span) + } + + /// Frees `b` with purging on, so its span is purged (decommitted on + /// Windows), and checks that it was. + fn purge_right(b: *mut u8) { + let seg = segment_of(b); + // SAFETY: `b` is live and freed once; its slot is read after. + unsafe { + let ib = page_index(seg, page_of(seg, b)); + free(b); + collect(true); + assert!((*seg).pages[ib].purged, "setup: b's span was not purged"); + } + } + + /// After `a` merged with the purged `b`: the merged span must still say + /// purged, and reusing the whole of it must be backed. On Windows the + /// write is the check (a decommitted page faults); the flag assertion + /// fails first everywhere. + fn merged_span_is_backed(a: *mut u8, a_idx: usize, span: usize) { + let seg = segment_of(a); + // SAFETY: `seg` is pinned live; slot reads only. + unsafe { + assert!( + (*seg).pages[a_idx].purged, + "a merge kept a clear flag over a purged neighbour: the next tenant gets unbacked memory" + ); + } + let whole = 2 * span; + let c = malloc(whole); + assert!(!c.is_null()); + assert_eq!(c, a, "setup: the merged span should be reused first-fit"); + // SAFETY: a live block of `whole` bytes. + unsafe { + core::ptr::write_bytes(c, 3, whole); + free(c); + } + } + + /// Mechanism B (`docs/plans/recommit-failure-ignored.md`): a freed span + /// merges with a PURGED right neighbour, and re-purging the merged span + /// FAILS. The merged start is the freed span's slot, whose flag is clear, + /// so without carrying the neighbour's flag the decommitted half is + /// handed out as if it were backed. + #[test] + fn a_failed_repurge_of_a_merge_keeps_the_span_purged() { + // Serialised with each other (they set the PROCESS-WIDE `purge_delay`) + // and with the arena adoption tests (they allocate segments, which + // can take a chunk from an arena one of those tests just adopted). + let _g = crate::arena::adopt_tests::lock(); + crate::options::set(15, 0); // purge_delay: purge at once + let (pin, a, b, span) = adjacent_pair(); + let seg = segment_of(a); + // SAFETY: `a` is live. + let ia = unsafe { page_index(seg, page_of(seg, a)) }; + purge_right(b); + crate::os::test_hooks::fail_next_purges(1); + // SAFETY: `a` is live and freed once. + unsafe { free(a) }; + collect(true); + crate::os::test_hooks::fail_next_purges(0); + merged_span_is_backed(a, ia, span); + // SAFETY: live, freed once. + unsafe { free(pin) }; + crate::options::set(15, -1); + } + + /// The same merge with purging switched OFF at runtime after the + /// neighbour was purged: the merged span is not re-purged at all. + #[test] + fn a_merge_after_purging_is_turned_off_keeps_the_span_purged() { + // Serialised with each other (they set the PROCESS-WIDE `purge_delay`) + // and with the arena adoption tests (they allocate segments, which + // can take a chunk from an arena one of those tests just adopted). + let _g = crate::arena::adopt_tests::lock(); + crate::options::set(15, 0); + let (pin, a, b, span) = adjacent_pair(); + let seg = segment_of(a); + // SAFETY: `a` is live. + let ia = unsafe { page_index(seg, page_of(seg, a)) }; + purge_right(b); + crate::options::set(15, -1); // purging off from here + // SAFETY: `a` is live and freed once; `collect` flushes the large + // cache, so its span reaches `span_free` and merges. + unsafe { free(a) }; + collect(true); + merged_span_is_backed(a, ia, span); + // SAFETY: live, freed once. + unsafe { free(pin) }; + } } diff --git a/crates/rusty_alloc_api/CHANGELOG.md b/crates/rusty_alloc_api/CHANGELOG.md index 339006c..a822243 100644 --- a/crates/rusty_alloc_api/CHANGELOG.md +++ b/crates/rusty_alloc_api/CHANGELOG.md @@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [2.2.6](https://github.com/Remade-With-Rust/rusty_alloc/compare/rusty_alloc-api-v2.2.5...rusty_alloc-api-v2.2.6) - 2026-10-09 + +### Fixed + +- Picks up `rusty_alloc` 2.2.6: a merged free span no longer hides a + decommitted neighbour (Windows, purging on). This crate's code is + unchanged. + ## [2.2.5](https://github.com/Remade-With-Rust/rusty_alloc/compare/rusty_alloc-api-v2.2.4...rusty_alloc-api-v2.2.5) - 2026-10-08 ### Fixed diff --git a/crates/rusty_alloc_api/Cargo.toml b/crates/rusty_alloc_api/Cargo.toml index 3626fee..7e0e20e 100644 --- a/crates/rusty_alloc_api/Cargo.toml +++ b/crates/rusty_alloc_api/Cargo.toml @@ -14,7 +14,7 @@ readme = "README.md" [dependencies] # A published crate needs a VERSION alongside the path: the path is what the # workspace builds against, the version is what crates.io resolves. -rusty_alloc = { path = "../rusty_alloc", version = "2.2.5", default-features = false } +rusty_alloc = { path = "../rusty_alloc", version = "2.2.6", default-features = false } [features] # Mirrors the core's (P3, docs/plans/small-metal.md). Default-on, so every diff --git a/docs/plans/recommit-failure-ignored.md b/docs/plans/recommit-failure-ignored.md index 15e83e5..ec0918c 100644 --- a/docs/plans/recommit-failure-ignored.md +++ b/docs/plans/recommit-failure-ignored.md @@ -1,8 +1,10 @@ # An unbacked page reached `page_extend` on Windows, and a re-commit whose failure is ignored -**Status:** one crash in a shipping consumer, **not reproduced**; mechanism A -**fixed in the working tree 2026-10-04, uncommitted** (section 8); which of the -two mechanisms caused the crash is still **open** · +**Status (2026-10-09):** mechanism A **fixed and released in 2.2.3** +(`e6475c1`; `mata-master` pins `=2.2.3`). Mechanism B **found, reproduced as +`STATUS_ACCESS_VIOLATION` and fixed** on branch `fix/purged-merge-flag`, not +yet released (section 9). Which one caused the field crash is still unknown; +the `secure` guard-page route is the one path not traced · **Date:** 2026-10-04 · **Seen on:** 1.1.6 (`secure` on, `purge_delay = 0`) · **Code checked against:** HEAD `86980ec` (2.2.2) · **Platform:** Windows 11, x86-64 · **From:** the MATA desktop session (`mata-master`, branch @@ -222,3 +224,51 @@ the `bench/alloc-eval` replays of a recorded Endless Sky battle in WSL (about exhaustion. That load very likely contributed to the 98.5 % commit charge the dump recorded, which makes A's precondition more plausible than section 4 assumed. It does not rule out B. + +## 9. Mechanism B, found (2026-10-09) + +**Where.** `span_recommit` is the only reader of the `purged` flag, and it +reads a free span's FIRST slot only. `span_free` merges a freed span with free +neighbours on both sides, then purges the WHOLE merged span and flags its +start, but only if it is at least `MEDIUM_PAGE_SLICES` long and purging is on. +When that re-purge does not happen, the merged span kept the flag of its first +part, which can be clear while a merged-in neighbour is decommitted. Two ways: + +1. **The re-purge fails** (`os::purge` returns an error) after a merge with a + purged neighbour. +2. **Purging was switched off at runtime** (`options::set`) after the + neighbour was purged, so the merge is not re-purged at all. + +Every other reuse path was checked and re-commits or never holds purged +memory: the first-fit split (it re-commits the whole free span before +splitting), the large-span cache, adoption's dead-span retirement, the 2.2.4 +huge purge (`restore_for_reuse`), and the heap-box cache. A stale TRUE flag on +an interior slot is harmless (a re-commit of committed memory). + +**Reproduced.** `segment::recommit_tests::a_failed_repurge_of_a_merge_keeps_the_span_purged` +(case 1, through a new test-only "fail the next N purges" hook beside the +commit one) and `a_merge_after_purging_is_turned_off_keeps_the_span_purged` +(case 2). With the fix disabled, both fail at the flag check. With the flag +check also skipped, writing the merged span on Windows ends the test process +with `0xc0000005 STATUS_ACCESS_VIOLATION`: the field crash's exception, from +the same kind of write. + +**Fixed.** `span_free` ORs the merged neighbours' flags and, when the merged +span is not re-purged, flags its start if any part was purged. Cost: about 2 +instructions per `span_free`. Opscan unchanged on every op; allocator Ir +perl +455 (0.002 %), Endless Sky +68. + +**Does it explain the field crash?** Possibly, not provably. The MATA app sets +purging once at start-up (no case 2), and case 1 needs a failed decommit, +which is rare. The `failed commits` counter still separates A (non-zero) from +the rest; a `MiniDumpWithFullMemoryInfo` dump would still settle it. + +**Test isolation.** The new tests set the process-wide `purge_delay`, as the +mechanism-A test does, so all three now hold the arena adoption tests' lock +(`arena::adopt_tests::lock`). Without it, `arena::adopt_tests::adjacent_adoption_extends_in_place` +and the new tests each failed about once in a dozen runs (a segment allocation +on another test thread can take a chunk from an arena that test just adopted). +With it, 30 consecutive runs (15 default, 15 `secure`) were clean. + +**Still open:** the guard-page route (`secure`, a guard left no-access), not +traced. diff --git a/tools/unsafe-baseline.txt b/tools/unsafe-baseline.txt index 02a3289..cb4d095 100644 --- a/tools/unsafe-baseline.txt +++ b/tools/unsafe-baseline.txt @@ -13,7 +13,7 @@ 6 crates/rusty_alloc/src/prim/wasm.rs 34 crates/rusty_alloc/src/prim/windows.rs 2 crates/rusty_alloc/src/random.rs - 45 crates/rusty_alloc/src/segment.rs + 55 crates/rusty_alloc/src/segment.rs 3 crates/rusty_alloc/src/stats.rs 16 crates/rusty_alloc_api/src/lib.rs 15 crates/rusty_alloc_bench/src/kernels.rs