perf(regex): resume JS-level searches on non-ASCII strings from the previous call's position

[proggeramlug]

Part of #10164 and #10165. A JavaScript-level exec / test / search / matchAll loop over one non-ASCII string now resumes each search from where the previous call's search stopped, instead of seeking from an end of the string.

Problem

Within one compound operation (split, replace, global match), #10181 already carries the search position. A JS-level loop instead runs one search per call and binds its subject afresh each time (#10183). On a non-ASCII (WTF-8) string, a search that starts at lastIndex without a position pays a seek from the nearer end, so a loop over one string does quadratic work. At n = 40,000 records, the #10183 measurements recorded 42–44 s for these loops, against 7–8 ms on Node. The reruns against the pre-Perex revision on #10164 and #10165 show both non-ASCII workloads timing out at 100k.

Change

Position table (regex/perex_position_hint.rs): a per-thread table of four plain-data entries. Each entry holds a concealed address, a heap generation, byte and UTF-16 lengths, and a perex::input::Position.

• execute_with_resources looks a position up for a freshly bound non-ASCII subject and records the search's final position afterwards. The identity is re-read after the search, because a collection during it may have moved the string.
• The identity comes from the same header read that binding already does.
• ASCII strings never consult the table; they seek in constant work.
• Home, and why: a RegExp gains no state, so RegExpHeader stays one 56-byte record. The collector has nothing new to scan and no traced edge, and the address is compared for equality only, never read as a pointer.
• Keying by string rather than by RegExp also serves loops that alternate regexps over one string.

Same string, decided without per-object state: the address, both lengths and the thread's heap generation must all match.

• An unchanged generation means nothing at that address was freed (so no other string can have taken it) and the string did not move.
• The lengths reject an in-place append, the only way a live string's bytes change.
• A wrong position could only give wrong answers, never unsafety; that is the Position contract, and Perex still refuses a mismatched layout.

Heap generation (gc/heap_generation.rs): HeapChange::begin(kind) advances the generation when the scope opens and again when it closes. Opening covers addresses recorded before the event; closing covers any recorded by a JS callback during it. Scopes may nest.

Scopes:

kind	where
CopyingMinor	run_copied_minor_attempt
Sweep	the Sweep arm of GcCycleState::step, and IncrementalSweepState::finish_unbounded (synchronous sweeps)
Reclaim	the Reclaim arm of GcCycleState::step
Evacuation	atomic_finalize_minor_prelude's evacuation branch, which includes forwarding-stub release
Compaction	nested around evacuate_selected_old_pages_collecting
Promotion	finish_in_place_promotion
Realloc	gc_realloc

Funnel enforcement: debug builds panic if a primitive that makes object memory reusable, or evacuates a young object, runs with no scope open. The asserted primitives:

• the arena region and block resets (reset_region_to_zero, copying_reset_from_spaces_and_flip, arena_reset_empty_blocks and its incremental step, survivor and old dead-block reclaim, the free-list filter);
• the promotion young reset;
• old_free_push;
• the malloc sweep's dealloc;
• move_young;
• release_evacuated_original_forwarding_stubs.

The first full debug run with the assertions in place found 41 hits, every one a test calling a primitive directly. Every production path was already inside a scope. Those tests now open a scope.

Not asserted, and why:

• Old-generation evacuation functions: a relocation leaves a forwarding stub, and the stub's memory is freed only by the scoped, asserted release or sweep. The events that call them are scoped. gc/oldgen.rs is also at the 2,000-line cap.
• Mutator-side forwarding (array growth, async frames): the old address stays occupied by a stub until a collection.
• Recycling an already-empty block (block pool, from-space quarantine, an arena dropped at thread exit): the objects there were freed by an event that already advanced the generation.

scripts/gc_runtime_root_holders.json records the #[cfg(test)] HINT_USES counter as test_only. It also re-audits the PASS1_MARKED census window for the gc/cycle.rs and gc/mod.rs hunks. The Sweep scope opens before step_sweep takes the snapshot, and only increments two thread-local integers.

Tests

• gc::tests::heap_generation, one test per kind through its production entry point: copying minor, in-place promotion, full sweep, emergency full, malloc free, incremental reclaim, minor-prelude evacuation, old-page compaction, moving realloc. Each asserts that the generation advanced and that its own kind's scope opened, so removing a nested scope still fails.
• Funnel check: the funnel assertion must fire with no scope open.
• perex_position_hint::cross_call_positions_keep_a_js_level_non_ascii_loop_linear: work at 2n/n is below 2.2× with positions (and positions were used) and above 3.0× with positions disabled.
• a_moved_string_does_not_reuse_its_position: a copying minor moves the string, and the next call does not use the old position.
• another_string_at_the_same_address_after_a_free_does_not_use_the_position: a string with the same byte and UTF-16 lengths but a different arrangement is allocated at the freed string's address; the address equality is asserted as a precondition. It must not use the position, and its match must be correct.
• perex_reuse_positions_keep_a_non_ascii_global_loop_linear's unpositioned control now disables cross-call positions too.

Fault injections

Each ran as one cargo test, with the source restored byte-identical afterwards. All 13 FAILED, as required.

injection	failing test
remove the CopyingMinor scope	copying minor
remove the Promotion scope	in-place promotion
remove the cycle Sweep scope	full sweep, emergency full, malloc free (3 runs)
remove the Reclaim scope	incremental reclaim
remove the Evacuation scope	evacuation
remove the Compaction scope	compaction
remove the Realloc scope	realloc
disable the funnel assertion	funnel check
identity ignores the generation	wrong string at the same address
identity ignores address and generation	moved string
positions never looked up	loop linearity

Measurements (perrymaster, release builds, main 5d3bf85f9 vs this branch)

The #10183 reproducer at n = 40,000 is a single run on a shared host; results match Node.

loop	main	this PR	Node
re.exec loop, /([ä中Ö漢]+)([0-9]+)/gu over "ä中12 Ö漢345😀".repeat(n)	28,486 ms	314 ms	6.6 ms
matchAll, same subject	26,899 ms	463 ms	7.8 ms

Both loops are now linear: 43 → 90 → 314 ms at 10k / 20k / 40k for exec, against main's 1,647 → 6,480 → 28,486 ms. They are still 40–60× Node, which is the per-call cost tracked in #10166.

ASCII per-call cost, measured as perf stat -e instructions:u over 1,000,000 short-string calls minus the build-only program (the #10166 probes, 2 runs per arm):

probe	main	this PR	added per call
hoisted re.test(v)	23,694	23,723 / 23,749	+29 / +55 (0.1–0.2 %)
re.exec(v) with captures	40,150	40,185 / 40,185	+34 / +33
/_[0-9]+/g test with lastIndex = 0	26,671	26,698 / 26,713	+27 / +42

Validation (perrymaster, --locked)

• cargo fmt --all -- --check
• cargo check -p perry-runtime --no-default-features --features full --lib (regex feature off): 0 warnings
• cargo check -p perry-runtime --lib --tests: no warnings outside the known global_this_webassembly.rs dead code
• cargo test -p perry-runtime --lib -- --test-threads=1: 3724 passed, 1 failed. The failure is native_stack::tests::stack_top_respects_custom_thread_stack_sizes, red on main.
• cargo clippy -p perry-runtime --lib --tests: identical to main (train 181), with no new warnings
• scripts/run_lint_gates.sh script tier: 76 of 77 pass. The failure is public benchmark evidence freshness, identical on main. Also checked: gc_runtime_root_holders.py, check_file_size.sh.

https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

[coderabbitai]

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/regex-cross-call-position

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[proggeramlug]

Landed on main via merge train 182 (#10206) as 7a2788d1ab..20f6fe3207, with the version bump 0956673b5e (0.5.1555). Closing, since this landed through the train.

https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

[proggeramlug]

Heads-up from a merge-train validation on current main (26ed55cb74, with this PR landed): RUST_TEST_THREADS=1 cargo test --release -p perry-runtime fails one test this PR added:

gc::tests::heap_generation::a_free_or_move_outside_every_scope_is_caught_in_debug_builds ... FAILED
panicked at crates/perry-runtime/src/gc/tests/heap_generation.rs:285:5:
the funnel assertion must fire with no scope open

The test expects debug_assert_heap_change_open() to panic, but it isn't gated on debug_assertions, so under --release the assertion is compiled out and catch_unwind sees no panic. 3763 other runtime tests pass on that tree. #[cfg(debug_assertions)] on the test, or if !cfg!(debug_assertions) { return; } at its top, would match its name.