perf(regex): bind a RegExp's program and subject in constant work across calls (#10166) - #10183
Closed
proggeramlug wants to merge 9 commits into
Closed
proggeramlug wants to merge 9 commits into
proggeramlug wants to merge 9 commits into
Conversation
added 3 commits
September 13, 2026 07:43
…10165) String split, replace and global match run a search at every position or match of one string with one matcher, but execute_with_resources bound both afresh for each search. Binding a subject decodes the entire string (perex Input::wtf8, uncharged) and binding a program revalidates every word, so each of those operations did O(n) binding work per search and O(n²) overall. On ASCII input this is why `str.split(/[,; ]+/)` took 8.1 s for a 150,000-unit string. The three loops already held a BoundSubject over their input. A new perex_api::Reuse carries it, plus a program binding taken from the receiver before the loop, into execute_with_resources. Perex's binding contract allows a binding to outlive allocation, collection and JS callbacks: Perry's owners hold registered roots and reacquire their base on every view. A search uses a reused binding only while it is provably the same object (same string; same receiver still holding the same program cell); otherwise it binds afresh exactly as before, which covers an exec override, RegExp.prototype.compile and a different string. Reuse is built before each loop because runtime handle scopes are a stack. matchAll's next(), JS-level exec/test and search are one search per JS call and are unchanged; cross-call reuse is a separate contract. The non-ASCII seek charge behind the work-limit RangeError (#10164) is on the Perex side and needs its search-from-position API; this change does not affect it. Tests: gc::tests::runtime_roots::perex_reuse covers a whole global loop with a minor collection at every poll under forced evacuation (subject and program cell both relocate; fresh work equals reused work plus six program validations, proving reuse engages), a recompile between searches, and a different string. Each test fails when its guard is sabotaged. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9
…10165) RegExp.prototype[@@split] tries a sticky match at every position q. Each attempt starts a whole search, so split pays a search's fixed setup per subject unit: about 27.6 work units per unit for `/[,; ]+/`, against about 9 for a global exec loop over the same subject. A non-sticky search from q returns the leftmost position s >= q where the pattern matches, with the same match a sticky attempt at s finds. The attempts at q..s-1 can therefore be skipped without changing any piece or capture, and empty matches and Unicode advancement line up. Skipping them is unobservable only when nothing can see a RegExpExec: - the species is absent or the intrinsic RegExp (recognised by its call thunk), so the splitter is a fresh object no user code holds and its skipped lastIndex writes cannot be seen; a user species could return a real RegExp and read lastIndex afterwards; - the splitter's exec resolves, without running a getter, to the builtin data property (regexp_view_uses_builtin), so the skipped Get(exec) calls cannot be seen either. When both hold, split compiles a program from the splitter's own internal source and canonical flags without `y` (never from the receiver, whose program a limit valueOf could replace via RegExp.prototype.compile after the splitter was built) and searches forward with it, reusing the operation's subject binding. Anything else runs the unchanged per-position sticky loop. Tests (gc::tests::runtime_roots::perex_split): - forward search matches ten results derived by hand from the sticky algorithm (repeated and unmatched captures, empty matches, `$` at the end, limits inside captures, Unicode empty-match advancement, the non-ASCII #10164 record), each asserted to take the forward path; - a user species returning a real RegExp keeps the sticky loop and leaves the splitter's lastIndex at 2, as the specification requires; - the existing species-factory and custom-exec tests now also assert the sticky loop ran. Sabotage: admitting any species fails the user-species test; trying the end of the input or dropping captures fails the forward-search test. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9
|
Warning Review limit reachedNext included review available in 12 seconds. View limit detailsLimit details: You’ve used all 8 included reviews currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (26)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
proggeramlug
pushed a commit
that referenced
this pull request
Sep 13, 2026
added 6 commits
September 13, 2026 10:46
gc_runtime_root_holders.py flags the new #[cfg(test)] FORWARD_SPLITS Cell<usize> under rule B. It is a test-only count, never an address. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9
…tion (#10164) On non-ASCII (byte) storage a Perex search seeks to its start from the nearer end of the subject, up to half its length. A global replace or match starts a search per match, and split one per match or position, so the seeks summed to about n²/4: at 32,000 units that exceeded the former 100,000,000-unit allowance and threw, and without a cap it is quadratic time. Materializing captures seeked from an end the same way. Perex 0.1.2 adds a search-from-position API: Search::new_near and Search::position (the match end, or the start of the last attempt), and BoundSpan::new_near. perex_runtime::find_near and perex_strings::copy_span_near take an optional position; find and copy_span delegate to them with none. - perex_api::Reuse carries the last position of the reused subject, set only from and used only with that binding, because a position from another string with the same layout cannot be detected. execute_with_resources seeds each search and its capture materialization from it; global match copies each result from its search's end. - Split's forward search seeds each search from the previous one and materializes captures from the search's position. Tests: - perex_reuse: a non-ASCII global loop's work roughly doubles when the subject doubles (3.71x without positions); the #10164 reduction, a 32,000-unit split (6,001 pieces) and a 60,000-unit global replace (76,000 units), completes. - perex_split: a non-ASCII forward split's work roughly doubles when the input doubles (3.97x when it never resumes). Sabotage: each of those fails when positions are not used. Requires perex 0.1.2, published 2026-09-13 from PerryTS/perex d9f395d88931f0cfee1fe89ddf455cda606fd9e1 (crates.io checksum 21df239ee18f99de6abff50953f6f15be1b5ebd11e6ae9661acdd93026e983db). It is inside the workspace's 7-day min-publish-age window, so Cargo.lock was resolved once with CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE=allow, as for perex 0.1.0, with the maintainer's approval. Ordinary --locked builds use the locked version without the override. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9
gc_runtime_root_holders.py flags the new #[cfg(test)] LAST_FORWARD_WORK Cell<usize> under rule B. It is a test-only work count, never an address. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9
…oss calls (#10166) A search per JavaScript call (an exec or test loop, matchAll's next(), search) bound its subject and program from scratch every time: binding a subject decodes the entire string and binding a program revalidates every word. A loop over one string therefore did O(n) binding work per call and O(n²) overall, and `.test()` paid a program validation per call. Perex 0.1.3 adds constant-work rebinding for what the host already validated: BoundSubject::new_counted(storage, utf16_len) and a ProgramWitness from BoundProgram::witness() that BoundProgram::new_witnessed checks against the program's length and header. Perry now keeps both, as plain data, with no allocation and nothing new traced: - Program: ProgramCell gains `witness: Option<ProgramWitness>` in its pointer-free prefix, beside the words it describes. The first validating bind records it; later binds use new_witnessed, falling back to validation (recording a fresh witness) if it does not match. A recompile emits a new cell that starts with none, so a witness can never describe other words. RegExpHeader stays 56 bytes. Debug builds assert no witness is written while a view of the cell's words is live. - Subject: StringHeader gains STRING_FLAG_WTF8_VALIDATED. Perry strings are not all valid WTF-8 (raw Buffer/FFI payloads reach regex operations), so nothing is assumed: the first bind decodes, and only if that succeeds and the decoded UTF-16 length equals the header's is the header marked; later binds use new_counted. A validated header is already shared, so its payload is never mutated in place. init_string_header strips the bit from every constructed string, and the in-place writers (js_string_append, js_string_append_chain) clear it, so it never reaches another string; concat's memo check ignores it. Deliberately excluded: a cross-call lastIndex position hint for non-ASCII exec loops. Recognising the same string across calls without a traced reference would need a heap generation counter bumped on every free and move path, and one missed path gives silent wrong answers. The remaining cost is one seek from the nearer end of the subject per call on non-ASCII subjects; it is charged but uncapped (#10176), so those loops finish, but are not linear. Tests: - string::tests_validated_flag: slice, substring, trim, concat, repeat, padStart, toUpperCase, join, string_copy_range and js_string_from_bytes_known_utf16 (passed the source's whole flags word) never inherit the bit; both in-place append paths clear it. - gc::tests::runtime_roots::perex_cross_call: a program cell records its witness on first bind and a recompiled program's new cell has none; a foreign equal-header witness (x(b) on x(a)) still answers with the cell's own words; a mismatched witness falls back and is replaced; a subject is marked only with an exact length, never with a corrupted utf16_len or malformed bytes; marks survive moving collections; a witness write under a live view is caught. - perex_reuse's accounting now expects one validation per program. Fault injection, each confirmed to fail its test: marking without the length check; never recording a witness; a mismatch without fallback; removing the view guard; keeping the whole flags word in init_string_header; keeping the bit in the in-place writers. Requires perex 0.1.3 (crates.io checksum 060b4682849d20ebcba05d68f9584a1bba20af4b7838c688cfb37af572f562f4), resolved once with CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE=allow with the maintainer's approval; ordinary --locked builds use it without the override. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9
proggeramlug
force-pushed
the
perf/10166-regex-cross-call
branch
from
September 13, 2026 10:47
98b20b3 to
cf63ad1
Compare
This was referenced Sep 13, 2026
Contributor
Author
MeasurementsThese are release builds from source on perrymaster (Linux x86_64, 16 cores, Node 26.8.1): Method: three alternating rounds, one arm after the other, reporting the median. The host was shared (load average 11–30 during the runs), so absolute times are noisy. The signal is how each row scales with
Short-string per-call cost (instruction counts)Load-independent:
No regression on short subjects. An early wall-clock round suggested one; the counts show it was host load. A remaining quadratic outside this PRA class quantifier followed by required text is still O(lastIndex) per search on ASCII subjects. For example, Full-suite and lint replay (
|
This was referenced Sep 13, 2026
proggeramlug
pushed a commit
that referenced
this pull request
Sep 13, 2026
This was referenced Sep 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the cross-call part of #10166 and #10165. Stacked on #10181 (and so on #10174): please review only the top commit,
perf(regex): bind a RegExp's program and subject in constant work across calls.Problem
A search per JavaScript call bound its subject and program from scratch every time. That covers a JS-level
execortestloop,matchAll'snext(), andsearch.Input::wtf8, uncharged).A loop over one string therefore did O(n) binding work per call and O(n²) overall, and
.test()paid a full program validation on every call. #10174 fixed this within one operation (split/replace/global match); this PR fixes it across calls.Change
Perex 0.1.3 adds constant-work rebinding for what the host already validated:
BoundSubject::new_counted(storage, utf16_len), and aProgramWitnessthatBoundProgram::new_witnessedchecks against the program's length and header. Perry keeps both as plain data, with no allocation and nothing new for the collector to trace.Program witness, in the program cell.
ProgramCellgainswitness: Option<ProgramWitness>in its pointer-free prefix, beside the words it describes.new_witnessed, and a mismatch falls back to validation, which records a fresh witness.RegExpHeader, which stays at the 56 bytes pinned byregexp_header_is_one_56_byte_per_object_record. That header is allocated per regex-literal evaluation; the cell is shared per compiled program.Subject, validated once per string.
StringHeadergainsSTRING_FLAG_WTF8_VALIDATED. Perry strings are not all valid WTF-8 — raw Buffer/FFI payloads reach regex operations — so validity is never assumed.new_counted.js_string_addref), so its payload is never mutated in place.init_string_headerstrips it from every constructed string (a single funnel, however a constructor computesflags), and the in-place writers (js_string_append,js_string_append_chain) clear it. Concat's memoization check ignores it.STRING_FLAG_JSON_ESCAPE_FREEpasses through constructors exactly as before.Deliberately excluded: a cross-call
lastIndexposition hintA non-ASCII JS-level
execloop could also resume each call's seek from the previous call's match end. Recognising the same string across calls without a traced reference would need a heap generation counter bumped on every free and move path. Frees and moves are spread across the old-gen sweep, incremental reclaim, malloc-tracked frees, emergency reclaim, copying minors and compaction, and one missed path gives silent wrong answers. The maintainer and the Perex owner agreed to leave it out.Remaining fallback: on non-ASCII subjects each call still seeks from the nearer end of the string once. That seek is charged but uncapped (#10176), so such loops finish, but they are not linear. ASCII subjects seek in constant work.
Tests
string::tests_validated_flag: slice, substring, trim, concat, repeat, padStart, toUpperCase, join,string_copy_rangeandjs_string_from_bytes_known_utf16(passed the source's whole flags word) never inherit the bit; both in-place append paths clear it.gc::tests::runtime_roots::perex_cross_call:x(b)'s onx(a), equality asserted first) still answers with the cell's own words;b's ona) falls back and is replaced;utf16_len, never with malformed bytes, which still error as before;perex_reuse: accounting now expects one validation per program.init_string_header;gc::tests::runtime_roots::perex*tests, all 71regex::tests (including the 56-byte header ratchet) and all 149string::tests pass, and rustfmt is clean.--no-default-features --features full).Dependency
perex = "0.1.3", published 2026-09-13 from PerryTS/perex8eaa3b0, crates.io checksum060b4682849d20ebcba05d68f9584a1bba20af4b7838c688cfb37af572f562f4.Cargo.lockwas resolved once withCARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE=allow, with the maintainer's approval for this version; ordinary--lockedbuilds use it without the override.Measurements and the full-suite / lint-gate replay follow in comments.
https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9