fix(regex): do not cap RegExp operations by work (#10164) by proggeramlug · Pull Request #10176 · PerryTS/perry

proggeramlug · 2026-09-13T08:18:14Z

Fixes the RangeError half of #10164.

Problem

Every RegExp operation ran under one fixed Budget of 100,000,000 work units (perex_api::WORK), so valid programs that Node completes threw RangeError: Regular expression work limit exceeded:

a 32,000-unit non-ASCII split and a 60,000-unit global replace — the per-search seek charge summed to about n²/4 (the cost placement is on bug(regex): split and replace throw "Regular expression work limit exceeded" on 32,000-unit strings Node handles in under a millisecond #10164);
once perf(regex): bind once per split/replace/match, and split searches forward (#10165) #10174 makes split and replace linear, linear work too: at n = 1,000,000 records the benchmark split (15M units) and replace (11M units) reach the end of the input and throw, where Node takes 240 and 281 ms.

Why no cap

No finite allowance separates valid programs from pathological ones. Perex's charge per subject unit is set by the program, not the subject:

linear pattern	work units per subject unit
`/x/g`, no match	1.0
`/\w+/g`	9.4
split `/[,; ]+/`, sticky attempt per position	27.6
`/([a-z]+)([0-9]+)/g`	59.2
`/\w(?=[\w,; ]{32})/g`	223.6

Any cap throws on some large linear input, and a quadratic pattern on a short subject never reaches it. JavaScript engines never abort matching for work. The Perex maintainer set out three options — no cap, a length-scaled cap, or a wall-clock/cancellation guard — and the maintainer chose no cap, for parity.

Change

WORK becomes usize::MAX, with the reasoning documented on the constant.

Searches still run in QUANTUM slices with a GC poll between them, so collection and cancellation keep working.
The scratch, program and output memory limits are unchanged.
A catastrophic pattern now runs as long as it does in Node instead of throwing.

The existing tests that exercise ExecError::WorkLimit all pass their own small budgets (0, 3, 37, …), so they still cover the error mapping and accounting.

Test

gc::tests::runtime_roots::perex_work_policy runs one valid, linear search that charges about 1.2e8 units and asserts that it completes and charged more than the former limit. The search is a failing 32-unit lookahead at every position of 540,000 units.

The pattern deliberately has no required literal. My first candidate, \w(?=\w{32}!), charged only 600,040 units, because admission rejects a search whose required ! never occurs in one pass. So a witness has to defeat that shortcut to prove anything.

Sabotage: with the old 100,000,000 cap restored, the test fails with "a valid search must complete without a work-limit error".
All 78 gc::tests::runtime_roots::perex* tests and all 71 regex:: unit tests pass, and rustfmt is clean.
Verified locally (GitHub runners are not in use).

perf(regex): bind once per split/replace/match, and split searches forward (#10165) #10174 — binds the subject and program once per split/replace/match, plus a forward-search split. That makes those operations linear, and this PR lets linear work of any size complete.
The quadratic time half of bug(regex): split and replace throw "Regular expression work limit exceeded" on 32,000-unit strings Node handles in under a millisecond #10164 (the per-search seek on non-ASCII storage) needs Perex 0.1.2's search-from-position API. That wiring waits for perex 0.1.2 to clear Perry's 7-day publish-age window.

https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

Summary by CodeRabbit

Bug Fixes
- Regular expression operations can now process valid workloads exceeding the previous work limit without being prematurely aborted.
- Large, valid searches continue to respect memory safeguards, cancellation, and runtime scheduling while completing successfully.
- Repeated split, replace, and global-match operations on the same input can run more efficiently.

coderabbitai · 2026-09-13T08:18:45Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: ab6ba07e-d378-4849-9b66-1b9feb714901

📥 Commits

Reviewing files that changed from the base of the PR and between c61d617 and b895f32.

📒 Files selected for processing (3)

crates/perry-runtime/src/gc/tests/runtime_roots.rs
crates/perry-runtime/src/gc/tests/runtime_roots/perex_work_policy.rs
crates/perry-runtime/src/regex/perex_api.rs

Included review availability: Your plan provides up to 8 included reviews per hour; 4 remain after this review.

📝 Walkthrough

Walkthrough

The regex work allowance changes to usize::MAX. The API can reuse bound programs and subjects for eligible operations. Memory limits and quantum-based GC polling remain unchanged. A feature-gated runtime-root test verifies a large linear search.

Changes

Regex work policy

Layer / File(s)	Summary
Work allowance and binding reuse `crates/perry-runtime/src/regex/perex_api.rs`	The regex work allowance changes from `100_000_000` to `usize::MAX`. `execute_with_resources` accepts optional reuse state for program and subject bindings, while retaining fresh binding behavior when reuse is unavailable.
Runtime-root regression coverage `crates/perry-runtime/src/gc/tests/runtime_roots.rs`, `crates/perry-runtime/src/gc/tests/runtime_roots/perex_work_policy.rs`, `changelog.d/10176-regex-no-work-cap.md`	The feature-gated test executes an expensive linear no-match regular expression and verifies successful completion with work usage above the former limit. The changelog records the removed cap and retained memory and polling behavior.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant RuntimeRootTest
  participant PerexAPI
  participant GC
  RuntimeRootTest->>PerexAPI: Execute regex with resource budgets
  PerexAPI->>GC: Poll during QUANTUM slices
  PerexAPI-->>RuntimeRootTest: Return no-match result and charged work

Merge Risk: ⚪ Minimal · up to b895f

The feature-disabled build path excludes the Perex-specific test module, so no actionable merge risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly states that RegExp operations no longer use the fixed work cap. It matches the primary change.
Description check	✅ Passed	The description explains the problem, implementation, rationale, testing, and related issues. It does not use the repository template headings and omits the checklist, but it contains the required tec…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/10164-regex-no-work-cap

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

proggeramlug · 2026-09-13T08:53:32Z

Full perry-runtime lib suite, replayed locally at `f9073c461`

GitHub runners are not in use, so this ran on an Ubuntu x86_64 build host with nightly-2026-08-20 and LLVM 22.1.8: cargo test -p perry-runtime --lib -- --test-threads=1.

3,678 passed, 1 failed, 4 ignored.

The single failure is native_stack::tests::stack_top_respects_custom_thread_stack_sizes (native_stack.rs:53, "bound must belong to this worker"). It is red on main itself: reproduced identically on origin/main 50e08e9, as recorded on #10115, and main's own CI reports it too. Nothing in this PR touches native_stack.rs.

https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

…oss calls (#10166) A search per JavaScript call (an exec or test loop, matchAll's next(), search) bound its subject and program from scratch every time: binding a subject decodes the entire string and binding a program revalidates every word. A loop over one string therefore did O(n) binding work per call and O(n²) overall, and `.test()` paid a program validation per call. Perex 0.1.3 adds constant-work rebinding for what the host already validated: BoundSubject::new_counted(storage, utf16_len) and a ProgramWitness from BoundProgram::witness() that BoundProgram::new_witnessed checks against the program's length and header. Perry now keeps both, as plain data, with no allocation and nothing new traced: - Program: ProgramCell gains `witness: Option<ProgramWitness>` in its pointer-free prefix, beside the words it describes. The first validating bind records it; later binds use new_witnessed, falling back to validation (recording a fresh witness) if it does not match. A recompile emits a new cell that starts with none, so a witness can never describe other words. RegExpHeader stays 56 bytes. Debug builds assert no witness is written while a view of the cell's words is live. - Subject: StringHeader gains STRING_FLAG_WTF8_VALIDATED. Perry strings are not all valid WTF-8 (raw Buffer/FFI payloads reach regex operations), so nothing is assumed: the first bind decodes, and only if that succeeds and the decoded UTF-16 length equals the header's is the header marked; later binds use new_counted. A validated header is already shared, so its payload is never mutated in place. init_string_header strips the bit from every constructed string, and the in-place writers (js_string_append, js_string_append_chain) clear it, so it never reaches another string; concat's memo check ignores it. Deliberately excluded: a cross-call lastIndex position hint for non-ASCII exec loops. Recognising the same string across calls without a traced reference would need a heap generation counter bumped on every free and move path, and one missed path gives silent wrong answers. The remaining cost is one seek from the nearer end of the subject per call on non-ASCII subjects; it is charged but uncapped (#10176), so those loops finish, but are not linear. Tests: - string::tests_validated_flag: slice, substring, trim, concat, repeat, padStart, toUpperCase, join, string_copy_range and js_string_from_bytes_known_utf16 (passed the source's whole flags word) never inherit the bit; both in-place append paths clear it. - gc::tests::runtime_roots::perex_cross_call: a program cell records its witness on first bind and a recompiled program's new cell has none; a foreign equal-header witness (x(b) on x(a)) still answers with the cell's own words; a mismatched witness falls back and is replaced; a subject is marked only with an exact length, never with a corrupted utf16_len or malformed bytes; marks survive moving collections; a witness write under a live view is caught. - perex_reuse's accounting now expects one validation per program. Fault injection, each confirmed to fail its test: marking without the length check; never recording a witness; a mismatch without fallback; removing the view guard; keeping the whole flags word in init_string_header; keeping the bit in the in-place writers. Requires perex 0.1.3 (crates.io checksum 060b4682849d20ebcba05d68f9584a1bba20af4b7838c688cfb37af572f562f4), resolved once with CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE=allow with the maintainer's approval; ordinary --locked builds use it without the override. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

proggeramlug · 2026-09-13T11:13:32Z

Heads-up for landing this on current main (6874a9eb73, which now has #10174): the new test crates/perry-runtime/src/gc/tests/runtime_roots/perex_work_policy.rs calls api::execute_with_resources with 6 arguments, but #10174 added a 7th reuse: Option<&Reuse> parameter, so perry-runtime's lib test target stops compiling:

error[E0061]: this function takes 7 arguments but 6 arguments were supplied
   --> crates/perry-runtime/src/gc/tests/runtime_roots/perex_work_policy.rs:41:13
    = argument #7 of type `Option<&Reuse<'_, '_>>` is missing

The fix is one line, passing None after &mut || Ok(()),. The two PRs compile on their own, but not together. I hit this assembling a combined train and have dropped #10176/#10181 from my train, since your lane is landing the regex stack ("Next in order: #10176, #10181, #10183" on #10187).

Every RegExp operation ran under one fixed Budget of 100,000,000 work units. Valid programs Node completes threw `RangeError: Regular expression work limit exceeded`: a 32,000-unit non-ASCII split and a 60,000-unit global replace (from the per-search seek charge), and after #10165's fixes even linear splits and replaces of 11-15 million units. No finite allowance separates valid programs from pathological ones. Perex charges per subject unit an amount set by the program, not the subject: about 1 for `/x/`, 9 for `/\w+/g`, 60 for `/([a-z]+)([0-9]+)/g`, over 200 for a 32-unit lookahead. Any cap therefore throws on some large linear input, while a quadratic pattern on a short subject never reaches it. JavaScript engines never abort matching for work. WORK becomes usize::MAX. Searches still run in QUANTUM slices with a GC poll between them, so collection and cancellation keep working, and the scratch, program and output memory limits are unchanged. A catastrophic pattern now runs as long as it does in Node instead of throwing. The existing tests that exercise ExecError::WorkLimit all pass their own small budgets, so they still cover the error mapping and accounting. Test: gc::tests::runtime_roots::perex_work_policy runs one valid, linear search that charges about 1.2e8 units (a failing 32-unit lookahead at every position, with no required literal that admission could reject up front) and asserts it completes and charges more than the former limit. It fails when the old 100,000,000 cap is restored. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

proggeramlug · 2026-09-13T11:18:27Z

Rebased onto main at 6874a9eb73, now that #10174 has landed (train 177, #10187), and pushed as b895f32e5.

Both PRs merged cleanly as text, but the combined tree did not compile. This PR's perex_work_policy test calls api::execute_with_resources with 6 arguments, and #10174 added a seventh reuse: Option<&Reuse> parameter. The fix passes None there, which is what a standalone call already means. No runtime code changed.

Re-verified on the rebased head (perrymaster, --locked, no publish-age override):

fmt; cargo check -p perry-runtime --lib --tests: no warnings outside the known global_this_webassembly.rs dead code
gc::tests::runtime_roots::perex: 83 passed; regex::: 71 passed
Fault injection: restoring WORK = 100_000_000 fails perex_operation_allowance_admits_linear_work_beyond_the_former_limit. The source was restored byte-identical afterwards.

Before the rebase, the full replay of the old head c61d617cb matched main exactly: perry-runtime --lib 3678 passed, the only failure being native_stack::tests::stack_top_respects_custom_thread_stack_sizes, which is red on main; lint script tier 76/77 with the same public-baseline freshness failure as main; regex-off check with 0 warnings. The merge train revalidates the combined tree before landing.

https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

proggeramlug · 2026-09-13T11:38:10Z

Landed on main via merge train 178 (#10192) as 0ec7ee1a81..ca1f65deaa, with the version bump df886c6445 (0.5.1550). Validation of the combined tree is in #10192. Closing, since this landed through the train.

https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

…oss calls (#10166) A search per JavaScript call (an exec or test loop, matchAll's next(), search) bound its subject and program from scratch every time: binding a subject decodes the entire string and binding a program revalidates every word. A loop over one string therefore did O(n) binding work per call and O(n²) overall, and `.test()` paid a program validation per call. Perex 0.1.3 adds constant-work rebinding for what the host already validated: BoundSubject::new_counted(storage, utf16_len) and a ProgramWitness from BoundProgram::witness() that BoundProgram::new_witnessed checks against the program's length and header. Perry now keeps both, as plain data, with no allocation and nothing new traced: - Program: ProgramCell gains `witness: Option<ProgramWitness>` in its pointer-free prefix, beside the words it describes. The first validating bind records it; later binds use new_witnessed, falling back to validation (recording a fresh witness) if it does not match. A recompile emits a new cell that starts with none, so a witness can never describe other words. RegExpHeader stays 56 bytes. Debug builds assert no witness is written while a view of the cell's words is live. - Subject: StringHeader gains STRING_FLAG_WTF8_VALIDATED. Perry strings are not all valid WTF-8 (raw Buffer/FFI payloads reach regex operations), so nothing is assumed: the first bind decodes, and only if that succeeds and the decoded UTF-16 length equals the header's is the header marked; later binds use new_counted. A validated header is already shared, so its payload is never mutated in place. init_string_header strips the bit from every constructed string, and the in-place writers (js_string_append, js_string_append_chain) clear it, so it never reaches another string; concat's memo check ignores it. Deliberately excluded: a cross-call lastIndex position hint for non-ASCII exec loops. Recognising the same string across calls without a traced reference would need a heap generation counter bumped on every free and move path, and one missed path gives silent wrong answers. The remaining cost is one seek from the nearer end of the subject per call on non-ASCII subjects; it is charged but uncapped (#10176), so those loops finish, but are not linear. Tests: - string::tests_validated_flag: slice, substring, trim, concat, repeat, padStart, toUpperCase, join, string_copy_range and js_string_from_bytes_known_utf16 (passed the source's whole flags word) never inherit the bit; both in-place append paths clear it. - gc::tests::runtime_roots::perex_cross_call: a program cell records its witness on first bind and a recompiled program's new cell has none; a foreign equal-header witness (x(b) on x(a)) still answers with the cell's own words; a mismatched witness falls back and is replaced; a subject is marked only with an exact length, never with a corrupted utf16_len or malformed bytes; marks survive moving collections; a witness write under a live view is caught. - perex_reuse's accounting now expects one validation per program. Fault injection, each confirmed to fail its test: marking without the length check; never recording a witness; a mismatch without fallback; removing the view guard; keeping the whole flags word in init_string_header; keeping the bit in the in-place writers. Requires perex 0.1.3 (crates.io checksum 060b4682849d20ebcba05d68f9584a1bba20af4b7838c688cfb37af572f562f4), resolved once with CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE=allow with the maintainer's approval; ordinary --locked builds use it without the override. Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

…h limit (#10164) `String.prototype.replace`, `replaceAll` and `split` threw "Regular expression memory limit exceeded" once an operation's list reached 8,388,608 entries: `List::push` refused past `SCRATCH_BYTES / 8`. A replacement stores three entries per output piece and a callback replace appends two pieces per match, so `'ab12 cd345;'.repeat(n).replace(/[0-9]+/g, cb)` threw from exactly 699,050 records, where Node finishes in about 235 ms. That limit is for native scratch buffers; these lists are ordinary GC arrays whose length follows the subject. The cap is removed. The lists are bounded by what can be allocated, and output by the existing maximum-string-length check on piece units. The work budget has been unlimited since #10176 for the same reason. Test: a `replaceAll` over one match more than the former cap allows now produces the complete output; with the cap restored the same test aborts on the thrown error. Claude-Session: https://claude.ai/code/session_01RJkA4Fhqz9J5F5fzDk5HWv

proggeramlug mentioned this pull request Sep 13, 2026

perf(regex): bind once per split/replace/match, and split searches forward (#10165) #10174

Closed

proggeramlug pushed a commit that referenced this pull request Sep 13, 2026

changelog: add fragment for #10176

c61d617

Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

proggeramlug mentioned this pull request Sep 13, 2026

perf(regex): bind a RegExp's program and subject in constant work across calls (#10166) #10183

Closed

proggeramlug mentioned this pull request Sep 13, 2026

Merge train 177: #10174 #10187

Merged

Ralph Küpper added 2 commits September 13, 2026 11:15

changelog: add fragment for #10176

b895f32

Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

proggeramlug force-pushed the fix/10164-regex-no-work-cap branch from c61d617 to b895f32 Compare September 13, 2026 11:18

This was referenced Sep 13, 2026

Merge train 179: #10168, #10171, #10147, #10150, #10136, #10145 #10188

Merged

Merge train 178: #10176 #10192

Merged

proggeramlug pushed a commit that referenced this pull request Sep 13, 2026

changelog: add fragment for #10176

ca1f65d

Claude-Session: https://claude.ai/code/session_01Da12JXeG5XuVBma5yWp5C9

proggeramlug closed this Sep 13, 2026

proggeramlug mentioned this pull request Sep 13, 2026

Merge train 180: #10183 #10195

Merged

Navigation Menu

Uh oh!

Uh oh!

fix(regex): do not cap RegExp operations by work (#10164) - #10176

fix(regex): do not cap RegExp operations by work (#10164)#10176
proggeramlug wants to merge 2 commits into
mainfrom
fix/10164-regex-no-work-cap

proggeramlug commented Sep 13, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Sep 13, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

proggeramlug commented Sep 13, 2026

Uh oh!

proggeramlug commented Sep 13, 2026

Uh oh!

proggeramlug commented Sep 13, 2026

Uh oh!

proggeramlug commented Sep 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

proggeramlug commented Sep 13, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Why no cap

Change

Test

Related

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

proggeramlug commented Sep 13, 2026

Full perry-runtime lib suite, replayed locally at f9073c461

Uh oh!

proggeramlug commented Sep 13, 2026

Uh oh!

proggeramlug commented Sep 13, 2026

Uh oh!

proggeramlug commented Sep 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

proggeramlug commented Sep 13, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 13, 2026 •

edited

Loading

Full perry-runtime lib suite, replayed locally at `f9073c461`