perf(runtime): add ASCII fast path to toLowerCase/toUpperCase

[proggeramlug]

Summary

• Fixes perf(string): toLowerCase/toUpperCase have no ASCII fast path, costing 33x Node on pure-ASCII input #10090: toLowerCase()/toUpperCase() cost 30-33x Node on pure-ASCII input because case_convert ran every input through a scalar wtf8_step decode / per-char char::to_lowercase()/to_uppercase() iterator / re-encode loop, even when the input was pure ASCII.
• case_convert now checks bytes.is_ascii() (a real per-byte scan) and, when true, produces the result with a single to_ascii_lowercase()/to_ascii_uppercase() byte-table transform, skipping the scalar decode loop entirely. Everything else (Unicode special casing, WTF-8/lone-surrogate handling, locale-aware casing in locale.rs) is unchanged.
• Deliberately does not reuse the existing is_ascii_string(s) helper (byte_len == utf16_len) as the gate — that's only an aggregate proxy and can be true for malformed WTF-8 where a stray continuation byte and a truncated multi-byte lead cancel out in the unit count, even though the bytes aren't ASCII. A regression test (case_convert_rejects_the_aggregate_ascii_lie) locks this in; I verified it actually fails when the gate is swapped back to is_ascii_string(s).

Testing

• RUST_TEST_THREADS=1 cargo test --release -p perry-runtime string:: — all 129 string-module tests pass, including the new ones covering: basic ASCII correctness (empty/single-char/already-in-target-case/flags/utf16_len), an ASCII prefix followed by a lone surrogate (must NOT take the fast path, flag preserved), and the aggregate-lie regression above.
• Compiled test-files/test_gap_10090_string_case_ascii_fastpath.ts with the release binary and diffed byte-for-byte against node --experimental-strip-types (Node v26.5.1, matching .node-version): identical output for ASCII fast-path cases, ß→SS/ẞ→ß, default-locale i/I (not the Turkish rule), the default-locale İ→i+combining-dot special casing, Cherokee, Deseret (astral), an ASCII string with one trailing multi-byte character, and an ASCII prefix + lone surrogate.
• cargo fmt --all -- --check clean.
• Not run: the full gap suite / release perf benchmark (scoped out for this pass — the Rust unit tests plus the targeted byte-for-byte comparison above cover this change's correctness surface, which is confined to perry-runtime).

Note

While writing the correctness gap test I found that Greek Σ-at-word-end lowercasing doesn't apply the Unicode conditional Final_Sigma rule (toLowerCase() always produces medial σ, never final ς) — this is a pre-existing bug in the untouched scalar path (Rust's char::to_lowercase() has no notion of that conditional rule), unaffected by this fix since Greek text is non-ASCII either way. Filed separately as #10116 and excluded from this PR's gap test with a comment pointing there.

Per instruction, this PR does not bump [workspace.package] version in Cargo.toml or the Current Version line in CLAUDE.md.

https://claude.ai/code/session_013naeTjgijAXt8PwpQEKkbu

Summary by CodeRabbit

• Performance

  • Improved toLowerCase() and toUpperCase() performance for pure-ASCII strings.

• Bug Fixes

  • Preserved correct Unicode case conversion, malformed character handling, and lone-surrogate data during case conversion.
  • Added coverage for ASCII, Unicode special cases, locale behavior, and truncated multibyte input.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 89db58b2-f9f8-4ff2-aeeb-38ae1f18c4fa

📥 Commits

Reviewing files that changed from the base of the PR and between e8f9123 and aa1e9e1.

📒 Files selected for processing (5)

• changelog.d/10117-string-case-ascii-fastpath.md
• crates/perry-runtime/src/string/slice_ops.rs
• crates/perry-runtime/src/string/tests.rs
• crates/perry-runtime/src/string/tests_guard_page.rs
• test-files/test_gap_10090_string_case_ascii_fastpath.ts

Included review availability: Your plan provides up to 8 included reviews per hour; 5 remain after this review.

---

📝 Walkthrough

Walkthrough

case_convert now uses a byte-level ASCII fast path for pure-ASCII strings. Unicode, malformed WTF-8, locale behavior, string metadata, and special casing retain scalar conversion. Runtime, guard-page, and gap tests cover these paths.

Changes

ASCII case conversion

Layer / File(s)	Summary
ASCII fast path crates/perry-runtime/src/string/slice_ops.rs, changelog.d/10117-string-case-ascii-fastpath.md	case_convert detects pure ASCII bytes, applies ASCII casing, and returns known UTF-16 metadata. Non-ASCII and malformed WTF-8 input uses the existing scalar path.
Case conversion validation crates/perry-runtime/src/string/tests.rs, crates/perry-runtime/src/string/tests_guard_page.rs, test-files/test_gap_10090_string_case_ascii_fastpath.ts	Tests cover ASCII conversion, Unicode special cases, truncated multibyte input, lone surrogates, string lengths, and flags.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to aa1e9

The ASCII optimization preserves the existing non-ASCII conversion path and is covered for the relevant Unicode and malformed-string boundaries. No actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The ASCII fast path uses bytes.is_ascii() and byte transforms. The non-ASCII path remains in place. The added tests cover ASCII inputs, a truncated WTF-8 tail, special mappings, lengths, and a trail…	Add automated coverage for stray continuation bytes and the required final-sigma behavior. Run all four issue benchmark reproducers at 100K and 1M inputs, verify matching checksums, and report the ASCII and Unicode performance comparisons.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: adding an ASCII fast path for runtime case conversion.
Description check	✅ Passed	The description is complete and directly related to the change. It covers the implementation, linked issue, tests, validation results, known limitations, and repository metadata constraints. It does n…
Out of Scope Changes check	✅ Passed	The changed runtime code, runtime tests, gap test, and changelog entry directly support issue #10090. No unrelated index_of work or other unrelated implementation is present in the provided change s…
Docstring Coverage	✅ Passed	Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 4 files. (1 skipped: 1 u…

Full details: Linked Issues check

Explanation

The ASCII fast path uses bytes.is_ascii() and byte transforms. The non-ASCII path remains in place. The added tests cover ASCII inputs, a truncated WTF-8 tail, special mappings, lengths, and a trailing lone surrogate. However, the issue requires a stray-continuation test, and no such test is shown. The gap test also explicitly omits final-sigma coverage. The required four benchmark reproducers, checksum comparisons, and ASCII performance results were not run or reported; the PR summary states that performance benchmarks were not run.

• Fix all pre-merge checks with AI

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/10090-string-case-ascii-fastpath

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[proggeramlug]

Landed on main via merge train #10122 (rebase-merged, per-commit authorship preserved).

Your commits are on main starting at 254902ecda; the train tree was verified identical to main after the merge (git diff origin/main HEAD --stat empty).

Closing this PR as landed — GitHub cannot auto-close it because the train merges as its own branch.