fix(string): apply context-dependent final sigma lowercasing

[proggeramlug]

Summary

String.prototype.toLowerCase() now applies Unicode's Final_Sigma rule: "ΑΣ" becomes "ας", while "ΑΣΑ" remains "ασα". Combining marks and other case-ignorable characters are skipped when determining the surrounding cased letters.

Changes

• Use Rust's context-aware whole-string lowercasing for validated UTF-8.
• Lower decoded scalar runs together for WTF-8, preserving lone surrogates and undecodable bytes as context boundaries. Keep the ASCII fast path and uppercase mappings.
• Add runtime coverage for 28 Node-verified mappings, surrogate boundaries, UTF-16 lengths, and truncated payloads at guard pages, plus a 32-case TypeScript parity fixture.
• Add a changelog fragment; no version bump.

Related issue

Fixes #10116.

Test plan

• Both new runtime regression tests fail on the original implementation.
• RUST_TEST_THREADS=1 cargo test -p perry-runtime --lib string:: -j 3: 153 passed after the fix (unoptimized profile, debug information disabled).
• Built perry, perry-runtime-static, and perry-stdlib-static together, verified the artifacts postdate the fix, and pinned PERRY_RUNTIME_DIR for compilation.
• Compiled test_gap_10116_final_sigma.ts and test_gap_10090_string_case_ascii_fastpath.ts with --no-auto-optimize; both outputs match Node 26.5.1 byte-for-byte.
• python3 scripts/check_node_version_consistency.py and git diff --check: passed.
• bash scripts/pre-tag-check.sh --quick: all checks passed except public benchmark evidence freshness. Its checker, artifact, and every fingerprint input are unchanged from base bb9aa5a641.

The same base has unrelated CI failures in unused registered_extern_handle / wasm_memory_descriptor_maximum functions and an unregistered raw thread-local in regex/perex_owner.rs (also reproduced locally). Those files are unchanged here.

Summary by CodeRabbit

• Bug Fixes

  • Improved String.prototype.toLowerCase() handling for context-sensitive Greek final sigma across Unicode text, combining marks, punctuation, symbols, and mixed scripts.
  • Preserved correct behavior around lone surrogates, malformed or truncated UTF-8 sequences, and supplementary characters.
  • Maintained surrogate boundaries, flags, and string lengths during lowercase conversion.

• Tests

  • Added comprehensive coverage for Greek final-sigma casing and boundary conditions.
  • Retained the ASCII lowercase fast-path coverage.

CI follow-up

CI is not green. The runtime suite fails in native_stack::tests::stack_top_respects_custom_thread_stack_sizes (bound must belong to this worker); the warnings gate reports unused WebAssembly helpers; and the checker self-test rejects a raw thread_local! in regex/perex_owner.rs. Those files are unchanged by this PR. Lint also reports the benchmark artifact freshness failure noted above.

The check job additionally reports generated API docs drift (manifest and generated docs are unchanged here). The new test_gap_10116_final_sigma fixture passes in Linux CI. Gap shard 2's new regression is test_gap_iterator_prototype_next_patch: the Node oracle exits 1; the other listed parity failures were already in the snapshot. The remaining shards were still running when this note was added.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 50ee9927-9900-4bbc-b2db-0c0639eff7eb

📥 Commits

Reviewing files that changed from the base of the PR and between bb9aa5a and d8fd8ba.

📒 Files selected for processing (7)

• changelog.d/10226-final-sigma.md
• crates/perry-runtime/src/string/case_tests.rs
• crates/perry-runtime/src/string/mod.rs
• crates/perry-runtime/src/string/slice_ops.rs
• crates/perry-runtime/src/string/tests_guard_page.rs
• test-files/test_gap_10090_string_case_ascii_fastpath.ts
• test-files/test_gap_10116_final_sigma.ts

Included review availability: Your plan provides up to 8 included reviews per hour; 6 remain after this review.

---

📝 Walkthrough

Walkthrough

Changes

The string runtime now applies context-dependent Greek final-sigma lowercasing. Valid UTF-8 uses whole-string lowercasing. WTF-8 scalar runs are lowercased as units, while lone surrogates and malformed tails remain unchanged. Tests cover Unicode and boundary cases.

Greek final-sigma lowercasing

Layer / File(s)	Summary
Context-aware lowercase conversion crates/perry-runtime/src/string/slice_ops.rs	case_convert uses whole-string lowercasing for valid UTF-8 and buffered scalar-run lowercasing for WTF-8 input. Lone surrogates and undecodable bytes are copied unchanged.
Runtime regression coverage crates/perry-runtime/src/string/mod.rs, crates/perry-runtime/src/string/case_tests.rs, crates/perry-runtime/src/string/tests_guard_page.rs	Runtime tests cover final-sigma context, combining characters, surrogate preservation, UTF-16 lengths, flags, and truncated UTF-8 tails.
External regression coverage and release note test-files/test_gap_10116_final_sigma.ts, test-files/test_gap_10090_string_case_ascii_fastpath.ts, changelog.d/10226-final-sigma.md	The test suite records expected final-sigma results and the changelog documents the new behavior and preserved ASCII fast path.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to d8fd8

The final-sigma behavior and WTF-8 boundary handling are covered without an unresolved actionable issue, so the change is ready to merge.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The PR meets the coding requirements in [#10116]. case_convert lowercases valid UTF-8 as a whole string and lowercases contiguous WTF-8 scalar runs together. This preserves Unicode Final_Sigma con…
Out of Scope Changes check	✅ Passed	The changed files stay within [#10116]. The runtime change implements context-dependent lowercasing. The Rust and TypeScript tests verify the required behavior. The guard-page regression test covers m…
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 6 files. (1 skipped: 1 …
Title check	✅ Passed	The title clearly and concisely describes the main change: context-dependent Final Sigma lowercasing in strings.
Description check	✅ Passed	The description covers the change, implementation details, related issue, test plan, CI status, and scope. The optional screenshots section and template checklist are not included, but the core requir…

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/10116-final-sigma

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[proggeramlug]

Landed via merge train 185r (#10242) at 9fda98d on main.