host: cutting a long text must not leave half a surrogate pair (U+FFFD) #1907

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: 3eaaaee750d548febad742405686b3df
Branch: dev/1-p267-surrogate-safe-cut
Base: 2ec1687
At: 2026-10-31T07:11:50Z

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment.

Written-by: FlyBot Lead

Symptom

Host-side text cutting slices by UTF-16 units, so a cut can land between the halves of a surrogate pair. The lone surrogate is then encoded as U+FFFD wherever the line goes out — a clipped emoji becomes a replacement character instead of a whole character.

• core/src/host/text.js:50 clip: text.length > max ? ${text.slice(0, max)}… (cut; ${rest}) : text
• core/src/host/text.js:38-42 excerpt: at = Math.max(0, text.toLowerCase().indexOf(wanted) - 120) then text.slice(at, at + max) — both ends can land mid-pair
• The same rule was already fixed on the client in #1642 (core/src/client/cells.js:139-147, comment: "the cut never lands inside a surrogate pair, so a clipped emoji cannot show up as U+FFFD"); the host was not updated.

Reachable through parts.js:344/:355/:465 (handoff / checkpoint lines, 4,000 and 400), memory.js:1154 (review input lines), memory.js:1338/:1347 (error sentences in failure lines) and chat-index.js:189/:271 (read_own_chat output).

Reproduction (on main 2ec16870)

const cut = clip('a'.repeat(3999) + '😀' + 'z'.repeat(100), 4000, 'r')
cut.isWellFormed()                       // false — half a surrogate pair kept
Buffer.from(cut, 'utf8').toString('utf8') // the cut point is EF BF BD (U+FFFD)
excerpt('x'.repeat(119) + '😀' + 'y'.repeat(119) + 'needle' + 'z'.repeat(300), 'needle', 400)
  .isWellFormed()                        // false — the window starts inside the pair

Expected

A cut always lands on a code point boundary: a high surrogate is only kept when its low surrogate is kept too, and a window never starts on a low surrogate. The … (cut; N) wording, the argument shapes and the limits stay as they are.

Acceptance (red first)

1. On 2ec16870 with only the new tests in place, node --test core/test/text.test.js is 12 tests / 10 pass / 2 fail — clip never leaves half of a surrogate pair behind and excerpt never starts inside a surrogate pair.
2. Each test asserts isWellFormed() === true, !cut.includes('\ufffd'), and Buffer.from(cut, 'utf8').toString('utf8') === cut; for clip, the text left after the … (cut; N) tail equals the code-point prefix of the original and its UTF-8 bytes equal the original's UTF-8 prefix; for excerpt, the window starts on a code point boundary and matches the original bytes for that window.
3. After the fix: core/test/text.test.js 12/12, (cd core && npm test) 2,297/2,297 exit 0, check ok, canonical ok (generate a no-op after the regenerated patches/ds-bot/0001-core-fixes.patch is committed), arch-docs ok.
4. The two existing ASCII cases in text.test.js stay untouched.

Scope

core/src/host/text.js (two cuts), core/test/text.test.js (two tests), the docs/subsystems/core-host.md line the arch-docs gate requires, and the regenerated patches/ds-bot/0001-core-fixes.patch. No limit or wording change.

Not in scope

memory.js:1148 ends(text, max) cuts at both ends (slice(0, ceil(max*2/3)) + slice(-floor(max/3))) and has the same class of bug in a different shape; it is a separate follow-up, not this PR.

Dedup

Checked live main 2ec16870 (text.js/cells.js/memory.js) and the issue tracker: surrogate / U+FFFD hits are the closed #1642 (client-side clipping), #1588/#1643 (workflow titles) and #1188/#1191 (CI scan) — none covers host-side cutting, and no open issue does.