core: cut() stores a lone surrogate the input already carried #1963

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: 4196415b273940ff9cb53457a1af6eaf
Branch: dev/2-p308-cut-wellformed
Base: 4accf1e
At: 2026-11-01T01:44:26Z

What happens today (live main 4accf1e2cb40445f4dd7dd37b4a2704bb1cc6508)

core/src/shared/text-boundary.js:16 is the one shared bound:

export const cut = (text, max) => ({ text: text.slice(0, codePointEnd(text, max)), cut: text.length > max })

codePointEnd only moves the cut point back off a pair it would otherwise split. A string that already carries an unpaired \ud800 / \udc00 is copied straight through, so no bound ever repairs its input. #1958/#1959 fixed the half that cut itself creates; the half the caller hands in is still stored.

Card text goes through it at core/src/host/tasks.js:105 (oneLine: title / note / links / target), :371 (Done summary), :1430 (criteria), :1485 (return summary) and core/src/host/work-dispatch.js:17; roster.js:265/:268 (notice, name), hooks.js:48, reach.js:158 take the same helper.

The host convention elsewhere is to normalise at the entry: core/src/host/delivery.js:41, core/src/shared/remote-label.js:7, core/src/shared/room-share.js:241, core/src/host/room-membership.js:1986/:2523 all call .toWellFormed(). On this ref toWellFormed appears in exactly those five source files — none of them is on the cut path, so nothing normalises card text before it is bounded and stored.

Reproduction (reported by FlyBot Product, verified on this ref)

A task-create with note: 'a' + String.fromCharCode(0xd800) + 'b' and the same shape in title. The probe is a copy of tasks.test.js plus one case, run with node --test --test-name-pattern="PROBE: an unpaired surrogate":

# PROBE note="a\ud800b" wellFormed=false titleWellFormed=false utf8="a\uFFFDb"

The value read back from state.json still holds the lone surrogate, and Buffer.from(note, 'utf8') renders it as U+FFFD — the stored text is not well formed. The entry is reachable over HTTP: the request body is JSON, "\ud800" is legal JSON and pure ASCII on the wire, and JSON.parse turns it into a lone surrogate (a model's tool-argument JSON, a paste or an old file truncated mid-emoji all produce it).

By inspection on this ref the path from JSON.parse to state.json crosses no .toWellFormed(): cut is the last stop before the value is stored.

Expected

• cut makes its input well formed before bounding it: flatten with String(text).toWellFormed(), then codePointEnd on the flat string. U+FFFD is one UTF-16 unit for the one unit it replaces, so .cut (length > max) and every rendered length stay exactly what they are today for well-formed input.
• Fixing the shared bound covers every caller at once: tasks.js :105/:371/:1430/:1485, work-dispatch.js:17, roster.js:265/:268, hooks.js:48, reach.js:158.
• Well-formed input must stay byte-identical: the same text, the same .cut flag, the same ellipsis.

Decision: fix inside cut(), and the behaviour change it brings

Owner-side review asked which of the two places to change. It is the shared helper, not the call sites: cut is the one bound every card and room writer goes through, and a call-site fix would have to be repeated at tasks.js:105/:371/:1430/:1485, work-dispatch.js:17, roster.js:265/:268, hooks.js:48, reach.js:158 — each of them free to drift again.

So the PR must state the visible change plainly, in these terms:

• Well-formed input: no change at all — the same text, the same .cut flag, the same length and ellipsis. This is the case every current caller passes, and it needs an assertion, not a promise.
• Input that already carries an unpaired surrogate: the value now comes back as U+FFFD in that position instead of the raw lone surrogate. Length is unchanged (one UTF-16 unit replaced by one), so .cut (length > max) and every budget calculation stay the same; only the character differs, and it differs by being well formed where it previously was not.
• Whether existing callers can tell: they cannot tell from .cut, from lengths, or from any branch they take — only from the text itself, and only when they handed in an invalid string. The PR should show that with the stored value: what state.json holds for such a note before and after the change (the JSON escape \ud800 versus \ufffd), not just an in-memory assertion.

Scope

• core/src/shared/text-boundary.js — normalise inside cut (one line plus the comment that says why).
• core/test/text-boundary.test.js — one case: cut('a\uD800b', 8) yields a well-formed string (.isWellFormed() === true) and .cut === false; and one case that an already well-formed input comes back byte-identical with the same flag (the guard against drifting the normal path).
• patches/ds-bot/ regenerated (generate, then check).
• changelog.d/<this issue>-cut-wellformed.fixed.md.

Not in scope: adding .toWellFormed() at each call site (the shared boundary is the point); changing codePointEnd / codePointStart semantics; the entry-point normalisations already in delivery.js / remote-label.js / room-share.js / room-membership.js; any change to what a bound renders for already-well-formed input.

Dedupe (against live main 4accf1e2)

• Open issues searched for surrogate and toWellFormed: no hit describes the shared cut passing an already-ill-formed input through. The nearest open issue is #1962 (a settings line in the room), unrelated.
• Closed issues in the family, all one entry point each: #1907/#1908/#1912 (a cut that splits a pair), #1937/#1940/#1941 (a peer display name), #1943/#1944 (an inbound line's body), #1945/#1947 (a notice or name), #1950/#1951 (a seat line), #1956, #1958/#1959 (oneLine), #1960/#1961 (room-context cuts). None covers "the input already carried the lone surrogate, the shared bound stored it".

Found by FlyBot Product; the probe and the run above are theirs, and the Lead verified the code path on this ref.