perf(lsp): semantic tokens up to 10x faster, symbols up to 2.8x on large files

[noah-emp]

First LSP perf pass, following the baseline taken after #201 (which already made every LSP request 2–4x faster than before it).

What changed

• Semantic tokens (src/lsp/semantic.bend): each use's binder was found by scanning every binder of the document with no early exit (Bind.kind_at, Bind.kind_of_item), which is O(uses × binders). The binders are now indexed once per request in a binary trie on the low 16 bits of a key: a local's line, or a hash of an item's or constructor's name. Each leaf keeps source order, so a lookup returns exactly what the old front-to-back scan returned. Bind.kind_at / Bind.kind_of_item had no other caller and are deleted.
• Positions (src/lsp/enc.bend): every position sent or read looked its line up with List.get from line 0, even under utf-32. Enc.cols now checks the text once, and a document with no char past U+FFFF converts as utf-32, where UTF-16 agrees. out / into never touch the lines under utf-32.
• New requirement BOLT-LSP-8 in SPEC.md, proved by the new law enc_narrow in src/lsp/LAWS.bend: on a line with no character past U+FFFF, a column counted in UTF-16 units equals the column counted in characters, both sent and read. That is what makes the utf-32 shortcut safe. ⚠️ SPEC.md and LAWS.bend are human-owned, so the row's wording and the law need your review. No existing law changed, and every proof still checks.

Measurements

bolt lsp --gpu off driven like an editor; best of 2 runs, in ms.

file	size	semantic tokens	symbols
src/syntax/bind.bend	35 KB	99 → 51	28 → 22
src/LAWS.bend	68 KB	145 → 97	52 → 42
src/syntax/PROOF.bend	147 KB	1525 → 396	221 → 148
src/rules/LAWS.bend	192 KB	1640 → 549	356 → 224
src/rules/PROOF.bend	1.4 MB	95,286 → 9,750	7,429 → 2,637

Change (lint), hover, definition, completion and references are unchanged within noise. Their cost is re-linting and re-binding the whole document on every request, not position conversion. That is the next pass: cache each document's analysis, and skip superseded didChanges.

Verification

• bend src/lsp/PROOF.bend: All terms check.
• bolt on the tree: 0 errors. The new index defs add L001 warnings. They are left in, not noqa'd, because a lookup must always return what a scan of the binders returns, so a law should cover them eventually. The rest of semantic classification (classify, data, …) already warns on main for the same reason.
• Every LSP answer (diagnostics, semantic tokens, symbols, and hover / definition / references at ~200 positions per file) is byte-identical between main and this branch. Checked on 5 repo files and a file with chars past U+FFFF, under both utf-16 and utf-32.

🤖 Generated with Claude Code