Sheetwrite v0.5.0

Remote tracker for the completed performance work and the agreed showcase and formula expansion. The optional @sheetwrite/formulas package and agreed data-analysis function expansion must land in v0.5.0; they are not deferred to a later release. All work is in the stacked pull requests below; nothing is merged into develop or released yet. Unchecked items are planned, not implemented.

Previous release tracker: #33. The open Changesets version PR #56 is separate; this tracker does not change its versions or publish packages.

Pull requests (GitHub stack #69)

The v0.5.0 work is one stack of pull requests. Each pull request targets the branch of the one below it and shows only its own changes. Merge from the bottom up; when a lower pull request merges, the next one targets develop.

Merging only #60 must not close this tracker.

CI status

• 2026-10-08: chore(deps): pin patched versions of audited build tools #75 merged into develop (audit pins). The stack was updated by merging develop up every layer (fast-forward pushes, no rewritten history; every layer's merged lockfile installs frozen and passes audit:js).
• Full remote CI then passed on layers 1–21 (perf: improve engine and grid hot paths for v0.5.0 #60–fix(core): repaint built-in widgets on theme change #83), the first full run for bench: one-pass release evidence with lower-noise engine comparisons #72–fix(core): repaint built-in widgets on theme change #83.
• docs: rework the landing and showcases, add the 0.5.0 guides, and cut per-page transfer #84 failed Browser Smoke only: specs imported showcase modules that load @sheetwrite/core at runtime, and that job runs without built packages. Fixed by moving the spec-facing data builders into type-only modules (engine-live-data.ts, collaboration-seed.ts, database-seed.ts); rerunning.
• docs(agents): add agent instructions for issues, triage labels, and domain docs #85 merged into develop (agent instructions). The stack will take it with the next update.

Completed: performance and correctness

• Add formula, query, render and core-path benchmark cases with full result checks and replay/output options.
• Avoid formula dependency rebuilds when references do not change; update dirty spill ranges in one pass.
• Reuse lookup ranges within a recalculation; stream reductions and criteria; read literal cells without formula/spill lookups; evaluate each LET binding once.
• Deduplicate distinct values by string-pool ID; index value filters; build multi-key sort keys once.
• Cache number/date text with bounded storage; stop wrapped layout at cell height without changing visible output.
• Index search matches and notes by view row; reduce visible-column lookup and position-only DOM work.
• Index row identities for adapter inserts; derive command state only when a handler exists.
• Rebase in the working copy; skip validation rules outside sparse writes.
• Stream CSV into target columns; avoid temporary CSV field/cell arrays.
• Stream XLSX worksheet rows and shared strings; scan XML tags/attributes without temporary regex arrays; build snapshot error paths only on failure.
• Pack block text, load columns column-major, batch change reads and reduce repeated loaded-span checks.
• Measure transaction payload once per commit; invalidate cached admission across mutable callbacks.
• Validate packed text before interning it; preserve resolved values in bulk rollback events.
• Keep every highlight on a partly visible top row; deliver command state when a callback installs its handler.
• Report oversized undo/redo through mutation-rejected; discard undo entries rejected by fixed transaction limits so older edits remain undoable. Keep entries for temporary sync-queue rejection. Host applyTransaction behavior is unchanged.
• Keep the large-clear undo benchmark under the payload limit and verify every restored cell.
• Pin Rust 1.99.0 in the toolchain and CI; use the newer iterator form where useful.
• Add changesets, refresh API/docs contracts and document deprecations/removals.

API change

The accepted raw WASM cutover removes CellStore.setBlock in favor of setBlockPacked. The raw store contract also adds CellSnapshot, cellSnapshots, loadedSpans and setColumnBlockPacked. This is documented and has a WASM minor changeset; it is not a compatibility shim.

Measured results

Baseline: develop at fac48bd. Speed/size/memory candidate: ed66b6d; the undo fix and 500K-cell undo measurement use the later branch tip. Paired runs use the same benchmark code, alternate base/candidate and pin core 4. These are measured scenarios, not claims for every workbook.

Delivery size and runtime memory are separate

• WASM: 700,739 → 758,385 raw bytes; 213,832 → 228,901 Brotli bytes (+57,646 raw / +15,069 Brotli).
• Node cold initialization: 3.83 → 4.05 ms (+5.6%). This is an accepted, reported cost, not a hidden regression.
• Data store at 1M rows: WASM memory 436.4 → 306.8 MiB; JS heap 874.3 → 614.8 MiB.
• XLSX import peak in the separate import measurement: 476 → 312 MiB.
• Bun/JSC has a measured warm-up-order cost on the first high-cardinality distinct call after low-cardinality calls (about +16 ms). Cold first calls were comparable; later calls were faster.

Committed evidence: bench/results/formula-results.json, bench/results/render-diagnostics.json, bench/results/render-diagnostics.md, bench/results/xlsx-results.json, and docs/src/content/docs/guides/performance-resources.md. Additional paired captures were measured locally and have not been uploaded.

Evaluated and not shipping

These are decisions, not unfinished implementations:

• Hasher replacement: no proof of safety against crafted keys.
• Per-operation aggregate specialization: below the required gain.
• Text truncation: could not prove pixel-identical output.
• Capture gating/projection skipping: would change the public capture contract.
• Sync optimization: about 2% gain, below the gate.
• Async Node WASM loader: heartbeat gap rose from 3.6 to 23.5 ms; reverted.
• Conditional-format/window packing: about 0.2% gain.
• Worker scroll reuse: result validation and a regression test fail; frozen scrolling is 6.2% slower. Not merged.
• Native local undo fast path: Grid's own listener always needs the patches.
• SIMD build: no target meets the gain requirement; some formula cases are 24–36% slower. Not merged.
• panic=abort: already the WASM target default.

Showcase redesign (in #61)

• Shared shell: compact heroes, no load-time scroll jump, theme-aware controls, consistent mobile gutters.
• Hub cards without clipped previews; framework cards show the real import and mount code.
• 1-billion-cell page: honest 1,000,000 × 1,000 framing, position ruler, Start/Middle/End/Random jumps, even 0–100% landmarks, side panel no longer leaves blank space under the grid.
• Database, interoperability, collaboration, engine and host-rows pages: named headers, columns that fill the stage, grids sized to their content.
• Vanilla, React, Vue and Svelte examples: named headers, columns fit the stage, untruncated text, readable tabs.
• Verified: automated overflow/scroll/cut-off-text check on all 11 routes at 1440×900 and 390×844 in light and dark; screenshot review of every route at desktop in both themes and most routes on phones (exact list in feat(docs): redesign the showcases and fix paged formula loading #61); Chromium browser suite 68/68.

Core bugs found and fixed in #61, each with a regression test that fails on the old code:

• Paged formulas stuck on #LOADING! when they read an off-screen column (the "stuck after fast scrolling" bug on the 1-billion-cell page). Adds CellStore.formulaReadBands (@sheetwrite/wasm minor).
• Row numbers painted into the column header.
• Sheet tabs scrolled the whole page on mount.
• Sheet options menu closed on unrelated scrolls and dropped focus.
• Toolbar color swatches showed black. Adds grid.getTheme() and the theme-change event (@sheetwrite/core minor).

Modular formula engine (#62)

Agreed package name: @sheetwrite/formulas. Two complete engine builds from one crate, not runtime-linked WASM plugins. @sheetwrite/wasm keeps the existing function set; the optional package supplies the full engine. Apps select one engine with initSheetwrite(undefined, formulas). Nothing downloads or initializes both by default.

• Cargo analysis feature, @sheetwrite/formulas package, loaders, build/size/release/API tooling; engine selection in core; both engines verified in Node and in a Chromium Grid.
• Statistical distributions family (36 functions), checked against Microsoft's published examples.
• Registration, formula assist (selected engine only), contract inventory and generated docs stay aligned per build; contributor guide for adding a family.
• Descriptive statistics and simple regression (18 functions): AVEDEV, DEVSQ, HARMEAN, KURT, SKEW, SKEW.P, TRIMMEAN, PERCENTILE.EXC, QUARTILE.EXC, PERCENTRANK.INC/EXC, RANK.AVG, PEARSON, RSQ, SLOPE, INTERCEPT, STEYX, FORECAST.LINEAR.
• Regression arrays (11): LINEST, LOGEST, TREND, GROWTH, FREQUENCY, PROB, MODE.MULT and legacy PERCENTILE, PERCENTRANK, QUARTILE, RANK.
• Dates (4) and finance (15).
• Text and regex (10). Regex uses regex-lite: no lookaround, backreferences or Unicode classes; such patterns return #VALUE!.
• Database (12) and math, aggregate and matrix functions (20).
• Array reshaping (8): TOCOL, TOROW, WRAPROWS, WRAPCOLS, EXPAND, HSTACK, VSTACK, SORTBY. Fixed a FILTER debug-assert panic found on the way.
• Array constants such as {1,2,3} in both engines (fix(wasm): support array constants in formulas #64). Default engine +6,937 raw / +2,137 Brotli bytes.
• LAMBDA, MAP, REDUCE, SCAN, BYROW, BYCOL, MAKEARRAY and ISOMITTED, with bounded call depth and work. Parser changes are compiled only into the full engine.
• GROUPBY, PIVOTBY and PERCENTOF with eta-reduced built-ins (SUM) or LAMBDA arguments. Vectors of functions such as HSTACK(SUM,AVERAGE) return #VALUE! (documented).
• A1# spill references in both engines, with XLSX _xlfn.ANCHORARRAY round trip (feat(formulas): support spill references and fix spill read order #68). Fixed spills that read a later spill (old values) or each other (no error); they now get current values or #CYCLE!.
• Performance pass over every family (MMULT −28%, PERCENTRANK.INC −33%; full engine −5,819 bytes). Rejected candidates that measured slower.
• Full vs default engine benchmarks (bench: compare the default and full formula engines #67). Matched rounds (51 pairs) show no shared workload more than 5% slower with the full engine. Func::Analysis uses a u16 ID (4 instead of 24 bytes).

Contract counts: 299 functions, 302 full-engine spellings. Release sizes, rebuilt at the top of the stack (#83/#84; #84 changes no engine code): default 791,937 raw / 238,071 Brotli; full 1,074,934 raw / 326,278 Brotli. At #74 they were 793,973 / 237,888 and 1,077,002 / 326,563. The default engine grew from 758,385 to 767,306 bytes in #61 (formulaReadBands), by 6,937 bytes in #64 (array constants), by 12,041 bytes in #68 (spill read order and A1#), by 1,722 in #70 (SEARCH/WORKDAY/UNIQUE fast paths), by 5,238 in #71 (array operators) and by 718 in #74; the formula families add nothing to it.

This is broad data-analysis coverage, not a promise of every Excel or Google Sheets function. Record the exact supported names, semantics and unsupported cases in the inventory. Current Google Sheets export evidence is metadata-only; do not claim verified workbook compatibility without actual fixtures.

Large undo with sync (#63)

• Chosen design: a compact restoreBlock operation plus atomic multi-version batches. The per-version 8 MiB cap stays.
• Undo of a million-number clear restores every cell locally, at the server store and at a synced peer. A rejected batch member leaves the peer unchanged.
• Remote payload limits and the host applyTransaction result and event contract are unchanged.
• Hosts must support restoreBlock and batch metadata before they accept clients from this release. The @sheetwrite/core changeset (and so the CHANGELOG) says this.
• Core depends on fflate 0.8.3 for the restore codec (user decision). Consumer checks now verify that core and the adapters install without @sheetwrite/xlsx.

Formula kernels (#70)

• Faster kernels with bit-identical results: SEARCH −78%, WORKDAY/WORKDAY.INTL −41% to −82%, NETWORKDAYS.INTL −19% to −52%, XIRR −46% to −80%, XNPV −51%, regex −44% to −50% (bounded 4-entry compiled-pattern cache), TEXTSPLIT −34%, PERCENTILE.EXC/QUARTILE.EXC −31%, FREQUENCY −28% to −30%, distribution inverses −29% to −36%, GROUPBY/PIVOTBY/PERCENTOF −16% to −25%, LAMBDA helpers −12% to −19%, SUMSQ family −44% to −46%, MDETERM/MINVERSE −24%/−31%.
• Gate: A/A control (noise about ±2.5%), ABBA rounds, ≥5% on the target, untouched workloads within ±3%. Rejected candidates (VSTACK move, generic range streaming, SUMX dispatch, DSUM loop) were reverted.

Array arithmetic (#71)

• Operators apply element by element over arrays with Excel broadcasting, in both engines: =A1:A3*10, =SUM(A1:A3*B1:B3), =FILTER(B1:B12,E1:E12>=1000), =SUMPRODUCT((A1:A3>5)*B1:B3), =A1#*10. Before, these returned #VALUE!.
• Behavior change: =VSTACK(B2,A1)+0 now spills both values, as in Excel.
• No measured slowdown on untouched recalculation (+0.21%, inside A/A noise).

Benchmark evidence (#72)

• bench:release captures every artifact the performance page needs in one pass on one clean commit, then re-records the machine- and harness-pinned regression baseline.
• Identical-work spread in default vs full engine comparisons: 8.99× → 1.59× (p90/p10). Rows above 1.5× are marked unsupported.
• Render failures name engine, size, step and error. The 20 unfinished runs are upstream: Handsontable 18 remove_row at 1,000,000 rows overflows its own call stack.
• New sections: core write path (packed 1M-cell block, 200k×10 CSV, 500k-cell clear + undo, paged residency, row-bridge insert) and XLSX codec vs the pre-0.5.0 codec.
• Run bench:release on the final integrated tree and regenerate the performance page (the committed page still shows the July and October captures as mixed vintages).

Regression check and remaining gaps (#73, #74)

• 0.5.0 vs 0.4.x matched render capture: no regression above A/A noise. A focused re-measurement of the editing scenarios (10 A/A + 10 ABBA rounds, ≥250 ms per sample, 100,000 rows) found +3.3% to +6.2% (about 20–40 µs per edit), inside the A/A band, with every 99% confidence interval including zero. Recorded as noise; a small regression cannot be fully ruled out.
• JSON byte accounting: clear + undo of 500,000 cells −7.2% (perf(core): cheaper JSON byte accounting #73).
• Write paths skip empty formula and spill metadata: 1,000,000-row ingestion −35% (perf(wasm): skip empty metadata probes on cell writes #74).

Showcase and documentation rework (#84)

Committed as the top layer after maintainer review on the dev server.

• Every showcase page rebuilt on one stage layout with realistic data, and each shows one live 0.5.0 behavior: large undo and atomic batches (database, collaboration), streamed CSV/XLSX import and spill round trip (interoperability), off-screen formula reads (engine), row-bridge bulk insert (host rows), theme-following toolbar swatches (workbenches).
• New Formula analysis showcase: thirteen live panels over 20,000 orders, including distributions, finance and dates, a database query, LAMBDA and regex.
• Billion-cell flight deck: dive, diagonal cross to ALL1000000, teleports, with live frame rate and residency.
• Docs: What's new in 0.5.0 with an upgrade checklist, Analysis formulas guide (engine-verified examples), Calculation engine concept page, updated persistence/collaboration/interaction/styling/configuration guides, refreshed overview and first-grid quickstart.
• Fixed on the way: StrictMode double session and stalled autosave after an external write on the database page, conflict recovery 4.3 s → about 1 s, stale LAMBDA/dynamic-array compatibility records.
• Maintainer review on the dev server, then commit as its own layer (docs: rework the landing and showcases, add the 0.5.0 guides, and cut per-page transfer #84).
• Landing page rework: seekable hero and architecture illustrations (click a numbered step to start there), richer feature illustrations; sectioned showcase hub.
• Page weight on the production build: uncompressed −21% to −56%, after Brotli −7% to −18% (Brotli already removes most repeated markup). Changes: one popover panel per distinct hovered symbol (they were 63–81% of docs HTML), Expressive Code styles served once instead of inlined into 509 pages, and landing snippets, the docs page index, the capability inventory, and landing and showcase styles kept off pages that do not use them.
• Deployment: content-hashed assets and Pagefind shards are cached as immutable (the live site sends max-age=0, must-revalidate for them); HTML and stable entry files still revalidate. Prebuilt-only deployment is unchanged.
• Paste across hidden columns silently did nothing (found during the architecture review); fixed with a regression test in fix(core): paste across hidden columns #76.

Architecture review (in progress)

Deepening candidates found by a read-only review and checked by hand; decisions are being made with the maintainer. Planned for 0.5.0, in order:

• One source-replacement path in the engine (perf(wasm): keep the dependency index for constant-only source rewrites #77). Grid edits went through setSparseBlock, which always rebuilt the dependency index, so the "constant formula edit 34.6 ms → 0.03 ms" result held only for direct setFormula callers; constant-only rewrites now keep the index on that path too.

• Release evaluation caches when the pass ends (fix(wasm): release lookup ranges when recalculation ends #78).

• One publication policy (build: enforce map-free library publication #79): stop shipping unusable source maps (about 218 KB unpacked in @sheetwrite/xlsx); declare CSS-only side effects for the adapters.

• Bounded echo identity in row projection; delete a write-only set (fix(core): bound row bridge echo identities with an adjustable window #80).
  Decided on 2026-10-08: all six remaining candidates land in 0.5.0, each as its own stack layer. Every layer must pass a performance gate: A/A control plus at least five alternating rounds, target workload not slower, untouched workloads within 3% beyond noise; a failing layer is reworked or dropped.

• Prepare each transaction once (perf(core): prepare each restore block once per transaction #86): each restoreBlock is decoded once per apply (was 3–5), compact undo decodes 0 times (was 3), rebase with 10 transforms decodes once (was 20). Gate: near-ceiling restore 2,213 → 1,346 ms (−39%), 1M-cell undo 2,536 → 1,833 ms (−28%), peak RSS down; untouched controls −0.9% and −0.1%.

• Interaction sessions (refactor(core): one owner for interaction sessions, editors and context menus #87): one private owner for pointer, editor and context-menu sessions. Resize previews no longer mutate the document before admission, gestures end on sheet/view/structure/read-only changes, a second pointer cannot take over a drag, stock and custom editors commit to the original cell through sorting and filtering, selection changes precede edit-begin, and a rejected fill leaves the selection unchanged. Edit notifications are built only for subscribed consumers. Gate (5 A/A + 5 alternating rounds): edit 0.202 → 0.194 ms (−3.7%), column resize −10.4%, row resize −34.2%, selection/get-cell within band; memory ceilings and owned cold long task unchanged.

• Shared adapter lifetime: one shared lifetime controller for React, Vue and Svelte; a ref/onReady exception retires the new Grid in every adapter; readiness waits for the latest live options (likely Svelte defect).

• Formula/spill ownership: Rust owns source writes, spill admission and recalculation through one commit interface; calculated results move to a derived overlay separate from source storage.

• Window ownership: one ReadWindowController owns the visible/paint plan, datasource demand, formula closure and engine residency. Fixes bulk reads replacing the viewport's residency pins and formula loading that skipped frozen rows and merge anchors.

• String-pool lifetime: the pool never reclaimed text (edit-heavy sessions grew without bound). Threshold collection after commits with versioned string IDs; undo snapshots own their text.

Test cleanup (#65)

• 13 suites that no CI step ran now run in CI.
• Shell, sheet tabs, formula assist, controllers, adapters, tooling scripts, browser specs, core grid and rendering, core data errors and XLSX warnings: tests check behavior and contracts instead of copy, layout, call traces and copied inventories. Tests +1,285 / −1,896 lines.
• Found and fixed: landing page structured data claimed version 0.1.0; two XLSX fixtures listed a stale hidden-sheet warning.
• Rust, bench, sync and admission tests reviewed and kept (reasons in test: remove hyper-specific tests and run unrouted suites in CI #65).
• Interoperability warning codes come from the fixture manifest. Sync, offline sync, persistence and history: 45 tests reviewed one by one (36 kept, 8 changed, 1 merged), with mutation checks.

Verification and release gates

Remote CI passed on layers 1–10 before the dependency advisories (see CI status). Top layer (#84), locally: Chromium browser suite 82/82, docs tests 21, tooling tests 104, docs typecheck, docs:check, lint. Local checks on the top layer (#74): Rust 147 (default) / 223 (--features analysis); packages 869 pass / 6 skip / 0 fail; build, typecheck, lint, api:check, docs:check.

• Run checks and real browser smoke verification after the remaining implementation.
• WebKit: passes in CI (Browser Smoke, Ubuntu). It cannot run on the local development host, which lacks its system libraries.
• Review public API changes, package selection, changesets and release versions. Re-checked on 2026-10-08 across all 22 layers: every layer that changes package code has a changeset (bench: one-pass release evidence with lower-noise engine comparisons #72 and bench: measure current interaction metrics in the release capture #82 change only benches, chore(deps): pin patched versions of audited build tools #75 only root tooling); bump types match the changes; changeset status moves all seven published packages from 0.4.0 to 0.5.0. docs: rework the landing and showcases, add the 0.5.0 guides, and cut per-page transfer #84 corrects three stale changesets: final engine sizes in full-formula-engine, the obsolete 1,000,000-cell undo threshold in report-oversized-undo (superseded by fix(core): restore large undo steps with atomic sync batches #63), and the superseded Rust 1.98.1 pin in steady-toolchain-refresh (the release pins 1.99.0). Intentional exports: core adds PersistenceBatchCommitRequest, SheetwriteEngine and VersionBatchMember; @sheetwrite/formulas is a new entry point; nothing is removed. The raw WASM setBlock → setBlockPacked cutover is documented (perf: improve engine and grid hot paths for v0.5.0 #60). Changesets move every package to 0.5.0 (wasm and formulas are fixed together; a coordinated changeset releases xlsx and the adapters).
• Release notes draft, including the host requirement for restoreBlock and batch metadata and the new fflate dependency of core.
• Keep the full data comparison's Handsontable DOM memory limit explicit: it exceeds the local 10 GiB cap; the Sheetwrite part was run separately.
• Merge the dependency audit fix, rebase the stack, and confirm CI on every layer.
• Release layer like release: publish Sheetwrite 0.4.0 #53: changeset version to 0.5.0 also needs the consumer fixture locks and tarball names (test/consumer, test/release-locks/*, test/bundler-fixtures/*) updated, or consumer-lock.test.ts fails; regenerate docs so showcase evidence shows 0.5.0.
• Review and merge the stack from the bottom up (maintainer decision), then publish.
• Publish final remote evidence for speed, delivery size, runtime memory and compatibility limitations after the merge.

Do not close this tracker when only part of the stack merges. The showcase rework, the architecture work planned above and the release gates remain open.