Repository navigation
Scenario plans run totals; grid API fixes; browser tests; an Effect recipe; grid packages 0.5.0 - #98
Merged
Merged
Conversation
The grid pages gain a cookbook (grid/cookbook/ and grid/react/cookbook/): four recipes, each running on the page with its source under it. The source shown is the recipe's own file, imported with ?raw, so what is read is what runs, and npm run check type-checks it. - A first grid: a table, a grid, a total outside it, edits from code. - Rows from an API: fetch DummyJSON's products and putMany them, with loading and error states, fetching again, and aborting on unmount. - Load pages as you scroll: loadMore over a paged API, starting the next page at table.size, so a page asked for twice is harmless, and retrying a page that failed. - Charts from the table: GroupChart, and a chart of your own drawn from groups(), both following edits. Both pages share the words (web/grid/src/cookbook/text.ts) and styles. The grid pages and the three package READMEs link to them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Bun reads the same .npmrc, so the setup shows bun add beside npm install, and the bunfig.toml scope for projects that keep their settings there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
"A large table in a worker", in Solid and React: start a WorkerTable, load 300,000 rows into it in blocks with putMany while the page stays free, and move a thousand prices in one batch, with the total and the last recalculation's cost shown as they come. It disposes the worker when the component goes, including while the worker is still starting. The notes say which reads answer at once from the snapshot and which ask the worker, that a write's promise resolves before the totals it changes arrive, and what Vite needs for the worker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Two recipes, in Solid and React, that write a stream to the table the same way: gather messages as they come, and write them once a frame inside batch, so a burst costs one recalculation and one redraw. - Prices over a WebSocket: Coinbase's public ticker sets the price of a few coins, keeping only each coin's latest price between frames; each row is added with its first price, and the price column draws its trend. - Events over SSE: Wikimedia's stream of recent changes, through EventSource, puts a row an edit and drops the oldest in the same batch, keeping the latest 2,000, grouped by wiki. Each connects when asked and closes its connection when the component goes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
"Edit and save back to an API", in Solid and React. The table keeps what the server last had; after each recalculation that changed an input, the rows are compared with it, so whatever a person did in the grid (typing, pasting, filling a block, undoing) shows as unsaved changes, and undoing back to the saved value makes a row clean again. Save sends one PATCH a changed row with only its changed fields; a row that fails stays unsaved to retry, and only what was sent is marked saved. Discard puts the saved values back in one batch. Recent requests are listed under the grid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Prism highlights each recipe's source (TSX) and the setup's snippets (.npmrc, bash, bunfig.toml, TypeScript), with token colours for the light and dark page in cookbook.css. Both pages share web/grid/src/cookbook/highlight.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
drawGrid (and the Solid and React <Grid>) take onEdit, called with
{ cause, cells, removed } whenever a person changes the grid: each cell's
key, column, and value before and after, for typing, pasting, filling a
block, undoing and doing again, and the keys of rows they delete. Changes
made from code (put, set, drop) are not reported, so an app can save
exactly what people did.
The engine notes the cells that set_cell, undo and redo change between
record_edits and take_edits (Grid::record_edits, Grid::take_edits). The
view runs each of a person's edits between the two; with a WorkerTable
the three go as one message (RemoteEngine.sequence), so nothing else is
set in between and the same edits are reported as with a Table. Pasting
goes the same way, and RemoteEngine.setCells, which it replaces, is gone.
The cookbook's edit-and-save recipe now uses onEdit instead of comparing
every row after each recalculation, so it costs only what was edited.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@surefold/grid, grid-solid and grid-react at 0.4.0, for the tag that publishes them: the grid's onEdit says what a person edited (cells with their values before and after, and deleted rows), and RemoteEngine.setCells gives way to RemoteEngine.sequence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Each cookbook page's header (its one h1, the intro and the Solid/React switch) is now in the page's HTML rather than drawn by its script, which renders the recipes under it. So the heading shows before the script runs, and the page passes surefold.dev's site checker, which reads the HTML for one h1, a title, a language and a description. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Prism nests the JavaScript in a JSX attribute ({…}) inside the tag's token,
so names there that have no colour of their own (orders in onClick, title
and decimals in keyColumn) took the tag's red. They now take the code's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Its code background is --code-bg, the name surefold.dev's watershed theme sets, so on the site code blocks have the site's code colour rather than the cookbook's own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
… claude/stoic-pascal-v0q6pz
"Live tables in the browser" says the cookbook shows the grid in an app, and the Browser part of getting started says what it has and links it in Solid and React. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
If loadMore threw, the grid said so through onError and then took it as
having no more rows, so it never asked again. Now a failed page leaves the
grid able to ask: the next scroll near the end, or view.fetchMore(), calls
loadMore again. onStatus says it failed, with { loading: false, done: false,
failed: true, error }.
The cookbook's paging recipes no longer catch the error and resolve to true
to keep the grid asking: loadMore throws, and onStatus shows that the page
did not load.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The worker replied to a write before the recalculation it set off, so after await table.putMany(...), total() and size read the snapshot from before it. Now the worker recalculates before it replies, so the snapshot with the write reaches the page first, and a write's promise resolves once the page has it. A write that changes nothing resolves when the worker answers, as before. Writes inside batch resolve once the batch is answered, rather than at once. sort, filter, search and group resolve the same way. Table gains flushWaiting (internal), which recalculates only if writes are waiting, so a reply to a read sets off no snapshot. The worker recipe counts the rows loaded with table.size after each block, and its note says what the promise waits for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
useReading compares what it reads with ===, so a list such as groups()
had to be read as a string and parsed. It now takes isEqual: the hook
keeps the value it gave until a new one compares unequal, so the same
array goes to a chart or memo until something in it changed. useGroups
(table) is that for groups(), a new array only when a group, its row count
or a total changed.
In Solid, createReading takes the same isEqual, and createGroups follows
groups() and is set only when they changed. Both keep the table's type for
the read function, so (t) => t.total("pnl") is checked against its columns.
The chart recipes use them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
useTable(() => new Table({ fields: { … } })) gave a table of any fields:
its type parameter extended Table<any, any>, which TypeScript took as the
type new Table(…) should have, so the fields and formulas were not
inferred. It now asks only for what it uses, dispose(), and the table
keeps its names and kinds.
test/types.tsx (React) and test/types.ts (Solid) are compiled by npm run
check and not built: a misspelt field or column, a value of the wrong kind
and a row missing a field must each be an error. They fail with the old
useTable.
The React edit-and-save recipe makes its table inline, as the other
recipes do, rather than in a function outside the component.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
drawGrid takes rowClass, (key) => classes, and gives a row what it returns each time the row is drawn, so the classes follow the row's key as people scroll and sort, with a Table or a WorkerTable. When what it would return changes, view.redrawRows() draws the rows in view again. The Solid and React <Grid> pass it on, read when called, as they do the handlers. The edit-and-save recipes class rows with unsaved edits "unsaved", which cookbook.css tints with a warm colour mixed into the page's surface, and a mark by the key, in light and dark. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A new workspace, web/e2e, with Playwright 1.63.0. `npm run e2e` builds a small fixture page into web/dist/e2e and runs the tests in Chromium against web/dist, served by python3 -m http.server (after web/build.sh). - onEdit, with a Table and a WorkerTable: typing, undo, redo, a 2x2 fill, a paste that skips a value that is not a number, deleting a selected row, and changes from code (set, patch, put, drop) that must not be reported. The exact cells, before and after, are checked. - Every recipe in the Solid and React cookbooks, with DummyJSON, Coinbase's WebSocket, Wikimedia's SSE stream and Google Fonts mocked; any other request off the test server, and any page or console error, fails the test. - Both grid pages load, and an edited cell is put back by Ctrl+Z. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
After web/build.sh and the type check: install Playwright's Chromium and run `npm run e2e`. The suite takes about 25 seconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A total over a collection or a grouping was refused by a plan whenever a varied input reached it, so models built the sum as a tree of + nodes by hand. A plan now lowers a total (sum, min or max of f64) into the fold the graph itself keeps: the balanced tree with room for the next power of two, each slot its two halves combined. So its result is the graph's, bit for bit. - Room no member holds, and members absent in every scenario, are the operation's identity, which combined with anything gives it back unchanged, so they are left out without changing a bit. - Halves whose members are the same in every scenario are combined when the plan is made, in the same order, into one constant. - A member present in only some scenarios is when(present, value, identity). - Varied halves are combined by three new instructions, AGG_SUM, AGG_MIN and AGG_MAX (26 to 28; 25 is fuse::LOCAL), on the CPU, in the generated kernels and in the general kernel: a NaN wins, the left one first, as the graph's totals do (MIN and MAX let a number win over a NaN). Counts stay refused: they are whole numbers. A plan's file is version 3, so an older crate refuses one that may hold the new instructions; versions 1 and 2 are still read. A shape longer than 1,024 instructions (a large total is one) is run by the general kernel rather than written as its own. Tests: forty random totals over members that vary, are shared, are fixed, or come and go, with NaN, signed zeros and infinities, against the graph bit for bit after shared inputs change; a collection's sum, max and min with a removed row, read by a later node and saved and read back; a file of the version before. The kernels' C for the three instructions gives the graph's combine bit for bit on all 49 pairs of 0, -0, 1.5, -2, +-inf and NaN (checked with gcc; no GPU here to run them). On a 1,000,000-row collection the plan compiled in 0.9 s and ran a scenario in 7.3 ms on the CPU, where the graph took 130 ms. Not yet: a total is one node, so a GPU adds its members in one thread per scenario, and members' values are kept until the total reads them. docs/scenario-plans.md#totals says how both would go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
…ed writes - The paged recipe: a page answered with a 500 says it did not load, and the next scroll asks for it again and loads the rest. - Edit and save: the rows with something to save have the "unsaved" class (rowClass), a row undone back to clean loses it, and saving clears it. - onEdit's fixture: an awaited set, put or drop is in the totals and size read straight after it, with a Table and a WorkerTable. With the WorkerTable fix taken back out, the WorkerTable case fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A failed page is asked for again on the next scroll near the end, but a list already at its end cannot scroll further, so no scroll comes: the recipe's "Scroll to try again" could not be done there. It now says the page did not load and shows a Try again button that calls fetchMore(), until a page loads. The note says why. The browser test makes the page fail until it is told to stop, waits for the button (which shows once the failure has settled), then clicks it; scrolling then loads the rest. It passed 20 runs on its own and 114 tests in a stress run of the whole suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The WebSocket recipe again, in Solid and React, with Effect doing what the hand-written one leaves out. The Coinbase connection is a Stream.callback whose socket is opened with acquireRelease, so it is closed however the stream stops, and the server closing it fails the stream. Stream.retry opens another on a Schedule: exponential from 0.5 s, capped at 10 s, jittered, starting again once a message comes through; the schedule's tap puts "Reconnecting in N s (attempt k)" on the page. groupedWithin(500, 50 ms) batches the ticks, and each batch is one table.batch keeping the latest price of each coin. Connect runs it with Effect.runFork; Disconnect, onCleanup and the React effect's cleanup interrupt the fiber. Same table, formulas and look as the WebSocket recipe. effect 4.1.2, pinned, is a dependency of web/grid and web/grid-react only; the published packages do not depend on it. The notes say what to install, what it costs (Effect's part is 55 KB minified, 19 KB gzipped; each cookbook page's JavaScript went from 55 to 77 KB gzipped in Solid, 118 to 140 KB in React) and how the same shape fits SSE and polling. Browser tests, both frameworks, all mocked: rows and gain follow the ticks; a connection the server drops is opened again after the first wait and subscribes again, the waits grow while the server refuses and start again after a connection that worked; Disconnect closes the socket and stops a pending reconnect; unmounting does the same, on a fixture page per framework that mounts the recipe with an Unmount button (the e2e build now compiles Solid and React JSX for them); leaving the page while waiting makes no more connections. The Coinbase mock counts connections and subscriptions, and can drop a connection or refuse new ones. With the cleanup taken out of either recipe, the unmount test fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The random totals (sum, min and max over members that vary, are shared, are fixed, or come and go) now also run on the GPU, by the generated kernels and by the general kernel, and in clusters; and a 2,000-row total, longer than a shape gets its own kernel for, runs by the general one. A clustered plan's totals and the long total are checked on the CPU too, so they run where there is no GPU. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@surefold/grid, grid-solid and grid-react at 0.5.0, for the tag that publishes them: a failed loadMore page is asked for again; an awaited WorkerTable write resolves once its totals are in; useGroups and createGroups, and an isEqual for useReading and createReading; useTable keeps an inline table's types; rowClass and redrawRows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
matthewjones372
pushed a commit
that referenced
this pull request
Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Five pieces of work:
GPU: the CUDA tests (
cargo test -p watershed-gpu --features cuda) were run on an NVIDIA GPU machine under WSL2 on this branch, and all passed, including the new GPU tests for totals. CI has no GPU, so it doesn't run them.1. Scenario plans run totals: sum, min and max, bit for bit as the graph
Until now a plan refused any total (a
sum,minormaxover a collection or a grouping) that a varied input reached, so models built totals as hand-written trees of+nodes. A plan now lowers a total itself, and the results are the graph's, bit for bit.How: the graph keeps each total as a balanced binary tree of partial results, with room for the next power of two, and each slot combines its two halves. Its result is exactly what folding that tree from scratch gives (
watershed-core/src/aggregate.rs).lowered_totalinplan.rslowers a total into that same fold:-0.0,+∞or-∞). Combining that with anything gives the other value back unchanged, so they're left out without changing a bit.SELECT(present, value, identity).AGG_SUM,AGG_MINandAGG_MAX(codes 26 to 28; 25 isfuse::LOCAL), in the CPU interpreter, the generated kernels (in_c) and the general kernel (kernel.cu). NaN wins, left first, as the graph's totals do. The existingMINandMAXlet a number win over a NaN, which would make a total's result depend on its tree.Also:
docs/scenario-plans.mdgets a "Totals" section, anddocs/limitations.mdand the crate docs are updated.Tested:
cargo fmt,cargo clippy --workspace -D warnings, clippy with--features cuda,cargo test --workspaceandcargo doc -D warningsall pass.Not yet:
docs/scenario-plans.md#totalssets out how both would be fixed.2. Grid API fixes, one commit each
loadMorethrows,onErrorgets the message andonStatusgets{ loading: false, done: false, failed: true, error }.donestays false, so the next scroll near the end, orview.fetchMore(), retries. It doesn't retry by itself, so a failing server isn't hammered.fetchMore(). The new browser test found this: "Scroll to try again" couldn't work at the end of the list.loadMorethrows on purpose to stop loading will now be asked again.WorkerTablewrite resolves once its totals are in. The worker recalculates before it replies, and the snapshot is posted before the reply, soawait table.putMany(...); table.total(...)sees the new totals. A write that changes nothing resolves at once, and writes insidebatchresolve with the batch.sort,filter,searchandgroupbehave the same way. Promises now resolve one recalculation later than before.useGroups(table)(React) andcreateGroups(table)(Solid) return groups that only change when a group's keys, counts or totals change.useReadingandcreateReadingtake an optionalisEqual. The chart recipes no longer round-trip the groups through JSON.useTablekeeps an inline table's types: its constraint is nowT extends { dispose(): void }. New type-level tests (packages/grid-react/test/types.tsx,packages/grid-solid/test/types.ts, compiled bynpm run check) require errors for a misspelt field, a wrong kind, a row missing a field and a misspelt column.rowClass(key)option, called whenever a row is drawn, andview.redrawRows()for when the classes change. Classes follow their keys through virtual scrolling and sorting, for bothTableandWorkerTable. The edit-and-save recipes use it to tint unsaved rows, in light and dark.READMEs,
view.d.tsand the cookbook notes are updated.3. Browser tests:
web/e2e, in CIA new
web/e2eworkspace.npm run e2ebuilds its fixture pages and runs Playwright againstweb/dist. There are 48 tests, which take about 31 s:onEdit(onedit.spec.ts), against both aTableand aWorkerTable: typing with undo and redo, a 2×2 fill, a paste with a value that's skipped, a row delete, and changes from code that must not be reported. Each checks the exact{cause, cells, removed}. Plus: an awaited write is in the totals read straight after. With fix 2 taken out, theWorkerTablecase fails.cookbook.spec.ts):pages.spec.ts): they load with no errors, and editing a cell then undoing restores it.CI's
grid-jsjob installs Playwright's Chromium and runsnpm run e2e. CI has run it on this PR, and all tests passed.Tested: after a clean
npm ciandweb/build.sh,npm run checkandnpm run build:packagespass. All 48 tests passed in CI mode, twice. Before the Effect recipe was added, the 38 tests then in the suite also passed a stress run of three repeats with 4 workers (114 tests).4. A cookbook recipe: streams and retries with Effect
A new recipe in both cookbooks, after the SSE one (
EffectPrices.tsx). It rebuilds the live-prices feed with Effect (effect4.1.2), to compare with the hand-written WebSocket recipe:Stream.callback, opened withEffect.acquireRelease, so it's closed however the stream stops.Stream.retryuses aSchedulethat waits 0.5 s, then 1, 2, 4… up to 10 s, with ±20% jitter. It resets once a message comes through, and the page shows "Reconnecting in N s (attempt k)".Stream.groupedWithin(500, "50 millis")collects 50 ms of messages, and each batch is written to the table with onetable.batch.onCleanup, React's effect cleanup).Scope:
effectis added only to the private cookbook workspaces (web/grid,web/grid-react). The published packages don't depend on it. Effect adds 18.8 KB gzipped (54.7 KB minified) to each cookbook page, which the recipe's notes state.Tests: in both frameworks:
Against the real Coinbase feed, both versions went live, and Disconnect stopped them.
5. Grid packages 0.5.0
@surefold/grid,grid-solidandgrid-reactare at 0.5.0, for thegrid-v0.5.0tag that publishes them. The@surefold/griddependency in the other two, and in the three web workspaces, is updated to match, along with the lockfile. It's a minor version because section 2 adds API and changes behaviour slightly.The publish workflow's checks pass locally: the versions match, and a clean
npm ci,npm run build:packagesandnpm run checkall pass. A dry-runnpm publishshows all three packages at 0.5.0.As with 0.4.0, the tag goes on the merge commit on
mainafter this PR merges.🤖 Generated with Claude Code
https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr