Repository navigation
The Effect recipe uses Effect's Socket and Schema; its notes give Effect's SSE and HTTP - #99
Merged
Merged
Conversation
The grid pages gain a cookbook (grid/cookbook/ and grid/react/cookbook/): four recipes, each running on the page with its source under it. The source shown is the recipe's own file, imported with ?raw, so what is read is what runs, and npm run check type-checks it. - A first grid: a table, a grid, a total outside it, edits from code. - Rows from an API: fetch DummyJSON's products and putMany them, with loading and error states, fetching again, and aborting on unmount. - Load pages as you scroll: loadMore over a paged API, starting the next page at table.size, so a page asked for twice is harmless, and retrying a page that failed. - Charts from the table: GroupChart, and a chart of your own drawn from groups(), both following edits. Both pages share the words (web/grid/src/cookbook/text.ts) and styles. The grid pages and the three package READMEs link to them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Bun reads the same .npmrc, so the setup shows bun add beside npm install, and the bunfig.toml scope for projects that keep their settings there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
"A large table in a worker", in Solid and React: start a WorkerTable, load 300,000 rows into it in blocks with putMany while the page stays free, and move a thousand prices in one batch, with the total and the last recalculation's cost shown as they come. It disposes the worker when the component goes, including while the worker is still starting. The notes say which reads answer at once from the snapshot and which ask the worker, that a write's promise resolves before the totals it changes arrive, and what Vite needs for the worker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Two recipes, in Solid and React, that write a stream to the table the same way: gather messages as they come, and write them once a frame inside batch, so a burst costs one recalculation and one redraw. - Prices over a WebSocket: Coinbase's public ticker sets the price of a few coins, keeping only each coin's latest price between frames; each row is added with its first price, and the price column draws its trend. - Events over SSE: Wikimedia's stream of recent changes, through EventSource, puts a row an edit and drops the oldest in the same batch, keeping the latest 2,000, grouped by wiki. Each connects when asked and closes its connection when the component goes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
"Edit and save back to an API", in Solid and React. The table keeps what the server last had; after each recalculation that changed an input, the rows are compared with it, so whatever a person did in the grid (typing, pasting, filling a block, undoing) shows as unsaved changes, and undoing back to the saved value makes a row clean again. Save sends one PATCH a changed row with only its changed fields; a row that fails stays unsaved to retry, and only what was sent is marked saved. Discard puts the saved values back in one batch. Recent requests are listed under the grid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Prism highlights each recipe's source (TSX) and the setup's snippets (.npmrc, bash, bunfig.toml, TypeScript), with token colours for the light and dark page in cookbook.css. Both pages share web/grid/src/cookbook/highlight.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
drawGrid (and the Solid and React <Grid>) take onEdit, called with
{ cause, cells, removed } whenever a person changes the grid: each cell's
key, column, and value before and after, for typing, pasting, filling a
block, undoing and doing again, and the keys of rows they delete. Changes
made from code (put, set, drop) are not reported, so an app can save
exactly what people did.
The engine notes the cells that set_cell, undo and redo change between
record_edits and take_edits (Grid::record_edits, Grid::take_edits). The
view runs each of a person's edits between the two; with a WorkerTable
the three go as one message (RemoteEngine.sequence), so nothing else is
set in between and the same edits are reported as with a Table. Pasting
goes the same way, and RemoteEngine.setCells, which it replaces, is gone.
The cookbook's edit-and-save recipe now uses onEdit instead of comparing
every row after each recalculation, so it costs only what was edited.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@surefold/grid, grid-solid and grid-react at 0.4.0, for the tag that publishes them: the grid's onEdit says what a person edited (cells with their values before and after, and deleted rows), and RemoteEngine.setCells gives way to RemoteEngine.sequence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Each cookbook page's header (its one h1, the intro and the Solid/React switch) is now in the page's HTML rather than drawn by its script, which renders the recipes under it. So the heading shows before the script runs, and the page passes surefold.dev's site checker, which reads the HTML for one h1, a title, a language and a description. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Prism nests the JavaScript in a JSX attribute ({…}) inside the tag's token,
so names there that have no colour of their own (orders in onClick, title
and decimals in keyColumn) took the tag's red. They now take the code's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Its code background is --code-bg, the name surefold.dev's watershed theme sets, so on the site code blocks have the site's code colour rather than the cookbook's own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
… claude/stoic-pascal-v0q6pz
"Live tables in the browser" says the cookbook shows the grid in an app, and the Browser part of getting started says what it has and links it in Solid and React. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
If loadMore threw, the grid said so through onError and then took it as
having no more rows, so it never asked again. Now a failed page leaves the
grid able to ask: the next scroll near the end, or view.fetchMore(), calls
loadMore again. onStatus says it failed, with { loading: false, done: false,
failed: true, error }.
The cookbook's paging recipes no longer catch the error and resolve to true
to keep the grid asking: loadMore throws, and onStatus shows that the page
did not load.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The worker replied to a write before the recalculation it set off, so after await table.putMany(...), total() and size read the snapshot from before it. Now the worker recalculates before it replies, so the snapshot with the write reaches the page first, and a write's promise resolves once the page has it. A write that changes nothing resolves when the worker answers, as before. Writes inside batch resolve once the batch is answered, rather than at once. sort, filter, search and group resolve the same way. Table gains flushWaiting (internal), which recalculates only if writes are waiting, so a reply to a read sets off no snapshot. The worker recipe counts the rows loaded with table.size after each block, and its note says what the promise waits for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
useReading compares what it reads with ===, so a list such as groups()
had to be read as a string and parsed. It now takes isEqual: the hook
keeps the value it gave until a new one compares unequal, so the same
array goes to a chart or memo until something in it changed. useGroups
(table) is that for groups(), a new array only when a group, its row count
or a total changed.
In Solid, createReading takes the same isEqual, and createGroups follows
groups() and is set only when they changed. Both keep the table's type for
the read function, so (t) => t.total("pnl") is checked against its columns.
The chart recipes use them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
useTable(() => new Table({ fields: { … } })) gave a table of any fields:
its type parameter extended Table<any, any>, which TypeScript took as the
type new Table(…) should have, so the fields and formulas were not
inferred. It now asks only for what it uses, dispose(), and the table
keeps its names and kinds.
test/types.tsx (React) and test/types.ts (Solid) are compiled by npm run
check and not built: a misspelt field or column, a value of the wrong kind
and a row missing a field must each be an error. They fail with the old
useTable.
The React edit-and-save recipe makes its table inline, as the other
recipes do, rather than in a function outside the component.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
drawGrid takes rowClass, (key) => classes, and gives a row what it returns each time the row is drawn, so the classes follow the row's key as people scroll and sort, with a Table or a WorkerTable. When what it would return changes, view.redrawRows() draws the rows in view again. The Solid and React <Grid> pass it on, read when called, as they do the handlers. The edit-and-save recipes class rows with unsaved edits "unsaved", which cookbook.css tints with a warm colour mixed into the page's surface, and a mark by the key, in light and dark. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A new workspace, web/e2e, with Playwright 1.63.0. `npm run e2e` builds a small fixture page into web/dist/e2e and runs the tests in Chromium against web/dist, served by python3 -m http.server (after web/build.sh). - onEdit, with a Table and a WorkerTable: typing, undo, redo, a 2x2 fill, a paste that skips a value that is not a number, deleting a selected row, and changes from code (set, patch, put, drop) that must not be reported. The exact cells, before and after, are checked. - Every recipe in the Solid and React cookbooks, with DummyJSON, Coinbase's WebSocket, Wikimedia's SSE stream and Google Fonts mocked; any other request off the test server, and any page or console error, fails the test. - Both grid pages load, and an edited cell is put back by Ctrl+Z. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
After web/build.sh and the type check: install Playwright's Chromium and run `npm run e2e`. The suite takes about 25 seconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A total over a collection or a grouping was refused by a plan whenever a varied input reached it, so models built the sum as a tree of + nodes by hand. A plan now lowers a total (sum, min or max of f64) into the fold the graph itself keeps: the balanced tree with room for the next power of two, each slot its two halves combined. So its result is the graph's, bit for bit. - Room no member holds, and members absent in every scenario, are the operation's identity, which combined with anything gives it back unchanged, so they are left out without changing a bit. - Halves whose members are the same in every scenario are combined when the plan is made, in the same order, into one constant. - A member present in only some scenarios is when(present, value, identity). - Varied halves are combined by three new instructions, AGG_SUM, AGG_MIN and AGG_MAX (26 to 28; 25 is fuse::LOCAL), on the CPU, in the generated kernels and in the general kernel: a NaN wins, the left one first, as the graph's totals do (MIN and MAX let a number win over a NaN). Counts stay refused: they are whole numbers. A plan's file is version 3, so an older crate refuses one that may hold the new instructions; versions 1 and 2 are still read. A shape longer than 1,024 instructions (a large total is one) is run by the general kernel rather than written as its own. Tests: forty random totals over members that vary, are shared, are fixed, or come and go, with NaN, signed zeros and infinities, against the graph bit for bit after shared inputs change; a collection's sum, max and min with a removed row, read by a later node and saved and read back; a file of the version before. The kernels' C for the three instructions gives the graph's combine bit for bit on all 49 pairs of 0, -0, 1.5, -2, +-inf and NaN (checked with gcc; no GPU here to run them). On a 1,000,000-row collection the plan compiled in 0.9 s and ran a scenario in 7.3 ms on the CPU, where the graph took 130 ms. Not yet: a total is one node, so a GPU adds its members in one thread per scenario, and members' values are kept until the total reads them. docs/scenario-plans.md#totals says how both would go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
…ed writes - The paged recipe: a page answered with a 500 says it did not load, and the next scroll asks for it again and loads the rest. - Edit and save: the rows with something to save have the "unsaved" class (rowClass), a row undone back to clean loses it, and saving clears it. - onEdit's fixture: an awaited set, put or drop is in the totals and size read straight after it, with a Table and a WorkerTable. With the WorkerTable fix taken back out, the WorkerTable case fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A failed page is asked for again on the next scroll near the end, but a list already at its end cannot scroll further, so no scroll comes: the recipe's "Scroll to try again" could not be done there. It now says the page did not load and shows a Try again button that calls fetchMore(), until a page loads. The note says why. The browser test makes the page fail until it is told to stop, waits for the button (which shows once the failure has settled), then clicks it; scrolling then loads the rest. It passed 20 runs on its own and 114 tests in a stress run of the whole suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The WebSocket recipe again, in Solid and React, with Effect doing what the hand-written one leaves out. The Coinbase connection is a Stream.callback whose socket is opened with acquireRelease, so it is closed however the stream stops, and the server closing it fails the stream. Stream.retry opens another on a Schedule: exponential from 0.5 s, capped at 10 s, jittered, starting again once a message comes through; the schedule's tap puts "Reconnecting in N s (attempt k)" on the page. groupedWithin(500, 50 ms) batches the ticks, and each batch is one table.batch keeping the latest price of each coin. Connect runs it with Effect.runFork; Disconnect, onCleanup and the React effect's cleanup interrupt the fiber. Same table, formulas and look as the WebSocket recipe. effect 4.1.2, pinned, is a dependency of web/grid and web/grid-react only; the published packages do not depend on it. The notes say what to install, what it costs (Effect's part is 55 KB minified, 19 KB gzipped; each cookbook page's JavaScript went from 55 to 77 KB gzipped in Solid, 118 to 140 KB in React) and how the same shape fits SSE and polling. Browser tests, both frameworks, all mocked: rows and gain follow the ticks; a connection the server drops is opened again after the first wait and subscribes again, the waits grow while the server refuses and start again after a connection that worked; Disconnect closes the socket and stops a pending reconnect; unmounting does the same, on a fixture page per framework that mounts the recipe with an Unmount button (the e2e build now compiles Solid and React JSX for them); leaving the page while waiting makes no more connections. The Coinbase mock counts connections and subscriptions, and can drop a connection or refuse new ones. With the cleanup taken out of either recipe, the unmount test fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The random totals (sum, min and max over members that vary, are shared, are fixed, or come and go) now also run on the GPU, by the generated kernels and by the general kernel, and in clusters; and a 2,000-row total, longer than a shape gets its own kernel for, runs by the general one. A clustered plan's totals and the long total are checked on the CPU too, so they run where there is no GPU. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@surefold/grid, grid-solid and grid-react at 0.5.0, for the tag that publishes them: a failed loadMore page is asked for again; an awaited WorkerTable write resolves once its totals are in; useGroups and createGroups, and an isEqual for useReading and createReading; useTable keeps an inline table's types; rowClass and redrawRows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The recipe wired the browser's WebSocket into a Stream by hand: a Stream.callback, acquireRelease to close the socket, onmessage offering to a queue, and an Effect.callback waiting on onclose to fail the stream. Effect has this already: Socket.makeWebSocket (effect/socket) dials when its reader is acquired, closes the socket when the scope ends, and fails every read once the socket closes, the server's close included. So the connection is now a Stream.unwrap that dials, sends the subscription through the socket's writer (once a connection, between dialling and the first read) and reads the frames with Stream.fromEffectRepeat. The retry schedule, the batching and the fiber are as before. Socket costs bundle: it brings in Effect's channels and layers, and each cookbook page's JavaScript grew by 19 KB gzipped (Solid 77 to 96 KB, React 140 to 159 KB). The notes say so, and that effect/socket is marked unstable. The SSE note now says how to build the stream by hand, since Effect has no EventSource. All 48 browser tests pass, among them the mocked reconnect, backoff, Disconnect, unmount and leaving-the-page tests in both frameworks. Against Coinbase's real feed, both pages stream (300+ messages in 6 s) and stop on Disconnect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
…d HTTP
Going over the recipe for anything Effect already does:
- Each message was JSON.parse'd, cast, and checked by a hand-written type
guard, with the price turned into a number and checked for NaN later.
Now a Schema (fromJsonString of a Struct with FiniteFromString) decodes
it, and Stream.filterMap keeps what decodes. Socket already brings in
Schema, so this adds under 1 KB gzipped.
- The SSE note said Effect has no EventSource of its own and showed a
Stream.callback by hand. Effect has HttpClient and Sse.decode
(effect/encoding), and the note now shows those. Checked against
Wikimedia's stream in Chromium: it fails with a transport error until
HttpClient's trace headers are turned off (the browser's preflight for
traceparent and b3 is refused), so a note says how, with
HttpClient.TracerPropagationEnabled.
- The polling note uses HttpClient.get rather than fetch in tryPromise.
- The unstable note covers HttpClient and Sse too.
- Both price recipes kept a Set of the coins with a row beside the table;
they ask the table (coins.get) instead.
Every other claim the notes make about Effect was checked against its
4.1.2 source: Schedule.upTo({ times }), jittered's 0.8 to 1.2, Stream.retry
resetting its schedule after the first element, and the abort signal.
All 48 browser tests pass. Against Coinbase, both pages stream and stop on
Disconnect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cookbook's "Streams and retries with Effect" recipe built several things by hand that Effect already provides. This PR switches them to Effect's own modules. It changes only the cookbook, not the packages, so no version bump is needed.
What changes
Socket.makeWebSocket(effect/socket). The old version used aStream.callbackwithacquireRelease,onmessagepushing into a queue, and anEffect.callbackwaiting foronclose.Socketcloses the socket when the stream's scope ends, and every close fails the read (including the server's), soStream.retry(backoff)reconnects as before.Schema, replacingJSON.parse+ a cast + a hand-written type guard + a laterisFinitecheck:EventSourceand showed a hand-built stream. It now showsHttpClient+Sse.decode(effect/encoding). I tested this against Wikimedia's stream in Chromium, and it fails with a transport error untilHttpClient's tracing headers (traceparent,b3) are turned off, because the browser's preflight for them is refused. A note says how:Effect.provideService(HttpClient.TracerPropagationEnabled, false).HttpClient.getrather thanfetchinsidetryPromise.Setof the coins that have a row, alongside the table. They now ask the table (coins.get).I checked the notes' other claims about Effect against its 4.1.2 source:
Schedule.upTo({ times }),jitteredscaling by 0.8 to 1.2,Stream.retryresetting its schedule after the first element, and the abort signal passed totryPromise.Cost
Socketpulls in Effect's channels, layers andSchema. Each cookbook page's JavaScript grows by about 19 KB gzipped:Adding
Schemaon top costs under 1 KB, sinceSocketalready includes it. The notes say this, and thateffect/socket,effect/httpandeffect/encodingare marked unstable (Effect is pinned at 4.1.2).Checks
npm run checkinweb/gridandweb/grid-react;web/build.sh.npm run e2e: 48/48 pass, including the mocked Effect tests in both frameworks. They cover rows following ticks; a dropped connection reopening after a backoff and subscribing again; waits growing while refused; Disconnect and unmount closing the socket and cancelling a pending reconnect; and no reconnects after leaving the page.🤖 Generated with Claude Code
https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr