Skip to content

The Effect recipe uses Effect's Socket and Schema; its notes give Effect's SSE and HTTP - #99

Merged
matthewjones372 merged 33 commits into
mainfrom
claude/stoic-pascal-v0q6pz
Nov 16, 2026
Merged

matthewjones372 merged 33 commits into
mainfrom
claude/stoic-pascal-v0q6pz

Conversation

@matthewjones372

@matthewjones372 matthewjones372 commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

The cookbook's "Streams and retries with Effect" recipe built several things by hand that Effect already provides. This PR switches them to Effect's own modules. It changes only the cookbook, not the packages, so no version bump is needed.

What changes

  1. The WebSocket: it now uses Socket.makeWebSocket (effect/socket). The old version used a Stream.callback with acquireRelease, onmessage pushing into a queue, and an Effect.callback waiting for onclose.
    const connection = Stream.unwrap(
      Effect.gen(function* () {
        const socket = yield* Socket.makeWebSocket(FEED);
        const read = yield* Socket.readerString(socket);   // dials, waits for open
        const { write: send } = yield* socket.writer;
        yield* send(JSON.stringify({ type: "subscribe", … })); // once per connection
        return Stream.fromEffectRepeat(read).pipe(Stream.flattenIterable);
      }),
    );
    Socket closes the socket when the stream's scope ends, and every close fails the read (including the server's), so Stream.retry(backoff) reconnects as before.
  2. Decoding: messages are decoded with Schema, replacing JSON.parse + a cast + a hand-written type guard + a later isFinite check:
    const Ticker = Schema.Struct({ type: Schema.Literal("ticker"), product_id: Schema.String, price: Schema.FiniteFromString });
    Stream.filterMap(Filter.fromPredicateOption(Schema.decodeUnknownOption(Schema.fromJsonString(Ticker))))
  3. The SSE note was wrong. It said Effect has no EventSource and showed a hand-built stream. It now shows HttpClient + Sse.decode (effect/encoding). I tested this against Wikimedia's stream in Chromium, and it fails with a transport error until HttpClient's tracing headers (traceparent, b3) are turned off, because the browser's preflight for them is refused. A note says how: Effect.provideService(HttpClient.TracerPropagationEnabled, false).
  4. The polling note uses HttpClient.get rather than fetch inside tryPromise.
  5. Both price recipes (Effect and plain WebSocket) kept a Set of the coins that have a row, alongside the table. They now ask the table (coins.get).

I checked the notes' other claims about Effect against its 4.1.2 source: Schedule.upTo({ times }), jittered scaling by 0.8 to 1.2, Stream.retry resetting its schedule after the first element, and the abort signal passed to tryPromise.

Cost

Socket pulls in Effect's channels, layers and Schema. Each cookbook page's JavaScript grows by about 19 KB gzipped:

  • Solid: 77 → 96 KB.
  • React: 140 → 159 KB.

Adding Schema on top costs under 1 KB, since Socket already includes it. The notes say this, and that effect/socket, effect/http and effect/encoding are marked unstable (Effect is pinned at 4.1.2).

Checks

  • npm run check in web/grid and web/grid-react; web/build.sh.
  • npm run e2e: 48/48 pass, including the mocked Effect tests in both frameworks. They cover rows following ticks; a dropped connection reopening after a backoff and subscribing again; waits growing while refused; Disconnect and unmount closing the socket and cancelling a pending reconnect; and no reconnects after leaving the page.
  • Against Coinbase's real feed, both cookbook pages stream with no page errors. After Disconnect the message count stays fixed.
  • The SSE snippet streams Wikimedia's real events in Chromium once tracing headers are off.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr

The grid pages gain a cookbook (grid/cookbook/ and grid/react/cookbook/):
four recipes, each running on the page with its source under it. The
source shown is the recipe's own file, imported with ?raw, so what is read
is what runs, and npm run check type-checks it.

- A first grid: a table, a grid, a total outside it, edits from code.
- Rows from an API: fetch DummyJSON's products and putMany them, with
  loading and error states, fetching again, and aborting on unmount.
- Load pages as you scroll: loadMore over a paged API, starting the next
  page at table.size, so a page asked for twice is harmless, and retrying
  a page that failed.
- Charts from the table: GroupChart, and a chart of your own drawn from
  groups(), both following edits.

Both pages share the words (web/grid/src/cookbook/text.ts) and styles.
The grid pages and the three package READMEs link to them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Bun reads the same .npmrc, so the setup shows bun add beside npm install,
and the bunfig.toml scope for projects that keep their settings there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
"A large table in a worker", in Solid and React: start a WorkerTable,
load 300,000 rows into it in blocks with putMany while the page stays
free, and move a thousand prices in one batch, with the total and the
last recalculation's cost shown as they come. It disposes the worker when
the component goes, including while the worker is still starting.

The notes say which reads answer at once from the snapshot and which ask
the worker, that a write's promise resolves before the totals it changes
arrive, and what Vite needs for the worker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Two recipes, in Solid and React, that write a stream to the table the
same way: gather messages as they come, and write them once a frame
inside batch, so a burst costs one recalculation and one redraw.

- Prices over a WebSocket: Coinbase's public ticker sets the price of a
  few coins, keeping only each coin's latest price between frames; each
  row is added with its first price, and the price column draws its trend.
- Events over SSE: Wikimedia's stream of recent changes, through
  EventSource, puts a row an edit and drops the oldest in the same batch,
  keeping the latest 2,000, grouped by wiki.

Each connects when asked and closes its connection when the component goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
"Edit and save back to an API", in Solid and React. The table keeps what
the server last had; after each recalculation that changed an input, the
rows are compared with it, so whatever a person did in the grid (typing,
pasting, filling a block, undoing) shows as unsaved changes, and undoing
back to the saved value makes a row clean again. Save sends one PATCH a
changed row with only its changed fields; a row that fails stays unsaved
to retry, and only what was sent is marked saved. Discard puts the saved
values back in one batch. Recent requests are listed under the grid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Prism highlights each recipe's source (TSX) and the setup's snippets
(.npmrc, bash, bunfig.toml, TypeScript), with token colours for the light
and dark page in cookbook.css. Both pages share web/grid/src/cookbook/highlight.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
drawGrid (and the Solid and React <Grid>) take onEdit, called with
{ cause, cells, removed } whenever a person changes the grid: each cell's
key, column, and value before and after, for typing, pasting, filling a
block, undoing and doing again, and the keys of rows they delete. Changes
made from code (put, set, drop) are not reported, so an app can save
exactly what people did.

The engine notes the cells that set_cell, undo and redo change between
record_edits and take_edits (Grid::record_edits, Grid::take_edits). The
view runs each of a person's edits between the two; with a WorkerTable
the three go as one message (RemoteEngine.sequence), so nothing else is
set in between and the same edits are reported as with a Table. Pasting
goes the same way, and RemoteEngine.setCells, which it replaces, is gone.

The cookbook's edit-and-save recipe now uses onEdit instead of comparing
every row after each recalculation, so it costs only what was edited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@surefold/grid, grid-solid and grid-react at 0.4.0, for the tag that
publishes them: the grid's onEdit says what a person edited (cells with
their values before and after, and deleted rows), and RemoteEngine.setCells
gives way to RemoteEngine.sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Each cookbook page's header (its one h1, the intro and the Solid/React
switch) is now in the page's HTML rather than drawn by its script, which
renders the recipes under it. So the heading shows before the script runs,
and the page passes surefold.dev's site checker, which reads the HTML for
one h1, a title, a language and a description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Prism nests the JavaScript in a JSX attribute ({…}) inside the tag's token,
so names there that have no colour of their own (orders in onClick, title
and decimals in keyColumn) took the tag's red. They now take the code's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
Its code background is --code-bg, the name surefold.dev's watershed theme
sets, so on the site code blocks have the site's code colour rather than
the cookbook's own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
"Live tables in the browser" says the cookbook shows the grid in an app,
and the Browser part of getting started says what it has and links it in
Solid and React.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
If loadMore threw, the grid said so through onError and then took it as
having no more rows, so it never asked again. Now a failed page leaves the
grid able to ask: the next scroll near the end, or view.fetchMore(), calls
loadMore again. onStatus says it failed, with { loading: false, done: false,
failed: true, error }.

The cookbook's paging recipes no longer catch the error and resolve to true
to keep the grid asking: loadMore throws, and onStatus shows that the page
did not load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The worker replied to a write before the recalculation it set off, so
after await table.putMany(...), total() and size read the snapshot from
before it. Now the worker recalculates before it replies, so the snapshot
with the write reaches the page first, and a write's promise resolves once
the page has it. A write that changes nothing resolves when the worker
answers, as before. Writes inside batch resolve once the batch is
answered, rather than at once. sort, filter, search and group resolve the
same way.

Table gains flushWaiting (internal), which recalculates only if writes
are waiting, so a reply to a read sets off no snapshot.

The worker recipe counts the rows loaded with table.size after each
block, and its note says what the promise waits for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
useReading compares what it reads with ===, so a list such as groups()
had to be read as a string and parsed. It now takes isEqual: the hook
keeps the value it gave until a new one compares unequal, so the same
array goes to a chart or memo until something in it changed. useGroups
(table) is that for groups(), a new array only when a group, its row count
or a total changed.

In Solid, createReading takes the same isEqual, and createGroups follows
groups() and is set only when they changed. Both keep the table's type for
the read function, so (t) => t.total("pnl") is checked against its columns.

The chart recipes use them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
useTable(() => new Table({ fields: { … } })) gave a table of any fields:
its type parameter extended Table<any, any>, which TypeScript took as the
type new Table(…) should have, so the fields and formulas were not
inferred. It now asks only for what it uses, dispose(), and the table
keeps its names and kinds.

test/types.tsx (React) and test/types.ts (Solid) are compiled by npm run
check and not built: a misspelt field or column, a value of the wrong kind
and a row missing a field must each be an error. They fail with the old
useTable.

The React edit-and-save recipe makes its table inline, as the other
recipes do, rather than in a function outside the component.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
drawGrid takes rowClass, (key) => classes, and gives a row what it
returns each time the row is drawn, so the classes follow the row's key
as people scroll and sort, with a Table or a WorkerTable. When what it
would return changes, view.redrawRows() draws the rows in view again.
The Solid and React <Grid> pass it on, read when called, as they do the
handlers.

The edit-and-save recipes class rows with unsaved edits "unsaved", which
cookbook.css tints with a warm colour mixed into the page's surface, and a
mark by the key, in light and dark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A new workspace, web/e2e, with Playwright 1.63.0. `npm run e2e` builds a
small fixture page into web/dist/e2e and runs the tests in Chromium
against web/dist, served by python3 -m http.server (after web/build.sh).

- onEdit, with a Table and a WorkerTable: typing, undo, redo, a 2x2
  fill, a paste that skips a value that is not a number, deleting a
  selected row, and changes from code (set, patch, put, drop) that must
  not be reported. The exact cells, before and after, are checked.
- Every recipe in the Solid and React cookbooks, with DummyJSON,
  Coinbase's WebSocket, Wikimedia's SSE stream and Google Fonts mocked;
  any other request off the test server, and any page or console error,
  fails the test.
- Both grid pages load, and an edited cell is put back by Ctrl+Z.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
After web/build.sh and the type check: install Playwright's Chromium
and run `npm run e2e`. The suite takes about 25 seconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A total over a collection or a grouping was refused by a plan whenever a
varied input reached it, so models built the sum as a tree of + nodes by
hand. A plan now lowers a total (sum, min or max of f64) into the fold the
graph itself keeps: the balanced tree with room for the next power of two,
each slot its two halves combined. So its result is the graph's, bit for
bit.

- Room no member holds, and members absent in every scenario, are the
  operation's identity, which combined with anything gives it back
  unchanged, so they are left out without changing a bit.
- Halves whose members are the same in every scenario are combined when the
  plan is made, in the same order, into one constant.
- A member present in only some scenarios is when(present, value, identity).
- Varied halves are combined by three new instructions, AGG_SUM, AGG_MIN
  and AGG_MAX (26 to 28; 25 is fuse::LOCAL), on the CPU, in the generated
  kernels and in the general kernel: a NaN wins, the left one first, as the
  graph's totals do (MIN and MAX let a number win over a NaN).

Counts stay refused: they are whole numbers. A plan's file is version 3, so
an older crate refuses one that may hold the new instructions; versions 1
and 2 are still read. A shape longer than 1,024 instructions (a large total
is one) is run by the general kernel rather than written as its own.

Tests: forty random totals over members that vary, are shared, are fixed,
or come and go, with NaN, signed zeros and infinities, against the graph bit
for bit after shared inputs change; a collection's sum, max and min with a
removed row, read by a later node and saved and read back; a file of the
version before. The kernels' C for the three instructions gives the graph's
combine bit for bit on all 49 pairs of 0, -0, 1.5, -2, +-inf and NaN
(checked with gcc; no GPU here to run them). On a 1,000,000-row collection
the plan compiled in 0.9 s and ran a scenario in 7.3 ms on the CPU, where
the graph took 130 ms.

Not yet: a total is one node, so a GPU adds its members in one thread per
scenario, and members' values are kept until the total reads them.
docs/scenario-plans.md#totals says how both would go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
…ed writes

- The paged recipe: a page answered with a 500 says it did not load, and the
  next scroll asks for it again and loads the rest.
- Edit and save: the rows with something to save have the "unsaved" class
  (rowClass), a row undone back to clean loses it, and saving clears it.
- onEdit's fixture: an awaited set, put or drop is in the totals and size
  read straight after it, with a Table and a WorkerTable. With the
  WorkerTable fix taken back out, the WorkerTable case fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
A failed page is asked for again on the next scroll near the end, but a
list already at its end cannot scroll further, so no scroll comes: the
recipe's "Scroll to try again" could not be done there. It now says the page
did not load and shows a Try again button that calls fetchMore(), until a
page loads. The note says why.

The browser test makes the page fail until it is told to stop, waits for the
button (which shows once the failure has settled), then clicks it; scrolling
then loads the rest. It passed 20 runs on its own and 114 tests in a stress
run of the whole suite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The WebSocket recipe again, in Solid and React, with Effect doing what the
hand-written one leaves out. The Coinbase connection is a Stream.callback
whose socket is opened with acquireRelease, so it is closed however the
stream stops, and the server closing it fails the stream. Stream.retry opens
another on a Schedule: exponential from 0.5 s, capped at 10 s, jittered,
starting again once a message comes through; the schedule's tap puts
"Reconnecting in N s (attempt k)" on the page. groupedWithin(500, 50 ms)
batches the ticks, and each batch is one table.batch keeping the latest
price of each coin. Connect runs it with Effect.runFork; Disconnect, onCleanup
and the React effect's cleanup interrupt the fiber. Same table, formulas and
look as the WebSocket recipe.

effect 4.1.2, pinned, is a dependency of web/grid and web/grid-react only;
the published packages do not depend on it. The notes say what to install,
what it costs (Effect's part is 55 KB minified, 19 KB gzipped; each
cookbook page's JavaScript went from 55 to 77 KB gzipped in Solid, 118 to
140 KB in React) and how the same shape fits SSE and polling.

Browser tests, both frameworks, all mocked: rows and gain follow the ticks;
a connection the server drops is opened again after the first wait and
subscribes again, the waits grow while the server refuses and start again
after a connection that worked; Disconnect closes the socket and stops a
pending reconnect; unmounting does the same, on a fixture page per
framework that mounts the recipe with an Unmount button (the e2e build now
compiles Solid and React JSX for them); leaving the page while waiting
makes no more connections. The Coinbase mock counts connections and
subscriptions, and can drop a connection or refuse new ones. With the
cleanup taken out of either recipe, the unmount test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The random totals (sum, min and max over members that vary, are shared,
are fixed, or come and go) now also run on the GPU, by the generated kernels
and by the general kernel, and in clusters; and a 2,000-row total, longer
than a shape gets its own kernel for, runs by the general one. A clustered
plan's totals and the long total are checked on the CPU too, so they run
where there is no GPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@surefold/grid, grid-solid and grid-react at 0.5.0, for the tag that
publishes them: a failed loadMore page is asked for again; an awaited
WorkerTable write resolves once its totals are in; useGroups and
createGroups, and an isEqual for useReading and createReading; useTable
keeps an inline table's types; rowClass and redrawRows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The recipe wired the browser's WebSocket into a Stream by hand: a
Stream.callback, acquireRelease to close the socket, onmessage offering to
a queue, and an Effect.callback waiting on onclose to fail the stream.
Effect has this already: Socket.makeWebSocket (effect/socket) dials when
its reader is acquired, closes the socket when the scope ends, and fails
every read once the socket closes, the server's close included. So the
connection is now a Stream.unwrap that dials, sends the subscription
through the socket's writer (once a connection, between dialling and the
first read) and reads the frames with Stream.fromEffectRepeat. The retry
schedule, the batching and the fiber are as before.

Socket costs bundle: it brings in Effect's channels and layers, and each
cookbook page's JavaScript grew by 19 KB gzipped (Solid 77 to 96 KB,
React 140 to 159 KB). The notes say so, and that effect/socket is marked
unstable. The SSE note now says how to build the stream by hand, since
Effect has no EventSource.

All 48 browser tests pass, among them the mocked reconnect, backoff,
Disconnect, unmount and leaving-the-page tests in both frameworks. Against
Coinbase's real feed, both pages stream (300+ messages in 6 s) and stop
on Disconnect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
…d HTTP

Going over the recipe for anything Effect already does:

- Each message was JSON.parse'd, cast, and checked by a hand-written type
  guard, with the price turned into a number and checked for NaN later.
  Now a Schema (fromJsonString of a Struct with FiniteFromString) decodes
  it, and Stream.filterMap keeps what decodes. Socket already brings in
  Schema, so this adds under 1 KB gzipped.
- The SSE note said Effect has no EventSource of its own and showed a
  Stream.callback by hand. Effect has HttpClient and Sse.decode
  (effect/encoding), and the note now shows those. Checked against
  Wikimedia's stream in Chromium: it fails with a transport error until
  HttpClient's trace headers are turned off (the browser's preflight for
  traceparent and b3 is refused), so a note says how, with
  HttpClient.TracerPropagationEnabled.
- The polling note uses HttpClient.get rather than fetch in tryPromise.
- The unstable note covers HttpClient and Sse too.
- Both price recipes kept a Set of the coins with a row beside the table;
  they ask the table (coins.get) instead.

Every other claim the notes make about Effect was checked against its
4.1.2 source: Schedule.upTo({ times }), jittered's 0.8 to 1.2, Stream.retry
resetting its schedule after the first element, and the abort signal.

All 48 browser tests pass. Against Coinbase, both pages stream and stop on
Disconnect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@matthewjones372 matthewjones372 changed the title The Effect recipe reads its WebSocket with Effect's Socket The Effect recipe uses Effect's Socket and Schema; its notes give Effect's SSE and HTTP Nov 16, 2026
@matthewjones372
matthewjones372 merged commit f37bd93 into main Nov 16, 2026
9 checks passed
@matthewjones372
matthewjones372 deleted the claude/stoic-pascal-v0q6pz branch November 16, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants