Skip to content

A collection's groupings by the same key share their membership nodes - #102

Merged
matthewjones372 merged 2 commits into
mainfrom
claude/stoic-pascal-v0q6pz
Nov 16, 2026
Merged

matthewjones372 merged 2 commits into
mainfrom
claude/stoic-pascal-v0q6pz

Conversation

@matthewjones372

Copy link
Copy Markdown
Collaborator

Follows #101. A row's membership node (present && key == group) doesn't depend on what a grouping adds up, yet every grouping had its own. So a sum and a count per group made two nodes per row, and the grid, which makes one grouping per totalled column, made six for six totals.

Changes

  • Shared membership nodes. A collection's groupings by the same field or column now share them, through a Pool per key. The first grouping makes a row's node; the others reuse it, each adding it to its own group's total.
  • Taken-over nodes, with sharing. When A collection's group membership nodes are reused with its rows #101 takes a removed row's node for another group:
    • the pool recompiles and renames it once;
    • each grouping, as it follows the row, empties its member in the old group's total and joins the new one.
  • Smaller bookkeeping.
    • Group keys are numbered once per pool, so a grouping holds a row's group and its counts as numbers, not strings.
    • A row's membership entries (nearly always one) are held in place, with a separate list only for rows that have more.
  • The grid's group budget rises from 100,000 row-and-column pairs to 300,000. That costs about the memory the old budget allowed.

Numbers

50,000 rows, six totals, grouped by one column with 100 groups. Measured against main built in a worktree on the same machine:

before after
core collection: nodes added 300,600 50,600
core collection: memory 226 MB 54 MB
core collection: time to add the groupings 1.44 s 0.42 s
moving 1,000 rows between groups 47 ms 16 ms
grid set_group: memory 256 MB 74 MB
grid set_group: time 1.21 s 0.40 s

Tests

  • New: groupings by one key make one membership node per row between them, and a row that moves leaves all of their old group's totals.
  • Existing property test: collections_match_a_plain_map_added_up_from_scratch runs a sum and a count by a field and by a calculated column, with removals and reuse. It passes, and fails at step 7 when a grouping doesn't empty its old group's member on a shared node taken for another group. That was checked by hand and recorded in docs/testing.md.
  • Workspace: cargo test --workspace passes. Clippy, including --features cuda, and cargo doc are clean.
  • Browser: the 48 browser tests pass against the rebuilt WebAssembly.

Docs

  • watershed-core's README.
  • The grid README.
  • limitations.md.
  • benchmarks.md: the grid's grouping figure has been re-measured, so it's off the "not re-measured" list.
  • testing.md.

Generated by Claude Code

A grouping puts a row in a group through a membership node, present && key
== that group, which does not depend on what the grouping adds up. Yet each
grouping had its own: a sum and a count of each group made two a row, and
the grid, which makes a grouping per totalled column, six for six totals.

Now a collection's groupings by the same field or column share them (a
Pool per key): the first grouping makes a row's membership node, the others
use it, each adding it to its own group's total. When a removed row's node
is taken for another group (#101), the pool compiles and renames it once,
and each grouping, as it follows the row, empties its member in the old
group's total and joins the new one.

The bookkeeping is smaller too. Group keys are numbered once a pool, so a
grouping holds a row's group and its counts as numbers, not strings, and a
row's membership entries (nearly always one) are held in place, with a list
only for rows that have more.

Fifty thousand rows, six totals grouped by one column of 100 groups:

  core collection: 300,600 nodes, 226 MB, 1.44 s -> 50,600 nodes, 54 MB,
  0.42 s; moving 1,000 rows between groups 47 ms -> 16 ms.
  grid: 256 MB, 1.21 s -> 74 MB, 0.40 s.

The grid's budget for keeping group totals in the graph rises from 100,000
row-and-column pairs to 300,000, about the memory the old budget allowed.

Tests: groupings by one key make a membership node a row between them, and
a row that moves leaves all their old group's totals.
collections_match_a_plain_map_added_up_from_scratch (a sum and a count by a
field and by a calculated column) passes, and fails when a grouping does not
empty its old group's member on a shared node taken for another group. All
workspace tests and the 48 browser tests pass.

Docs: watershed-core's README, the grid README, limitations.md,
benchmarks.md (the grid's grouping figure is measured again), testing.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
The React cookbook's "a page that fails to load is asked for again" timed
out in CI waiting to click Try again: the button had been visible, then
went before the click. Showing the failure changes the bar above the grid
(a longer message, and the button), which can move the grid and give it one
more scroll event near the end; the grid then asks for the page again, as
it does on any scroll near the end. If that request came after the test had
stopped failing the page, the page loaded and the button rightly went.

The test meant to wait until the failure had settled ("with nothing asked
for since") but only waited for the button. It now waits until half a
second passes with no request for the page and the button showing, then
stops failing it and clicks. It passed 40 runs of its own here, and the
suite passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@matthewjones372
matthewjones372 merged commit 985c843 into main Nov 16, 2026
9 checks passed
@matthewjones372
matthewjones372 deleted the claude/stoic-pascal-v0q6pz branch November 16, 2026 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants