Skip to content

Plans: totals in pieces with no slot a member, counts, choices, deep expressions - #100

Merged
matthewjones372 merged 1 commit into
mainfrom
claude/stoic-pascal-v0q6pz
Nov 16, 2026
Merged

matthewjones372 merged 1 commit into
mainfrom
claude/stoic-pascal-v0q6pz

Conversation

@matthewjones372

Copy link
Copy Markdown
Collaborator

This fixes the two totals limits left open by #98 and lets plans run more of what they used to refuse. It adds no new instructions, so the CUDA kernels and the plan file format are unchanged.

1. Totals: pieces, and no slot per member

  • Pieces. A large total is cut into whole subtrees of the same number of members: a power of two, as many as keep a piece under about 512 instructions, at most 256. Each piece is a node, pieces of pieces are cut the same way, and the total combines the top ones. Every piece is a whole subtree of the graph's tree, so the order of combining, and therefore the result, is still the graph's bit for bit. The pieces at a level share one shape and are short enough for a kernel each, so a GPU works them out side by side instead of using one thread per total per scenario.
  • Members computed inside their piece. A node read only by totals (typically a column read by its sum, min and max) and at most 24 instructions long gets no slot. Each total computes it inside its piece.

1,000,000-row collection, a plan of sum + max, same machine, old code built in a worktree for comparison:

before after
compile ~0.88 s ~0.78 s
memory per scenario 8 MB 254 KB (31,750 pieces)
CPU run per scenario (4 cores) ~6.0 ms ~4.6 ms

2. What plans refused that they now run

  • Counts. A count adds 1 for each member present. That's exact in an f64 up to 2^53, and it comes out as the f64 of the same number. A whole number can be compared, and the smaller or larger of two taken. Arithmetic on whole numbers is still refused, because the graph's integer arithmetic doesn't round and can overflow. Outputs may now be whole numbers.
  • choose. It's lowered to a chain of conditionals on index == k. The index must be the same in every scenario and in range; otherwise the plan refuses the node by name (as HoldsError), because the graph would hold an error. The constant folding below then leaves only the chosen option. A choose whose index varies (only possible via a count) is refused by name.
  • Constant folding. Any part of a node's program that reads nothing varied or shared is computed when the plan is made, by running it, so it gives exactly what running it would. A constant condition keeps only the branch taken. Nodes with nothing to fold are passed through without being copied.
  • Deep expressions. Programs needing a stack deeper than 32 used to fail with TooDeep. Now the operand that needs the most stack becomes a node of its own until every piece fits. A 300-row lookup written as nested conditionals (how the spreadsheet example writes VLOOKUP) runs as about 20 nodes. TooDeep stays in the enum but is no longer returned.

Tests

  • Random totals: the 40 random totals now include counts. They also passed with every total forced into 2-member pieces, the most pieces possible (checked by hand, not kept in CI).
  • New CPU tests:
    • a 100,000-row total in pieces with no slot per row, also read back from a file;
    • counts of groups whose rows change with the varied input (the test asserts the counts really vary), also in clusters;
    • choose for each option, out of range, and by a varying index;
    • a 300-row lookup split into nodes.
  • New GPU tests (--features cuda):
    • the 100,000-row total (also in clusters), replacing the "too long for its own kernel" test, since that case no longer happens;
    • counts;
    • the deep lookup.
  • Unit tests: every piece of a 100k-member total is short and there are few shapes, and constant folding works as intended.
  • Workspace: cargo test --workspace: 238 passed. cargo clippy --workspace --all-targets and with --features cuda are clean. cargo fmt --check passes.
  • Examples: every watershed-gpu example, run at a small size against the old code, gives identical output (timings aside), and plan-build times are on par.

The GPU tests haven't been run here: there's no GPU in this environment. They need running on a CUDA machine.

Docs

  • docs/scenario-plans.md: compilation steps, totals, pieces, and members inside their piece.
  • docs/limitations.md.
  • The README's limits, which still said collection totals were incremental only.
  • The crate README and lib.rs.
  • docs/testing.md.

Generated by Claude Code

…expressions

Totals in pieces. A total was one node of a plan: on a GPU each scenario
added its members in one thread, and every member's value kept a slot until
the total read it (8 MB a scenario at a million rows). Now a large total is
cut into whole subtrees of the same number of members (a power of two, as
many as keep a piece under about 512 instructions, at most 256), each a
node, and those again; the total combines the top ones. Each piece is a
whole subtree of the graph's tree, so the order of combining, and the
result, are the graph's bit for bit. The pieces of a level are one shape and
short enough for a kernel each, so a GPU works them out side by side.

Members worked out in their piece. A node read only by totals (a column
read by its sum, min and max, say), and at most 24 instructions, gets no
slot: each total works it out inside its piece. On a 1,000,000-row
collection, a plan of the sum and the maximum now compiles in 0.8 s (0.9 s
before), needs 254 KB a scenario (8 MB before) and runs a scenario in 4.6 ms
on the CPU on four cores (6 ms before), with the same results.

Counts. A count adds 1 for each member present, exact in an f64 up to 2^53,
and comes out as the f64 of the same number. A whole number can be compared
and the smaller or larger of two taken; arithmetic on whole numbers is still
refused (the graph's does not round, and can overflow). An output may now be
a whole number.

Choices. choose(index, options) is a chain of conditionals on index == k.
The index must be the same in every scenario and in range, or the plan
refuses the node by name (the graph would hold an error); then the choice
folds to the option it picks.

Constant folding. A part of a node's program that reads nothing varied or
shared is worked out when the plan is made, by running it, so it gives what
running it would; a constant condition keeps only the branch taken. Nodes
with nothing to fold are passed through without being copied.

Deep expressions. A program that needs a stack deeper than 32 was refused
(TooDeep). Now the operand that needs most is made a node of its own until
every piece fits, so a lookup written as hundreds of nested conditionals, as
the spreadsheet example writes VLOOKUP, runs. TooDeep is no longer returned.

No new instructions: kernels and the plan file format are unchanged.

Tests: random totals now include counts, and passed with every total cut
into pieces of two members (checked by hand). New: a 100,000-row total in
pieces with no slot a row, read back from a file; counts of groups whose rows
change with the varied input, also in clusters; choices of each option, out
of range, and by a number that varies; a 300-row lookup split into nodes;
on the GPU, the large total (also in clusters), the counts and the lookup,
replacing the test of a total too long for its own kernel; unit tests that
every piece is short and few shapes, and of constant folding. Every example
in watershed-gpu, at a small size, gives the same output as before.

Docs: scenario-plans.md (compilation, totals, pieces), limitations.md, the
README's limits (which still said totals were incremental only), the crate
README and lib.rs, testing.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AdzBXcxXfE7iKtPMVoTTfr
@matthewjones372
matthewjones372 merged commit daff1b3 into main Nov 16, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants