Repository navigation
The node chunk is sized at translation, whatever the device, and each chunk is a kernel launch #361
Description
What happens
A source's node chunk is sized by SimulationSettings.node_budget at translation, whatever the backend: min(n_columns, node_budget // column_bytes), with 64 MiB on modal routes and 1 GiB on spatial ones. On a device a chunk is a kernel launch.
SEP 16 measured the 128 x 128 Cartesian filament on a data-centre GPU: 161 chunks a call at the then default of 64 MiB, and with every column in one chunk IP-DOPRI5 ran 4 times faster and the exponential formulation 2.4 times.
Those numbers are out of date. The spatial default has since gone to 1 GiB. By arithmetic, not run, with column bytes 20 * 16 * n_pol * n_to and n_pol * n_to = 2048, which reproduces SEP 16's 102 columns a chunk, that case now takes 1638 columns a chunk, so 11 chunks a call. So most of that gain may already be had, and the remaining gain is unknown.
To do
- Re-measure the filament on a data-centre GPU at the default budget and with one chunk, on Runge-Kutta and Adams, with
SOLPROP_BENCH_NODE_BUDGETor the setting. - If it still pays, let the device backend choose its chunk from the device's memory when it is built.
Constraints: translation must not import the device runtime, since a scan must not initialise it before it forks. A chunk of any size gives the same fields to rounding, which the chunked record rows assert. An explicit node_budget must still win. #89's buffer plan should see the same number.
Needs a decision: this changes a device run's default memory use, so it is the maintainer's. I recommend it only if the re-measurement shows at least 10 %.
Done when
The ratio at the current default is recorded with device and date. Then either the backend chooses its own chunk, with pytest -m gpu passing on a CUDA and a HIP build and a scan still forking before the runtime starts, or the issue closes on that number.
Checked against commit 8a9f764. Takes over the roadmap's entry "A device-sized node chunk".
Also
#59's rejected list includes "a device whole-node chunk", which the numbers above from SEP 16 predate. On one radial case the ledger also records one chunk in place of two (48 to 31 kernels a trial) as not moving it, and not adopted. Read those before re-measuring. Tracked in #367.
Activity
Split a free-space envelope's collar in place on the state
Take a device's split collar a whole state at a time
jtravs commentedon Nov 5, 2026
One part of this is measured, and it doesn't pay. #520's temporal collar now works in place on the state's own rows, so its block holds no buffer and costs only launches. So I tried the extreme, every row in one block on a device by default. At 768 x 768 x 600, 8 steps, the median of three runs on one runner, against the node chunk:
The plans' device growth is the same to 6 MiB on both. That's well short of the 10 % you asked of a default-memory change, so I dropped it, along with the
Transformfield it needed. The commit isae82a2a9on #520's branch if you want it back. It goes from 119 kernels and 220 transforms a step to 14 and 10, so launches aren't what bounds this step any more.That says nothing about the filament row this issue starts from, on Runge-Kutta and Adams, which I didn't re-measure:
cartesian_filament_128waits on #389. So this stays open for that.Keep a device's split collar at the node chunk, and draw the handler
Take a device's split collar a whole state at a time again
jtravs commentedon Nov 6, 2026
The whole-state collar is back in #542 for a CUDA or HIP build, with no seam field, as you asked. It passes your rule that an interleaved run on an AMD card has to gain in every round.
The run was on an AMD Instinct MI300X, ROCm 10.0.0, at
c0bd23f2, against the same tree at the node chunk. Each point is field-free, 8 steps with the check off, and the median of a runner's second and third runs. Three rounds of fresh processes alternated between the two builds:The plans' memory was the same, 5.76 GiB at 768.
python -m benchmarks ab --device --length 0.3, 3 rounds, giveskerr_beam_384x0.952 (0.912–0.999), with the fields identical. On an NVIDIA A40 the same comparison gavekerr_beam_384x0.989 andkerr_beam_192x0.971.So it stays. That's about 1.5 % of a point, well short of 10 %, but it costs no memory. A CPU keeps the node chunk. The rest of this issue, the chunk being sized at translation for every other source, is untouched, so I've left it open.
SEP 46: response to the AI review, Part I revised
SEP 46 phase 6: SOLPROP_BENCH_NODE_BUDGET, a node budget for one arm …
jtravs commentedon Nov 8, 2026
Re-measured on an NVIDIA H200 (CUDA 13.1), 2026-11-08, for SEP 46 phase 6 (#582), at commit
c8545546. The candidate arm runs the same build withSOLPROP_BENCH_NODE_BUDGET(new in #582);benchmarks ab --device, a fifth of each case, three rounds; ratio candidate / default (1 GiB on these spatial routes); fields equal to rounding.radial_filament_512cartesian_filament_256cartesian_filament_128kerr_beam_384(fixed-step RK4IP)kerr_beam_768(fixed-step RK4IP)The A/A control on the tracking set was within about ±3 % per case. So the chunk still pays: up to 25 % with Runge-Kutta and 6–12 % with Multistep, past this issue's 10 % bar on three of the six rows. An 8 GiB budget takes nearly all of it.
SEP 46 closes phase 6 without a calibrated device setting. The chunk is a better default, not a per-machine measurement, so it lands here as its own PR after SEP 46 (the maintainer's decision, as it raises a device run's default memory use). That PR still has to be qualified on an AMD Instinct device. The same session found graph replay paying (off:
ctc1.08,h2_raman_averaged1.07 slower) and neither a team and reduction width of 128/256 nor of 512/1024 paying beyond noise.SEP 46: Final
jtravs commentedon Nov 9, 2026
John Travers has asked for this work to be included directly in SEP 47, tracked by #588 and proposed in #589 (scope revision 2a66238). No separate prerequisite chunk PR is needed.
Phase 3 puts resource-aware chunk policy in backend lowering; phase 4 completes actual CUDA/HIP timing and qualification before adopting a changed production device rule. Explicit user budgets remain authoritative, live buffers drive the ledger, and translation must not initialize the device runtime before a scan forks. The H200 results from SEP 46 motivate the candidate; they do not constitute a landed default or HIP qualification. Preserve a qualified rule if concurrent work lands first, and measure the memory cost as well as timing.
Separate chunk-policy gains from session gains. This includes the work in the implementation plan without promising a particular fixed budget or claiming the issue's acceptance criteria have already been met.
The SEP remains Draft pending the maintainer's reading and explicit acceptance. No implementation or new measurement is claimed here.