Repository navigation
Description
What happens
Two costs at a saved plane have not been explained.
A large Cartesian plane costs about 3 s, in copies. #59 measured what repeats per saved plane of kerr_beam_384's 1.77 GB field (ledger):
| step | time |
|---|---|
| the saved plane's copy | 1.1 s |
spectrum_to_optics, the conversion to the stored convention |
1.0 s |
| the output store's copy | 0.6 s |
| the native observation | 0.35 s |
A capillary or radial plane costs 7–25 ms. #59 left this as output-layer work for a pull request of its own. Each copy is also another field-sized allocation at the plane, which matters on runs sized to memory.
A kept native save on Linux. SEP 29 recorded about 4.5 ms for a kept native Adams save, against numba_cpu's 1.6 ms, unexplained, with run totals level (1.398 s against 1.397 s, about 110 saves). The native march (SEP 39) and #59 have changed this path since, and the case it used is no longer in benchmarks/, so re-measure first with a run that saves each step.
Why it matters
A run keeping several planes of a large beam spends seconds per plane outside the physics, and its peak memory sits at the plane, not in the steps.
Context
_observed_fieldgives an owning host array. An in-memory store has to own what it keeps, while a file only has to write it, so the copies may differ per handler. The file writer and the in-memory store are where they end.- A translated problem keeps a full copy of its initial field for the life of a run #5 is the problem's copy of the initial field,
estimate_memorydoes not count the problem's copy of the initial field #6 the estimate that misses it, and The run record is rewritten at every plane:/meta/outcomeand Multistep'sorders#80 the run record rewritten at every plane. None of them covers this path. - Why
spectrum_to_opticsexists: stored spectra have the optics convention, and that must not change.
Read, not run.
To do
- Trace a plane from observation to store on
native, for the file and the in-memory handlers, and remove the copies nothing mutates. - See if the conversion can run in place on a buffer the plane already owns.
- Re-time a kept save on Linux on
native, and say what takes the time. - Check peak memory at a plane against
estimate_memory.
Stored fields stay bitwise equal, and a kept plane must not alias the live state.
Done when
On 2x Xeon Gold 6240 a saved plane of kerr_beam_384 costs at most half of the 3 s above, timed the same way, with bitwise-equal output and a test that a kept plane does not change when the run goes on. The kept save's time on the same machine is recorded, with what takes it.
Checked against commit 8a9f764. Takes over the roadmap's entry "The cost of a kept native save on Linux."
Activity
jtravs commentedon Nov 9, 2026
John Travers has asked for this work to be included directly in SEP 47, tracked by #588 and proposed in #589 (scope revision 2a66238). No separate prerequisite PR is needed.
The plan is to trace file and in-memory output in phase 1, take straightforward copy elisions early, and complete destination-based export in phase 3. Include unit/non-unit weights, checkpoint/plane export sharing, retained-array safety, peak live storage and the unexplained kept-save cost on Linux. Measure focused copy changes separately from the session ownership change.
The historical same-hardware saved-plane target still needs an actual measurement; the proposal's structural success criterion does not itself satisfy this issue. Use the current allocation ledger rather than restoring the retired memory estimator/admission machinery. Copies and storage remain exact; any fused arithmetic conversion is separately qualified under SEP 47's existing numerical contract. Record any required reconciliation of this issue's historical criteria before claiming it complete.
The SEP remains Draft pending the maintainer's reading and explicit acceptance. This is a scope/coordination update, not an implementation or closure.