Skip to content

A saved plane of a large Cartesian field costs about 3 s in copies, and a kept native save on Linux is unexplained #346

Open

Description

@jtravs

What happens

Two costs at a saved plane have not been explained.

A large Cartesian plane costs about 3 s, in copies. #59 measured what repeats per saved plane of kerr_beam_384's 1.77 GB field (ledger):

step time
the saved plane's copy 1.1 s
spectrum_to_optics, the conversion to the stored convention 1.0 s
the output store's copy 0.6 s
the native observation 0.35 s

A capillary or radial plane costs 7–25 ms. #59 left this as output-layer work for a pull request of its own. Each copy is also another field-sized allocation at the plane, which matters on runs sized to memory.

A kept native save on Linux. SEP 29 recorded about 4.5 ms for a kept native Adams save, against numba_cpu's 1.6 ms, unexplained, with run totals level (1.398 s against 1.397 s, about 110 saves). The native march (SEP 39) and #59 have changed this path since, and the case it used is no longer in benchmarks/, so re-measure first with a run that saves each step.

Why it matters

A run keeping several planes of a large beam spends seconds per plane outside the physics, and its peak memory sits at the plane, not in the steps.

Context

Read, not run.

To do

  • Trace a plane from observation to store on native, for the file and the in-memory handlers, and remove the copies nothing mutates.
  • See if the conversion can run in place on a buffer the plane already owns.
  • Re-time a kept save on Linux on native, and say what takes the time.
  • Check peak memory at a plane against estimate_memory.

Stored fields stay bitwise equal, and a kept plane must not alias the live state.

Done when

On 2x Xeon Gold 6240 a saved plane of kerr_beam_384 costs at most half of the 3 s above, timed the same way, with bitwise-equal output and a test that a kept plane does not change when the run goes on. The kept save's time on the same machine is recorded, with what takes it.


Checked against commit 8a9f764. Takes over the roadmap's entry "The cost of a kept native save on Linux."

Activity

  1. added
    performanceSlower or larger than it should be, with no wrong result
    area: outputOutput files, processing and plotting
    area: free spaceRadial, Cartesian, line and planar free-space routes
    area: gpuCUDA and HIP builds: qualification and performance
    on Nov 1, 2026
  2. jtravs commented on Nov 9, 2026

    @jtravs
    ContributorAuthor

    John Travers has asked for this work to be included directly in SEP 47, tracked by #588 and proposed in #589 (scope revision 2a66238). No separate prerequisite PR is needed.

    The plan is to trace file and in-memory output in phase 1, take straightforward copy elisions early, and complete destination-based export in phase 3. Include unit/non-unit weights, checkpoint/plane export sharing, retained-array safety, peak live storage and the unexplained kept-save cost on Linux. Measure focused copy changes separately from the session ownership change.

    The historical same-hardware saved-plane target still needs an actual measurement; the proposal's structural success criterion does not itself satisfy this issue. Use the current allocation ledger rather than restoring the retired memory estimator/admission machinery. Copies and storage remain exact; any fused arithmetic conversion is separately qualified under SEP 47's existing numerical contract. Record any required reconciliation of this issue's historical criteria before claiming it complete.

    The SEP remains Draft pending the maintainer's reading and explicit acceptance. This is a scope/coordination update, not an implementation or closure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: free spaceRadial, Cartesian, line and planar free-space routesarea: gpuCUDA and HIP builds: qualification and performancearea: outputOutput files, processing and plottingperformanceSlower or larger than it should be, with no wrong resultpriority: lowWorth doing, no hurry

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions