Native: keep the FFTW plans runs measure, by default (ADR-076)

changelog.d/fftw-plans-kept.changed.md

@@ -0,0 +1 @@
+The native backend now keeps the FFTW plans its runs measure by default, so each transform shape is measured once per machine and build instead of on every run. Plans are kept per transform in `wisdom.json` in the calibration store's directory for the machine and build, and a calibration profile's own plans take precedence. `SOLPROP_FFTW_WISDOM` now only moves this file, and the file it names no longer turns off a profile's FFT planning. `SOLPROP_CALIBRATION=off` keeps no default file, and `solprop calibrate --clear` removes the plans. Plans kept by earlier releases are not read; their files can be deleted. A name that cannot be used now logs a warning and the run plans without stored plans, where it used to raise. The store is keyed by the machine fingerprint, which tells Apple silicon models apart (#136).

docs/dev/backends.md

@@ -746,7 +746,8 @@ Threads are a property of the plan, chosen at construction: a plan of fewer real
 than `default_threaded_fft_min_samples` runs on one thread (§4), and the others take their
 width from the size rules of §3.4, within the process's thread budget (ADR-052: the affinity
 mask, the cgroup CPU quota on the process's own path rounded down, and `SOLPROP_THREADS`;
-never `os.cpu_count()`). Wisdom is explicit and opt-in (`SOLPROP_FFTW_WISDOM`, §4).
+never `os.cpu_count()`). The plans runs measure are kept by default, per machine and
+build (ADR-076, §4).
 
 ### 3.4 Threading regimes
 
@@ -1309,16 +1310,39 @@ threaded plans; free space's transverse planes are measured at any width. An exp
 planning, `measure` and experimental `patient` time it. Estimate can still consume
 matching wisdom already present in the process; the backend does not clear shared FFTW
 wisdom. Planning is destructive and precedes state initialization.
-Wisdom outlives the process only when `SOLPROP_FFTW_WISDOM` names a file: the adapter
-(`solprop_native._wisdom`) imports this build's file, `<name>.native-<key>`, before the
-state plans, and afterwards merges the file into the process's wisdom and replaces it
-whole when that adds a plan. The key digests FFTW's wisdom preamble (its version and
-planner signature, which FFTW checks again on import) and the processor model. Readers
-take no lock, since the file only ever changes by an atomic rename; writers serialise on
-`<file>.lock` (`flock` on POSIX). The C++ side only converts wisdom to and from text
-(`export_fftw_wisdom`, `import_fftw_wisdom` in `fft_plan.hpp`), registering FFTW's
-threaded solvers first, since they are part of the planner signature an import must
-match. Metadata records `wisdom_loaded` and `wisdom_digest`.
+The plans runs measure outlive the process, by default (ADR-076). The adapter
+(`solprop_native._wisdom`) keeps them by plan key (`fft_plan_key`: kind, sizes,
+strides, placement, alignment and threads), each with the wisdom of that transform
+alone, in `wisdom.json` in the calibration store's directory of the machine and build
+(`<store>/machines/<fingerprint>/`, ADR-074's fingerprint), or as
+`<name>.native-<fingerprint>` beside the file `SOLPROP_FFTW_WISDOM` names;
+`SOLPROP_CALIBRATION=off` keeps no default file, and the process keeps what it measured
+in memory for its later workspaces either way. A run's own planning (the policy's, or
+the profile's) is offered the stored plans (`FftPlanning::stored`) and asked to keep
+what it measures (`keep`). In `begin_seed` a key the profile holds is planned from the
+profile's wisdom, as before; any other seeded above estimate plans *alone*: the
+workspace's accumulated wisdom is exported and forgotten, only that key's known wisdom
+imported (stored, or measured earlier in this workspace, so a key seeded twice is
+measured once; a key to measure starts from what the workspace measured so far, whose
+sub-plans it shares), and after the seed (`release_seed`) the transform's own wisdom is exported
+into `measured` when the seed added an entry the stored text lacked, or the key is
+listed in `reused`; the workspace's wisdom is then imported back over the transform's,
+so that a profile's entries win where both hold one. A key left at estimate that the
+store holds is not seeded: its wisdom is imported beneath the workspace's for
+Kokkos-FFT's estimate plan to take, and it counts as reused. Seeds and their Kokkos-FFT
+plans interleave (a workspace seeds several transforms before planning them), so the
+wisdom accumulates within a workspace rather than being forgotten between keys; SEP 46's
+transaction still forgets it before and after a profiled workspace. After construction
+the adapter stores the measured plans not held by the profile, so a profile's winners
+are never overridden. Readers take no lock, since the file only ever changes by an
+atomic rename; writers serialise on `<file>.lock` (`flock` on POSIX), merge the file's
+plans with their own and replace it only when that adds or changes one. A calibration's
+own workspaces (given `fft_planning`) and its children (`off`) neither read nor write
+the store. The C++ side only converts wisdom to and from text (`export_fftw_wisdom`,
+`import_fftw_wisdom` in `fft_plan.hpp`), registering FFTW's threaded solvers first,
+since they are part of the planner signature an import must match. Metadata records
+`wisdom_loaded` (the file held plans) and `wisdom_digest` (of the plans the workspace
+built from wisdom or measured).
 Contiguous Raman transforms use the same planning policy, including long doubled
 rows and partial chunks. On macOS, NEON FFTW handles serial and long Raman rows;
 short parallel power-of-two rows (at most 16384 samples) retain Accelerate. Selection

docs/dev/cpu-performance.md

@@ -188,7 +188,8 @@ measured while it existed.
     happens before the first plan is made; the plasma streams' plans were made earlier
     and stayed at estimate.
 
-- **Measured plans kept across processes** (`SOLPROP_FFTW_WISDOM`, opt-in;
+- **Measured plans kept across processes** (by default since ADR-076, opt-in through
+  `SOLPROP_FFTW_WISDOM` before;
   [measurements](benchmarks.md#persistent-fftw-wisdom)). A scan of one-thread jobs
   measured the same plans in every job: 0.5–4 s of construction against windows of 0.1–8
   s on a Xeon Gold 6240. With the wisdom file in place a job constructs its backend in

docs/dev/decisions.md

@@ -2165,3 +2165,64 @@ Withdrawn (development-process record, not a design decision).
   reach a run through a calibration's stored wisdom, not the default. Stored profiles keyed
   on the old widths miss and replan (each plan key holds its threads); the calibration's
   version is bumped with this change.
+
+## ADR-076 — Runs keep the FFTW plans they measure, by transform, by default
+
+- **Status:** proposed with PR #602 (issue #136), for the maintainer.
+- **Context:** measured plans for threaded rows win in the step (0.94–0.97, ADR-075), but
+  planning them costs up to 12 s of a run's construction (`ctc` on 16 cores of an AMD EPYC
+  7763: 13.8 s against 1.6 s), so the policy left them at estimate. Wisdom outlived the
+  process only when `SOLPROP_FFTW_WISDOM` named a file, for three reasons: solprop wrote
+  nothing to user directories, which ADR-074's store ended; a bad stored plan could slow
+  every later run in silence (SEP 46 §4), which is why a profiled workspace forgets all
+  wisdom; and #136's robustness bugs. The opt-in file was also one blob of wisdom, which a
+  profiled workspace forgets, so it helped only runs without a profile. Wisdom planned
+  alone, outside the run, once ran `h2_raman_averaged` 1.17 times slower on an AMD EPYC
+  9B45; a plan measured by the run itself, in its own workspace, is the one the policy
+  already trusts.
+- **Decision:**
+  - *What is kept.* Each plan key (`fft_plan_key`: kind, sizes, strides, placement,
+    alignment and threads, as a calibration keys its winners) holds the wisdom of that
+    transform alone, measured in place by the first run that planned it: the seed plans
+    with the workspace's other wisdom set aside, from the key's stored wisdom only (one to
+    measure from what the workspace itself measured so far, whose sub-plans it shares, so
+    that a cold run measures as much as before), and its own wisdom is exported after it (`FftPlanning::stored`, `keep`, `measured`,
+    `reused`). A stored plan applies at any effort, as FFTW applies wisdom: a transform
+    left at estimate takes it for its estimate plan, unseeded.
+  - *Where.* `wisdom.json` in the calibration store's directory of the machine and build
+    (ADR-074's fingerprint: machine, operating system, build digest, calibration version),
+    so machines and builds never share plans, and Apple silicon models are told apart.
+    `SOLPROP_FFTW_WISDOM` moves it, beside the named file as
+    `<name>.native-<fingerprint>`, and no longer turns off a profile's FFT planning;
+    `SOLPROP_CALIBRATION=off` keeps no default file. `solprop calibrate --clear` removes
+    both. A process also keeps what it measured in memory for its later workspaces.
+  - *Precedence.* A key the profile holds is planned from the profile's wisdom, never
+    from the store, and never written to it. SEP 46's transaction stays: a profiled
+    workspace forgets wisdom before and after it plans, and imports the store's wisdom key
+    by key, beneath the workspace's, so no unrelated wisdom reaches a profiled plan. This
+    amends SEP 46 §6.7's "runs no longer merge wisdom into the store when a profile
+    applies": they merge the plans the profile does not hold.
+  - *Writes and robustness.* Only a run's own planning (the policy's or the profile's)
+    reads and writes the store; a calibration's workspaces and children do neither. The
+    file is replaced whole by an atomic rename, under a `flock` writer lock, merging what
+    others stored. A name that cannot be used (an empty file name, an unknown user's
+    home, no home at all) and a file that cannot be read or written log a warning and the
+    run plans without stored plans (#136).
+- **Consequences:** under `SOLPROP_FFTW_PLAN=measure` on 16 cores (one NUMA domain) of an
+  AMD EPYC 7763, five interleaved rounds against PR #598, a cold run plans as before
+  (`ctc` 13.7 s, `h2_raman_averaged` 5.1 s, `grin_nonlinear` 3.6 s, `rdw` 1.8 s, both
+  builds) and a warm one builds its backend in 0.18, 0.13, 0.36 and 0.16 s; on an Intel
+  Xeon Gold 6240 at 18 and 36 threads `ctc` took 23 s cold and 0.12 s warm. Windows with
+  the stored plans ran as those with plans measured in place: `h2_raman_averaged` 0.99–1.02
+  across four comparisons on the EPYC (same build and against #598, two independent
+  stored measurements), no slower than plans measured in place; `ctc` 1.003–1.017, its
+  range spanning 1 with a second stored measurement, so the plan a single measurement
+  picks; `rdw` 1.00; the Xeon's geometric mean 1.015, every range spanning 1. Fields agree
+  to rounding. A first attempt that planned each transform from nothing measured a key
+  seeded twice twice and lost shared sub-plans (`rdw` 6.7 s cold on the Xeon against 1.4 s);
+  a transform to measure now starts from the workspace's own measurements. The store
+  holds about 20 kB a case at one width (80 kB for four cases on the EPYC) and grows with
+  the shapes a user runs; nothing evicts a plan, and `--clear` empties it. Plans kept by
+  the earlier opt-in file are not read. Measured threaded rows stay off by default: a
+  plan measured once now costs nothing after, which leaves the policy's threaded-rows
+  effort for the maintainer.

docs/dev/tooling.md

@@ -176,7 +176,8 @@ build works offline and survives a flaky mirror; refresh the copies occasionally
   `SOLPROP_OVERSUBSCRIBE`, which lets an explicit count exceed the CPUs (ADR-052).
 - The native backend's plans and neighbours: `SOLPROP_FFTW_PLAN` (`estimate` in CI and
   tests, `measure` by default, read by `native/include/solprop_native/fft_plan.hpp`),
-  `SOLPROP_FFTW_WISDOM` (a file name, beside which the native backend keeps its wisdom)
+  `SOLPROP_FFTW_WISDOM` (a file name, beside which the native backend keeps the plans its
+  runs measure instead of the calibration store, ADR-076)
   and `SOLPROP_BLAS_THREADS` (the width of the other BLAS libraries while a native
   backend lives).
 - In the test suite: `SOLPROP_TEST_BACKEND` (the backend the driver tests select; by

docs/guide/performance/calibration.md

@@ -50,7 +50,9 @@ never runs your own simulation; for that, see
    plans made as a run without a profile makes them. A more exhaustive plan is not
    always a faster one, and a plan chosen alone not always the best in the step. Only
    a winning effort's wisdom is kept, so later runs build those plans at once instead
-   of measuring again; without one, runs plan as they would without a profile.
+   of measuring again; without one, runs plan as they would without a profile. The
+   calibration's problems are not yours: the plans of your own transforms are measured
+   by the first run that needs them and kept (below).
 
 A candidate replaces the built-in value only if it **wins in every round** of
 interleaved timings by more than the noise measured by timing the built-in setting
@@ -93,6 +95,14 @@ A job takes its own allocation's row. Without one, it takes the workspace settin
 the row with the same layout and the nearest smaller core count, and otherwise the
 built-in table.
 
+The same directory keeps the **FFT plans runs measure**, in `wisdom.json` beside the
+machine and build's profiles. The first run of a transform shape measures its plan in
+place, inside its own workspace, and stores it; every later run of that shape, in any
+process, builds the plan from it at once. A plan the profile holds keeps precedence:
+runs never read or overwrite one of the profile's winners from this file. The file is
+kept with or without a profile, and goes with the profile when you clear it
+([Performance tuning](tuning.md#the-native-backends-pools) says more).
+
 ## The first run
 
 With no profile for the machine, the build and the job's allocation, the first native
@@ -116,7 +126,7 @@ operation grain at once and the new waiting and team from its next session.
 |---|---|
 | `auto` (default) | use a profile; calibrate quickly on first use when there is none |
 | `use` | use a profile; never measure implicitly |
-| `off` | the built-in table and no stored FFT wisdom, whatever the store holds; for bisecting and for holding settings fixed while code changes |
+| `off` | the built-in table and no stored FFT plans, whatever the store holds (a file `SOLPROP_FFTW_WISDOM` names is still kept); for bisecting and for holding settings fixed while code changes |
 
 The notice is a Python warning of its own category, shown once; silence it with the
 standard warnings filters (`warnings.filterwarnings("ignore",
@@ -171,7 +181,9 @@ built-in table for them. Each candidate's score is its median time relative to t
 built-in setting. Every run records the settings it actually applied in its metadata
 (`calibration` in the backend's settings), so a result says what produced it.
 
-`solprop calibrate --clear` forgets this machine and build's profiles.
+`solprop calibrate --clear` forgets this machine and build's profiles, their
+refinements and the FFT plans runs stored, and the plans kept beside a file
+`SOLPROP_FFTW_WISDOM` names.
 
 ## Refining one simulation
 
@@ -196,8 +208,9 @@ stays the same.
 
 Explicit settings keep their precedence over any profile: `SOLPROP_THREADS`,
 `SOLPROP_NATIVE_THREADS` and the pool variables, `OMP_WAIT_POLICY`, `GOMP_SPINCOUNT` and
-`KMP_BLOCKTIME`, `SOLPROP_FFTW_PLAN`, `SOLPROP_FFTW_WISDOM` and
-`SOLPROP_NATIVE_THREADED_FFT_MIN_SAMPLES` ([Performance tuning](tuning.md)). A setting
+`KMP_BLOCKTIME`, `SOLPROP_FFTW_PLAN` and `SOLPROP_NATIVE_THREADED_FFT_MIN_SAMPLES`
+([Performance tuning](tuning.md)). `SOLPROP_FFTW_WISDOM` only moves the plans runs
+store; the profile's own plans still apply. A setting
 measured under conditions an override changes is not used with it: operation thresholds
 fitted under spinning threads give way to the built-in table when you make the threads
 sleep.