[AdamCorrao]

re/04: native GSAS-II schemas (gsasii.rietveld, gsasii.spf) and gateway adoption

Merges re/04-gsasii-schema (7 commits on 2810c82, tip 65b95ca) into
refactor/multi-engine. Not a gate branch. Plan: devkit
dossiers/multi-engine/subplans/re-04-gsasii-schema.md; decisions A81–A92,
A93–A123 as they apply to gsasii, A124, A125 (master plan §3).

What this does

• Native gsasii engine schema 1.0.0 (gateways/gsasii/schema.py, engine-free), built on core 1.0.0:
  • one phase block per phase (core Phase[RefinableParameter] plus scale and peak_broadening);
  • a single-source instrument in blocks (radiation, geometry, corrections, broadening);
  • every refinable is [value, refine_flag], with no bounds (A84).
• The gateway runs native recipes.
  • run_refinement consumes the validated model (A39). The input-side edits to frozen code are ledger rows R1–R14, all signed in devkit tasks/re04-frozen-edits.md.
  • GSASII_* 0.26.0 recipes still run, through a verbatim legacy copy of every touched function (_legacy_executor.py, generated from 2810c82). The 0.26.0 path runs only unedited code until re/07.
• Converter scripts/convert_recipe_026.py (0.26.0 → gsasii 1.0.0). It states explicitly what v0.1.1 refined and reports every change. Anything the new schema rejects stops the conversion with every problem listed; nothing is fixed silently.
• Results on the native path:
  • standard rwp, r_exp, gof, chi2_red from core fitstats (A125);
  • GSAS-II's own values under engine_details, with parameters_requested and parameters_varied;
  • structured warnings: the documented defaults applied, and GSAS-II's refinement messages.

A recipe, before and after (LaB6, abridged)

0.26.0 (GSASII_Rietveld): the structure and parameterization are separate blocks, and the instrument has two sources:

"instrument": {
  "initialization": [{"Lam": [0.1665, 0.1665, false], "SH/L": [0.0005, 0.0005, false],
                      "U": [18.717, 18.717, false], "Zero": [0.0, 0.0, false], /* + 9 more GSAS-II keys */}, {}],
  "parameterization": {"wavelength": [0.1665, false, null, null],
                       "broadening": {"U": [null, true, null, null], /* V..Z alike */},
                       "corrections": {"zero_shift": null, "axial_divergence": null}}
},
"phases": {"LaB6": {
  "structure": {"phase_name": "LaB6", "space_group": "P m -3 m",
                "unit_cell": {"a": 4.15682, /* ... */ "volume": 71.826},
                "atoms": {"B": {"element": "B", "x": 0.5, "y": 0.5, "z": 0.2021, "occupancy": 1.0,
                                "Multiplicity": 6, "ADP": "Uiso", "Uiso": 0.009}, /* La */}},
  "parameterization": {"scale": [1, true, null, null],
                       "unit_cell": {"a": [null, false, null, null], /* ... */},
                       "atoms": {"B": {"x": [null, false, null, null], /* ... */ "ADP": "Uiso"}},
                       "peak_broadening": {"size_broadening": {"model": "isotropic", "isotropic_size": null, "LG_eta": null}, /* strain */}}
}},
"background": {"single_peaks": {"pv_gaussian_sigma": [[1000, true, null, null]], /* ... */}}

gsasii 1.0.0 (gsasii.rietveld, produced by the converter): one phase block, one instrument, every value explicit:

"schema_name": "gsasii.rietveld", "core_schema_version": "1.0.0", "engine_schema_version": "1.0.0",
"instrument": {
  "description": "Pair distribution function (PDF) beamline (port 28-ID-1) at NSLS-II, BNL",
  "radiation":   {"type": "PXC", "wavelength": [0.1665, false]},
  "geometry":    {"bank": 1, "azimuth": 0.0},
  "corrections": {"zero_shift": [0.0, false], "polarization": [0.99, false], "axial_divergence": [0.002, false]},
  "broadening":  {"U": [18.717, true], "V": [0.6003, true], /* W, X, Y, Z */}
},
"phases": {"LaB6": {
  "space_group": "P m -3 m",
  "unit_cell": {"a": [4.15682, false], "b": [4.15682, false], "c": [4.15682, false],
                "alpha": [90.0, false], "beta": [90.0, false], "gamma": [90.0, false]},
  "atoms": {"B":  {"element": "B", "x": [0.5, false], "y": [0.5, false], "z": [0.2021, false],
                   "occupancy": [1.0, false], "Multiplicity": 6, "ADP": "Uiso", "Uiso": [0.009, false]},
            "La": {/* ... */}},
  "scale": [1, true]
  /* peak_broadening left out: 10 µm / 0 microstrain, fixed, with a warning (A107) */
}},
"background": {"chebyshev": {"num_coefficients": 6, "coefficients": [10, 5, 1, 0, 0, 0], "refine_flag": true},
               "single_peaks": {"pv_gaussian_sigma_sq": [[1000, true]], /* positions, intensities, gamma */}}

The converter's report for it:

payload.xrd_data.UID: moved to metadata.xrd_data.UID
payload.instrument.corrections.axial_divergence: SH/L 0.0005 -> 0.002 (GSAS-II computed with 0.002 anyway, EB-37; A91, deliberate change)
payload.background.single_peaks: pv_gaussian_sigma renamed pv_gaussian_sigma_sq (the value always was σ², A89)
payload.phases.LaB6.unit_cell.volume: dropped (derived from a..gamma, A75)
payload.phases.LaB6.peak_broadening.size_broadening: left out: the documented default 10 um applies, where v0.1.1 used GSAS-II's 1 um (A87, A107; deliberate change)
payload.phases.LaB6.peak_broadening.strain_broadening: left out: the documented default 0 applies, where v0.1.1 used GSAS-II's 1000 microstrain (A87, A107; deliberate change)

(The committed LaB6 example itself stops in the converter: its background peak
has γ = 1e-5, below GSAS-II's silent floor of 0.1 (A86, EB-35). The example
above raises it to 0.1 first. The example is fixed at re/07.)

Commits

Commit	Content
e2481db	gsasii schemas; schema_core.collect_warnings; validate() routes gsasii.* recipes; schema tests (77) + GSAS-II proof tests (1054)
809c329	verbatim legacy executor (generated, not edited)
ba068d2	ledger R1–R10: recipe reads in run_refinement, setters, project; routing (gateways/gsasii/routing.py)
c936153	0.26.0 → gsasii converter + 43 tests
38d6007	legacy copy extended for batch 2 (21 functions + SCHEMA_EXECUTORS)
b8923dd	R13: GSAS-II site check after phase setup (A117); core symmetry.orbit
65b95ca	R11 fit statistics / engine details / warnings, R12 peak-fit error (EB-38), R14 core report writers (native only, A83); kicker.run warnings and validate_only summary; server response fields; SCHEMA_HISTORY gsasii 1.0.0

Rules the gsasii schema enforces

• Space group: the canonical gemmi name, translated to GSAS-II's symbol. Refused settings:
  • origin choice 1 (EB-03);
  • 9 settings GSAS-II cannot express: 7 it doesn't parse, and 2 (P 21 n m, A b a m) it reads as another setting (EB-41).
  • Proven by a general-position orbit test over all 564 canonical names.
• Cell refinement: refining part of one of GSAS-II's coarser cell groups is an error (monoclinic {a, c, β}; triclinic and :R: all six). Matches GSAS-II's own cellVary dispatch for all 522 settings (A95).
• GSAS-II floors: SH/L ≥ 0.002 (A91); background-peak floors (A86) and Peak List floors (A90) are errors.
• Simulation: every flag must be false (A88).
• Other: scale ≥ 0 and required; phase names and atom labels unique ignoring case (A112).
• Documented defaults (A119):
  • size/strain left out → 10 µm / 0, with a warning (A87, A107);
  • background left out → none (one fixed 0.0 term), instead of GSAS-II's constant 1.0.
• At run time (A117): GSAS-II misreads some 2-fold sites in R32, R-3m and R-3c. That gives 4× the intensity with nothing refined (EB-40, proven with two equivalent orientations).
  • Such a site is an error that names an equivalent position GSAS-II reads correctly.
  • So is a site GSAS-II can't name ('sp', EB-05), but only when a coordinate or Uaniso is refined.

Evidence

• Default suite: 2731 passed, 4 skipped (baseline at 2810c82: 1518 passed, 4 skipped). Every baseline test passes; the 21 test_run_refinement_missing_keys tests now run as [legacy] and [native].
• easydiff env: 55 passed, same IDs as the baseline.
• Equivalence (devkit probes/re04_equivalence_probe.py): 8 examples run on both paths, each 0.26.0 recipe edited only to state the new defaults. Results:
  • identical GSAS-II Rwp, y_calc (maximum difference 0.0) and refined values;
  • every report file byte-identical between the paths;
  • the Iparm1 built from the instrument blocks equals each example's.
  • The examples: LaB6, LaB6_simulation, LaB6_singlepeakfit, and the DRX_33 variants (base, atomrefine with 39 refined parameters, anisoADP, strainonly, simulation).
• Native runs (tests/test_gsasii_native_run.py):
  • standard statistics vs GSAS-II's (LaB6: 6.5345 vs 6.5346);
  • refined zero_shift, polarization and axial_divergence are actually varied (A103);
  • the site-check errors, and the SPF run.
• Checks: server mode verified with a fresh server; undefined-name probe 0; LF only; examples/ untouched; code hash updated (manifest now includes the legacy copy).

Behaviour changes vs v0.1.1

Rows 6–17 in devkit tasks/deliberate-changes.md, reviewed at G2. Notably:

• new size/strain and background defaults;
• the SH/L and peak floors are errors;
• strict refine flags (retires KI-02);
• on the native path, rwp is the uniform Rwp. For the two simulation examples, GSAS-II reports wR 100 %, while the uniform Rwp against their placeholder observations is 1787 % / 6820 % (EB-52; to be documented in re/10).

Left for later branches

• re/07: migrate the examples with the converter (fix the LaB6 background peak and SH/L); remove the legacy copy, the routing's legacy branch and the server's null-key cleanup.
• re/10: user docs for the statistics differences (EB-52), the defaults and their warnings.
• re/05, re/06: reuse collect_warnings, the fit-report pattern, the A125 keys and the canary registration.

Assisted-by: claude-code:claude-opus-5-5
Assisted-by: claude-code:claude-sonnet-5-5