Skip to content

Reorganize catalog validation and documentation by topic - #161

Open
ptgrogan wants to merge 9 commits into
mainfrom
catalog-validation-layout
Open

ptgrogan wants to merge 9 commits into
mainfrom
catalog-validation-layout

Conversation

@ptgrogan

@ptgrogan ptgrogan commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Reorganizes the validation of the instrument catalog and the documentation around it, and applies two reviews of tatc.catalog.

Catalog validation

  • Moves the catalog validation scripts to validation/catalog and retires the GOES notebook. Fitted values now come only from each script's --fit run, and notebook tests that repeated the scripts' tests are removed.
  • Configurations are selected by platform and grade. The holdout gap is computed from the periods of the fitting and test data instead of being declared. Results on sibling platforms are recorded apart as transfers, which do not count toward validation.
  • InstrumentEntry.validation lists every gap of an entry: missing grades, each configuration's gaps, and evidence gaps apart from gates.
  • catalog.build_payload and InstrumentPayload.to_satellite build satellites that carry instruments from several entries; build_constellation accepts payloads too.
  • Records and transfers are written by entry, through the schema. A transfer to a stale record is refused.
  • Adds xarray to the validation extra, which the ABI, AHI, and KaRIn scripts need.

Documentation

  • Groups the validation notebooks into folders by capability (orbits, instruments, coverage, limb_and_space, ground_systems, trade_studies), each with its own page. They share one download cache in docs/validation/data.
  • Groups the example notebooks into folders by topic (getting_started, coverage, limb_and_space, ground_systems, next to osse and tradespace). Each notebook's input files sit beside it.
  • The validation standard (docs/validation/standard.rst) gains a Metrics section that says what each gated metric measures and how, and that a gate takes the worst value over scopes.

Breaking changes

  • attitude_schedule is renamed tilt_schedule.
  • Configuration.is_fitted is removed.
  • The exports of tatc.catalog are trimmed.
  • The paths of documentation notebooks change, so links to the old pages break.

…GOES notebook

The catalog validation scripts, their helpers, and the element sets only they
read move out of docs/validation to a top-level validation/catalog folder, with
a README on running them. Scripts take the record, fixture, and element set
folders and their own source path from validation_io instead of repeating them,
and the validation records cite the new paths.

ValidateGeostationaryGOES is removed, as validate_catalog_abi tests the same
products; the remaining validation notebooks are grouped by capability.
…book tests

The CrIS, AMSR2, GMI, MODIS, and SMAP radiometer validation scripts gain a
--fit mode that measures their entries' fitted values as the notebooks did,
and the entries' provenance cites the scripts. A shared fit_cone helper
measures conical scanners, MODIS reductions keep the reported velocity, and
the scripts log in to NASA Earthdata with EARTHDATA_TOKEN or ~/.netrc.

The ATMS, CrIS, VIIRS, MODIS, AMSR2, GMI, NISAR, and Sentinel-1 notebooks drop
the pass, epoch, and swath area tests that the catalog's validation now runs
on held-out data; the summary, standard, and scripts cite what remains.
…yout

# Conflicts:
#	.gitignore
#	CHANGELOG.md
#	docs/validation/ValidateImagerCrIS.ipynb
#	docs/validation/index.rst
…grade, compute the holdout gap from the periods of fitting and test data, record transfers apart and the standard version of each configuration, check record platforms and class versions, build payloads of several entries (also in build_constellation, with start and end for tilt schedules), give the analysis index of unions, rename attitude_schedule to tilt_schedule, move the subsolar latitude to utils, default the provenance of tailored and swept views, keep the gates in one table, trim the exports, write records through the schema, pass JPSS archives explicitly, deduplicate the conical scripts, and test the status of entries from one table
…with evidence gaps apart from gates), build satellites from payloads (InstrumentPayload.to_satellite), write records and transfers by entry (refusing transfers to stale records, keeping scopes), key class versions by class, drop Configuration.is_fitted, and require a grouping only for several instruments
…struments, coverage, limb_and_space, ground_systems, trade_studies), each with its own page, sharing one cache of downloads in docs/validation/data, with the element sets of the occultation and satellite-to-satellite notebooks in limb_and_space/elements
…overage, limb_and_space, ground_systems, beside osse and tradespace), each with its own page, with the files they read beside them and the GNSS element sets of CollectRO in limb_and_space/elements
…, grouped by grade, and state that a gate takes the worst value over scopes
@ptgrogan ptgrogan changed the title Catalog validation layout Reorganize catalog validation and documentation by topic Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant