Skip to content

Add EarthCARE and TRMM instruments to the catalog - #160

Merged
ptgrogan merged 1 commit into
mainfrom
catalog-earthcare-trmm
Nov 15, 2026
Merged

ptgrogan merged 1 commit into
mainfrom
catalog-earthcare-trmm

Conversation

@ptgrogan

Copy link
Copy Markdown
Contributor
  • Add seven catalog entries: EarthCARE ATLID, BBR, CPR, and MSI (earthcare_msi, as msi is Sentinel-2), and TRMM PR, TMI, and VIRS. Each was fitted on one day and tested on data held out 39 to 41 days later; both days were fixed before any result.
  • Validated: TMI (three feeds), VIRS, and EarthCARE MSI.
  • Partial:
    • TRMM PR: the swath is validated, but the published footprint at the scan's edge (5.0 km) lies 7.9% below the size its beam gives, so 75% of beams are within 3% against the 90% gate. The threshold was not changed.
    • ATLID and CPR: the profiles lie 0.72 km (median) from the predicted track, against a 0.5 km gate. The cause is the element set nearest the held-out day's middle, whose inclination differs by 0.01 deg from its neighbors'. A retest with the day's other set passes (0.33 km); it is stated in the limitations and does not count toward the tier.
    • BBR: 95.5 to 97.9% of passes are predicted, against a 99% gate, and the nadir view's swath IoU is 0.921, against 0.95. With the day's other element set, swath IoU passes (0.963 to 0.978), but recall still fails (97.5 to 98.6%); this is stated and not counted. BBR's data break for about 4.3 s every 93 s, so it is scored with breaks over 1 s as periods without data. That rule was chosen after a first score on the day of the fit, as its limitations say.
  • Version 6 of the standard adds a profiler class, for ATLID and CPR, with the altimeter's gates (segment recall and track offset). No earlier threshold changes, and earlier versions remain compatible.
  • Add the validation scripts (docs/validation/validate_catalog_{atlid,bbr,cpr,earthcare_msi,tmi,trmm_pr,virs}.py, with the shared reader earthcare_io.py) and the element set files they read. EarthCARE product times include leap seconds and are corrected to UTC.
  • gesdisc_scanner.observe_orbit now bounds its nearest-neighbor query. The cells compared are the same (checked on a sample), and the search takes about a minute rather than hours.
  • Not fixed, documented: observe_orbit reads TAT-C's mean motion (deg/s) as revolutions per day, so swath IoU compares a span from noon to the end of the day rather than one orbit. This applies to every record scored through it, about 15 existing entries and the new ones. The code is unchanged here; its docstrings describe the span.
  • Add fixture and entry tests for each entry (with an instrument index for BBR's nadir view), NOTICE credits (ESA for ESA Level 1 products, JAXA/NICT for CPR), rows in docs/validation/summary.rst, and a CHANGELOG entry.

Checks: the full unit test suite passes (one skip that was already there), black is clean, and pylint rates src/tatc/catalog 9.95. The catalog page generates with all seven entries. I have not built the full documentation.

…the catalog

TMI, VIRS, and EarthCARE MSI are validated on data held out 39 to 41 days
from the fit. TRMM PR is partial, as its published footprint at the
scan's edge (5.0 km) lies 7.9% below the size its beam gives, while its
swath is validated. ATLID and CPR, of a new profiler class (version 6 of
the standard, with the altimeter's gates), are partial on track offset
(0.72 km) with the element set nearest the held-out day's middle; the
day's other set passes, stated in their limitations and not counted. BBR
is partial on pass recall and, with that set, swath IoU; it is scored
with breaks in its data over 1 s as periods without data, chosen after a
first score on the day of the fit, and stated in its limitations.

EarthCARE product times include leap seconds and are corrected to UTC.
observe_orbit bounds its nearest-neighbor query, which gives the same
cells compared in minutes rather than hours. Its span of the day is
documented as it is: TAT-C's mean motion, in degrees per second, is read
as revolutions per day, so swath IoU compares from noon to the end of the
day rather than one orbit, for every record scored through it.
@ptgrogan
ptgrogan merged commit 95adf28 into main Nov 15, 2026
9 checks passed
@ptgrogan
ptgrogan deleted the catalog-earthcare-trmm branch November 15, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant