Skip to content

scientific(coverage): bind rolling-origin cutoff geometry into the prospective design #753

Open

Description

@seonghobae

Finding

#750/#751 bind the declared successful-window cardinality (5) into CoverageCalibrationDesign::tepp_nominal_95_v1() and make analysis_engine consume that owner value. The actual cutoff geometry that defines those five windows is still partly caller-local in analysis_engine::coverage_calibration_execution: FIRST_TRAINING_EVENT_INDEX: usize = 3 selects the first event admitted to the cutoff sequence, recovery_cutoffs(...) then uses each selected event's latest document available_time as its cutoff, and adjacent cutoffs form expanding rolling-origin windows.

The current design fingerprint v8 binds only the count. A future executor can move the first admitted event or change the cutoff-anchor rule while preserving five windows and therefore retain the same prospective validation-design fingerprint while evaluating a different temporal training/evaluation geometry. Exact source_head would show that code changed, but the prospective scientific design would no longer identify the window geometry declared before outcomes were observed.

Required repair

  • keep actual snapshot/partition construction in analysis_engine and canonical admission in corpus_split;
  • give CoverageCalibrationDesign::tepp_nominal_95_v1() a stable rolling-origin geometry identity for the current rule: start at event index 3, derive one cutoff per selected event from that event's latest document availability time, and use adjacent cutoffs with expanding-history admission;
  • expose the first training event index from the validation owner and remove the analysis-local FIRST_TRAINING_EVENT_INDEX authority;
  • bind the geometry identity and first event index into a new coverage-design fingerprint domain before the 10,000-DGP run;
  • make execution consume the owner-issued first event index while preserving the existing five-window arithmetic, scenario, seed schedule, training-fit ALR estimand, interval rule, percentile/MCSE methods, denominator semantics, and undefined numerical-failure acceptability;
  • pin the new fingerprint and add owner/executor regression evidence;
  • do not claim this repair executes or passes scientific(selection): add leakage-safe rolling-origin K validation #680 acceptance.

RED boundary

The current owner and executor both encode the same effective geometry, so changing a literal solely to fabricate a failing numerical test would be synthetic. Treat this as an ownership/prospective-design drift finding: add regression contracts that would fail on future owner/executor divergence, and do not claim a hosted failing RED unless an actual exact-head run demonstrates one.

Refs #639 #680 #733 #740 #741 #750 #751.

Activity

  1. seonghobae commented on Sep 24, 2026

    @seonghobae
    ContributorAuthor

    Implemented ordinary-forward on #639 at exact head 5eaa422e318903407c1ed8a2ee0a7e82c07d2b0f.

    • validation owner now issues geometry ID tepp.coverage.rolling_origin_geometry.latest_event_availability_adjacent_expanding.v1 and first training event index 3;
    • coverage-design fingerprint domain is v9 and binds geometry ID + first event index before window count; canonical fingerprint is f6fd3855ec7ef413309cbb0b48b3572853d4fa87d4c46e47f265e06074aeffa0;
    • analysis_engine removed its local first-event literal and consumes validation_design.first_training_event_index() while retaining actual cutoff/snapshot/partition construction;
    • regression evidence binds the first selected event to its latest-availability cutoff and the complete owner-issued window count;
    • CHANGELOG records the boundary.

    No synthetic RED is claimed because old executor and new owner both use index 3; this is prospective-design/ownership drift repair. The 10,000-DGP study and #680 acceptance remain unexecuted. Keep #753 open until protected-main inheritance/exact-head gates and review are satisfied.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions