Skip to content

E1b: replay fix outcomes outside the E1 sample #648

Open

Description

@hyperpolymath

Why

E1 replayed 130 "success" rows from 2 repositories, and their recipes' own predicates confirmed none of them. Its measured claim covers only those rows. The structural claim, that labels outside the sample are unverified, rests on the recorders, not on measurement. E1b would measure how many of the other "success" labels are actually false. That decides the size of the label loss for the Hypatia pools, though not whether a restart is needed (ultraplan v0.2.3).

Scope

  • Repositories beyond batch-template-fixes and the second E1 repo, sampled by recipe volume.
  • recipe-heredoc-to-install (2,863 rows) and recipe-todo-to-fill (1,832). Both are robot-repo-automaton rows with no predicate, so a predicate must be defined first (see the reporter/fixer issue for those two scripts).
  • scripts/audit-fix-outcomes.sh already handles repo-scoped rows, . markers (fix(outcomes): replay "."-marked rows; let shellcheck parse dispatch-runner #623), NA on predicate failure, and the fixers' third-party exclusions.

Output

docs/experiments/E1b-…adoc plus a TSV, reproducible byte for byte from the replay script, like E1.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions