Skip to content

test: pin the lineage.csv reader/writer contract and flag legacy-scale runs in plots - #22

Merged
rasros merged 4 commits into
mainfrom
test/lineage-roundtrip
Jul 28, 2026
Merged

test: pin the lineage.csv reader/writer contract and flag legacy-scale runs in plots#22
rasros merged 4 commits into
mainfrom
test/lineage-roundtrip

Conversation

@rasros

@rasros rasros commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Two loose ends from the complexity split.

The reader half of the lineage.csv schema was unpinned. Both writers go through one LINEAGE_COLUMNS list, so a row cannot misalign with the header, and tests assert that. Nothing checked the other direction: plot_run.load_lineage and clean_runs.load_complexities_from_lineage address columns by string literal and fall back to 0.0, so a reader looking for a column the writer no longer emits degrades to zeros in silence.

That is not hypothetical. I introduced a stale reader deliberately, and the entire pre-existing suite passed:

reader-side drift, full suite minus the new file:  444 passed
the same drift, new round-trip tests:              1 failed

The new tests write with the real FileStorageAdapter and read with the real script loaders, so both sides have to agree. They cover the score and both complexity objectives round-tripping, an eviction row resolving to the correct final pool, clean_runs reading the same values, and one guard that every key the reader produces corresponds to a column the writer actually emits — which is what catches an invented column name.

For completeness I checked the opposite direction too: a writer-side rename was already caught by the existing storage tests, so only the reader direction was exposed.

Legacy runs are now labelled in plots. Splitting complexity changed the scale of that axis. Pre-split runs read their single complexity column into the visual objective, so old and new runs both plot fine but on incomparable scales, and an overlay silently mixed them. uses_legacy_complexity detects the old header and _label appends (legacy complexity); routing it through _label means all five call sites (score history, Pareto scatter, both convergence series, and the summary panel) pick it up at once. Verified on a mixed overlay:

2026-01-01_00-00-00: legacy=True   label='2026-01-01_00-00-00 (legacy complexity)'
2026-01-02_00-00-00: legacy=False  label='2026-01-02_00-00-00'

I chose labelling over the stronger option of excluding legacy runs from the Pareto panel: showing the data with a caveat is less surprising than silently dropping a run the user explicitly pointed at.

The new test module skips rather than fails if scripts/ or matplotlib is unavailable, since the analysis scripts are not part of the installed package.

451 passed, 13 skipped, ruff clean.

@rasros
rasros merged commit 90e9f6b into main Jul 28, 2026
1 check passed
@rasros
rasros deleted the test/lineage-roundtrip branch July 28, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant