Skip to content

refactor: derive metrics, objectives, and run artifacts from one registry - #23

Merged
rasros merged 5 commits into
mainfrom
refactor/metrics-registry
Jul 28, 2026
Merged

refactor: derive metrics, objectives, and run artifacts from one registry#23
rasros merged 5 commits into
mainfrom
refactor/metrics-registry

Conversation

@rasros

@rasros rasros commented Jul 28, 2026

Copy link
Copy Markdown
Owner

The NSGA selection machinery was already arity-agnostic, but the pipeline feeding it was not: adding a metric meant editing 53 sites across 9 files, and the README's extensibility claim had to be hedged because of it. Now that count is zero, and the claim is verified rather than asserted.

METRICS in score/complexity.py maps a metric name to a measure over (rendered PNG, source text). Everything downstream derives from it: SearchNode/Result carry a metrics dict instead of named fields, build_objectives emits (score, *METRIC_NAMES), LINEAGE_COLUMNS splices the metric names into the header, and both analysis scripts read their columns, Pareto keys, plot axis, and summary table from the registry.

Visual error stays deliberately outside it: it comes from the configured scorer, not a complexity measure, and it is the constraint-gated primary objective rather than one of the interchangeable tiebreakers.

Verified by actually adding a fourth metric. One entry, no other edits, then observed it propagate:

METRIC_NAMES     : ('visual_complexity', 'structural_complexity', 'element_count')
LINEAGE_COLUMNS  : [..., 'visual_complexity', 'structural_complexity', 'element_count', ...]
objective arity  : {4}
lineage row      : element_count written = 3
plot_run read    : all three read back
pareto over 4    : works

Reverted afterwards; a permanent test now pins the properties instead (registry contents, measure_all covering every metric, lineage columns containing every metric, and objective arity equal to len(METRIC_NAMES) + 1).

resume.py's positional 7-tuple is gone, replaced by a PreppedNode dataclass. This was the worst of the 53 sites: metrics were read as prepped_nodes[i][4] and [i][5], so registering a metric would have silently shifted every later index. That is the same misalignment class already fixed for stats.csv and lineage.csv, and it was the last instance.

The round-trip test added in #22 caught a bug I introduced here. Generalizing load_complexities_from_lineage to node.update(read_metrics(row)) dropped an implicit guard: the old if row.get("visual_complexity") check also skipped sparse eviction rows. Without it, an eviction row zeroed the metrics its node's real row had already supplied — node 2 read back 0.0 instead of 2000.0. Fixed with an explicit row_has_metrics predicate, which is clearer about why the check exists. Worth noting because the previous suite would not have caught this.

read_metrics defaults absent columns to 0.0, so a row written before a metric was registered stays readable, and the pre-split legacy complexity column mapping now lives next to the registry instead of being restated in both scripts.

The plot axis and summary table are registry-driven too: the scatter plots the first registered metric with a label derived from its name, and the star annotation reports the true objective count. The Pareto front is still a 2-D projection of an N-objective front, which the docstring already explains.

README updated: the extensibility paragraph no longer hedges, and it now points at the registry and explains why score sits outside it. The prior warning about dominance diluting past ~4 objectives is retained and repeated as a comment on the registry, since making objectives cheap to add is not the same as making them free.

Verified end to end after the refactor: all three formats complete an offline multi-epoch search with --tournament-size 3 and --epoch-variance 0.02 (svg 0.0027, graphviz 0.1467, typst 0.0586), the lineage header and rows are correct, and both scripts run clean against a real run.

459 passed, 13 skipped, ruff clean.

@rasros
rasros merged commit dad9d26 into main Jul 28, 2026
@rasros
rasros deleted the refactor/metrics-registry branch July 28, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant