Skip to content

Make the comparison behind ADR 0002's Evidence table reproducible #231

Open

Description

@jmacd867

Is your feature request related to a problem? Please describe.
The Evidence table in ADR 0002 (feature sets × log/linear, mean adjusted Rand index over seeds 0 to 11) came from a scratch script that is not in the repo. The ADR describes the steps, but the table cannot be regenerated or extended without rewriting them, and it already went stale once when the generator changed (#225).

Describe the solution you'd like
A small script under inference/ that, for a seed range, runs generate_pilots, compute_features and the clustering steps for each feature set and scale, and prints the table. It should reproduce the ADR's figures at seeds 0 to 11.

Describe alternatives you've considered
Keep the steps in the ADR prose only. Or make the table a test; a table is not pass/fail, and 12 seeds × 8 fits is slow.

Additional context
Pair with #230 so the table can include a cohort that separates mad. Also note why the k sweep ranks k=2 best on the default 15-pilot cohort (silhouette 0.47, stability 1.0) despite 3 hidden tiers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions