Repository navigation
Conversation
avalyset
force-pushed
the
fix/run-meta-followups
branch
from
November 4, 2026 13:11
9721918 to
0972e70
Compare
dev already passes meta=run_data.get("meta") in RepeatedExperimentResults.load (584f120). This keeps the regression test from the original fix, which also checks that a run saved without meta loads with an empty dict.
dev already sets result.meta["scenario_pack"] when AuditExperiment is given a pack name (584f120). This keeps the regression test from the original fix, which also checks that a run given a scenario list records none.
AuditResults.score is None when nothing was graded, so the example printed "Score: None/100". It now prints the same "n/a (nothing graded)" that summary() prints in that case.
avalyset
force-pushed
the
fix/run-meta-followups
branch
from
November 6, 2026 20:00
0972e70 to
0919c58
Compare
avalyset
changed the title
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the run metadata from #138. The two code fixes this PR first carried came in with #143 (584f120):
RepeatedExperimentResults.load()keeps each run'smeta, andAuditExperimentrecordsscenario_packwhen it is given a pack name. After the rebase on dev, this PR has the rest.Show a missing score in the README results example (
README.md)AuditResults.scoreisNonewhen nothing was graded, so the example printedScore: None/100. It now printsn/a (nothing graded), the same textsummary()uses.Two regression tests (
tests/test_repeated_experiments.py)TestSerialization::test_save_load_preserves_each_runs_meta: each run'smetasurvivessave()/load(), and a run saved withoutmetaloads with{}.TestAuditExperimentRepetitions::test_pack_name_is_recorded_in_each_runs_meta: a run started with a pack name records it, and a run started with a list of scenarios records none.dev has its own tests for both fixes (
test_audit_regressions_experiment.py:161-168,test_audit_regressions_pass6.py:84-86). These two add the run withoutmetaand the scenario-list case. If you would rather not keep them, the README change stands on its own.Testing: full suite in a fresh Python 3.13 venv with
.[dev,tracing], run withpytest -p no:randomly -n 4.dev(6b108cc) gave 2441 passed and 13 skipped; this branch gives 2443 passed and 13 skipped. The only per-test difference is the two new tests.ruff check --select E9,F821 .is clean.AI disclosure: implemented with Claude Code.