Skip to content

fix(verify): normalise the pytest summary duration when it carries a (H:MM:SS) suffix (AGT-4680) - #821

Merged
unohee merged 1 commit into
mainfrom
fix/agt-4680-pytest-summary-duration
Oct 3, 2026
Merged

unohee merged 1 commit into
mainfrom
fix/agt-4680-pytest-summary-duration

Conversation

@unohee

@unohee unohee commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • The pytest -q summary line gains (0:02:04) once a run passes a minute. The duration normaliser required the line to end right after the seconds, so the suffix kept the run time in the failure fingerprint.
  • Every full cgf-portal run is 113-149 s, so base and head fingerprints always differed, hasSameFailure was false, and a failure already on main was reported to the worker as a new failure.
  • The suffix is now optional in the regex in src/verify/runner.ts.

Evidence

  • After the 2026-10-03 23:24 restart, AX-1644 and AX-1767 (unrelated issues, neither touched B5) both got tests failed for tests/test_b5_repository.py::test_the_empty_result_has_the_same_shape_as_a_real_one.
  • Clean origin/main fails that test deterministically (2 of 2 runs), so it is not the workers' regression.
  • A probe of normalizeFailureOutput on in 124.27s (0:02:04) vs in 113.47s (0:01:53) returned equal=false before the change.

Tests

  • fingerprintStability.test.ts: two summaries differing only in duration and suffix normalise equal; a different failure count still differs.
  • Mutation: reverting the regex makes only the new test fail.
  • npx vitest run src/verify src/agents/deterministicTester: 202 passed. tsc --noEmit clean.

@unohee
unohee merged commit 26d0bfc into main Oct 3, 2026
7 checks passed
@unohee
unohee deleted the fix/agt-4680-pytest-summary-duration branch October 3, 2026 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant