Skip to content

ci: fail when the UI test fixture is out of date - #523

Merged
jsenecal merged 1 commit into
mainfrom
ci/522-fixture-drift
Nov 20, 2026
Merged

jsenecal merged 1 commit into
mainfrom
ci/522-fixture-drift

Conversation

@jsenecal

Copy link
Copy Markdown
Collaborator

Closes #522

Problem

The frontend tests read hyperglass/ui/__tests__/fixtures/config.json, which .tests/ui-fixture/generate.py builds from the backend models (#504). Nothing checked that the committed file is current, so a change to the UI parameters could leave the frontend tests passing against parameters the real UI build no longer receives.

Change

  • ci.yml, backend job, Python 3.12 only: a "Check the UI test fixture" step after lint. It runs the generator and fails with an error annotation and the regeneration command if git diff shows a change. The generator uses fakeredis and 192.0.2.0/24 addresses, so it needs neither the Redis service nor DNS. The job installs the project in editable mode, so it imports hyperglass without PYTHONPATH.
  • CONTRIBUTING.md: under "Local checks", when and how to regenerate the fixture.

Tests

Ran the step's commands locally on this branch:

  • Unchanged tree: the generator rewrites the file byte for byte, and git diff --exit-code passes.
  • After setting web.text.title: Drifted in .tests/ui-fixture/config.yaml without committing a new fixture: config.json | 2 +-, so the step fails.
  • After restoring the YAML: passes again.

If the generator itself fails, the step fails too, because Actions runs run: with bash -eo pipefail.

Changelog

  • CI fails when the frontend test fixture is out of date with the backend models.

Notes

Only the 3.12 leg runs the check. The output doesn't depend on the Python version, and one failure is enough.

The frontend tests read hyperglass/ui/__tests__/fixtures/config.json,
which .tests/ui-fixture/generate.py builds from the backend models. When
the UI parameters change and nobody regenerates it, the frontend tests
keep passing against parameters the real UI build no longer receives.

The backend job now regenerates the fixture on one Python version and
fails with a hint if the committed file differs. #508 made the output
deterministic, so an unchanged tree regenerates byte for byte.
CONTRIBUTING documents the regeneration step.
@github-actions github-actions Bot added docs Improve documentation ci Build, test and release pipelines labels Nov 20, 2026
@jsenecal
jsenecal requested a review from WilhelmZA November 20, 2026 20:29
@coveralls

Copy link
Copy Markdown

Coverage Status

coverage: 85.11%. remained the same — ci/522-fixture-drift into main

@WilhelmZA WilhelmZA left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 06f3a1d0.

The "Check the UI test fixture" step runs on the Python 3.12 backend leg after the dependencies are installed, and it passed in the CI run for this commit. It adds no trigger, permission or dependency change. Both "Done when" items of #522 are met, so Closes #522 is right. The commit is verified and CI gate passes.

#528 changes the same two files, .github/workflows/ci.yml and CONTRIBUTING.md. Whichever merges second needs a check that this step and the "Local checks" paragraph are still in place.

Nits:

  • The description says Actions runs run: with bash -eo pipefail. Without an explicit shell:, the default is bash -e {0}. The step has no pipe, so the result is the same.

@WilhelmZA WilhelmZA added the reviewing Issue is being reviewed by maintainers label Nov 20, 2026
@WilhelmZA WilhelmZA added the frontend Next.js UI and the runtime UI build label Nov 20, 2026
@jsenecal
jsenecal merged commit 6e358a9 into main Nov 20, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Build, test and release pipelines docs Improve documentation frontend Next.js UI and the runtime UI build reviewing Issue is being reviewed by maintainers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fail CI when the frontend test fixture is out of date

3 participants