Skip to content

ci: per-commit fast-check gate (catch broken intermediate commits under rebase-merge) - #1190

Open
vringar wants to merge 2 commits into
masterfrom
ci/per-commit-checks
Open

vringar wants to merge 2 commits into
masterfrom
ci/per-commit-checks

Conversation

@vringar

@vringar vringar commented Jun 15, 2026 •

Copy link
Copy Markdown
Contributor

Rebase-and-merge lands every PR commit on master individually, but CI only
tests the tip, so a commit that is broken on its own can still ship. This runs
the fast checks on each commit in the range: pre-commit run --all-files, the
commit-msg hooks, import openwpm, pytest --collect-only, plus npm ci /
npm run build if it touched Extension/ and ./install.sh if it changed
environment.yaml. No browser suite — that stays on the tip and the merge
queue.

The logic is in scripts/per_commit_checks.py rather than inline in the YAML,
so it can be linted and run locally:

python scripts/per_commit_checks.py --base master --head HEAD

It reads the range from the event payload, so nothing event-controlled reaches
a command line. Every commit is visited and every failure reported before the
non-zero exit. A failed conda rebuild is the exception and ends the walk, since
install.sh removes the env before re-creating it.

Only gates once it's a required status check in branch protection.

First commit: the commitlint hook was dead

It never linted anything. commitlint.config.js extends
@commitlint/config-conventional, a root package.json devDependency that
nothing installs in CI (MODULE_NOT_FOUND); and for anyone who has run root
npm install, rev: v9.11.0 bundles commitlint 18 against config-conventional
21, which loads no rules. Bumped to v9.26.0 with the preset as an
additional_dependency.

Consequence: release: vX.Y.Z is rejected, release not being a conventional
type. Not from the bump — it fails identically under 18. 5 of the last 40
master messages would fail, 4 of them release:. Release checklist now says
chore(release):.

@vringar
vringar force-pushed the ci/per-commit-checks branch from f361334 to 1041f4f Compare June 15, 2026 22:14
@codecov

codecov Bot commented Jun 15, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.34%. Comparing base (14745c4) to head (f0c19f8).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1190   +/-   ##
=======================================
  Coverage   62.34%   62.34%           
=======================================
  Files          40       40           
  Lines        3930     3930           
=======================================
  Hits         2450     2450           
  Misses       1480     1480           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vringar
vringar marked this pull request as ready for review June 19, 2026 23:03
Copilot AI review requested due to automatic review settings June 19, 2026 23:03
@vringar
vringar force-pushed the ci/per-commit-checks branch from 1041f4f to 24076a0 Compare June 19, 2026 23:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new GitHub Actions workflow to gate rebase-and-merge PRs by running a fast static-check suite on every individual commit in the PR (and in merge-queue merge_group candidates), preventing “broken intermediate commits” from landing on master.

Changes:

  • Introduces per-commit-checks.yaml, triggered on pull_request and merge_group.
  • Computes a commit range and iterates commit-by-commit, running pre-commit (changed-files), conditional Extension build, import openwpm, and pytest --collect-only.
  • Aggregates failures across commits (doesn’t stop at first failure) and fails the job at the end if any commit failed.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/per-commit-checks.yaml Outdated
Comment thread .github/workflows/per-commit-checks.yaml Outdated
@vringar
vringar force-pushed the ci/per-commit-checks branch from c4533be to 4183b69 Compare July 20, 2026 22:54
@vringar
vringar force-pushed the ci/per-commit-checks branch 7 times, most recently from ce797fb to 81f4d2c Compare September 6, 2026 20:35
The commit-msg hook has never linted anything. commitlint.config.js extends
@commitlint/config-conventional, which is a devDependency of the root
package.json, so the hook fails with MODULE_NOT_FOUND for anyone who has not
run `npm install` at the repo root — and nothing installs root node_modules in
CI. For a developer who has, rev v9.11.0 bundles commitlint 18 while the root
manifest provides config-conventional 21, so the preset loads with no rules and
commitlint errors out with "Please add rules to your commitlint.config.js".

Bump the hook to v9.26.0, which bundles commitlint 21, and declare the preset
as an additional_dependency so the hook no longer depends on root node_modules.

With the hook working, `release: vX.Y.Z` is rejected, as release is not one of
the conventional commit types (it was not one under commitlint 18 either, the
hook was simply never running). Note the `chore(release):` form in the release
checklist instead.
…er rebase-merge)

Add a lightweight CI job that runs the fast static checks on every commit in
a PR / merge-queue candidate, not just the tip. Because the project uses
rebase-and-merge, every commit lands on master individually, but the existing
tests job only validates the tip — a commit that is broken in isolation can
reach master. This gate closes that gap cheaply: per commit it runs the
pre-commit hooks (black/isort/mypy/actionlint) over the whole tree, checks the
commit message with the commit-msg hooks, rebuilds the extension when its
sources changed, imports the package, and collects the test suite. It
intentionally does not run the browser test suite — runtime and
browser-behavior regressions stay covered by the tests job on the tip plus the
merge queue.

The logic lives in scripts/per_commit_checks.py rather than inline in the
workflow, so it is readable, reviewable and runnable locally
(`python scripts/per_commit_checks.py --base master --head HEAD`) instead of
being a shell loop embedded in YAML. The workflow is reduced to checkout,
setup and one call. The script reads the commit range from the GitHub event
payload, so no untrusted event data is interpolated into a shell command.

A failed conda env rebuild ends the walk rather than blaming every later
commit, as install.sh removes the environment before re-creating it and leaves
nothing behind to check the remaining commits against.
@vringar
vringar force-pushed the ci/per-commit-checks branch from 81f4d2c to f0c19f8 Compare September 6, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants