Repository navigation
Fix - Let org members waive the spec gate with qa-waived, report refusals as neutral, and build assets in CI - #19
Merged
Conversation
…sals as neutral, and build assets in CI
… not PHPUnit or a manual script
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three changes to the PR gates in
suite.ymlandpro-suite.yml.1. A
qa-waivedlabel can waive the spec gate. Some changes cannot have the spec the gate asks for (a Stripe or PayPal webhook, a cron handler, a subscription row with no order), because the harness cannot create that state. Reviewers were already waiving these by comment, with nothing to say who agreed. Now a member of the organisation applies theqa-waivedlabel.gate-refusal.mjs, checks that the label is on the PR and who applied it. The person must be an organisation member, or have write access on the repo. A bot never counts.labeledinpull_request.types, and the reusable workflows let onlyqa-waivedthrough on that event, so any other label is a skipped job. Existing callers need that one line added.@claudegrill suitere-runs for any caller without it.2. A refused PR no longer fails the job. Nothing was tested and nothing broke, so a red check read like a test failure.
suite.yml: the job passes with a warning, and a separate grey check run, "QA suite — refused", says why. This needschecks: writeon thesuitejob. Without it the check run is skipped with a warning and the job still passes.pro-suite.yml: thegatejob passes and reports no modes, so the whole matrix is skipped (grey). No new permission.3. CI builds the product's assets. A new "Build the product's assets" step runs a
buildcommand declared in the product's.themegrill-qa/suite.json, after install and before the site boots. A product with nobuildskips it and behaves as before. This is documented in SUITE.md.Why: CI tested whatever compiled CSS/JS was committed, so a source fix failed until the rebuilt files were committed too (user-registration #1535, #1548). Anything the build emits that is gitignored never existed on a runner at all. User Registration's React bundle (
chunks/) is one, and its setup-wizard specs failed or skipped in CI because of it.The matching
buildlines for user-registration and user-registration-pro are already on theirdevelopbranches.Verification
gate-refusal.mjsagainst a stubbedgh, 10 cases: member, write access, read-only access, a bot, a label re-applied by an outsider, no event found, no label, API failure, two gates tripped, nothing tripped. Plus the usage errors.gh, for both workflows, including a token withoutchecks: write.actionlinton both workflows: nothing new, apart from two info-level shellcheck notes (SC2016) on anode -epatternmainalready uses.setup-product.mjs check-workflowon the updated templates: clean.synchronizeand an unknown type are still flagged.build:no-makepottakes about 20 seconds and changes no tracked files. With the bundle built, the setup-wizard specs pass 6 of 6 on a fresh Playground site. The pro command (UR_PRO=true) also builds in about 15 seconds with no tracked changes.Not verified
Nothing has run in GitHub Actions. Callers pin
@main, so it can only be exercised after merge. Specifically unproven:ifallow-list forlabeled+qa-waivedGITHUB_TOKEN(it may not see private members, which is why write access is the fallback)checks: writemodes=[]output skipping the pro matrix, and theanswerjob staying quiet on a refusalTo test after merge: open a PR with a source change and no spec, confirm the grey refusal, apply
qa-waived, and confirm the suite runs.Notes
labeledfires on every label. Other labels cost one skipped job and no runner minutes.labeledrun ofqa-waivedon a PR that was never refused will run the suite again.🤖 Generated with Claude Code