feat(gate): explain why a file was flagged - #220
Merged
Conversation
There was a problem hiding this comment.

Query Doctor — 6 successful checks
More details via MCP → get_ci_run({ runId: "019fd2a1-cab4-7c41-aefd-a6fae4da71ff" }) · view run · docs
3 queries read against main on assumed statistics of 10,000,000 rows per table. Sync production stats for costs measured against your real data.
veksen
force-pushed
the
refact-gate-drop-config-seam
branch
from
August 5, 2026 15:37
99771cb to
bb8a796
Compare
veksen
force-pushed
the
feat-gate-evidence
branch
from
August 5, 2026 15:37
f128c15 to
33857b2
Compare
veksen
force-pushed
the
refact-gate-drop-config-seam
branch
from
August 5, 2026 15:47
bb8a796 to
916551d
Compare
veksen
force-pushed
the
feat-gate-evidence
branch
from
August 5, 2026 15:47
33857b2 to
c3868d8
Compare
The gate decided but could not explain. Six precision fixes in four weeks were each diagnosed the same way: clone the analyzer, write a throwaway script, replay the PR's changed-file list, and print which pattern matched what text. The module had that information and dropped it on the way out. Give every content pattern a stable name and return the match. findQueryCode reports the rule, the line within the added lines, and an excerpt of the text. patchAddsQueryCode stays as the boolean form, now defined in terms of it. Rule order is precedence, so a line matching both `.execute(` and the raw select shape reports the first, which names the cause better. Line numbers required a change to the three strippers. They removed comment, import and block-comment text, which shifted every later line, so an offset in the stripped text pointed at the wrong place. They now replace that text with spaces and keep the newlines, leaving positions intact. No behaviour change. The full suite passes, and replaying 42 PRs from Site, Nutcracker, this repo and d2armory classifies the same files as before. Two of those PRs now explain themselves. The unresolved false positive on apps/api/src/db/test-db.ts reports rule `drizzle-init` at line 2 on the text `drizzle(`, which is the testcontainers harness being read as production data access. The one true positive in the corpus, project-queries.repository.ts, reports `drizzle-sql-tag` at line 35.
The gate now knows which rule matched and where, but kept it to itself: the
verdict carried a list of paths, so a reader still had to clone the analyzer to
find out why their file was flagged.
Carry the evidence through. `dataAccessFiles` becomes a list of `{path,
evidence}` rather than strings, and both renderings use it. The PR comment for
Site#3530 goes from
- `apps/api/src/db/test-db.ts`
to
- `apps/api/src/db/test-db.ts` — matched `drizzle-init` on line 2: `drizzle(`
which is enough to see that the flag is the testcontainers harness being read
as production data access, without leaving the PR.
Evidence is absent when GitHub supplied no patch, which happens for large and
binary files. The filename prior decides those alone, there is no matched text
to point at, and the line renders as a bare path. A test covers each branch.
The check annotation built in main.ts gets the same treatment. It renders
separately from the template, which is its own problem, but not this change's.
Full suite 429/429, typecheck clean. Verified by rendering the real comment for
Site#3530 and #3573 through the reporter's own nunjucks config.
veksen
force-pushed
the
feat-gate-evidence
branch
from
August 5, 2026 15:53
c3868d8 to
5b8eff7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Make the untested-data-access gate explain itself, so a false positive can be judged from the PR comment instead of by cloning the analyzer.
The gate has taken six precision fixes in four weeks (Query-Doctor/Site#3531, #3550, #3615, #3650, plus two cases nobody filed). Every one was diagnosed the same way: check out the analyzer, write a throwaway script, replay the PR's changed-file list, print which pattern matched what text. The module had that information and discarded it before anyone could see it.
An audit of all 15 PRs the gate has ever fired on, across Site, Nutcracker, this repo and veksen-enterprises/d2armory, found 2 correct flags. One false positive is still open:
apps/api/src/db/test-db.tson Query-Doctor/Site#3530 and #3573.What
Before, the PR comment listed paths:
After, it names the cause:
That line is enough to see the flag is the testcontainers harness being read as production data access. It is the open false positive above, diagnosed without leaving the PR.
How
Read
src/gate/test-presence.tsfirst, then the two rendering sites.The 20 anonymous regexes in the content matcher become named rules:
raw-select-from,drizzle-sql-tag,ddl-create-index, and so on.findQueryCodereturns the first that matches, with the line and an excerpt.patchAddsQueryCodestays as the boolean form, defined in terms of it. Rule order is precedence, so a line matching both.execute(and the raw select shape reports the first, which names the cause better.Line numbers required a change to the three strippers. They removed comment, import and block-comment text, which shifted every later line, so an offset into the stripped text pointed at the wrong place. They now replace that text with spaces and keep the newlines. This is the one behavioural risk in the change: blanking leaves whitespace where deletion closed the gap, and the raw select shape has a 300-character budget between
selectandfrom, so blanking is marginally more conservative. I checked that against the corpus below.dataAccessFilesbecomes{path, evidence}rather than strings. Evidence is absent when GitHub supplied no patch, which happens for large and binary files: the filename prior decides those alone, and the line renders as a bare path.The check annotation in
main.tsrenders separately from the template rather than sharing a formatter. That predates this change and I left it alone.Tests
Six new cases in
src/gate/test-presence.test.ts, each written before its implementation and observed failing:Full suite 429/429.
npm run typecheckclean.Beyond the unit tests, I replayed 42 real PRs from Site, Nutcracker, this repo and d2armory through
classifyChangedFiles. The set of files classified as data access is identical before and after, so the stripper change altered no outcome. I also rendered the comment for Site#3530 and #3573 through the reporter's own nunjucks config, with a patchless file appended so both branches of the new loop render.