feat(engines): GT-716 AC4 — the named backlog is implemented or declared; nothing coverage-only by omission - #797
Conversation
…red; guard 73 refuses coverage-only by omission Implemented, so the OPA-only side of engine-coverage-parity.baseline.json is EMPTY on both scenarios (was 7 / 7): - TelemetryEvidenceRuleHandler decides OBS-EVD-01..03 from the satellite's dependencies — the same proxy telemetry-evidence.rego reads, the same package lists, so the same verdict. The rules declare `satellite.packageJson` and nothing they do not read; `telemetryBackend` leaves the vocabulary (84 facets). - McpRuleHandler decides MCP-05 from the server source (the tokens mcp.rego looks for) and FAILS MCP-01..03 on absent smoke evidence with the policy's own words — the evidence is an observed fact of the checkout. Declared, in the rule where the declaration belongs: - SEC-INJ-01/02, SEC-PATH-01/02, SEC-TIMING-01/02, SEC-RL-03 declare `satellite.findings`: a scanner's finding over the AST, not a regex over the tree (MM-R10). unimplemented-native 21 → 14, needs-external-system 27 → 33. - QT-05 declares `satellite.testing` and SdlcRuleHandler no longer claims it — "always passes (requires runtime analysis)" was a fixed answer. It leaves the native-only baseline (82 → 81 / 49 → 48). Recorded, and enforced: `.harness/scripts/ci/engine-coverage-decisions.json`. Every baseline entry whose class is debt (no-policy-in-bundle, unimplemented-native, handler-declined, opa-gave-no-reason, undecided) must carry a decision — native-only / opa-only / neither — or guard 73 fails it as coverage-only BY OMISSION; a decision the runs contradict (its rule decided by both engines, or by the engine it said would not) fails too. Eight decisions: SSDF, SLSA, SEC-RL-01/02, MM-R* (11 — MM-R03 is decided by both, and the guard said so on its first run), the enforcer-routed HXA-01/02/04/05, the 138 generated ADR-conformance rules as a `neither` pattern (documentation-only on both engines — OpaEvaluator now states the declaration's class for a policy-less rule whose `facts: []` say there is nothing to check), the seven scanner rules and QT-05 as `neither`. Guard 68: GOV-RULE-NON-EXECUTABLE registered with a reason of its own — on 68's `--core`-less run the native engine resolves the corpus to the CLI's bundled copy and fails the 138 ADR-conformance rules (0 non-executable) while OPA now counts 138; with `--core .` both engines emit the identical row. Moving 68 onto the export is AC5. Pins moved: native-handler 171 → 173, needs-runtime 23 → 22; blocking rules that do not run 73 → 71 (5 handlers, 21 adapters, 14 runtime, 27 postures, 4 authoring); unclaimed 106 → 104. Snapshot, ISO 5055 mapping and the standards README re-derived (backlog 14: 5 adoptable, 0 partial, 9 to author). Parity fixtures follow the handlers. OPA README, known-limitations and the CLI's bundled copy updated (EN/ES). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
…ster is there by omission Catalog (EN/ES): AC4 ticked with what was implemented (OBS-EVD-01..03, MCP-05, MCP-01..03 on both engines), what was declared in the rule (the seven scanner rules, QT-05), what was recorded and enforced (engine-coverage-decisions.json, eight decisions, guard 73 refusing a debt entry without one), the falsifier observed before it was trusted (MM-R03, decided by both engines, flagged on the guard's first run), the measurements (opa-only 0 / 0, native-only 81 / 48), what was NOT done as first phrased (no .rego twins for the 11 no-policy rules — a recorded decision each) and the costs (backlog 21 → 14, blocking non-running 73 → 71, one registered conflict on guard 68 that AC5 removes by moving 68 onto the export). Board (EN/ES): row note and Last Updated; counters unchanged (688 / 715 · 3 in progress · 3 pending · 21 deferred), AC5 open. maturity-evidence.json asOf follows the board date (guard 09). Derived chain replayed (guard 46 at a fixed point). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
…ld has no production paths CI (Test infra-providers) caught the native twin doing what `--engine opa` always did: failing a freshly initialised satellite on OBS-EVD-01..03 — "no tracing / logging / metrics package" about a repository with no request path yet. The GT-571 invariant is the right one: a scaffold does nothing wrong. The rules speak of PRODUCTION request paths and services, so they now declare `appliesFromSdlcPhase: 3` (Construction: the first phase at which a codebase with request paths and dependencies exists) — the applicability fact their prose always stated, the same move MTN-05 made for Design. It excludes them BEFORE either engine runs, so both engines agree on the scaffold too. Deferred, not disabled: from Construction on, a satellite without those packages is failed for exactly the reason the rule states. Deferral test added beside MTN-05's; catalog (EN/ES) says so. Guards 73 / 68 / 46 / 08 green; infra-providers 189/189. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
|
CI caught the native twin of |
… Core's server, never a satellite CI (Test infra-providers, on a checkout with no `.harness/evidence/`) caught the second half of the same lesson: once the native engine decides MCP-01..03 the way the policy does (absent smoke evidence FAILS), a freshly initialised satellite validated against a Core without evidence got three blocking findings addressed to the Core — the GT-571 complaint verbatim. The laptop had the gitignored evidence file, so the local run passed; reproduced by hiding it. `scope: core-cli` always said whom the pack speaks to; `audience: core` now says it where applicability reads it (as the CLI packs do), so both engines exclude MCP-* on a satellite before running and still decide all five on the Core. MCP-01 pinned in the GT-571 exclusion list; catalog (EN/ES) says so. infra-providers 189/189 with the evidence hidden; guards 73 / 68 / 46 / 08 green. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: aarroyo <beyondnet.peru@gmail.com>
|
Second catch from the same job, visible only on a checkout without the gitignored |
What
GT-716 AC4: the named backlog is implemented or declared, none of it left in
coverageOnlyby omission.Implemented — the OPA-only side of
engine-coverage-parity.baseline.jsonis empty on both scenarios (was 7 / 7):TelemetryEvidenceRuleHandlerdecidesOBS-EVD-01..03from the satellite's dependencies — the same proxy and package liststelemetry-evidence.regoreads, so the same verdict. The rules declaresatellite.packageJsonand nothing they do not read;telemetryBackendleaves the vocabulary (84 facets).McpRuleHandlerdecidesMCP-05from the server source (the tokensmcp.regolooks for) and failsMCP-01..03on absent smoke evidence with the policy's own words — the evidence is an observed fact of the checkout; its absence is the finding.Declared, in the rule where the declaration belongs:
SEC-INJ-01/02,SEC-PATH-01/02,SEC-TIMING-01/02,SEC-RL-03declaresatellite.findings: a scanner's finding over the AST, not a regex over the tree (MM-R10forbids that kind of handler).unimplemented-native21 → 14,needs-external-system27 → 33.QT-05declaressatellite.testingandSdlcRuleHandlerno longer claims it — "always passes (requires runtime analysis)" was a fixed answer, not a verdict. It leaves the native-only baseline (82 → 81 / 49 → 48).Recorded, and enforced —
.harness/scripts/ci/engine-coverage-decisions.json. Every baseline entry whose class is debt (no-policy-in-bundle,unimplemented-native,handler-declined,opa-gave-no-reason,undecided) must carry a decision —native-only/opa-only/neither— or guard 73 fails it as coverage-only by omission; a decision the runs contradict (its rule decided by both engines, or by the engine it said would not) fails too. Eight decisions:SSDF-*,SLSA-*,SEC-RL-01/02,MM-R*×11, the enforcer-routedHXA-01/02/04/05(native-only, each with why and what reopens it); the 138 generated ADR-conformance rules as aneitherpattern — documentation-only on both engines, now thatOpaEvaluatorstates the declaration's class for a policy-less rule whosefacts: []say there is nothing to check; the seven scanner rules andQT-05asneither.Falsifier, observed before it was trusted
The extended guard's first run went red on
MM-R03 (decision modular-monolith-native-only)—modular-monolith.regodecides it and both engines agree, which the register had not said — and green once the id left the decision.Not done as the criterion first phrased it
No
.regotwin was authored for the 11 no-policy rules. Each is a recorded decision instead:SSDF/SLSA— the input builder projects none of what the handler reads, so a twin is the same check behind a second projection;SEC-RL-01/02— a source scan;MM-R*— an import graph the OPA path does not carry until GT-514's seam feeds it.Costs and side effects
GOV-RULE-NON-EXECUTABLE: on its--core-less run the native engine resolves the corpus to the CLI's bundled copy and fails the 138 ADR-conformance rules (0 non-executable) while OPA now counts 138; with--core .both engines emit the identical row. Moving 68 onto the export is AC5's.ea736a75; AC5 stays open.Verified locally
core-domain 2098 specs green (incl. 19 guard-73 unit tests); guards 73, 68, 27, 46 (fixed point), 08, 09, 01, 04, 66, 47, 49, 50, 42, 43 green;
npm run build:policygreen (415 rules checked, 84 facets).🤖 Generated with Claude Code