fix(ci): run the integration tests, the fresh-clone canary, and the uncovered paths - #429
Merged
Merged
Conversation
…ncovered paths Three CI gates reported success without doing the work they claim to. 1. Integration tests never ran. Every `cargo test` in the workflows was `--lib`-scoped, so all 154 real integration tests in src-tauri/tests/ had never executed in CI on any workflow — including all 12 migration tests and the only forward-migration coverage, against a TARGET_VERSION=103 chain with no checksums and no downgrade path. Both workflows now use `--tests` (lib + bins + integration). Measured before changing anything: all pass, 0 failures, and all are hermetic by construction (pipeline_integration uses an in-memory DB via test_utils::test_db(), migration_tests uses tempfile::tempdir()). The count floor now SUMS every binary instead of reading the last one, which would otherwise have tripped on every run, and a new binary-count assertion fails loudly if this is ever re-scoped to --lib. 2. The hermetic fresh-clone canary never ran outside PRs. `fresh-clone` needs the PR-only `changes` job, and GitHub skips a job whose `needs` was skipped unless its `if:` contains a status function. Push-to-main, the nightly cron and manual dispatch therefore all reported success in 6-26 SECONDS without cloning anything (observed 7s, 7s, 9s vs ~19min for a real PR run) — the cron had never built a single fresh clone. Wrapped in `!cancelled()`. The identical defect silently disabled workflow_dispatch for Frontend, MCP Server and the whole Rust matrix in validate.yml; same fix. 3. Path filters were an allowlist that failed open. site/, paddle-webhook/, mcp-memory-server/, editors/vscode/ and .husky/ matched no filter, and an unmatched path meant "skip == pass" while `Validate Success` is the only required check and auto-merge is enabled repo-wide. Adds .github/** and .husky/**, plus an `uncovered` fail-safe filter (predicate-quantifier: every, verified against the action source at the pinned SHA) so an unrecognised path RUNS the generic gate instead of passing silently — including directories added in future. Also wires `pnpm run test:scripts` into the Frontend job: the guards' own 53 self-tests ran in no hook and no workflow, so nothing verified the guards still detect what they claim. Deliberately NOT included: no branch-protection or ruleset change. Making `Hermetic Success` required is the correct end state but is a separate, human-reviewed step — see the PR body. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AUeKTKwNmdow8yUk3q8RB2
Today's outage made this concrete. #423 was a Rust-only PR, so the path filter skipped its Frontend job — and `scripts/check-file-sizes.cjs --ci` ran ONLY inside that job, even though it scans BOTH trees (SCAN_DIRS = ['src', 'src-tauri/src']). A 1032-line src-tauri/src/analysis_rerank.rs merged past the 1000-line hard error threshold with the gate never executing. check-file-sizes then exited 1 on main, and because .husky/pre-commit treats that as blocking, every developer in the fleet was unable to commit until #430 landed. A gate that guards Rust files must not be reachable only through a filter that excludes Rust. Adds a `repo-guards` job with NO path filter and NO `needs:`, so it runs on every pull request and dispatch whatever changed. It carries check-file-sizes, check-no-window-spawns, check-release-channel and the guard self-tests, which are removed from `frontend` — they were never frontend-specific. It needs no pnpm install (all three guards use only node builtins) and runs hosted in ~40s. It is in `validate-success`'s needs, so it actually gates the merge: since it never skips, it is the only leg guaranteed to have run. Also folds in `pnpm run test:scripts` — 53 tests across 6 files that verify the guards still detect what they claim, and which previously executed in no hook and no workflow. package.json is deliberately untouched: a peer holds a claim on it and #418 edits the same line, and wiring it here achieves the same goal. Rust job timeout 30 -> 45: that job now compiles the integration test targets, and Swatinem's cache is only saved from main, so until this lands every PR run pays a cold link for 5 extra binaries. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AUeKTKwNmdow8yUk3q8RB2
runyourempire
force-pushed
the
worktree-agent-a0963da5be8b5a2ec
branch
from
August 14, 2026 14:23
cee6a86 to
31cc69e
Compare
This was referenced Aug 14, 2026
Merged
runyourempire
added a commit
that referenced
this pull request
Aug 14, 2026
…he ghost gate (#434) check-remove-by.cjs shipped in #421 wired to nothing; running it caught osv::Affected::versions falling due 2026-08-15. Removed after verifying inert (no .versions read in osv/, matching uses affected_ranges with an assume-affected fallback, serde ignores unknown fields) plus its five constructors. Also backlogged the 13 commands #425's seed missed, which had been failing the ghost gate for every Rust-touching commit; entries record that they are not a #421 regression (their callers were components nothing mounted). Flags that Standing Queries and Cross-Project Intelligence are marketed on /signal with no UI despite complete, tested backends. No always-on invariants workflow: #429 already shipped Repo guards.
runyourempire
added a commit
that referenced
this pull request
Aug 15, 2026
…pply-chain blind spot (#433) ## The premise, verified first An audit lane claimed the quick-xml suppression in `deny.toml:96-112` / `.cargo/audit.toml:15-29` had gone stale. It rested on this justification: > "NO consumer in our tree has a released version against >=0.41 yet" **Confirmed false.** Read straight out of the registry index (`rust_version` and `deps` per published version): | consumer | we had | latest | quick-xml req | zip req | |---|---|---|---|---| | `calamine` | **0.25.0** | 0.36.1 | `^0.31` → **`^0.41`** | `^1.0` → `^8.6` | | `docx-rs` | **0.4.20** | 0.4.22 | `^0.36` → **`^0.41`** (since 0.4.21) | `^0.6.3` → `^8.6` | | `plist` | **1.9.0** | 1.10.0 | `^0.39.2` → **`^0.41`** | — | All three shipped support. The ignores were suppressing a live, fixable advisory pair on a parser that reads **user-supplied `.xlsx` / `.docx`**. ## What changed **quick-xml (RUSTSEC-2026-0194 / -0195) — resolved, not re-justified.** Bumping the three consumers collapses `quick-xml` **0.31.0 + 0.36.2 + 0.39.4 → a single 0.41.0**. Both advisories stop firing on their own, so both ignores are **deleted** from `deny.toml` *and* `.cargo/audit.toml` (they had diverged; both were checked). `RUSTSEC-2023-0071` (`rsa`) left alone as instructed. **`office.rs` needed no edit** — and that is a verified claim, not an absence of errors: - `sheet_names()` and `worksheet_range()` have byte-identical signatures in 0.25 and 0.36. - `Data` still has exactly the same nine variants with the same payloads. `cell_to_string` matches it **exhaustively with no wildcard arm**, so an added variant could not have compiled. - `ExcelDateTime`'s `Display` impl is byte-identical (`write!(f, "{}", self.value)`), so `DateTime` cells format the same. - Same for `docx-rs`: `TableChild` / `TableRowChild` are destructured irrefutably, so a new variant there could not have compiled either. The documented decompression-bomb weakness (the 100 MB cap is on the **compressed** size) is untouched — separate work, not regressed. **zip — partial.** `zip 1.1.4` retired as hoped. **`zip 0.6.6` did not** — it is our own direct `zip = "0.6"`, so retiring it is an 8-major API migration across `osv/cache.rs`, `extractors/archive.rs` and `embeddings_providers/fastembed.rs`. No advisory attaches to it, so it is staleness, not exposure. Left as follow-up rather than smuggled into a security PR. Tree is now `zip` 0.6.6 (ours) + 4.6.1 (tauri-plugin-updater) + 8.6.0 (calamine/docx-rs). **`relay/` — 5 vulnerabilities → 0.** A TLS-terminating server with no Dependabot entry, no cargo-audit, no CI. | crate | change | advisory | |---|---|---| | `rustls-webpki` | 0.103.9 → **0.103.14** | RUSTSEC-2026-0049 / -0098 / -0099 / -0104 (cert validation) | | `spin` | 0.9.8 → **0.9.9** | 0.9.8 was **yanked** | | `anyhow` | 1.0.102 → 1.0.104 | RUSTSEC-2026-0190 | | `event-listener` | 5.4.1 → 5.4.2 | RUSTSEC-2026-0221 | | `rand` | 0.8.5 → 0.8.7 | RUSTSEC-2026-0097 | `rsa 0.9.10` remains with no fix available, and is recorded in a new `relay/.cargo/audit.toml` with evidence that it is **not in the build graph**: it reaches `Cargo.lock` only via sqlx's optional `mysql` backend, which relay never enables — `cargo tree -i rsa` and `cargo tree -i sqlx-mysql` both report *nothing to print*. **Coverage, so it stops recurring.** `dependabot.yml` gains a `cargo` entry for `/relay` (not a `src-tauri` workspace member, so the existing entry never saw it), and `nightly-audit.yml`'s cargo-audit step now loops every `Cargo.lock` in the repo. **Workflow footprint is deliberately limited to those two files** — `validate.yml` is being reshaped by peer PRs and is untouched here. **`relay/Dockerfile`.** `cargo build --release --locked 2>/dev/null || cargo build --release` silently dropped lockfile enforcement and swallowed the reason. Fallback removed. Its base image also had to move **1.82 → 1.95**: the lockfile already required 1.88 via `time 0.3.47` (`jsonwebtoken` → `simple_asn1`), so that image could not have built this crate at all — the fallback was hiding a hard failure, not surviving a soft one. ## Two things found on the way **1. `main` was un-committable — independently confirmed, now fixed by #430.** `scripts/check-file-sizes.cjs` exits 1 on `src-tauri/src/analysis_rerank.rs` (**1032 lines against a 1000 hard limit**, arrived with #423). The gate scans the whole repo rather than staged paths, so `.husky/pre-commit` failed for *every* terminal on *every* commit — including this one. I hit it, diagnosed it, and fixed it the same way a peer did in **#430** (lift the test module into a sibling `analysis_rerank_tests.rs` via `#[path]`, 1032 → 866). #430 landed first, so **that commit has been dropped from this branch by rebase** — this PR now contains only the dependency work. Recording it here as an independent second confirmation of both the diagnosis and the chosen fix. **2. `cargo clippy --all-targets -- -D warnings` does not pass on `main`** (255 pre-existing errors at my branch point, ~all `unwrap_used`/`expect_used` in test code). This is **not** the gate — CI runs `cargo clippy ${{ matrix.cargo-features }} -- -D warnings` *without* `--all-targets`, so the numbers below are from the CI-equivalent invocation. Reported as an observation, not touched. ## Verification | check | result | |---|---| | `src-tauri` `cargo audit` | **exit 0** — zero vulnerabilities, zero warnings | | `src-tauri` `cargo deny check` | **exit 0** — `advisories ok, bans ok, licenses ok, sources ok` | | `relay` `cargo audit` | **exit 0** (was 5 vulns + 4 warnings + 1 yanked) | | `relay` `cargo check --locked --all-targets` | clean | | `cargo clippy -- -D warnings` (CI-equivalent, default) | **exit 0** | | `cargo clippy --features experimental -- -D warnings` | **exit 0** | | `cargo fmt --check` | **exit 0** | | `cargo test --lib` | **4300 passed, 0 failed, 10 ignored** | All re-run after rebasing onto `c1fd348c` (#425, #426, #427, #429, #430, #431 all landed mid-flight). `--features team-sync` and `--features enterprise` fail to compile — **pre-existing rot on `main`** (`chacha20poly1305::aead::OsRng` unresolved, then cascading `__cmd__*` macro failures), which is what #424 exists to repair. My lockfile diff touches no crypto crate. #424 is still open as of this push, and the CI clippy matrix on `main` still carries only the `default` and `experimental` legs — so the two legs verified above are exactly the gate. ### The extractor tests were `#[ignore]`d and had never run There are no `.xlsx`/`.docx` fixtures anywhere in the repo, so `test_real_docx_extraction` / `test_real_xlsx_extraction` were no-ops that returned early. To gain real confidence in an 11-minor-version parser bump I generated **real OOXML documents** — shared strings, an inline string, numeric and boolean cells, paragraphs and a table — confirmed both `#[ignore]`d tests pass against them, and separately asserted the extracted text matches the pre-bump formatting contract exactly: ``` === Sheet: Budget === Hello from 4DA Item | Cost Second paragraph Widget | 42 A1 | B1 Gadget | 3.50 | TRUE ``` That exercises every arm of `cell_to_string` that a document can reach (shared/inline string, integral float → `42`, fractional float → `3.50`, bool → `TRUE`) plus the docx paragraph and table paths. The scratch harness was deleted; **no test-file changes ship in this PR**. ## Deliberately left - **`zip 0.6.6`** — direct dep, 8-major API migration, no advisory. Follow-up. - **`office.rs` decompression bomb** — the 100 MB cap is on the compressed size. Out of scope, not regressed. - **`--all-targets` clippy backlog** — pre-existing, not the CI gate. - **`validate.yml`** — peer-owned right now, untouched on purpose. - **`--features team-sync` / `enterprise`** — pre-existing rot, #424's job. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Four CI gates reported success without doing the work they claim. Each was verified against live run data before anything was changed.
1. The integration tests had never run in CI — on any workflow
Every
cargo testin.github/workflows/was--lib-scoped (validate.yml:303/308,hermetic.yml:167/176). No--tests, no--all-targets. So all 154 real integration tests insrc-tauri/tests/had never executed in CI, including all 12 migration tests and the repo's only forward-migration coverage — against aTARGET_VERSION = 103migration chain that has no checksums and no downgrade path.I measured before changing anything. Full
cargo test --testsrun on this branch's base (a6ece843), isolated data dir, exit code 0:4da(cli bin)fourda/fourda-enginebinsmigration_testspipeline_integrationsource_resiliencestack_simulationvictauri_dogfoodVICTAURI_E2E=1)Nothing was broken and nothing had to be excluded. They are also hermetic by construction, which I verified rather than assumed:
pipeline_integrationuses an in-memory DB (test_utils::test_db()→:memory:) andmigration_testsusestempfile::tempdir(). Both workflows now run--tests(lib + bins + integration) under the same throwaway-data-dir isolationhermetic.ymlalready used.Two follow-on fixes were required to avoid landing a red gate:
tail -1of thetest result:lines. With--teststhere are 9 test binaries, so it would have read the last binary's total (157) and tripped the 2000 floor on every run. It now sums all binaries.--lib, it fails loudly instead of silently dropping the integration suite again.2. The hermetic fresh-clone canary never ran outside PRs
fresh-cloneneeds the PR-onlychangesjob. GitHub skips any job whoseneedswas skipped unless itsif:contains a status function — andhermetic.yml:94had none. So push-to-main, the nightly cron and manual dispatch all skipped the clone and reported success:The file's own comment at
:56-60claimed these paths "ALWAYS run the full canary". The nightly cron had never built a single fresh clone. Fixed with!cancelled()— notalways(), so a cancelled run doesn't spawn a 45-minute cold build.The identical defect silently disabled
workflow_dispatchfor Frontend, MCP Server and the entire Rust matrix invalidate.yml: thegithub.event_name == 'workflow_dispatch'clause on those three jobs had never once fired, whileValidate Success(if: always()) still went green. Same fix.3. A path-filter hole took the whole fleet down today
This stopped being hypothetical while this PR was being written:
Frontendjob was skipped by the path filter.scripts/check-file-sizes.cjs --ciran only inside the Frontend job — but it scansSCAN_DIRS = ['src', 'src-tauri/src'], i.e. it guards Rust files too.src-tauri/src/analysis_rerank.rspast the 1000-line hard error threshold, with the gate never executing.check-file-sizes.cjsthen exited 1 onmain, and.husky/pre-commit:38-41treats that as blocking — every developer in the fleet was unable to commit.A gate that guards Rust files must not be reachable only through a filter that excludes Rust. This PR adds a
repo-guardsjob with no path filter and noneeds:— it runs on every PR and dispatch, and carriescheck-file-sizes,check-no-window-spawns,check-release-channeland the guard self-tests. They are removed fromFrontend(they were never frontend-specific). Hosted, ~40s, nopnpm installneeded: all three guards use only node builtins.It is also in
validate-success'sneeds, so it actually gates the merge —repo-guardsis now the only leg guaranteed to have run.Additionally, "no filter matched" now means RUN, not PASS.
site/,paddle-webhook/,mcp-memory-server/,editors/vscode/and.husky/matched no filter, whileValidate Successis the only required check and auto-merge is enabled repo-wide — so a Dependabot bump into the payment webhook, or a PR weakening.husky/itself, could merge with nothing run. Added.github/**and.husky/**explicitly, plus anuncoveredfail-safe filter that catches anything unrecognised including directories added in future.4. The guards' own self-tests ran nowhere
pnpm run test:scripts(53 tests across 6 files) executed in no hook and no workflow — nothing verified the guards still detect what they claim.pnpm run validateisn't run by CI either (the Frontend job runs its steps individually), so wiring it intopackage.jsonalone would not have gated it. It is now a step inrepo-guards.Deliberately NOT done
Hermetic Successrequired is the correct end state — currentlyValidate Successis the only required check in the activemain-protectionruleset (verified via the API; classic branch protection is disabled). But with ~30 open PRs and Hermetic historically failing on Audit remediation: remove ~53k lines of dead code, fix legal/tier accuracy, close enforcement gaps #421, flipping it now would block the fleet. Recommended as an explicit follow-up once this lands and Hermetic is observed green on push-to-main for a few days — which, note, is the first time that signal will ever have existed.package.jsonuntouched.test:scriptswas going to be added to thevalidatechain, but a peer worktree agent holds a claim on that file and fix(ci): guard PR title/body — the one leak path no local hook can see #418 also edits that exact line. Wiring it intorepo-guardsachieves the real goal (it now runs in CI) without touching the claimed file.analysis_rerank.rs/check-file-sizes.cjsuntouched — a separate agent owns the immediate unblock. This PR fixes the structural cause only.site/,paddle-webhook/,mcp-memory-server/,editors/vscode/. Theuncoveredfail-safe means they now trigger the generic gate instead of passing silently, but that gate does not build them. Dedicated jobs are the right follow-up and belong in their own PR.main, so until this lands there every PR run pays a cold link for 5 extra binaries. 30 was too tight for that first window, and a timeout on a required gate is a red gate.Conflicts with open PRs
Checked
gh pr diff --name-onlyon every PR touching these files:actions/checkoutSHA pins only — different linestaiki-e/install-actionSHA pin onlyvalidate.yml:236-255; my edits are at 301+ and insidesteps:. I deliberately did not add a matrix key — an integration-floor key would have had to be added to its new legs. Verifiedcargo test --tests --features experimentalcompiles clean, so itstest-floor: 0compile-gate legs are unaffected.validate-success'sneeds:. #418 addspr-metadata, this addsrepo-guards. Resolution is a union of the two lists — whoever merges second takes both. Flagged rather than pre-empted.uncoveredwas also deliberately placed before the main filter step, to stay clear of the end-of-job boundary #418 inserts a job into.Live CI evidence from this PR's own run
The first run of this branch already proves the fix, on both platforms:
The hermetic job log shows all 9 test binaries executing on both legs —
migration_tests12,pipeline_integration13,source_resilience5,stack_simulation124,victauri_dogfood157, plus lib (4,290 windows / 4,284 ubuntu — a 6-test platform delta, far above the 2000 floor) and the 3 bin targets. Zero failures. That is the first time any of those integration tests has run in CI.Rust finished in ~12min against the old 30min cap, so the 45min bump is headroom for the first cold-cache window rather than a response to an observed timeout.
Verification
if:expressions and filter blocks inspected post-rebase.cargo test --testsmeasured green in full before any workflow edit, and re-run green after rebasing onto currentmain(table above) — Audit remediation: remove ~53k lines of dead code, fix legal/tier accuracy, close enforcement gaps #421 removed ~54k lines and fix(pipeline): four silent production failures — dead sources, notification storm, lobotomised reranker, lying metrics #423 changed pipeline code between those two runs.dorny/paths-filternegation +everysemantics confirmed from source at the pinned SHA, not assumed.check-no-window-spawns,check-release-channel,test:scriptsall verified exit 0 locally;check-file-sizescorrectly exits 1 (the live outage above).main; only the two workflow files differ.The
analysis_rerank.rsunblock (#430) has landed, sorepo-guardspasses; this branch is rebased on top of it.