Skip to content

Replace vacuous Windows shell tests and make the anti-drift guard structural - #432

Merged
coneilen merged 1 commit into
mainfrom
w415
Sep 23, 2026
Merged

coneilen merged 1 commit into
mainfrom
w415

Conversation

@coneilen

@coneilen coneilen commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Three tests were executing in CI while being structurally incapable of failing, and the guard meant to prevent exactly that could not see the regression it was built to catch. These were surfaced during the parity fleet and deferred as "flag, don't fix" while the relevant PRs were in flight; those have now merged, so this closes them out.

1. UpdateOfferDialog.zig asserted nothing

The single test read expectEqual(Action.later, .later) -- an enum compared to itself, invoking zero production code, under a name claiming to cover "install unavailable". The real contracts (Install presented but disabled, command routing) lived inside windowProc/createButton, unreachable from a unit test.

Extracted buttons and actionForCommand, and made windowProc consume them rather than open-coding the same decisions. The tests assert the same declarations the product reads, so they cannot drift. Incidentally removes the magic coupling if (id == later_id) 110 else 140.

2. QuickChats.zig was dead code with an unfailable test

Zero importers repo-wide. Availability had one variant and Controller returned it unconditionally, so "quick chat operations are available through the daemon controller" could not fail without a compile error. Meanwhile protocolGapMessage emitted "unavailable" strings that Controller could never produce.

Its one genuine assertion covered Wire.commandName, so that moved into Wire.zig -- the file owning the function -- and the dead scaffold was deleted.

3. The anti-drift guard could not detect the regression it exists to prevent

The guard (issue #424) compared test-bearing sources against a hand-maintained list, not against the zig test invocations that list claimed to describe. Deleting an invocation while leaving its name in the list silently stopped running those tests and still passed the guard.

Verified: removing the Navigation.zig invocation, old guard reported PASS. The wired set is now derived from the script's own invocations, and derivation yielding zero results is a hard failure rather than a vacuous pass.

Evidence

All under pinned Zig 0.15.2 (local 0.16.0 is actively misleading here).

RED: vacuous assertions could not fail -> 6 mutations of UpdateOfferDialog (Install silently enabled, Install claims real install, routing collapsed to later, unknown command resolves, dismiss implies install, duplicated command id) all passed against the old test and are all caught by the new ones
GREEN: UpdateOfferDialog 4/4, Wire 61/61, App 233/233 -> full consumer compiles and passes after the refactor
REGRESSION: removing the Navigation.zig invocation passed the old guard -> now fails; typo'd invocation, dropping a file from a multi-file invocation, and broken derivation are all caught too

Mutating Wire.commandName is caught by the relocated assertions, confirming the moved coverage is real rather than relocated text.

Scope

Wired suite count goes 40 -> 39 by deleting one dead file; no file that was executed before stops being executed. (Note: an earlier count of "39 wired suites" circulating from the #426 work was an undercount -- it used a [A-Za-z]+ file-name pattern that silently skipped Win32.zig. The guard now prints its own derived count, so this number is observed rather than asserted: CI reports PASS (39 source files executed).) Untouched: Sidebar.zig, GraphCanvas.zig, Accessibility.zig, AccessibilityProvider.cpp, uia-live-gate.ps1.

Wire.commandName has no product callers at all -- its only consumers were these tests. Left in place deliberately rather than cascading into an unrelated deletion.

…uctural

Three tests were executing in CI while being structurally incapable of
failing, and the guard meant to prevent exactly that could not see the
regression it was built to catch.

UpdateOfferDialog.zig asserted `expectEqual(Action.later, .later)` --
an enum compared to itself, invoking no production code at all, under a
name claiming to cover "install unavailable". The real contracts lived
inside windowProc and createButton, unreachable from a unit test. Extract
them into `buttons` and `actionForCommand`, have windowProc consume those
instead of open-coding the same decisions, and assert the actual
behaviour: Install is presented disabled and maps to install_unavailable,
the two honourable actions stay enabled, every command id routes to its
declared action, unknown commands return null, ids are distinct, and
dismissal defers rather than implying an install. Because windowProc now
reads the same declarations the tests assert on, the two cannot drift.

QuickChats.zig had zero importers anywhere in the repo. Its Availability
enum had a single variant and Controller returned it unconditionally, so
"quick chat operations are available through the daemon controller" could
not fail without a compile error, while protocolGapMessage emitted
"unavailable" strings the Controller could never produce. Its one genuine
assertion covered Wire.commandName, so move that into Wire.zig -- the file
that owns commandName -- and delete the dead scaffold.

The anti-drift guard (issue #424) compared test-bearing sources against a
hand-maintained list rather than against the zig test invocations it
claimed to describe. Deleting an invocation while leaving its name in the
list silently stopped running those tests and still passed: verified by
removing the Navigation.zig invocation, which the old guard reported as
PASS. Derive the wired set from the script's own invocations so the guard
observes reality instead of a description of it, and fail loudly if
derivation yields nothing rather than vacuously passing.

Evidence (pinned Zig 0.15.2): UpdateOfferDialog 4/4, Wire 61/61, App
233/233. Six mutations of the new dialog assertions all caught; mutating
Wire.commandName caught by the relocated assertions; removing an
invocation, typo'ing one, dropping a file from a multi-file invocation,
and breaking derivation are all now caught, where the first previously
passed.

Signed-off-by: Colin Neilens <coneilen@microsoft.com>

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@coneilen
coneilen merged commit 5dc7b21 into main Sep 23, 2026
11 checks passed
@coneilen
coneilen deleted the w415 branch September 23, 2026 09:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant