Skip to content

fix: CLI silently did nothing as a bin — a verifier failing OPEN - #6

Merged
dkitchell merged 1 commit into
mainfrom
fix/bin-invocation-fail-open
Aug 20, 2026
Merged

dkitchell merged 1 commit into
mainfrom
fix/bin-invocation-fail-open

Conversation

@dkitchell

Copy link
Copy Markdown
Contributor

fix: CLI silently did nothing when invoked as a bin — a verifier failing OPEN

The entry guard:

if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href)

Through a bin, process.argv[1] is the symlink npm creates
(node_modules/.bin/verify) while import.meta.url is the resolved real path
(node_modules/@certifieddata/verify/dist/cli.js). They never match, so main()
was never called. The process started, printed nothing, and exited 0.

./node_modules/.bin/verify 00000000-0000-0000-0000-000000000000
→ (no output) EXIT=0

node .../dist/cli.js 00000000-0000-0000-0000-000000000000
→ ✗ NOT_FOUND EXIT=3

A nonsense receipt id returned SUCCESS through the documented invocation path.
That is the worst failure mode a verifier can have: not a crash, not a false
negative, but silent assent. Every consumer the exit codes exist for —
verify $ID && deploy, CI gates, the audit pipeline in our own docs — got a
pass for a receipt that was never checked. The previous exit-127 bug was loud
and safe by comparison; this failed open.

It shipped because it is invisible on Windows. npm writes .cmd shims there that
pass the RESOLVED path as argv[1], so the comparison matched. On Linux and macOS
npm writes a true symlink and it never did. Reproduced locally on Windows via a
directory junction: EXIT=0 and zero bytes of output.

Fixed by removing the guard entirely rather than repairing it. A realpathSync()
comparison would also work, but it keeps a conditional on the critical path and
every way that conditional can be wrong fails silently and open. This file is
only ever an entry point — nothing in the repo imports it — so it just runs.

Why no existing test caught it: every other test spawns node dist/cli.js, the
resolved real path, which is the one invocation form no consumer uses.

src/bin-invocation.test.ts adds five tests that go through the bin, and asserts
NON-EMPTY OUTPUT as well as the exit code — the failure was a correct-looking
exit code with nothing behind it, so a code-only assertion would have passed
while the tool did nothing.

The load-bearing test invokes cli.js through a symlinked directory, reproducing
the argv[1]/import.meta.url divergence on every platform. The npm-install tests
alone cannot catch this on Windows, and a guard that only works on some CI legs
is half a guard. Verified both directions: that test FAILS against the old guard
(exit 0, zero bytes) and passes against the fix (exit 3, NOT_FOUND).

Verified through a real install of the packed tarball:

valid -> EXIT=0 ✓ VALID
tampered -> EXIT=1 ✗ INVALID
nonsense -> EXIT=3 ✗ NOT_FOUND

70 tests pass, up from 65.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com

The proof, both directions

The new symlink test was validated against the bug, not just against the fix:

build result
old guard ✖ FAIL — exit 0, zero bytes of output
this fix ✔ PASS — exit 3, ✗ NOT_FOUND

Reproduced on Windows with a directory junction, since npm's .cmd shims hide the bug there — which is exactly why it shipped.

Through a real tarball install

valid    -> EXIT=0  ✓ VALID    receipt 2492a060-8fbc-40ae-beab-7258aefb0608
tampered -> EXIT=1  ✗ INVALID  receipt 2492a060-8fbc-40ae-beab-7258aefb0608
nonsense -> EXIT=3  ✗ NOT_FOUND

Ready for re-test

npx github:certifieddata/verify 2492a060-8fbc-40ae-beab-7258aefb0608 --type receipt
npx github:certifieddata/verify 00000000-0000-0000-0000-000000000000

The second must not exit 0.

🤖 Generated with Claude Code

…ing OPEN

The entry guard:

  if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href)

Through a bin, process.argv[1] is the symlink npm creates
(node_modules/.bin/verify) while import.meta.url is the resolved real path
(node_modules/@certifieddata/verify/dist/cli.js). They never match, so main()
was never called. The process started, printed nothing, and exited 0.

  ./node_modules/.bin/verify 00000000-0000-0000-0000-000000000000
  → (no output)  EXIT=0

  node .../dist/cli.js       00000000-0000-0000-0000-000000000000
  → ✗ NOT_FOUND  EXIT=3

A nonsense receipt id returned SUCCESS through the documented invocation path.
That is the worst failure mode a verifier can have: not a crash, not a false
negative, but silent assent. Every consumer the exit codes exist for —
`verify $ID && deploy`, CI gates, the audit pipeline in our own docs — got a
pass for a receipt that was never checked. The previous exit-127 bug was loud
and safe by comparison; this failed open.

It shipped because it is invisible on Windows. npm writes .cmd shims there that
pass the RESOLVED path as argv[1], so the comparison matched. On Linux and macOS
npm writes a true symlink and it never did. Reproduced locally on Windows via a
directory junction: EXIT=0 and zero bytes of output.

Fixed by removing the guard entirely rather than repairing it. A realpathSync()
comparison would also work, but it keeps a conditional on the critical path and
every way that conditional can be wrong fails silently and open. This file is
only ever an entry point — nothing in the repo imports it — so it just runs.

Why no existing test caught it: every other test spawns `node dist/cli.js`, the
resolved real path, which is the one invocation form no consumer uses.

src/bin-invocation.test.ts adds five tests that go through the bin, and asserts
NON-EMPTY OUTPUT as well as the exit code — the failure was a correct-looking
exit code with nothing behind it, so a code-only assertion would have passed
while the tool did nothing.

The load-bearing test invokes cli.js through a symlinked directory, reproducing
the argv[1]/import.meta.url divergence on every platform. The npm-install tests
alone cannot catch this on Windows, and a guard that only works on some CI legs
is half a guard. Verified both directions: that test FAILS against the old guard
(exit 0, zero bytes) and passes against the fix (exit 3, NOT_FOUND).

Verified through a real install of the packed tarball:

  valid    -> EXIT=0  ✓ VALID
  tampered -> EXIT=1  ✗ INVALID
  nonsense -> EXIT=3  ✗ NOT_FOUND

70 tests pass, up from 65.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dkitchell
dkitchell merged commit 5b86c0c into main Aug 20, 2026
6 checks passed
@dkitchell
dkitchell deleted the fix/bin-invocation-fail-open branch August 20, 2026 23:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant