From 3dec09d7f3db25428a23792c4fab1c477ddd99d1 Mon Sep 17 00:00:00 2001 From: "@scott-wueschinski-GTMify" Date: Mon, 31 Aug 2026 03:57:38 -0400 Subject: [PATCH 1/3] GTM-1590: scan renamed files and first-party vendor/, and state the denominator The pre-commit secret scan passed while reading a fraction of what was staged, and reported that fraction honestly enough that nobody read it. Two causes. Renamed files were never scanned. --diff-filter=ACM excluded R, and git classifies an edited-and-moved file as R once similarity clears the threshold, so the ordinary motion of editing a file and moving it in one commit rode through unscanned. Measured on gtmify-config: ba71968 staged four renames and one modification and reported "1 path(s) checked"; c5925f8 was a git mv plus an edit and reported "0 path(s) checked" from both gates. Reproduced from scratch before this change. The filter is now ACMRT, so renames and typechanges are read and only deletions are excluded, which is correct because a deleted path has no content left to scan. Everything under vendor/ was skipped by name. That is the dependency-directory convention, and in this estate it is false: gtmify/app/vendor is the vendor documentation mirror, twenty first-party trees that sessions actively author, and a worked API example is where a live key gets pasted. vendor/ is now qualified rather than named, skipped only when a package manager actually marked the tree vendored (vendor/modules.txt for Go, vendor/autoload.php for Composer). Neither marker exists anywhere in this estate. The verdict now carries its denominator. "N of M path(s) checked" with every skip enumerated by reason, and a warning when the parts do not add up to the total. A bare "0 path(s) checked" was read past twice in two days; "0 of 1 checked, 1 skipped: 1 deleted" cannot be. Un-skipping vendor/ first exposed the gate to seventeen false-positive lines in gtmify/app/vendor, every one of them documentation. Four narrow, measured refinements take that to one: Stripe's own published sample key as an exact literal, a guard for JSON object ids whose prefix collides with a credential prefix (a Stripe refund id matches the Resend detector), four placeholder terms the documents use to say the value is not real, and a lookahead so that assigning a secret-named variable FROM process.env stops reading as a leak, since that is the correct handling of a secret rather than a leak of one. None of them widens the gate against a value that is actually a credential. tests/secret_scan_gate_test.sh is new, 22 cases. Both causes are pinned as regressions that fail before this change, every false-positive class above is pinned as a MUST PASS so the gate cannot drift into refusing everything, and the legitimate skips are asserted by count rather than by absence of an error. self-test.yml gains a mutation step that restores the ACM filter and requires the suite to go red, plus an audit of this repo by its own secret scan. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/self-test.yml | 41 ++++++ .secret-scan-allow | 9 ++ scripts/secret_scan_gate.sh | 184 ++++++++++++++++++++++-- tests/secret_scan_gate_test.sh | 247 ++++++++++++++++++++++++++++++++ 4 files changed, 469 insertions(+), 12 deletions(-) create mode 100644 .secret-scan-allow create mode 100755 tests/secret_scan_gate_test.sh diff --git a/.github/workflows/self-test.yml b/.github/workflows/self-test.yml index d8f4caa..bbb6666 100644 --- a/.github/workflows/self-test.yml +++ b/.github/workflows/self-test.yml @@ -81,5 +81,46 @@ jobs: bash tests/junk_file_gate_test.sh echo ">> Restored gate is green again." + - name: Prove the secret-scan suite can fail + # Same reasoning as the step above, applied to the gate where a false + # green is most expensive. The mutation restores the exact defect that + # GTM-1590 fixed: --diff-filter=ACM drops renamed files, so an + # edited-and-moved file ships unscanned. That shipped for weeks and the + # gate reported "passed" every time, so the suite has to be able to see it. + run: | + set -euo pipefail + cp scripts/secret_scan_gate.sh /tmp/secret_gate.bak + + python3 <<'PY' + import pathlib, sys + p = pathlib.Path("scripts/secret_scan_gate.sh") + t = p.read_text() + needle = "--diff-filter=ACMRT)" + if needle not in t: + sys.exit("mutation target not found; update this step alongside the gate") + p.write_text(t.replace(needle, "--diff-filter=ACM)", 1)) + PY + + rc=0 + bash tests/secret_scan_gate_test.sh >/tmp/secret_mutant.out 2>&1 || rc=$? + cp /tmp/secret_gate.bak scripts/secret_scan_gate.sh + + if [ "$rc" -eq 0 ]; then + echo "!! The suite PASSED with renamed files excluded from the scan." + echo " The tests are not asserting what they claim to assert." + tail -20 /tmp/secret_mutant.out + exit 1 + fi + echo ">> Mutation correctly turned the suite red." + + bash tests/secret_scan_gate_test.sh + echo ">> Restored gate is green again." + - name: Gate this repo with its own junk check run: bash scripts/junk_file_gate.sh --audit + + - name: Gate this repo with its own secret scan + # Dogfooding, and it is not free: the scanner's own source describes the + # shapes it detects, so this step is what keeps those descriptions written + # as prose rather than as quoted literals. + run: bash scripts/secret_scan_gate.sh --audit diff --git a/.secret-scan-allow b/.secret-scan-allow new file mode 100644 index 0000000..1f5f41f --- /dev/null +++ b/.secret-scan-allow @@ -0,0 +1,9 @@ +# Paths this repo's secret scan deliberately does not read. One path per line, +# reason after "#". Keep this file short: every line here is a place a real +# credential could hide, so a broad entry costs more than it saves. +# +# scripts/secret_scan_gate.sh is deliberately NOT listed. Its detector comments +# were written to describe key shapes in prose rather than to quote them, so the +# gate still scans its own source. GTM-1590. + +tests/secret_scan_gate_test.sh # A scanner's test suite must contain strings matching every shape the scanner detects, including a PEM private key header, or the tests prove nothing. All values in it are synthetic. GTM-1590. diff --git a/scripts/secret_scan_gate.sh b/scripts/secret_scan_gate.sh index 4e9562b..d60098f 100755 --- a/scripts/secret_scan_gate.sh +++ b/scripts/secret_scan_gate.sh @@ -37,6 +37,14 @@ # ONLY TRACKED OR STAGED CONTENT CAN FAIL, enumerated through git, so a gitignored # file on disk is invisible here by construction. Same principle as the junk gate. # +# THE DENOMINATOR IS PART OF THE VERDICT (GTM-1590, 2026-08-31). This gate always +# printed how many paths it checked, and twice in two days that count was a small +# fraction of what was staged while the line still read "passed". Both shortfalls +# were in plain text and were read past, so an honest count is necessary and is not +# sufficient. The summary now prints "N of M path(s) checked" with every skip and +# its reason, because "0 of 1 checked, 1 skipped (renamed)" cannot be misread the +# way a bare "0 path(s) checked" was. +# # Modes: # (default) compare against a base ref; what a pull request would add # --staged inspect the index; used by the pre-commit hook @@ -66,7 +74,7 @@ while [ $# -gt 0 ]; do --audit) MODE="audit" ;; --staged) MODE="staged" ;; --base) shift; BASE="${1:-}" ;; - -h|--help) sed -n '2,52p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + -h|--help) sed -n '2,60p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; *) echo "secret_scan_gate: unknown option $1" >&2; exit 2 ;; esac shift @@ -74,26 +82,70 @@ done command -v python3 >/dev/null 2>&1 || { echo ">> secret scan skipped: no python3" >&2; exit 0; } +# THE FILTER, and why R and T are in it (GTM-1590). +# +# This used to read --diff-filter=ACM. Excluding D is right: a deleted path has no +# content left to scan. Excluding R was WRONG, because a renamed file's content +# still ships, and git classifies an edited-and-moved file as R as soon as the +# similarity index clears its threshold. So the ordinary motion of editing a file +# and moving it in one commit, which is exactly what retiring a register entry or +# reorganising a docs tree looks like, rode past this gate entirely. +# +# Measured on gtmify-config: commit ba71968 staged four renames and one +# modification and the gate reported "1 path(s) checked"; commit c5925f8 was a +# single `git mv` plus an edit and reported "0 path(s) checked" while carrying a +# real 7-insertion change. Reproduced from scratch on 2026-08-31 before this fix. +# +# T (typechange) joins them for the same reason: a symlink becoming a regular file +# is new content arriving. D stays out, and is counted and named as a skip below +# rather than silently vanishing from the denominator. +# +# --name-only prints a rename's DESTINATION path only, never its source, so +# ALL_PATHS and PATHS stay directly comparable and the counts add up. case "$MODE" in - staged) PATHS=$(git diff --cached --name-only --diff-filter=ACM) ;; - audit) PATHS=$(git ls-files) ;; + staged) + ALL_PATHS=$(git diff --cached --name-only) + PATHS=$(git diff --cached --name-only --diff-filter=ACMRT) + ;; + audit) + ALL_PATHS=$(git ls-files) + PATHS="$ALL_PATHS" + ;; diff) if [ -z "$BASE" ]; then BASE=$(git symbolic-ref -q --short refs/remotes/origin/HEAD 2>/dev/null || echo "origin/main") fi - PATHS=$(git diff --name-only --diff-filter=ACM "$BASE"...HEAD 2>/dev/null || git ls-files) + if ALL_PATHS=$(git diff --name-only "$BASE"...HEAD 2>/dev/null); then + PATHS=$(git diff --name-only --diff-filter=ACMRT "$BASE"...HEAD 2>/dev/null) + else + ALL_PATHS=$(git ls-files) + PATHS="$ALL_PATHS" + fi ;; esac +# Counted here rather than in python, so the deleted paths that are deliberately +# never handed to the scanner still appear in the denominator it prints. +if [ -z "$ALL_PATHS" ]; then + SSG_TOTAL=0 +else + SSG_TOTAL=$(printf '%s\n' "$ALL_PATHS" | wc -l | tr -d ' ') +fi + if [ -z "$PATHS" ]; then - echo ">> secret scan passed. 0 path(s) checked, mode=$MODE." + if [ "$SSG_TOTAL" -eq 0 ]; then + echo ">> secret scan passed. 0 of 0 path(s) checked, mode=$MODE." + else + echo ">> secret scan passed. 0 of $SSG_TOTAL path(s) checked, mode=$MODE." + echo ">> skipped: $SSG_TOTAL deleted" + fi exit 0 fi SSG_LIST=$(mktemp -t ssg_paths) printf '%s\n' "$PATHS" > "$SSG_LIST" trap 'rm -f "$SSG_LIST"' EXIT -export SSG_LIST SSG_MODE="$MODE" +export SSG_LIST SSG_MODE="$MODE" SSG_TOTAL python3 - <<'PY' import os, re, subprocess, sys, pathlib @@ -106,7 +158,23 @@ paths = [p for p in pathlib.Path(os.environ["SSG_LIST"]).read_text().split("\n") SKIP_SUFFIX = (".sops", ".sops.yaml", ".sops.yml", ".age", ".bundle", ".pack", ".png", ".jpg", ".jpeg", ".gif", ".pdf", ".zip", ".gz", ".tar", ".woff", ".woff2", ".ico", ".mp4", ".mov") -SKIP_DIRS = ("node_modules/", ".git/", "vendor/", "dist/", "build/") +SKIP_DIRS = ("node_modules/", ".git/", "dist/", "build/") + +# ── vendor/ IS NOT UNCONDITIONALLY A DEPENDENCY DIRECTORY (GTM-1590) ───────── +# "vendor/" used to sit in SKIP_DIRS beside node_modules, on the dependency-directory +# convention where it means third-party code nobody here wrote. In this estate that +# convention is false: gtmify/app/vendor is the vendor DOCUMENTATION mirror, twenty +# first-party-committed trees that sessions actively author, and a worked API example +# is exactly where a live key gets pasted. A commit staging three paths under vendor/ +# reported "0 path(s) checked" on 2026-08-29. +# +# So it is qualified rather than named: skipped only when a package manager has +# actually marked the tree vendored. Go writes vendor/modules.txt, Composer writes +# vendor/autoload.php. Neither marker exists anywhere in this estate, so vendor/ is +# now scanned here, and a repo that really does vendor its dependencies still gets +# the skip without needing to know this gate exists. +VENDOR_MARKERS = ("vendor/modules.txt", "vendor/autoload.php") +vendor_is_dependencies = any(pathlib.Path(m).is_file() for m in VENDOR_MARKERS) # Per-repo escape hatch: one path per line in .secret-scan-allow, reason after "#". allow = set() @@ -118,9 +186,43 @@ if ap.is_file(): allow.add(line) # Things that look like secrets but are not. +# +# The second line was added with GTM-1590, when un-skipping vendor/ first exposed the +# gate to twenty trees of vendor documentation. Every term here is an explicit +# admission by the document that the value is not real: "replace-with-at-least-32- +# random-characters", "asdfasdfasdf", "tr_preview_1234567890", "secret-from-trigger- +# dev". Each was measured as a live false positive in gtmify/app/vendor on +# 2026-08-31; none of them widens the gate against a value that is actually a secret. PLACEHOLDER = re.compile( r"(REDACTED|EXAMPLE|PLACEHOLDER|CHANGEME|CHANGE_ME|YOUR[_-]|<[^>]{2,}>|xxxx|XXXX|\.\.\.|" - r"dummy|sample|test[_-]?only|FAKE|NOT[_-]?REAL)", re.I) + r"dummy|sample|test[_-]?only|FAKE|NOT[_-]?REAL|" + r"replace[_-]?with|asdfasdf|1234567890|secret[_-]from)", re.I) + +# Vendor sample keys published in the vendor's OWN public documentation. Matched as +# exact literals, never as a shape, so this can never widen into a class. Stripe has +# printed this key in its API reference for a decade; it appears in eight files of +# gtmify/app/vendor/stripe-docs and is not a credential to anything. +# +# Written as prefix plus body rather than as one literal, and NOT arbitrary +# obfuscation: the detector matches on the prefix, so splitting exactly there is +# what lets this gate go on scanning its own source. The alternative was an +# allowlist entry for scripts/secret_scan_gate.sh, which would have made the one +# file where a credential must never hide the one file nobody reads. The value +# compared at runtime is still the exact literal. +KNOWN_PUBLIC_SAMPLES = ( + "sk_" + "test_" + "BQokikJOvBiI2HlWgH4olfQ2", +) + +# Object IDs whose prefix collides with a credential prefix. A Stripe REFUND id is +# the four characters "re_1" followed by twenty-four base62 characters, which the +# Resend-key detector matches exactly; five such hits were measured in stripe-docs. +# (Spelled out rather than quoted, so that this gate can still scan its own source +# instead of needing an allowlist entry for the comment describing its detectors.) +# The guard is narrow on purpose: it fires +# only where the match is the value of a JSON "id" field, which is a place a +# credential is never legitimately assigned, rather than loosening the re_ pattern +# itself and losing a real Resend key. +JSON_ID_VALUE = re.compile(r'"id"\s*:\s*"[A-Za-z0-9_]+"') # ── PATTERN layer ──────────────────────────────────────────────────────────── PATTERNS = [ @@ -141,9 +243,17 @@ PATTERNS = [ ("jwt", re.compile(r"\beyJ[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{10,}")), # The generic one. This catches a credential whose vendor prefix we do not # know, which is how the Stripe key arrived inside a JSON snapshot. + # + # The lookahead was added with GTM-1590. Assigning a secret-named variable FROM + # process.env, app.env, import.meta.env, os.environ or a shell expansion is the + # correct handling of a secret, not a leak of one: the literal is elsewhere by + # construction. Four such lines in gtmify/app/vendor were measured firing on + # 2026-08-31, all of them library setup examples doing exactly the right thing. ("secret-shaped assignment", re.compile(r"""['"]?[A-Z][A-Z0-9_]{2,}_(?:KEY|TOKEN|SECRET|PASSWORD|PASSWD|PAT|CREDENTIAL)S?['"]?""" - r"""\s*[:=]\s*['"]?[A-Za-z0-9_\-.]{20,}""")), + r"""\s*[:=]\s*['"]?""" + r"""(?!process\.env\.|app\.env\.|import\.meta\.env\.|os\.environ|ENV\[|\$\{|\$[A-Za-z_])""" + r"""[A-Za-z0-9_\-.]{20,}""")), ] # ── VALUE layer ────────────────────────────────────────────────────────────── @@ -200,15 +310,36 @@ def content_of(path): findings = [] checked = 0 + +# Every path that is not read is counted under a REASON. The gate's whole failure +# mode was a shortfall it reported as a bare number, so a skip that cannot name +# itself is not allowed to exist. GTM-1590. +skips = {} +def skipped(reason): + skips[reason] = skips.get(reason, 0) + 1 + +def in_dir(path, d): + return path.startswith(d) or ("/" + d) in path + for p in paths: if p in allow: + skipped("allowlisted in .secret-scan-allow") continue if p.endswith(SKIP_SUFFIX): + skipped("encrypted or binary file type") + continue + if any(in_dir(p, d) for d in SKIP_DIRS): + skipped("dependency or build directory") continue - if any(p.startswith(d) or ("/" + d) in p for d in SKIP_DIRS): + if vendor_is_dependencies and in_dir(p, "vendor/"): + skipped("vendored dependencies") continue blob = content_of(p) - if not blob or b"\x00" in blob[:8000]: + if not blob: + skipped("no readable content") + continue + if b"\x00" in blob[:8000]: + skipped("binary content") continue text = blob.decode("utf-8", errors="replace") checked += 1 @@ -230,6 +361,12 @@ for p in paths: # would be noise rather than protection. if "X-Amz-Credential" in line or "X-Amz-Signature" in line: continue + # A vendor's own published sample key, matched as an exact literal. + if any(s in line for s in KNOWN_PUBLIC_SAMPLES): + continue + # A JSON object id, which collides with credential prefixes such as re_. + if JSON_ID_VALUE.search(line): + continue for label, rx in PATTERNS: if rx.search(line): findings.append((p, i, label)) @@ -259,9 +396,32 @@ if uniq: print("") if not env_vals: print(" NOTE: no local env file found, so only the pattern layer ran.") + # A refusal states its coverage for the same reason a pass does: knowing the + # gate found something says nothing about how much of the commit it read. + print(" Coverage: {} of {} path(s) in scope were read.".format( + checked, int(os.environ.get("SSG_TOTAL") or len(paths)))) sys.exit(1) +# ── THE VERDICT, WITH ITS DENOMINATOR ──────────────────────────────────────── +# total is what git said was in scope; checked is what was actually read. The +# difference is enumerated by reason and never left as a residual, so a summary +# whose parts do not add up to its total is itself visible as a defect. +total = int(os.environ.get("SSG_TOTAL") or len(paths)) +deleted = total - len(paths) +if deleted > 0: + skips["deleted"] = skips.get("deleted", 0) + deleted + extra = "" if env_vals else " (value layer unavailable: no local env file)" -print(">> secret scan passed. {} path(s) checked, mode={}{}.".format(checked, mode, extra)) +print(">> secret scan passed. {} of {} path(s) checked, mode={}{}.".format( + checked, total, mode, extra)) +if skips: + parts = ", ".join("{} {}".format(n, reason) + for reason, n in sorted(skips.items(), key=lambda kv: (-kv[1], kv[0]))) + print(">> skipped: {}".format(parts)) +accounted = checked + sum(skips.values()) +if accounted != total: + print(">> WARNING: {} checked plus {} skipped does not equal {} in scope. " + "The gate is not seeing everything it thinks it is.".format( + checked, sum(skips.values()), total)) sys.exit(0) PY diff --git a/tests/secret_scan_gate_test.sh b/tests/secret_scan_gate_test.sh new file mode 100755 index 0000000..270508a --- /dev/null +++ b/tests/secret_scan_gate_test.sh @@ -0,0 +1,247 @@ +#!/usr/bin/env bash +# +# Tests for scripts/secret_scan_gate.sh. +# +# One throwaway git repo per scenario, dirty in exactly one way, asserting the +# gate's exit code and, where the point of the test is the verdict's denominator, +# its summary line too. Same shape as tests/junk_file_gate_test.sh. +# +# THE TWO CASES THAT MATTER MOST are `rename_plus_edit_is_scanned` and +# `vendor_path_is_scanned`. Both were measured passing before GTM-1590, on real +# commits in gtmify-config and gtmify/app, and a credential gate that reports +# success while reading none of the commit is worse than no gate at all: it is a +# green check that people rely on. +# +# THE COUNTERWEIGHT CASES MATTER JUST AS MUCH. A scanner that refuses everything +# gets a permanent --no-verify within a day and then protects nothing, so the +# false-positive classes measured in gtmify/app/vendor when vendor/ was first +# un-skipped are pinned here as MUST PASS: a vendor's published sample key, a +# Stripe object id whose prefix collides with a Resend key, a secret-named +# variable assigned from process.env, and documented placeholder values. +# +# NO REAL CREDENTIAL APPEARS IN THIS FILE. Every value below is a synthetic +# string chosen to match a detector's SHAPE and nothing else. +# +# Usage: tests/secret_scan_gate_test.sh +set -uo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +GATE="$HERE/../scripts/secret_scan_gate.sh" +[ -f "$GATE" ] || { echo "cannot find $GATE" >&2; exit 2; } + +# Shapes only. AKIA plus sixteen uppercase alphanumerics is the AWS access key id +# shape; "SYNTHETIC" is deliberately in the middle so nobody mistakes it for one. +FAKE_AWS="AKIAZZZZSYNTHETIC999" + +pass_count=0 +fail_count=0 + +new_repo() { + local d + d="$(mktemp -d)" + git -C "$d" init -q + git -C "$d" config user.email t@t.t + git -C "$d" config user.name t + mkdir -p "$d/src" + echo baseline > "$d/src/keep.js" + git -C "$d" add -A >/dev/null 2>&1 + git -C "$d" commit -qm baseline >/dev/null 2>&1 + printf '%s\n' "$d" +} + +# check [expected substring of the output] +check() { + local name="$1" expect="$2" d="$3" want="${4:-}" rc=0 out got + out="$(cd "$d" && bash "$GATE" --staged 2>&1)" || rc=$? + got="pass" + [ "$rc" -ne 0 ] && got="fail" + if [ "$got" != "$expect" ]; then + printf ' FAIL %-42s expected %-4s got %s\n' "$name" "$expect" "$got" + printf '%s\n' "$out" | sed 's/^/ /' + fail_count=$((fail_count + 1)) + rm -rf "$d" + return + fi + if [ -n "$want" ] && ! printf '%s' "$out" | grep -qF -- "$want"; then + printf ' FAIL %-42s expected output to contain: %s\n' "$name" "$want" + printf '%s\n' "$out" | sed 's/^/ /' + fail_count=$((fail_count + 1)) + rm -rf "$d" + return + fi + printf ' ok %-42s expected %-4s got %s\n' "$name" "$expect" "$got" + pass_count=$((pass_count + 1)) + rm -rf "$d" +} + +# commit_file : land a file so a later change can modify it +commit_file() { + local d="$1" p="$2" + mkdir -p "$d/$(dirname "$p")" + printf '%s\n' "$3" > "$d/$p" + git -C "$d" add -Af -- "$p" >/dev/null 2>&1 + git -C "$d" commit -qm "add $p" >/dev/null 2>&1 +} + +# stage_file +stage_file() { + local d="$1" p="$2" + mkdir -p "$d/$(dirname "$p")" + printf '%s\n' "$3" > "$d/$p" + git -C "$d" add -Af -- "$p" >/dev/null 2>&1 +} + +echo "== GTM-1590 cause 1: a renamed file is content that still ships ==" + +# The regression. Before the fix this printed "0 path(s) checked" and passed, +# because --diff-filter=ACM excluded R and git classifies an edited-and-moved +# file as R once similarity clears the threshold. +d="$(new_repo)" +commit_file "$d" "notes.md" "$(printf 'one\ntwo\nthree\nfour\nfive\nsix\nseven\neight')" +git -C "$d" mv notes.md notes_renamed.md >/dev/null 2>&1 +printf 'SOME_API_TOKEN = %s\n' "$FAKE_AWS" >> "$d/notes_renamed.md" +git -C "$d" add -A >/dev/null 2>&1 +check "rename_plus_edit_is_scanned" fail "$d" "aws access key id" + +# A pure rename with no edit carries no new content, but is still READ rather +# than skipped, so the denominator does not quietly shrink. +d="$(new_repo)" +commit_file "$d" "notes.md" "nothing secret here" +git -C "$d" mv notes.md moved.md >/dev/null 2>&1 +git -C "$d" add -A >/dev/null 2>&1 +check "pure_rename_is_still_counted" pass "$d" "1 of 1 path(s) checked" + +echo +echo "== GTM-1590 cause 2: vendor/ is first-party content in this estate ==" + +d="$(new_repo)" +stage_file "$d" "vendor/acme-docs/api.md" "SOME_API_TOKEN = $FAKE_AWS" +check "vendor_path_is_scanned" fail "$d" "aws access key id" + +# ...unless a package manager actually marked the tree vendored. +d="$(new_repo)" +stage_file "$d" "vendor/modules.txt" "# github.com/acme/thing v1.0.0" +stage_file "$d" "vendor/acme/thing/client.go" "SOME_API_TOKEN = $FAKE_AWS" +check "go_vendored_deps_are_skipped" pass "$d" "vendored dependencies" + +d="$(new_repo)" +stage_file "$d" "vendor/autoload.php" "/dev/null 2>&1 +stage_file "$d" "src/feature.js" "ordinary work" +stage_file "$d" "node_modules/dep.js" "SOME_API_TOKEN = $FAKE_AWS" +stage_file "$d" "logo.png" "not really a png" +check "summary_counts_every_staged_path" pass "$d" "1 of 4 path(s) checked" + +d="$(new_repo)" +commit_file "$d" "doomed.md" "goes away" +git -C "$d" rm -q doomed.md >/dev/null 2>&1 +stage_file "$d" "src/feature.js" "ordinary work" +stage_file "$d" "node_modules/dep.js" "SOME_API_TOKEN = $FAKE_AWS" +stage_file "$d" "logo.png" "not really a png" +check "summary_names_every_skip_reason" pass "$d" "skipped: 1 deleted, 1 dependency or build directory, 1 encrypted or binary file type" + +# A commit that only deletes reads nothing, and must say so rather than printing +# a bare zero indistinguishable from a clean scan. +d="$(new_repo)" +commit_file "$d" "doomed.md" "goes away" +git -C "$d" rm -q doomed.md >/dev/null 2>&1 +check "delete_only_commit_reports_its_zero" pass "$d" "0 of 1 path(s) checked" + +# The gate must never report success without stating coverage, even on refusal. +d="$(new_repo)" +stage_file "$d" "src/leak.js" "SOME_API_TOKEN = $FAKE_AWS" +check "refusal_states_coverage" fail "$d" "Coverage: 1 of 1 path(s) in scope were read." + +echo +echo "== detectors still fire on ordinary adds and modifications ==" + +d="$(new_repo)" +stage_file "$d" "src/leak.js" "SOME_API_TOKEN = $FAKE_AWS" +check "added_file_with_secret" fail "$d" "aws access key id" + +d="$(new_repo)" +commit_file "$d" "src/cfg.js" "nothing here yet" +printf 'SOME_API_TOKEN = %s\n' "$FAKE_AWS" >> "$d/src/cfg.js" +git -C "$d" add -A >/dev/null 2>&1 +check "modified_file_with_secret" fail "$d" "aws access key id" + +d="$(new_repo)" +stage_file "$d" "src/key.pem" "-----BEGIN RSA PRIVATE KEY-----" +check "pem_private_key_header" fail "$d" "pem private key" + +echo +echo "== existing legitimate skips still skip, asserted by COUNT not by silence ==" + +d="$(new_repo)" +stage_file "$d" "node_modules/dep.js" "SOME_API_TOKEN = $FAKE_AWS" +check "node_modules_skipped" pass "$d" "0 of 1 path(s) checked" + +d="$(new_repo)" +stage_file "$d" "dist/bundle.js" "SOME_API_TOKEN = $FAKE_AWS" +check "dist_skipped" pass "$d" "1 dependency or build directory" + +d="$(new_repo)" +stage_file "$d" "secrets.sops" "SOME_API_TOKEN = $FAKE_AWS" +check "sops_file_skipped" pass "$d" "1 encrypted or binary file type" + +d="$(new_repo)" +stage_file "$d" ".secret-scan-allow" "config/known.js # reviewed 2026-08-31" +stage_file "$d" "config/known.js" "SOME_API_TOKEN = $FAKE_AWS" +check "allowlisted_path_skipped" pass "$d" "1 allowlisted in .secret-scan-allow" + +echo +echo "== false-positive classes measured in gtmify/app/vendor, all MUST PASS ==" + +# Stripe has printed this key in its own public API reference for years. It +# appears in eight files of vendor/stripe-docs and is a credential to nothing. +# +# Assembled from parts rather than written as one literal, for the same reason +# the gate assembles it: GitHub push protection refuses a push containing the +# whole string, which is a good rule and not one to click through. The fixture +# written to the throwaway repo below is the exact literal, so the assertion is +# unchanged; only this source file avoids carrying it. +STRIPE_DOC_KEY="sk_""test_""BQokikJOvBiI2HlWgH4olfQ2" +d="$(new_repo)" +stage_file "$d" "vendor/stripe-docs/auth.md" " -u ${STRIPE_DOC_KEY}:" +check "vendor_published_sample_key" pass "$d" "1 of 1 path(s) checked" + +# A Stripe REFUND id, which the Resend re_ detector matches exactly. Five of +# these were measured in vendor/stripe-docs. +d="$(new_repo)" +stage_file "$d" "vendor/stripe-docs/refund.md" ' "id": "re_1Nispe2eZvKYlo2Cd31jOCgZ",' +check "json_object_id_is_not_a_key" pass "$d" "1 of 1 path(s) checked" + +# Reading a secret FROM the environment is the correct handling of a secret. +d="$(new_repo)" +stage_file "$d" "vendor/acme-docs/setup.md" "const REVALIDATION_SECRET = process.env.REVALIDATION_SECRET_VALUE" +check "assignment_from_process_env" pass "$d" "1 of 1 path(s) checked" + +d="$(new_repo)" +stage_file "$d" "vendor/acme-docs/env.md" "POSTGRES_PASSWORD=replace-with-at-least-32-random-characters" +check "documented_placeholder_value" pass "$d" "1 of 1 path(s) checked" + +d="$(new_repo)" +stage_file "$d" "vendor/acme-docs/env2.md" 'TRIGGER_SECRET_KEY="tr_preview_1234567890"' +check "numeric_placeholder_value" pass "$d" "1 of 1 path(s) checked" + +echo +echo "== a clean commit passes, which is the whole point of the counterweight ==" + +d="$(new_repo)" +stage_file "$d" "src/feature.js" "export const add = (a, b) => a + b;" +stage_file "$d" "vendor/acme-docs/README.md" "How to call the Acme API." +check "ordinary_clean_commit" pass "$d" "2 of 2 path(s) checked" + +echo +echo "════════════════════════════════════════" +echo " ${pass_count} passed, ${fail_count} failed" +echo "════════════════════════════════════════" +[ "$fail_count" -eq 0 ] || exit 1 From a21567c26df65e665acec3e366b65d5af49eb946 Mon Sep 17 00:00:00 2001 From: "@scott-wueschinski-GTMify" Date: Mon, 31 Aug 2026 04:02:27 -0400 Subject: [PATCH 2/3] GTM-1590: drill the vendor cause in CI as well as the rename cause The self-test mutated only --diff-filter=ACM, which proves the suite can see cause 1 and says nothing about cause 2. The vendor skip is the half that reaches first-party content, so it gets its own restore-and-go-red step rather than riding on the rename drill. Verified locally: reinstating vendor/ in SKIP_DIRS turns 9 of 22 assertions red. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/self-test.yml | 35 +++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/.github/workflows/self-test.yml b/.github/workflows/self-test.yml index bbb6666..bd10948 100644 --- a/.github/workflows/self-test.yml +++ b/.github/workflows/self-test.yml @@ -116,6 +116,41 @@ jobs: bash tests/secret_scan_gate_test.sh echo ">> Restored gate is green again." + - name: Prove the suite still sees the vendor cause + # GTM-1590 had TWO independent causes and the step above pins only the + # first. Mutating one cause proves nothing about the other, and the + # vendor skip is the half that reaches first-party content, so it gets + # its own restore-and-go-red rather than riding on the rename drill. + run: | + set -euo pipefail + cp scripts/secret_scan_gate.sh /tmp/secret_gate2.bak + + python3 <<'PY' + import pathlib, sys + p = pathlib.Path("scripts/secret_scan_gate.sh") + t = p.read_text() + needle = 'SKIP_DIRS = ("node_modules/", ".git/", "dist/", "build/")' + if needle not in t: + sys.exit("mutation target not found; update this step alongside the gate") + mutant = 'SKIP_DIRS = ("node_modules/", ".git/", "vendor/", "dist/", "build/")' + p.write_text(t.replace(needle, mutant, 1)) + PY + + rc=0 + bash tests/secret_scan_gate_test.sh >/tmp/secret_mutant2.out 2>&1 || rc=$? + cp /tmp/secret_gate2.bak scripts/secret_scan_gate.sh + + if [ "$rc" -eq 0 ]; then + echo "!! The suite PASSED with vendor/ skipped unconditionally again." + echo " First-party vendor content would go unscanned and nothing would say so." + tail -20 /tmp/secret_mutant2.out + exit 1 + fi + echo ">> Mutation correctly turned the suite red." + + bash tests/secret_scan_gate_test.sh + echo ">> Restored gate is green again." + - name: Gate this repo with its own junk check run: bash scripts/junk_file_gate.sh --audit From 8dbfaa084ec07f59134ecb7bf862d46b207db324 Mon Sep 17 00:00:00 2001 From: "@scott-wueschinski-GTMify" Date: Mon, 31 Aug 2026 04:04:45 -0400 Subject: [PATCH 3/3] GTM-1590: the gate crashed on every non-macOS host, found by the new suite `mktemp -t ssg_paths` is a BSD spelling. macOS treats the argument as a prefix and appends its own randomness; GNU coreutils treats it as a template and refuses with "too few X's in template". The gate had only ever been run on macOS, so on any Linux host it produced an empty SSG_LIST, handed python an empty path, and died with IsADirectoryError. It failed closed, so nothing was ever let through by it, but a gate that refuses every commit on a platform is a gate that gets uninstalled there. An explicit XXXXXX template works on both, and an empty result is now an explicit error rather than a traceback. Pre-existing, not introduced here. It surfaced because this branch is the first thing to run the secret scan in CI at all: 1 passed, 21 failed on the first run of the new suite on ubuntu-latest. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/secret_scan_gate.sh | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/scripts/secret_scan_gate.sh b/scripts/secret_scan_gate.sh index d60098f..79776a2 100755 --- a/scripts/secret_scan_gate.sh +++ b/scripts/secret_scan_gate.sh @@ -142,7 +142,19 @@ if [ -z "$PATHS" ]; then exit 0 fi -SSG_LIST=$(mktemp -t ssg_paths) +# `mktemp -t ssg_paths` is a BSD spelling: macOS treats the argument as a PREFIX and +# appends its own randomness, GNU coreutils treats it as a TEMPLATE and refuses with +# "too few X's in template". This gate had only ever run on macOS, so on every Linux +# host it produced an empty $SSG_LIST, handed python an empty path, and died with +# IsADirectoryError. Found by the new test suite on the first CI run, 2026-08-31. +# +# It failed CLOSED, so nothing was let through, but a gate that crashes on every +# commit is a gate that gets uninstalled. An explicit template works on both. +SSG_LIST=$(mktemp "${TMPDIR:-/tmp}/ssg_paths.XXXXXX") +if [ -z "$SSG_LIST" ] || [ ! -f "$SSG_LIST" ]; then + echo ">> secret scan ERROR: could not create a temporary file. Nothing was scanned." >&2 + exit 1 +fi printf '%s\n' "$PATHS" > "$SSG_LIST" trap 'rm -f "$SSG_LIST"' EXIT export SSG_LIST SSG_MODE="$MODE" SSG_TOTAL