From c46eca11bad04c9b2545a1440a7ec3c75f2a59a8 Mon Sep 17 00:00:00 2001 From: Paolo Mottadelli Date: Fri, 25 Sep 2026 21:40:10 +0200 Subject: [PATCH 1/5] =?UTF-8?q?feat(stardust):=200.26.0=20=E2=80=94=20publ?= =?UTF-8?q?ished-origin=20gate=20hardened=20with=20element-level=20criteri?= =?UTF-8?q?a=20(#125)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A 96-page rollout passed /offers/offers at 6.7 % pixel while all 284 coupon cards were clipped and their "View details" links pushed under the Clip button: pixel PASS meant "right shapes", not "every element present and legible". New replica/scripts/gate-all.mjs runs every deployed page through stitched captures + pixel-compare AND two symmetric DOM probes, verdict = pixel ≤ 10 % AND |Δh| ≤ 5 % AND clipped = 0 AND MISSING+HIDDEN links/headings = 0 (+ required repeated units within 4 px); the pixel-only verdict is kept beside it as calibration data. New diff probes: clip-probe (D1), content-presence (D2, content-diff --published hands over), unit-geometry + measure-live (D3); pixel-compare --text-boxes / --pad auxiliary (D4); sbs-crop folds the crop helpers. Upstreamed walgreens feedback A1 (window-free real Chrome by default), A2 (Akamai 400 as challenge), A3 (devDependencies, never --no-save), B1, B7, and the DA 200-image cap via the admin x-error header. Docs: gate doc § all-pages gate, replica Phase 5, deploy Step 10, rollout Phase E, diff SKILL, scripts index. Tests for the four instruments. Calibration on the walgreens roster: 66 pixel-only PASS → 28 full PASS. Co-Authored-By: Claude Fable 5.1 --- plugins/stardust/.claude-plugin/plugin.json | 2 +- plugins/stardust/CHANGELOG.md | 91 ++++ plugins/stardust/skills/deploy/SKILL.md | 2 + .../skills/deploy/da-deploy-protocol.md | 5 + .../skills/deploy/scripts/deploy-batch.mjs | 7 +- plugins/stardust/skills/diff/SKILL.md | 47 ++- .../skills/diff/scripts/clip-probe.mjs | 390 +++++++++++++++++ .../skills/diff/scripts/content-diff.mjs | 11 + .../skills/diff/scripts/content-presence.mjs | 396 ++++++++++++++++++ .../skills/diff/scripts/live-session.mjs | 45 +- .../skills/diff/scripts/measure-live.mjs | 234 +++++++++++ .../diff/scripts/test/clip-probe.test.mjs | 113 +++++ .../scripts/test/content-presence.test.mjs | 97 +++++ .../diff/scripts/test/unit-geometry.test.mjs | 69 +++ .../skills/diff/scripts/unit-geometry.mjs | 265 ++++++++++++ plugins/stardust/skills/extract/SKILL.md | 6 +- plugins/stardust/skills/replica/SKILL.md | 38 +- .../replica/reference/source-fidelity-gate.md | 70 ++++ .../skills/replica/scripts/gate-all.mjs | 321 ++++++++++++++ .../skills/replica/scripts/pixel-compare.mjs | 80 +++- .../skills/replica/scripts/sbs-crop.mjs | 48 +++ .../replica/scripts/test/gate-all.test.mjs | 101 +++++ plugins/stardust/skills/rollout/SKILL.md | 10 + .../stardust/reference/scripts-index.md | 12 +- 24 files changed, 2432 insertions(+), 28 deletions(-) create mode 100644 plugins/stardust/skills/diff/scripts/clip-probe.mjs create mode 100644 plugins/stardust/skills/diff/scripts/content-presence.mjs create mode 100644 plugins/stardust/skills/diff/scripts/measure-live.mjs create mode 100644 plugins/stardust/skills/diff/scripts/test/clip-probe.test.mjs create mode 100644 plugins/stardust/skills/diff/scripts/test/content-presence.test.mjs create mode 100644 plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs create mode 100644 plugins/stardust/skills/diff/scripts/unit-geometry.mjs create mode 100644 plugins/stardust/skills/replica/scripts/gate-all.mjs create mode 100644 plugins/stardust/skills/replica/scripts/sbs-crop.mjs create mode 100644 plugins/stardust/skills/replica/scripts/test/gate-all.test.mjs diff --git a/plugins/stardust/.claude-plugin/plugin.json b/plugins/stardust/.claude-plugin/plugin.json index 7d56fd920..234d702fc 100644 --- a/plugins/stardust/.claude-plugin/plugin.json +++ b/plugins/stardust/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "stardust", "description": "Redesign an existing website to make it better. Higher-level guided flow on top of impeccable.", - "version": "0.25.2", + "version": "0.26.0", "author": { "name": "Adobe" }, diff --git a/plugins/stardust/CHANGELOG.md b/plugins/stardust/CHANGELOG.md index 1399087ea..72512be5d 100644 --- a/plugins/stardust/CHANGELOG.md +++ b/plugins/stardust/CHANGELOG.md @@ -4,6 +4,97 @@ This file starts at 0.14.0. Prior versions (0.3.0 – 0.13.1) are documented in git history only (plus the branch-scoped notes in `CHANGELOG-redesign-adobecom.md` and `CHANGELOG-delivery-media-fidelity.md`). +## 0.26.0 — published-origin gate hardened: element-level criteria beside the pixel number (#125) + +A 96-page rollout gated every deployed page on stitched 1440 captures and pixel-compare; the page +`/offers/offers` PASSED at 6.73 % / Δh 69 px while all 284 of its coupon cards were visibly broken — +a fixed 238 px card with `overflow: hidden`, the body ~30 px too low, the description clipped mid-glyph +and the "View details" link pushed under the Clip button on every card, the toolbar reading +"Expiration Date" where the origin reads "Recommended". The links WERE in the served DOM; pure CSS +geometry, which pixelmatch underweights (small text inside otherwise matching shapes). "PASS" meant +"right shapes at the right places", not "every element present and legible". Element presence and +legibility are checkable — so the gate checks them. Cross-references #115 (chrome crop gate), #124 +(content-cap row) and feedback A1–A3 of the same run, upstreamed here. + +- **New `replica/scripts/gate-all.mjs` — the all-pages published-origin gate (D0).** For every + `deployed` page in `stardust/state.json`: stitched origin + served captures (window-free real Chrome, + `--settle`), `pixel-compare`, then the two DOM probes below on both sides in the same tier, per-page + dir `/{origin,eds,diff}.png + pixel.json + content.json + clip.json [+ units.json]`, + `summary.{json,md}`, `runs/-.json` for `--only` runs. **Verdict = pixel % ≤ 10 AND |Δh| ≤ + 5 % of the origin height AND clipped ≤ 0 (+ documented allowance) AND content MISSING + HIDDEN + links / headings = 0** (+ required repeated units within 4 px). The pixel-only verdict is recorded + beside the full one per page and in the totals — every run is calibration data. Flags: `--only`, + `--skip-existing`, `--recapture-eds`, `--recapture-origin`, `--eds-host` (a code branch against the + same DA content — branch hosts serve the LITERAL branch name, `fix/x` → mirror `fix-x`), + `--blocked ` / `--try-blocked` / `--crawl-shots`, the three concurrencies, `--threshold`, + `--height-tol`, `--clip-max`, `--unit-tol`, `--no-clip` / `--no-content` / `--no-probes`, + `--units`, `--compare-only`, `--warmup`. Sidecars, each entry documented: `masks.json` (printed on + the verdict, never reported unmasked), `overrides.json` (shown BESIDE the measured number, never + replacing it), `clip-allow.json`, `presence.json` (session-variable regions), `units.json` + (repeated-unit declarations). Origin fallback chain: live stitch → previous origin → crawl fullPage + screenshot (`crawl-fullpage`, asymmetric, flagged). A padded/union pixel metric was tried as the + height guard and rejected (white gaps score as matches) — the guard is explicit; + `pixel-compare --pad` ships as an auxiliary number only. `replica/scripts/sbs-crop.mjs` folds the + project's three side-by-side crop helpers into one (one crop per fact). +- **New `diff/scripts/clip-probe.mjs` (D1).** Every text node's line rects and every control's box + against EVERY overflow-clipping ancestor (nearest first) and the page width: TEXT CLIPPED (a line + cut across), TEXT HIDDEN (whole lines behind an overflow-hidden ancestor without a line-clamp), + CONTROL HIDDEN / CLIPPED counted (exit 2); line-clamp, scrollable containers, horizontal cuts + (carousels, ellipsis) advisory; collapsed menus, sr-only boxes, hidden / opacity-0 subtrees, + off-page boxes never reported. Measured on the recorded page: 379 (284 anchors + 95 lines); 0 on the + live origin and on two known-good served pages. Emits the visible text LINE boxes (`textBoxes`). +- **New `diff/scripts/content-presence.mjs` (D2) — `content-diff.mjs --published` hands over to it.** + Origin vs served inventories of VISIBLE headings, links (visible text — sr-only suffixes broke + matching), buttons, images, text blocks, aligned into bands by the h1–h3 sequence (LCS), plus + control STATE (select values, checked radios, `aria-selected`, `aria-pressed`, `aria-current`, + `aria-haspopup` triggers, result counts). MISSING / HIDDEN LINK ×n and HEADING 🔴 (HIDDEN = in the + DOM, clipped past 50 %), MISSING / HIDDEN BUTTON and CONTROL STATE 🟠, MOVED / EXTRA / COUNT / + HEADING AS TEXT 🟡. Links and buttons are one pool on the served side; text-less image anchors are + the image's presence, not a link's. Scope decided symmetrically (root only when BOTH sides have + one — a live origin without `
` against a build with one compared chrome against content); + header / footer and everything above / below them left to the chrome crop gate unless `--chrome` + (three session-variable promo-strip links read MISSING on every page). `--variable ` + = counts only, HIDDEN still counts. The origin side fails loud on HTTP ≥ 400 (exit 4) — a 403 + Access-Denied page measured as the origin read "100 EXTRA" on the build. On the recorded page: + HIDDEN 284 "View details", CONTROL STATE sort by "recommended" → "expiration date". +- **New `diff/scripts/unit-geometry.mjs` (D3) + `diff/scripts/measure-live.mjs`.** measure-live lifts + the shared parts of the four project measure scripts that converged: window-free real Chrome, + optional home warm-up, slow-scroll settle until the document height is stable, shadow-DOM-aware + serialisation with `data-r`/`data-t`, rect + computed type per selector, cache under + `stardust/current/measure/.json`. unit-geometry dumps the inner elements of the first N + repeated units on both sides (role + text, shadow-aware), pairs them (key, text prefix, relaxed + link ↔ button, images by order, leftover text by position — a "1 day left" vs "2 days left" badge), + and reports Δx / Δy / Δw / Δh relative to the unit against `--tol` (default 4), `hidden` where the + served page clips. On the recorded card: body elements Δy +28…+47, the details link Δy +28 and + hidden, the badge 70×70 at (−10,−10) vs 64×64 at (−27,−30), the card 260 → 238 px. Optional in + gate-all (`units.json`), mandatory for units a project marks `required`. +- **`pixel-compare.mjs --text-boxes ` (D4, auxiliary)** — the differing-pixel share over the + origin's text boxes only, `textPct` in `--json`, printed beside the pixel % in summary.md: 16.5 % + vs 6.7 % on the recorded page; `--pad` (union-height %, auxiliary) merged from the project copy; + `textBoxPct` exported, main-module guard added. Backward compatible. +- **Upstreamed feedback.** A1: `live-session.mjs launchStealthHeaded` is WINDOW-FREE by default + (`channel: 'chrome'` headless — the binary cleared Akamai, not the window; four agents' windows + blocked the operator's desktop); `STARDUST_HEADED_WINDOW=1` opts into a visible window; a failed + real-Chrome launch falls back to bundled Chromium with a loud warning. A2: Akamai's HTTP 400 + `{"result":"Bad Request"}` escalation stamped by AkamaiGHost is a challenge marker. A3: the setup + steps (replica, extract, diff) install `playwright pixelmatch pngjs cheerio` as devDependencies + written into `package.json`, never `--no-save`, and say "run probes from the project root" once. + Deploy: `deploy-batch.mjs` appends the admin `x-error` header to every 4xx (`… 337 of 200 images` + — the DA pipeline caps a document at 200 images; protocol step 3a'). +- **Docs.** `replica/reference/source-fidelity-gate.md` § The all-pages published-origin gate (the + four criteria, sidecars, the offers example, calibration, open questions); `replica/SKILL.md` + Phase 5 delivery-gate bullet + the window-free `--headed` note; `deploy/SKILL.md` Step 10 item 7; + `rollout/SKILL.md` Phase E; `diff/SKILL.md` § The published-origin probes; `scripts-index.md` + rows. Tests: `diff/scripts/test/{clip-probe,content-presence,unit-geometry}.test.mjs`, + `replica/scripts/test/gate-all.test.mjs` (pure parts without a browser; clip-probe end-to-end + against a fixture card where playwright resolves). Lint clean (`npm run lint:stardust`, airbnb + eslint on the new scripts). +- **Open questions** (for the next calibration run): default tolerances (`--clip-max 0`, + `--unit-tol 4`, the 25 % count tolerance in content-presence), how a project declares + repeated-unit block families (today per page in `units.json`; candidate: a flag on the block + inventory), whether buttons and control state should join the blocking set once the + session-variable label classes (Clip / Clipped) are modelled. + ## 0.25.2 — content cap: the container sizing model is measured, persisted and gated at a derived wide width (#124) A hands-off replica run delivered a site whose live pages centre their content in two nested caps — diff --git a/plugins/stardust/skills/deploy/SKILL.md b/plugins/stardust/skills/deploy/SKILL.md index 00d87eeb4..a27c7b99e 100644 --- a/plugins/stardust/skills/deploy/SKILL.md +++ b/plugins/stardust/skills/deploy/SKILL.md @@ -967,6 +967,8 @@ After deploy, reconcile the EDS page against the source prototype on the **DEPLO **6. Geometry-fix verification hygiene (#117).** One field "parity verified" claim was wrong three ways at once; when verifying any geometry fix: (a) probe the **rule-bearing element**, not a heuristic match ("white column wider than 400px" happened to select a different box than the one carrying the lifted rule) — pair the element the fixed rule targets on the build with the element whose source rule was lifted on live; (b) verify **serving cache-free** — a reviewer's DevTools showed the old rule at its old line number while both hosts already served the fix; check out-of-band with `curl --compressed | grep ''` (block CSS is gzip-encoded — a bare `curl | grep` scans binary and silently matches nothing) and re-render in a **fresh headless context**, telling any human reviewer to hard-refresh before re-judging; (c) a reviewer's screenshot encodes their zoom — **back-compute their CSS viewport** from any element with a known percentage rule (a card at 851px under `width: 50%` → viewport 1702px) and reproduce THAT viewport headlessly; judged at face value their numbers contradicted a correct fix, reproduced at 1702 live and build matched to the pixel. +**7. All-pages published-origin gate at delivery — four criteria (#125).** Step 10 is per page; once the roster is deployed, run `node stardust/scripts/replica/gate-all.mjs [--only ] [--skip-existing] [--eds-host ]` from the project root (through `run-bg.mjs`) — stitched 1440 captures of the live origin and the served page, `pixel-compare`, then two DOM probes on both sides in the same window-free real-Chrome tier: `clip-probe` (text lines cut or hidden, links / buttons outside their overflow ancestor) and `content-presence` (visible headings / links / buttons / images per band + control state). A page is DELIVERED when pixel % ≤ 10 AND |Δh| ≤ 5 % of the origin height AND clipped = 0 AND MISSING + HIDDEN links / headings = 0 (repeated units declared in `units.json` within 4 px). A recorded page PASSED the pixel bar at 6.7 % with every one of its 284 cards clipped and their "View details" links pushed under the Clip button — a fixed-height block with `overflow: hidden` — so the pixel number alone is not the delivery verdict. Read `stardust/replica/gates/all-1440/summary.md`: the row's `reasons` names the criterion; `clip.json` groups name the clipper (fix that block's CSS, never the content); `content.json` HIDDEN = CSS, MISSING = encoder. Rules and sidecars: `../replica/reference/source-fidelity-gate.md` § The all-pages published-origin gate. + The retired `visual-diff` classes are covered elsewhere: stretched images by the `img { height:auto }` reset (#36) + eyeball; dropped max-width wraps and the outer cap by the qa-gate wide pass (#13/#124); blank/broken renders by the computed-style guard; imagery gaps by the `.plain.html` img/alt count (#75) + eyeball. `content-diff` is stack-agnostic via `skills/deploy/scripts/diff-profiles.mjs` (`--profile eds | generic`); it shares the role classifier with `block-roundtrip`/`section-schema` in `skills/deploy/scripts/content-inventory.mjs`. diff --git a/plugins/stardust/skills/deploy/da-deploy-protocol.md b/plugins/stardust/skills/deploy/da-deploy-protocol.md index bcbdba698..b031afcfe 100644 --- a/plugins/stardust/skills/deploy/da-deploy-protocol.md +++ b/plugins/stardust/skills/deploy/da-deploy-protocol.md @@ -56,6 +56,11 @@ done curl -sS -X POST -H "Authorization: Bearer $TOKEN" \ "https://admin.hlx.page/preview/$ORG/$REPO/$BRANCH/$P" # expect 200 +# 3a'. ANY admin 4xx: read the `x-error` response header first — the body is generic, the +# header carries the reason (deploy-batch appends it to the ledger's lastError as +# `[x-error: …]`). Known: `… 337 of 200 images` = the DA pipeline caps a document at +# 200 images (#125, recorded on a 284-card coupon page) → split the long grid into +# /fragments/-1, -2 documents and reference them from the page; no SVG hunt. # 3a. preview 409 "error from content-bus" — the error is OPAQUE (no per-asset # detail); do NOT dead-end on it. Two cheap diagnostics, in order: # (i) upload a known-good doc to the SAME path and re-preview — separates diff --git a/plugins/stardust/skills/deploy/scripts/deploy-batch.mjs b/plugins/stardust/skills/deploy/scripts/deploy-batch.mjs index 6c7ae714c..6ea5e620a 100644 --- a/plugins/stardust/skills/deploy/scripts/deploy-batch.mjs +++ b/plugins/stardust/skills/deploy/scripts/deploy-batch.mjs @@ -179,7 +179,12 @@ async function call(method, url, { token, body } = {}, { retries = 4, backoffMs try { const res = await fetch(url, { method, headers: { Authorization: `Bearer ${token}` }, body }); status = res.status; - text = status >= 400 ? (await res.text()).slice(0, 200) : ''; + // The admin API puts the REASON of a 4xx in the `x-error` header, not the body: a preview 409 whose + // body says only "error from content-bus" carries "… 337 of 200 images" there (the DA pipeline caps a + // document at 200 images — long grids ride /fragments/ documents; #125). Surface it, or the ledger + // records an opaque failure and the agent diagnoses SVGs for an hour. + const xErr = status >= 400 ? (res.headers.get('x-error') || '') : ''; + text = status >= 400 ? `${(await res.text()).slice(0, 200)}${xErr ? ` [x-error: ${xErr.slice(0, 200)}]` : ''}` : ''; } catch (err) { status = 0; text = String(err.message || err); diff --git a/plugins/stardust/skills/diff/SKILL.md b/plugins/stardust/skills/diff/SKILL.md index e8dbd9ae6..75f743010 100644 --- a/plugins/stardust/skills/diff/SKILL.md +++ b/plugins/stardust/skills/diff/SKILL.md @@ -40,9 +40,10 @@ root carries the `placeholder` / `aria-label` / `title` values and the icons. ```bash # Prereq 0: playwright importable from the project root — probe # node -e "import('playwright').then(()=>process.exit(0))" -# and re-install (npm i -D playwright --no-save --legacy-peer-deps) on failure: -# a --no-save install from extract is PRUNED by any later real npm i -# (extract SKILL.md § Setup). Run the copied scripts from the project, not the plugin. +# and on failure install it AS A devDependency (npm i -D playwright pixelmatch pngjs cheerio +# --legacy-peer-deps — never --no-save: a --no-save install is PRUNED by the next real npm i, +# recorded twice in one run; extract SKILL.md § Setup). Run the copied scripts from the project +# root, not the plugin: ESM resolves `playwright` from the script's own location. # Copy the WHOLE skills/diff/scripts/ dir: content-diff imports its local diff-profiles.mjs # AND content-inventory.mjs. (The deploy gates #93/#94 now use their OWN synced copies in # skills/deploy/scripts/ — A6/A2 are independent of this skill; the two copies must stay in @@ -110,6 +111,46 @@ measures the error page, and the flags (BLANK RENDER / content asymmetry) carry signal with **exit 0**. That is the probes' advisory contract: 0 = ran (flags advisory), 1 = probe error, 3 = bot challenge. +## The published-origin probes (#125) + +`content-diff` reconciles a prototype with its build; on a LIVE commerce origin (React apps, +session rails, coupon walls) its per-node findings were systematically false, and the pixel gate +passed a page whose 284 cards were all broken (`../replica/reference/source-fidelity-gate.md` +§ The all-pages published-origin gate). Three probes ask the smaller, checkable questions. All +three load both sides in the same window-free real-Chrome tier and settle them with the same +slow scroll (`scripts/measure-live.mjs`, the shared library) — the instrument stays symmetric. + +```bash +ORIGIN="https://www.example.com/offers" # the live source page +SERVED="https://main--repo--owner.aem.live/offers" # the published page +# D1 — clipping: text lines cut / hidden, links and buttons outside their overflow ancestor. Exit 2 on any count. +node stardust/scripts/diff/clip-probe.mjs "$SERVED" [--json clip.json] [--advisory] +# D2 — presence: visible headings / links / buttons / images / text per band + control state. Exit 2 on MISSING/HIDDEN link or heading. +node stardust/scripts/diff/content-presence.mjs "$ORIGIN" "$SERVED" [--variable ".origin-rail=.served-rail"] [--json content.json] +node stardust/scripts/diff/content-diff.mjs --published "$ORIGIN" "$SERVED" # same thing, from the older entry point +# D3 — repeated units: per-element Δx/Δy/Δw/Δh of the first N cards / rail items / FAQ rows. Exit 2 on any element off / hidden / missing. +node stardust/scripts/diff/unit-geometry.mjs "$ORIGIN" "$SERVED" --unit ".origin-card=.served-card" --n 2 [--tol 4] +# the shared measurement: settle + rect + computed type per selector, cached per slug +node stardust/scripts/diff/measure-live.mjs "$ORIGIN" ".hero" ".card" --all [--slug offers] +``` + +Reading them: `clip-probe` prints one line per group (kind × count in ``, sample path, +sample text, cut range) — a 🔴 group is a CSS geometry defect on the served page (a fixed-height +box with `overflow: hidden`, a body offset), never authored content; 🟡 groups (CLAMPED, +SCROLL-HIDDEN, X-CUT) are design or carousels. `content-presence` prints a band table (links +O/E with the hidden count in brackets) then findings: HIDDEN LINK ×n = in the DOM, clipped away +(fix the CSS); MISSING LINK = not served (fix the encoder); CONTROL STATE = the same control +shows another value ("Recommended" → "Expiration Date" — the served default differs); COUNT … += session-variable region deltas, confirm by eye. `--variable` marks regions whose content +differs per visit (rails, coupon walls, reviews) — counts only, HIDDEN still counts. The origin +side fails loud on HTTP ≥ 400 (exit 4) and on a bot challenge (exit 3): an Access-Denied page is +never measured as the origin. `unit-geometry` prints one table per unit with Δ relative to the +unit's corner; `≈ "…" (by position)` marks a session-variable text paired by place (a "1 day +left" badge vs "2 days left"). Traps: an infinite-scroll origin keeps loading under the settle +(285 → 458 coupons) — mark the region `--variable`; a live origin without `
` is compared +as a whole page against the build's whole page (the scope line says which), header / footer +left to the chrome crop gate unless `--chrome`. + ## Reading content-diff - 🔴 **MISSING CTA / HEADING / EYEBROW** — real dropped content. FIX. A missing eyebrow is most often a segmentation drop where the eyebrow precedes its heading; a missing CTA means the component never rendered the link. These are exactly what the pixel probe cannot see. diff --git a/plugins/stardust/skills/diff/scripts/clip-probe.mjs b/plugins/stardust/skills/diff/scripts/clip-probe.mjs new file mode 100644 index 000000000..0cd791dc9 --- /dev/null +++ b/plugins/stardust/skills/diff/scripts/clip-probe.mjs @@ -0,0 +1,390 @@ +#!/usr/bin/env node +/* eslint-disable import/no-extraneous-dependencies, import/extensions, no-await-in-loop, no-restricted-syntax, brace-style, object-curly-newline, max-len, no-console, no-continue, no-nested-ternary, no-plusplus, no-underscore-dangle, no-restricted-globals */ +/* global __clipChain, __controlOf, __isControl, __lineRects, __norm, __pageRect, __path, __rendered, __sel, __srOnly, __walk */ +/** + * skills/diff/scripts/clip-probe.mjs — the CLIPPING probe of the published-origin gate (#125, D1). + * + * Why: a page can PASS the pixel gate with every card broken. On the recorded offers page (walgreens, + * 2026-09-25) the served render measured 6.7 % / Δh 69 px against the live origin — inside the bar — + * while all 284 coupon cards clipped their description mid-glyph and pushed the "View details" link + * under the Clip button: the card is a fixed 238 px with `overflow: hidden`, the badge took its own + * row, the body started ~30 px too low. Pixelmatch underweights small text inside otherwise matching + * shapes, so "PASS" meant "right shapes at the right places", not "every element present and legible". + * The links WERE in the served DOM (284 in the fragments) — pure CSS geometry, invisible to a DOM + * count and nearly invisible to pixels. Geometry is checkable — so check it. + * + * What it measures. The served page is loaded headless (window-free real Chrome, live-session tier), + * settled with measure-live's slow scroll, then for every TEXT NODE with a non-empty box and every + * CONTROL (a[href], button, [role=button|link], summary, select) the probe compares its rect(s) with + * the rect of the nearest ancestor whose computed overflow / overflow-x / overflow-y is + * hidden | clip | scroll | auto, and with the page width: + * TEXT CLIPPED a text LINE cut across (partial line, > --min-cut px) counted 🔴 + * TEXT HIDDEN whole lines outside an overflow-hidden ancestor (no line-clamp) counted 🔴 + * CONTROL HIDDEN a link / button whose box is fully outside its clipping ancestor counted 🔴 + * CONTROL CLIPPED a link / button cut across by > --min-cut px counted 🔴 + * TEXT CLAMPED whole lines hidden by a `-webkit-line-clamp` ancestor advisory (design) + * TEXT|CONTROL COLLAPSED whole lines / controls behind a "Read more" / aria-expanded=false collapsible advisory (state) + * TEXT|CONTROL SCROLL-HIDDEN outside an overflow auto|scroll ancestor (reachable) advisory + * TEXT|CONTROL X-CUT | X-HIDDEN horizontal only (carousel tracks, ellipsis) advisory + * Not reported (recorded false-positive classes): collapsed containers (client box ≤ 2 px — a closed + * mega-menu with max-height 0), sr-only boxes (clip-path / clip / 1×1), visibility:hidden or opacity 0 + * subtrees, boxes wholly off-page (left:-9999px), text inside a control already reported. + * + * Symmetric use: content-presence.mjs runs THIS inventory on both sides and reports what is hidden on + * the build but visible on the origin; gate-all reads the build-side count into its `clipped` column + * and verdict criterion 3 (`--clip-max`, default 0; a documented allowance per page in clip-allow.json + * when a live page genuinely clips). The visible text LINE boxes come out as `textBoxes` — the input + * of pixel-compare `--text-boxes` (D4). + * + * Usage: + * node skills/diff/scripts/clip-probe.mjs [options] + * --width viewport width (default 1440) + * --min-cut partial cut below this many px is ignored (default 2 — sub-pixel rounding) + * --main restrict to this root (default: the whole document) + * --json [] JSON on stdout (or to ): counts, findings, groups, textBoxes + * --max-findings cap the printed / serialised finding list (default 400; groups are complete) + * --advisory also print the advisory kinds in the table + * --plain bundled Chromium instead of the window-free real-Chrome tier + * --warmup visit this URL first (home warm-up on bot-managed sites) + * --locale default en-US + * + * Exit: 0 nothing counted, 2 counted findings > 0, 1 error, 3 bot challenge. + * Requires: playwright (project devDependency). `clipInventoryInPage`, `IN_PAGE_LIB`, `inPage`, + * `summarize`, `formatTable` are exported; the browser is imported lazily in main so the contract test + * runs the pure parts without one. + */ +import { mkdirSync, realpathSync, writeFileSync } from 'node:fs'; +import { dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { openBrowser, openPage, visit } from './measure-live.mjs'; + +const HELP = `clip-probe — text and controls cut or hidden by an overflow ancestor on a served page (#125 D1) + +Usage: node clip-probe.mjs [options] + --width viewport width (default 1440) + --min-cut ignore partial cuts below this (default 2) + --main restrict to this root (default whole document) + --json [] JSON (counts, findings, groups, textBoxes) on stdout or to + --max-findings cap the finding list (default 400) + --advisory print the advisory kinds too (CLAMPED, COLLAPSED, SCROLL-HIDDEN, X-CUT, X-HIDDEN) + --plain bundled Chromium instead of the window-free real-Chrome tier + --warmup visit this URL first (bot-managed sites) + --locale default en-US + --help this text +Counted (exit 2): TEXT CLIPPED, TEXT HIDDEN, CONTROL HIDDEN, CONTROL CLIPPED. Exit 3 = bot challenge.`; + +// ---- in-page library -------------------------------------------------------------------------------- +// Helper declarations injected in front of every in-page function of the gate probes (content-presence +// and unit-geometry reuse them), so the clipping model is ONE piece of code. Playwright serialises a +// function's source only, so shared helpers must travel as text: inPage(fn, arg) builds the expression. +export const IN_PAGE_LIB = ` +const __cs = new WeakMap(); +function __style(el) { let s = __cs.get(el); if (!s) { s = getComputedStyle(el); __cs.set(el, s); } return s; } +function __parent(el) { return el.parentElement || (el.parentNode && el.parentNode.host) || null; } +function __rendered(el) { + for (let a = el; a && a.nodeType === 1; a = __parent(a)) { + const s = __style(a); + if (s.display === 'none' || s.visibility === 'hidden' || s.visibility === 'collapse' || parseFloat(s.opacity) === 0) return false; + if (a.tagName === 'BODY') break; + } + return true; +} +function __pageRect(r) { return { l: r.left + window.scrollX, t: r.top + window.scrollY, r: r.right + window.scrollX, b: r.bottom + window.scrollY, w: r.width, h: r.height }; } +// nearest clipping ancestor of el (el itself excluded): { el, rect, mode: 'hidden'|'scroll'|'collapsed'|'srOnly', clamp } or null +function __srOnly(el) { + const s = __style(el); const br = el.getBoundingClientRect(); + return (s.clipPath && s.clipPath !== 'none') || (s.clip && s.clip !== 'auto') || (br.width <= 1 && br.height <= 1 && s.overflow === 'hidden'); +} +// every clipping ancestor, nearest first (el itself included when inclusive — a text node's own element clips +// its lines: a 30 px 'p.desc { overflow: hidden }' inside a 170 px body). Stops at a collapsed / sr-only box +// (the whole subtree is a state, not a clip defect) by returning { skip: true }. +function __clipChain(el, inclusive) { + const chain = []; + for (let a = inclusive ? el : __parent(el); a && a.nodeType === 1 && a.tagName !== 'BODY' && a.tagName !== 'HTML'; a = __parent(a)) { + const s = __style(a); + const clips = (v) => v === 'hidden' || v === 'clip' || v === 'scroll' || v === 'auto'; + if (!clips(s.overflowX) && !clips(s.overflowY)) continue; + const c = __clipOf(a, true); + if (!c) continue; + if (c.mode === 'collapsed' || c.mode === 'srOnly') return { skip: true, chain }; + chain.push(c); + } + return { skip: false, chain }; +} +function __clipOf(el, inclusive) { + for (let a = inclusive ? el : __parent(el); a && a.nodeType === 1 && a.tagName !== 'BODY' && a.tagName !== 'HTML'; a = __parent(a)) { + const s = __style(a); + const ox = s.overflowX, oy = s.overflowY; + const clips = (v) => v === 'hidden' || v === 'clip' || v === 'scroll' || v === 'auto'; + if (!clips(ox) && !clips(oy)) continue; + const br = a.getBoundingClientRect(); + const cw = a.clientWidth || br.width, ch = a.clientHeight || br.height; + const l = br.left + (a.clientLeft || 0) + window.scrollX, t = br.top + (a.clientTop || 0) + window.scrollY; + const rect = { l, t, r: l + cw, b: t + ch, w: cw, h: ch }; + const srOnly = (s.clipPath && s.clipPath !== 'none') || (s.clip && s.clip !== 'auto') || (cw <= 1 && ch <= 1); + const mode = srOnly ? 'srOnly' : (cw <= 2 || ch <= 2) ? 'collapsed' : (ox === 'scroll' || ox === 'auto' || oy === 'scroll' || oy === 'auto') ? 'scroll' : 'hidden'; + const clamp = s.webkitLineClamp && s.webkitLineClamp !== 'none'; + const toggle = mode === 'hidden' && !clamp && __collapsible(a); + return { el: a, rect, mode, clamp, toggle, axes: { x: clips(ox), y: clips(oy) } }; + } + return null; +} +// a clipper that is a "Read more" / "Show all" collapsible: it (or its parent) holds or is followed by a +// control with aria-expanded="false" or a show-more label. Whole lines behind it are a STATE the visitor can +// open, not a defect — recorded: 13 product pages read 4–46 TEXT HIDDEN inside their collapsed accordions. +const __MORE = /\b(read|show|see|view|load)\s+(more|all|full|less)\b|\bmore\b\s*$|expand/i; +function __collapsible(clipEl) { + const scopes = [clipEl, __parent(clipEl)].filter(Boolean); + for (const sc of scopes) { + const controls = [...sc.querySelectorAll('button, [role="button"], a[href], summary, [aria-expanded]')]; + if (controls.some((c) => c.getAttribute('aria-expanded') === 'false' || c.getAttribute('aria-controls') || __MORE.test(__norm(c.textContent || c.getAttribute('aria-label') || '')))) return true; + const next = sc.nextElementSibling; + if (next && (next.getAttribute('aria-expanded') === 'false' || __MORE.test(__norm(next.textContent || '').slice(0, 40)))) return true; + } + return false; +} +function __lineRects(node) { + const range = document.createRange(); range.selectNodeContents(node); + return [...range.getClientRects()].filter((r) => r.width > 0 && r.height > 0).map(__pageRect); +} +function __sel(el) { + if (!el || el.nodeType !== 1) return ''; + const tag = el.tagName.toLowerCase(); + if (el.id) return tag + '#' + el.id; + const cls = (typeof el.className === 'string' ? el.className : '').trim().split(/\\s+/).filter(Boolean).slice(0, 2); + return tag + (cls.length ? '.' + cls.join('.') : ''); +} +function __path(el, depth) { + const parts = []; + for (let a = el, i = 0; a && a.nodeType === 1 && a.tagName !== 'BODY' && i < (depth || 4); a = __parent(a), i += 1) parts.unshift(__sel(a)); + return parts.join(' > '); +} +function __norm(s) { return (s || '').replace(/\\s+/g, ' ').trim().toLowerCase(); } +// text a sighted visitor can read inside el: text nodes whose element is rendered, not sr-only, on-page +function __visibleText(el) { + const pageW = document.documentElement.clientWidth; const parts = []; + const stack = [el]; + while (stack.length) { + const n = stack.pop(); + if (n.nodeType === 3) { if (n.textContent.trim()) parts.push(n.textContent); continue; } + if (n.nodeType !== 1) continue; + const tag = n.tagName; + if (tag === 'SCRIPT' || tag === 'STYLE' || tag === 'NOSCRIPT' || tag === 'TEMPLATE' || tag === 'SVG' || tag === 'svg') continue; + const s = __style(n); + if (s.display === 'none' || s.visibility === 'hidden' || parseFloat(s.opacity) === 0 || __srOnly(n)) continue; + const br = n.getBoundingClientRect(); + if (br.width > 0 && (br.right + window.scrollX <= 0 || br.left + window.scrollX >= pageW)) continue; + if (n.shadowRoot) for (let i = n.shadowRoot.childNodes.length - 1; i >= 0; i -= 1) stack.push(n.shadowRoot.childNodes[i]); + for (let i = n.childNodes.length - 1; i >= 0; i -= 1) stack.push(n.childNodes[i]); + } + return parts.join(' ').replace(/\\s+/g, ' ').trim(); +} +function __isControl(el) { + if (!el || el.nodeType !== 1) return false; + const t = el.tagName; + if (t === 'A') return el.hasAttribute('href'); + if (t === 'BUTTON' || t === 'SUMMARY' || t === 'SELECT') return true; + if (t === 'INPUT') return /^(button|submit|reset)$/i.test(el.type); + const role = el.getAttribute('role'); + return role === 'button' || role === 'link' || role === 'tab' || role === 'menuitem'; +} +function __controlOf(el) { for (let a = el; a && a.nodeType === 1; a = __parent(a)) { if (__isControl(a)) return a; if (a.tagName === 'BODY') break; } return null; } +function* __walk(root) { + const stack = [root]; + while (stack.length) { + const n = stack.pop(); + yield n; + if (n.nodeType === 1) { + if (n.shadowRoot) for (let i = n.shadowRoot.childNodes.length - 1; i >= 0; i -= 1) stack.push(n.shadowRoot.childNodes[i]); + const tag = n.tagName; + if (tag === 'SCRIPT' || tag === 'STYLE' || tag === 'NOSCRIPT' || tag === 'TEMPLATE' || tag === 'SVG' || tag === 'svg' || tag === 'SELECT' || tag === 'TEXTAREA' || tag === 'TITLE') continue; + for (let i = n.childNodes.length - 1; i >= 0; i -= 1) stack.push(n.childNodes[i]); + } + } +} +`; + +/** Build a page.evaluate expression: the library, then fn applied to a JSON-inlined arg. */ +export function inPage(fn, arg = {}) { return `(() => { ${IN_PAGE_LIB}\n return (${fn.toString()})(${JSON.stringify(arg)}); })()`; } + +/** + * In-page (needs IN_PAGE_LIB): page.evaluate(inPage(clipInventoryInPage, { minCut, rootSel, maxFindings })) + * → { docH, pageW, counts, findings, textBoxes } + */ +export function clipInventoryInPage({ minCut = 2, rootSel = null, maxFindings = 400 } = {}) { + const root = (rootSel && document.querySelector(rootSel)) || document.body; + const pageW = document.documentElement.clientWidth; + const docH = document.documentElement.scrollHeight; + const counts = { textClipped: 0, textHidden: 0, controlHidden: 0, controlClipped: 0, clamped: 0, collapsed: 0, scrollHidden: 0, horizontal: 0, offPage: 0, textNodes: 0, controls: 0, total: 0 }; + const findings = []; + const textBoxes = []; + const reported = new Set(); + const push = (f) => { if (findings.length < maxFindings) findings.push(f); }; + const overlap = (a0, a1, b0, b1) => Math.max(0, Math.min(a1, b1) - Math.max(a0, b0)); + const rnd = (o) => Object.fromEntries(Object.entries(o).map(([k, v]) => [k, typeof v === 'number' ? Math.round(v) : v])); + // per-rect visibility against EVERY clipping ancestor, nearest first (vertical = the defect axis) and the + // page width (horizontal). A line hidden by the nearest clipper is attributed to it (a line-clamp there is + // design); a line no single clipper hides but one cuts across is a partial cut by that clipper. + const judge = (rects, chain) => { + let partialCut = 0; let hiddenV = 0; let visibleV = 0; let hcut = 0; let hiddenH = 0; let offPage = 0; let by = null; let hiddenBy = null; + for (const r of rects) { + if (r.r <= 0 || r.l >= pageW) { offPage += 1; continue; } + if (r.r > pageW + minCut) hcut = Math.max(hcut, r.r - pageW); + let hidden = null; let cut = 0; let cutBy = null; let hHidden = false; + for (const clip of chain) { + const vis = clip.axes.y ? overlap(r.t, r.b, clip.rect.t, clip.rect.b) : r.h; + const hvis = clip.axes.x ? overlap(r.l, r.r, clip.rect.l, clip.rect.r) : r.w; + if (vis <= 0) { hidden = clip; break; } + if (r.h - vis > cut + 0.5 && r.h - vis > minCut) { cut = r.h - vis; cutBy = clip; } + if (hvis <= 0) hHidden = true; + else if (r.w - hvis > minCut) hcut = Math.max(hcut, r.w - hvis); + } + if (hidden) { hiddenV += 1; if (!hiddenBy) hiddenBy = hidden; } + else { visibleV += 1; if (cut > partialCut) { partialCut = cut; by = cutBy; } if (hHidden) hiddenH += 1; } + } + return { partialCut, hiddenV, visibleV, hcut, hiddenH, offPage, n: rects.length, by, hiddenBy }; + }; + for (const node of __walk(root)) { + if (node.nodeType === 1 && __isControl(node)) { + if (!__rendered(node) || __srOnly(node)) continue; + const br = node.getBoundingClientRect(); + if (br.width <= 0 || br.height <= 0) continue; + counts.controls += 1; + const r = __pageRect(br); + const cc = __clipChain(node, false); + if (cc.skip || !cc.chain.length) continue; + const j = judge([r], cc.chain); + if (j.offPage) { counts.offPage += 1; continue; } + const text = __norm(node.textContent || node.getAttribute('aria-label') || node.getAttribute('title') || (node.querySelector('img') && node.querySelector('img').alt) || '').slice(0, 60); + const clip = j.hiddenBy || j.by || cc.chain[0]; + const base = { path: __path(node), text, clipper: __sel(clip.el), rect: rnd(r), clip: rnd(clip.rect) }; + if (j.hiddenV) { + if (clip.mode === 'scroll') { counts.scrollHidden += 1; push({ kind: 'CONTROL SCROLL-HIDDEN', advisory: true, ...base }); } + else if (clip.toggle) { counts.collapsed += 1; reported.add(node); push({ kind: 'CONTROL COLLAPSED', advisory: true, ...base }); } + else { counts.controlHidden += 1; reported.add(node); push({ kind: 'CONTROL HIDDEN', cut: Math.round(r.h), ...base }); } + } else if (j.partialCut) { + if (clip.mode === 'scroll') { counts.scrollHidden += 1; push({ kind: 'CONTROL SCROLL-HIDDEN', advisory: true, cut: Math.round(j.partialCut), ...base }); } + else { counts.controlClipped += 1; reported.add(node); push({ kind: 'CONTROL CLIPPED', cut: Math.round(j.partialCut), ...base }); } + } else if (j.hiddenH) { counts.horizontal += 1; push({ kind: 'CONTROL X-HIDDEN', advisory: true, ...base }); } + else if (j.hcut) { counts.horizontal += 1; push({ kind: 'CONTROL X-CUT', advisory: true, cut: Math.round(j.hcut), ...base }); } + continue; + } + if (node.nodeType !== 3) continue; + const text = node.textContent; + if (!text || !text.trim()) continue; + const parent = node.parentElement || (node.parentNode && node.parentNode.host) || null; + if (!parent || !__rendered(parent)) continue; + const ptag = parent.tagName; + if (ptag === 'OPTION' || ptag === 'OPTGROUP' || ptag === 'TEXTAREA') continue; + if (__srOnly(parent)) continue; // the sr-only pattern carried by the text's own element + const lines = __lineRects(node); + if (!lines.length) continue; + counts.textNodes += 1; + const cc = __clipChain(parent, true); + if (cc.skip) continue; + const ctrl = __controlOf(parent); + if (ctrl && reported.has(ctrl)) continue; // the control already carries the finding + const j = judge(lines, cc.chain); + if (j.offPage === j.n) { counts.offPage += 1; continue; } + // visible line boxes → textBoxes (D4 input) + for (const r of lines) { + if (r.r <= 0 || r.l >= pageW) continue; + if (cc.chain.some((clip) => clip.axes.y && overlap(r.t, r.b, clip.rect.t, clip.rect.b) <= 0)) continue; + textBoxes.push({ x: Math.round(r.l), y: Math.round(r.t), w: Math.round(r.w), h: Math.round(r.h) }); + } + const clip = j.by || j.hiddenBy || cc.chain[0] || null; + const base = { path: __path(parent), text: __norm(text).slice(0, 60), clipper: clip ? __sel(clip.el) : null, rect: rnd(lines[0]), clip: clip ? rnd(clip.rect) : null, lines: lines.length }; + if (j.partialCut) { + if (j.by.mode === 'scroll') { counts.scrollHidden += 1; push({ kind: 'TEXT SCROLL-HIDDEN', advisory: true, cut: Math.round(j.partialCut), ...base }); } + else { counts.textClipped += 1; push({ kind: 'TEXT CLIPPED', cut: Math.round(j.partialCut), hiddenLines: j.hiddenV, ...base }); } + } else if (j.hiddenV) { + if (j.hiddenBy.mode === 'scroll') { counts.scrollHidden += 1; push({ kind: 'TEXT SCROLL-HIDDEN', advisory: true, hiddenLines: j.hiddenV, ...base }); } + else if (j.hiddenBy.clamp) { counts.clamped += 1; push({ kind: 'TEXT CLAMPED', advisory: true, hiddenLines: j.hiddenV, ...base }); } + else if (j.hiddenBy.toggle) { counts.collapsed += 1; push({ kind: 'TEXT COLLAPSED', advisory: true, hiddenLines: j.hiddenV, ...base }); } + else { counts.textHidden += 1; push({ kind: 'TEXT HIDDEN', hiddenLines: j.hiddenV, cut: Math.round(lines[0].h), ...base }); } + } else if (j.hiddenH || j.hcut) { counts.horizontal += 1; push({ kind: j.hiddenH ? 'TEXT X-HIDDEN' : 'TEXT X-CUT', advisory: true, cut: Math.round(j.hcut), ...base }); } + } + counts.total = counts.textClipped + counts.textHidden + counts.controlHidden + counts.controlClipped; + return { docH, pageW, counts, findings, textBoxes }; +} + +// ---- pure: grouping + formatting ---------------------------------------------------------------------- + +/** Group findings by kind + clipper + parent selector: [{ kind, advisory, n, clipper, path, text, cutMin, cutMax }] sorted by count. */ +export function summarize(findings) { + const groups = new Map(); + for (const f of findings) { + const key = `${f.kind}|${f.clipper || ''}|${(f.path || '').split(' > ').pop()}`; + let g = groups.get(key); + if (!g) { g = { kind: f.kind, advisory: !!f.advisory, n: 0, clipper: f.clipper, path: f.path, text: f.text, cutMin: Infinity, cutMax: -Infinity }; groups.set(key, g); } + g.n += 1; + if (Number.isFinite(f.cut)) { g.cutMin = Math.min(g.cutMin, f.cut); g.cutMax = Math.max(g.cutMax, f.cut); } + } + return [...groups.values()].map((g) => ({ ...g, cutMin: Number.isFinite(g.cutMin) ? g.cutMin : null, cutMax: Number.isFinite(g.cutMax) ? g.cutMax : null })).sort((a, b) => Number(a.advisory) - Number(b.advisory) || b.n - a.n); +} + +export function formatTable(groups, { advisory = false } = {}) { + const rows = groups.filter((g) => advisory || !g.advisory); + if (!rows.length) return ' (none)'; + const cut = (g) => (g.cutMin == null ? '-' : g.cutMin === g.cutMax ? `${g.cutMin}px` : `${g.cutMin}–${g.cutMax}px`); + return rows.map((g) => ` ${g.advisory ? '🟡' : '🔴'} ${g.kind.padEnd(22)} ×${String(g.n).padEnd(5)} in ${g.clipper || '(page)'} ${g.path} "${g.text}" cut ${cut(g)}`).join('\n'); +} + +export const verdictLine = (counts) => `Clipped: ${counts.total} (text clipped ${counts.textClipped}, text hidden ${counts.textHidden}, controls hidden ${counts.controlHidden}, controls clipped ${counts.controlClipped}; advisory: clamped ${counts.clamped}, collapsed ${counts.collapsed || 0}, scroll-hidden ${counts.scrollHidden}, horizontal ${counts.horizontal})`; + +// ---- CLI --------------------------------------------------------------------------------------------- + +export function parseArgs(argv) { + const rest = argv.slice(2); + if (!rest.length || rest.includes('--help') || rest.includes('-h')) { console.log(HELP); process.exit(0); } + const opts = { url: null, width: 1440, minCut: 2, main: null, json: false, jsonFile: null, maxFindings: 400, advisory: false, plain: false, warmup: null, locale: 'en-US' }; + for (let i = 0; i < rest.length; i += 1) { + const a = rest[i]; + if (a === '--width') opts.width = Number(rest[++i]); + else if (a === '--min-cut') opts.minCut = Number(rest[++i]); + else if (a === '--main') opts.main = rest[++i]; + else if (a === '--json') { opts.json = true; if (rest[i + 1] && !rest[i + 1].startsWith('--')) opts.jsonFile = rest[++i]; } + else if (a === '--max-findings') opts.maxFindings = Number(rest[++i]); + else if (a === '--advisory') opts.advisory = true; + else if (a === '--plain') opts.plain = true; + else if (a === '--warmup') opts.warmup = rest[++i]; + else if (a === '--locale') opts.locale = rest[++i]; + else if (a.startsWith('--')) { console.error(`unknown flag ${a}\n\n${HELP}`); process.exit(1); } + else if (!opts.url) opts.url = a; + } + if (!opts.url) { console.error(`need \n\n${HELP}`); process.exit(1); } + return opts; +} + +/** Load + settle a page and run the inventory (shared with gate-all's --no-content path). */ +export async function probe(page, url, opts) { + const v = await visit(page, url, { warmup: opts.warmup }); + const inv = await page.evaluate(inPage(clipInventoryInPage, { minCut: opts.minCut, rootSel: opts.main, maxFindings: opts.maxFindings })); + return { url, at: new Date().toISOString(), width: opts.width, status: v.status, settlePasses: v.passes, ...inv, groups: summarize(inv.findings) }; +} + +async function main() { + const opts = parseArgs(process.argv); + const { chromium } = await import('playwright'); + const browser = await openBrowser(chromium, { tier: opts.plain ? 'plain' : 'stealth' }); + let res; + try { + const { ctx, page } = await openPage(browser, { width: opts.width, locale: opts.locale }); + res = await probe(page, opts.url, opts); + await ctx.close(); + } finally { await browser.close(); } + const out = { ...res, textBoxes: res.textBoxes }; + if (opts.json && !opts.jsonFile) { console.log(JSON.stringify(out, null, 1)); } + else { + console.log(`clip-probe ${opts.url} @ ${opts.width}px — docH ${res.docH}, ${res.counts.textNodes} text nodes, ${res.counts.controls} controls (HTTP ${res.status})`); + console.log(formatTable(res.groups, { advisory: opts.advisory })); + console.log(verdictLine(res.counts)); + if (opts.jsonFile) { mkdirSync(dirname(opts.jsonFile) || '.', { recursive: true }); writeFileSync(opts.jsonFile, JSON.stringify(out, null, 1)); console.log(`json → ${opts.jsonFile}`); } + } + process.exitCode = res.counts.total > 0 ? 2 : 0; +} + +function safeRealpath(p) { try { return realpathSync(p); } catch { return p; } } +if (process.argv[1] && fileURLToPath(import.meta.url) === safeRealpath(process.argv[1])) { + main().catch((e) => { console.error(`clip-probe error: ${String(e.message).split('\n')[0]}`); process.exit(e.name === 'BotChallengeError' ? 3 : 1); }); +} diff --git a/plugins/stardust/skills/diff/scripts/content-diff.mjs b/plugins/stardust/skills/diff/scripts/content-diff.mjs index 8fe55d8dc..374014507 100644 --- a/plugins/stardust/skills/diff/scripts/content-diff.mjs +++ b/plugins/stardust/skills/diff/scripts/content-diff.mjs @@ -171,6 +171,8 @@ const USAGE = `usage: node skills/diff/scripts/content-diff.mjs pin Accept-Language + locale (e.g. en-GB) for geo determinism + --published live ORIGIN vs SERVED page: hands over to content-presence.mjs (visible + headings/links/buttons/images per band + control state; #125) exit codes: 0 ran (flags advisory; an HTTP-error side, e.g. a 404 build pre-propagation, is measured + flagged with a warning, not fatal), 1 error, 3 bot challenge (live side blocked — fail loud) @@ -478,6 +480,15 @@ async function grab(browser, url, opts, prof, roots) { } async function main() { + // --published: the PUBLISHED-ORIGIN case (live origin vs served page) is content-presence.mjs's job (#125 D2): + // this classifier is tuned to a prototype's DOM and read false per-node findings on live commerce origins. + // Hand the remaining arguments over unchanged so the older entry point keeps working in gate docs. + if (process.argv.includes('--published')) { + const { spawn } = await import('node:child_process'); + const args = [fileURLToPath(new URL('./content-presence.mjs', import.meta.url)), ...process.argv.slice(2).filter((a) => a !== '--published')]; + await new Promise((done) => { const ch = spawn(process.execPath, args, { stdio: 'inherit' }); ch.on('close', (code) => { process.exitCode = code ?? 1; done(); }); }); + return; + } const { proto, eds, opts } = parseArgs(process.argv); if (!proto || !eds) { process.stderr.write(USAGE); diff --git a/plugins/stardust/skills/diff/scripts/content-presence.mjs b/plugins/stardust/skills/diff/scripts/content-presence.mjs new file mode 100644 index 000000000..fc6403983 --- /dev/null +++ b/plugins/stardust/skills/diff/scripts/content-presence.mjs @@ -0,0 +1,396 @@ +#!/usr/bin/env node +/* eslint-disable import/no-extraneous-dependencies, import/extensions, no-await-in-loop, no-restricted-syntax, brace-style, object-curly-newline, max-len, no-console, no-continue, no-nested-ternary, no-plusplus, no-underscore-dangle, no-restricted-globals, newline-per-chained-call, object-property-newline */ +/* global __clipChain, __controlOf, __isControl, __norm, __pageRect, __parent, __path, __rendered, __sel, __srOnly, __visibleText, __walk */ +/** + * skills/diff/scripts/content-presence.mjs — the CONTENT-PRESENCE gate of the published-origin gate + * (#125, D2): live origin DOM vs served DOM, same minute, same settle, compared per band. + * + * content-diff.mjs reconciles a PROTOTYPE with its EDS build through a role classifier tuned to the + * prototype's `.ds-*` DOM; on a live commerce origin (React apps, session rails, coupon walls) its + * per-node findings were systematically false. This probe asks the smaller, checkable question the + * pixel gate cannot answer: is every VISIBLE heading, link, button, image and paragraph of the origin + * present AND visible on the served page, band by band — and does every control show the same state? + * Recorded motivation (walgreens offers page, 2026-09-25): 284 "View details" links present in the served + * DOM but pushed under the Clip button by an overflow-hidden card (pixel gate PASS at 6.7 %), and a + * toolbar reading "Expiration Date" where the origin read "Recommended". + * + * Method. Both pages load in the SAME window-free real-Chrome tier and settle with the same slow scroll + * (measure-live.mjs); each side yields an inventory of visible items — headings (level), links (text or + * aria-label / img alt, href path), buttons, images (alt / file name), text blocks (elements with own text) + * and control STATE (a 's selected option, checked radio / checkbox labels, [aria-selected] tabs, - * [aria-pressed] / [aria-current] items, [aria-haspopup] / [role=combobox] triggers, result-count phrases - * such as "284 coupons"). Visibility = rendered, non-zero box, on-page, and not clipped past 50 % by the - * nearest overflow ancestor (clip-probe's model — an item inside the DOM but cut away is HIDDEN, not - * present). Bands: the visible h1–h3 sequences are aligned by text in y-order (longest common - * subsequence); each aligned heading opens a band on both sides and items fall into bands by y. - * MISSING HEADING / HIDDEN HEADING 🔴 an origin h1–h3 with no aligned build heading (hidden = in the DOM but clipped) - * MISSING LINK ×n / HIDDEN LINK ×n 🔴 an origin link text (per band) short on the build; hidden first, then missing - * MISSING BUTTON / HIDDEN BUTTON 🟠 same for buttons (state-variable labels are common — Clip / Clipped) - * CONTROL STATE 🟠 same control (by label, else by ordinal in the band), different state text - * COUNT TEXT / COUNT IMAGES / COUNT LINKS 🟡 per-band count deltas beyond the tolerance; links only in variable regions - * MOVED LINK 🟡 short in its band but present visibly elsewhere on the build - * EXTRA HEADING / EXTRA LINK 🟡 build-only items - * Session-variable regions (rails, coupon walls, reviews): pass `--variable ` (`selO=selE` when the - * two sides differ) — items inside them are compared as COUNTS only, never as MISSING; gate-all reads the - * same lists from presence.json per page. Items hidden on the ORIGIN are excluded from the origin side - * (the origin's own clipping is not a conversion defect). + * MISSING / HIDDEN HEADING, MISSING / HIDDEN LINK ×n 🔴 (exit 2) + * MISSING / HIDDEN BUTTON, CONTROL STATE 🟠 + * COUNT TEXT|IMAGES|LINKS, MOVED LINK, EXTRA …, HEADING AS TEXT 🟡 * - * Usage: - * node skills/diff/scripts/content-presence.mjs [options] - * --width viewport width (default 1440) - * --main [=] content root(s) (default: main, else body — per side) - * --variable session-variable subtrees (counts only); `selO=selE` pairs allowed - * --json [] JSON on stdout or to : inventories, bands, findings, totals, - * clip (both sides), textBoxes (both sides — pixel-compare --text-boxes input) - * --min-cut clip-probe partial-cut floor (default 2) - * --chrome include header / footer items — default off: chrome repeats on every page, its - * promo strips and account state are session-variable, and the chrome crop gate - * already judges it (gate doc § Pass bar item 5) - * --settle-passes slow-scroll passes until the height is stable (default 4). Trap: an - * infinite-scroll origin keeps loading under the settle (recorded: 285 → 458 - * coupons) — that region is session-variable by nature, mark it --variable - * --max-findings cap per kind in the printed list (default 40; totals stay complete) - * --plain | --warmup | --locale as in measure-live + * Links and buttons are one pool on the served side; text-less image anchors count as images. Scope + * is symmetric (root only when BOTH sides have one, else whole page); header / footer and everything + * above / below them are left to the chrome crop gate unless --chrome. `--variable ` + * marks session-variable regions (counts only, HIDDEN still counts). The origin side fails loud on + * HTTP ≥ 400 (exit 4) and on a bot challenge (exit 3). Trap: an infinite-scroll origin keeps loading + * under the settle — mark that region --variable. * - * Exit: 0 no structural finding, 2 any MISSING / HIDDEN link or heading (the gate criterion — buttons, - * control state and counts are reported, not blocking), 1 error, 3 bot challenge on the origin, 4 origin - * HTTP ≥ 400 (never measured as the origin — gate-all records `content: n/a` and keeps the clip column). - * `presenceInventoryInPage` (in-page, needs clip-probe's IN_PAGE_LIB), `alignHeadings`, `diffPresence`, - * `formatReport` are exported; the browser is imported lazily so the contract test runs the differ alone. + * Usage: node skills/diff/scripts/content-presence.mjs [--width 1440] + * [--main [=]] [--variable ] [--chrome] [--json []] [--min-cut 2] + * [--settle-passes 4] [--max-findings 40] [--count-words ] [--plain] [--warmup ] + * `presenceInventoryInPage`, `alignHeadings`, `diffPresence`, `formatReport`, `norm` are exported. */ import { mkdirSync, realpathSync, writeFileSync } from 'node:fs'; import { dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; -import { clipInventoryInPage, inPage, summarize } from './clip-probe.mjs'; +import { clipInventoryInPage, inPage, morePrelude, summarize } from './clip-probe.mjs'; import { openBrowser, openPage, visit } from './measure-live.mjs'; const HELP = `content-presence — origin vs served page: visible headings / links / buttons / images / text per band + control state (#125 D2) @@ -76,6 +47,8 @@ Usage: node content-presence.mjs [options] --settle-passes slow-scroll passes until the height is stable (default 4; infinite-scroll origins keep loading — mark those regions --variable) --max-findings printed findings per kind (default 40) + --count-words result-count nouns (default English: results, items, products, coupons, offers, reviews, stores, matches) + --more-words "read more" toggle labels for the clip model (default English) --plain bundled Chromium instead of the window-free real-Chrome tier --warmup visit this URL first on the ORIGIN side (bot-managed sites) --locale default en-US @@ -84,17 +57,17 @@ Exit: 0 clean, 2 any MISSING/HIDDEN link or heading, 1 error, 3 bot challenge, 4 // ---- in-page inventory (needs IN_PAGE_LIB — run via inPage()) ----------------------------------------- /** page.evaluate(inPage(presenceInventoryInPage, { rootSel, variableSels })) → { docH, pageW, root, items[] } */ -export function presenceInventoryInPage({ rootSel = null, variableSels = [] } = {}) { - // Always walk the whole body; each item carries `inRoot` (inside
/ --main). The differ picks the - // scope SYMMETRICALLY: root only when BOTH sides have one, else the whole page — a live origin without - //
against a build with one (recorded) would otherwise compare chrome against content. +export function presenceInventoryInPage({ rootSel = null, variableSels = [], countWords = 'results?|items?|products?|coupons?|offers?|reviews?|stores?|matches' } = {}) { + const COUNT_RE = new RegExp(`\\b(\\d[\\d,]*)\\s+(${countWords})\\b`, 'i'); + // Always walk the whole body; each item carries `inRoot`. The differ picks the scope SYMMETRICALLY: root + // only when BOTH sides have one, else the whole page (a live origin without
against a build with + // one would otherwise compare chrome against content). const root = document.body; const rootEl = (rootSel && document.querySelector(rootSel)) || (!rootSel && document.querySelector('main')) || null; // chrome landmarks: items inside them are flagged so the differ can leave the header / footer to the chrome // crop gate (the chrome repeats on every page and its live promo strips / account state are session-variable) - // Chrome = the landmarks AND everything above the header's bottom edge / below the footer's top edge: - // promo strips and skip links often sit OUTSIDE
(recorded: three session-variable promo links - // at y 11 flagged MISSING on every page). The chrome crop gate judges that region; --chrome includes it. + // Chrome = the landmarks AND everything above the header's bottom edge / below the footer's top edge + // (promo strips often sit OUTSIDE
); the chrome crop gate judges that region, --chrome includes it. const chromeEls = [...document.querySelectorAll('header, [role="banner"], footer, [role="contentinfo"]')].filter((c) => !rootEl || !rootEl.contains(c)); const banner = chromeEls.filter((c) => c.matches('header, [role="banner"]')).map((c) => __pageRect(c.getBoundingClientRect())).filter((r) => r.h > 0); const contentinfo = chromeEls.filter((c) => c.matches('footer, [role="contentinfo"]')).map((c) => __pageRect(c.getBoundingClientRect())).filter((r) => r.h > 0); @@ -166,7 +139,7 @@ export function presenceInventoryInPage({ rootSel = null, variableSels = [] } = if (own.length >= 3 && !hasCountedAncestor(el) && !__controlOf(el)) { textBlocks.add(el); push('text', el, { text: own.slice(0, 80) }); - const m = own.match(/\b(\d[\d,]*)\s+(results?|items?|products?|coupons?|offers?|reviews?|stores?|matches)\b/i); + const m = own.match(COUNT_RE); if (m) push('control', el, { label: `count ${m[2].toLowerCase().replace(/s$/, '')}`, value: m[1].replace(/,/g, ''), via: 'count' }); } } @@ -175,10 +148,9 @@ export function presenceInventoryInPage({ rootSel = null, variableSels = [] } = } // ---- pure: alignment + diff -------------------------------------------------------------------------- -// Matching key: lower-case, trademark / footnote glyphs dropped (® ™ © † ‡ § * and a trailing footnote -// digit — "no annual fee †", "1-hour delivery *", "$50/month. 1"), a bare "tm" token dropped (a TM -// serialises as text on one side and as the glyph on the other — recorded: 17 false MISSING on one page), -// punctuation runs collapsed. Exported for the test. +// Matching key: lower-case; trademark / footnote glyphs and a trailing footnote digit dropped ("no annual +// fee †", "$50/month. 1"); a bare "tm" token dropped (a TM serialises as text on one side and as +// the glyph on the other — 17 false MISSING on one recorded page); punctuation runs collapsed. export const norm = (s) => (s || '') .toLowerCase() .replace(/[®™©†‡§*]/g, ' ') @@ -255,9 +227,8 @@ export function diffPresence(originInv, edsInv, { countTol = 0.25, chrome = fals // links and buttons are ONE pool on the served side (a "Sign in" button served as a link is the same // visible control); the ORIGIN kind decides the severity. const action = (it) => it.kind === 'link' || it.kind === 'button'; - // a text-less link (an image-only anchor without alt) has nothing a visitor reads; its presence is the - // image's (counted there) and its href does not survive the path rewrite of a migration — recorded: - // six product-rail image anchors read MISSING beside their matching text anchors. Text-keyed only. + // a text-less link (an image-only anchor without alt) is the image's presence (counted there); its href + // does not survive a migration's path rewrite — text-keyed only. const oFixed = oItems.filter((it) => it.kind === kind && !it.variable && norm(it.text)); const oVar = oItems.filter((it) => it.kind === kind && it.variable && norm(it.text)); const eVis = countBy(eItems.filter((it) => action(it) && it.state === 'visible'), linkKey); const eHid = countBy(eItems.filter((it) => action(it) && it.state === 'hidden'), linkKey); @@ -290,7 +261,7 @@ export function diffPresence(originInv, edsInv, { countTol = 0.25, chrome = fals let m = null; if (!generic) m = eC.find((x, idx) => !used.has(idx) && norm(x.label) === lab && (used.add(idx) || true)); // ordinal fallback only for GENERIC labels: a specifically labelled trigger with no served counterpart is - // CONTROL MISSING, never paired with an unrelated trigger (recorded: 16 "rebate help" buttons paired with nav menus) + // CONTROL MISSING, never paired with an unrelated trigger if (!m && generic) { const sameVia = eC.map((x, idx) => ({ x, idx })).filter(({ x, idx }) => !used.has(idx) && x.via === c.via); const exact = sameVia.find(({ x }) => norm(x.value) === norm(c.value)); const pick = exact || sameVia[0]; if (pick) { used.add(pick.idx); m = pick.x; } } if (!m) { if (/^count/.test(lab)) { add('🟠', 'CONTROL STATE', `${c.label}: origin "${c.value}", served: none`, { band: k, label: c.label, value: c.value }); continue; } @@ -332,7 +303,7 @@ export function formatReport(res, { maxPerKind = 40 } = {}) { export function parseArgs(argv) { const rest = argv.slice(2); if (rest.length < 2 || rest.includes('--help') || rest.includes('-h')) { console.log(HELP); process.exit(rest.includes('--help') || rest.includes('-h') || !rest.length ? 0 : 1); } - const opts = { origin: null, eds: null, width: 1440, main: null, mainEds: null, variable: [], variableEds: [], json: false, jsonFile: null, minCut: 2, maxFindings: 40, plain: false, warmup: null, locale: 'en-US', chrome: false, settlePasses: 4 }; + const opts = { origin: null, eds: null, width: 1440, main: null, mainEds: null, variable: [], variableEds: [], json: false, jsonFile: null, minCut: 2, maxFindings: 40, plain: false, warmup: null, locale: 'en-US', chrome: false, settlePasses: 4, countWords: null, moreWords: null }; for (let i = 0; i < rest.length; i += 1) { const a = rest[i]; if (a === '--width') opts.width = Number(rest[++i]); @@ -342,6 +313,8 @@ export function parseArgs(argv) { else if (a === '--min-cut') opts.minCut = Number(rest[++i]); else if (a === '--chrome') opts.chrome = true; else if (a === '--settle-passes') opts.settlePasses = Number(rest[++i]); + else if (a === '--count-words') opts.countWords = rest[++i].split(',').map((w) => w.trim()).filter(Boolean).join('|'); + else if (a === '--more-words') opts.moreWords = rest[++i]; else if (a === '--max-findings') opts.maxFindings = Number(rest[++i]); else if (a === '--plain') opts.plain = true; else if (a === '--warmup') opts.warmup = rest[++i]; @@ -354,12 +327,12 @@ export function parseArgs(argv) { } /** Inventory one side: visit + settle, presence inventory, clip inventory. */ -export async function inventorySide(browser, url, { width, locale, warmup, rootSel, variableSels, minCut, settlePasses = 4, httpError = 'measure' }) { +export async function inventorySide(browser, url, { width, locale, warmup, rootSel, variableSels, minCut, settlePasses = 4, httpError = 'measure', countWords = null, moreWords = null }) { const { ctx, page } = await openPage(browser, { width, locale }); try { const v = await visit(page, url, { warmup, settle: { passes: settlePasses }, httpError }); - const inv = await page.evaluate(inPage(presenceInventoryInPage, { rootSel, variableSels })); - const clip = await page.evaluate(inPage(clipInventoryInPage, { minCut, rootSel: null, maxFindings: 400 })); + const inv = await page.evaluate(inPage(presenceInventoryInPage, countWords ? { rootSel, variableSels, countWords } : { rootSel, variableSels }, morePrelude(moreWords))); + const clip = await page.evaluate(inPage(clipInventoryInPage, { minCut, rootSel: null, maxFindings: 400 }, morePrelude(moreWords))); return { url, at: new Date().toISOString(), status: v.status, settlePasses: v.passes, ...inv, clip: { counts: clip.counts, groups: summarize(clip.findings), findings: clip.findings }, textBoxes: clip.textBoxes }; } finally { await ctx.close(); } } @@ -370,11 +343,11 @@ async function main() { const browser = await openBrowser(chromium, { tier: opts.plain ? 'plain' : 'stealth' }); let o; let e; try { - // ORIGIN side fails loud on any HTTP ≥ 400 (a 403 Access-Denied page measured as the origin read "3 texts, - // 0 images → 100 EXTRA on the build" in a recorded run); the SERVED side is measured with a warning (a 404 - // build before preview propagation is the advisory contract). - o = await inventorySide(browser, opts.origin, { width: opts.width, locale: opts.locale, warmup: opts.warmup, rootSel: opts.main, variableSels: opts.variable, minCut: opts.minCut, settlePasses: opts.settlePasses, httpError: 'throw' }); - e = await inventorySide(browser, opts.eds, { width: opts.width, locale: opts.locale, warmup: null, rootSel: opts.mainEds, variableSels: opts.variableEds, minCut: opts.minCut, settlePasses: opts.settlePasses }); + // ORIGIN side fails loud on any HTTP ≥ 400 (a 403 page measured as the origin reads "100 EXTRA on the + // build"); the SERVED side is measured with a warning (a 404 build before preview propagation is the + // advisory contract). + o = await inventorySide(browser, opts.origin, { width: opts.width, locale: opts.locale, warmup: opts.warmup, rootSel: opts.main, variableSels: opts.variable, minCut: opts.minCut, settlePasses: opts.settlePasses, httpError: 'throw', countWords: opts.countWords, moreWords: opts.moreWords }); + e = await inventorySide(browser, opts.eds, { width: opts.width, locale: opts.locale, warmup: null, rootSel: opts.mainEds, variableSels: opts.variableEds, minCut: opts.minCut, settlePasses: opts.settlePasses, countWords: opts.countWords, moreWords: opts.moreWords }); } finally { await browser.close(); } const res = diffPresence(o, e, { chrome: opts.chrome }); const strip = (side) => ({ url: side.url, at: side.at, status: side.status, settlePasses: side.settlePasses, docH: side.docH, root: side.root, items: side.items, clip: { counts: side.clip.counts, groups: side.clip.groups } }); diff --git a/plugins/stardust/skills/diff/scripts/live-session.mjs b/plugins/stardust/skills/diff/scripts/live-session.mjs index 28dc744b4..8ab043fab 100644 --- a/plugins/stardust/skills/diff/scripts/live-session.mjs +++ b/plugins/stardust/skills/diff/scripts/live-session.mjs @@ -225,10 +225,9 @@ function challengeMarker(resp) { if (server.includes('big-ip') || server.includes('imperva') || h['x-iinfo']) return `HTTP ${status} + F5/Imperva edge signature`; // no edge signature — a genuine app-level status, not a challenge. } - // Akamai escalates to HTTP 400 with a JSON body {"result":"Bad Request"} after a - // burst of headless probes (recorded, walgreens 2026-09-18 — feedback A2): the - // instruments correctly refused to measure it but never suggested escalation. A - // 400 stamped by AkamaiGHost is a challenge marker, not an app-level status. + // Akamai escalates to HTTP 400 {"result":"Bad Request"} after a burst of headless + // probes (#125): the instruments refused to measure it but never suggested + // escalation. A 400 stamped by AkamaiGHost is a challenge marker. if (status === 400) { const server = (h.server || '').toLowerCase(); if (server.includes('akamaighost') || server.includes('akamai') || h['x-akamai-transformed']) return 'HTTP 400 + Akamai edge signature (bot-management escalation)'; @@ -320,14 +319,11 @@ export async function gotoLive(page, url, { waitUntil = 'domcontentloaded', time * probes for. Pair with newLiveContext so the navigator.webdriver spoof lands * on every context. Takes the caller's `chromium` so this module stays import-free. * - * WINDOW-FREE BY DEFAULT (#125, walgreens 2026-09-18 — feedback A1). The visible - * window was never the ingredient that cleared the block, only the binary was: - * verified `channel:'chrome'` + headless gets HTTP 200 from Akamai where the - * bundled headless Chromium gets 400 {"result":"Bad Request"}. Visible windows - * blocked the operator's desktop (four agents probing in parallel — "browsers - * keep popping up"). A visible window is opt-in: `STARDUST_HEADED_WINDOW=1`, - * for the rare challenge that genuinely needs a human. Every `--headed` flag in - * the skills means THIS tier; none of them opens a window on its own. + * WINDOW-FREE BY DEFAULT (#125). The window was never what cleared the block, the + * binary was: `channel:'chrome'` + headless gets HTTP 200 from Akamai where bundled + * headless Chromium gets 400. Visible windows blocked an operator's desktop under + * four parallel agents. A window is opt-in: `STARDUST_HEADED_WINDOW=1`, for a + * challenge that genuinely needs a human. Every `--headed` flag means THIS tier. * * `channel: 'chrome'` needs Google Chrome installed; when the launch fails the * bundled Chromium is launched instead with the same stealth args and a loud diff --git a/plugins/stardust/skills/diff/scripts/measure-live.mjs b/plugins/stardust/skills/diff/scripts/measure-live.mjs index 5989e0c10..3ba3f3e50 100644 --- a/plugins/stardust/skills/diff/scripts/measure-live.mjs +++ b/plugins/stardust/skills/diff/scripts/measure-live.mjs @@ -1,42 +1,19 @@ #!/usr/bin/env node /* eslint-disable import/no-extraneous-dependencies, import/extensions, no-await-in-loop, no-restricted-syntax, brace-style, object-curly-newline, max-len, no-console, no-continue, no-nested-ternary, no-plusplus, object-property-newline */ /** - * skills/diff/scripts/measure-live.mjs + * skills/diff/scripts/measure-live.mjs — measure a live page the way the converging passes did (#125): + * window-free real Chrome (live-session), optional home warm-up for bot-managed sites, a slow-scroll + * settle repeated until the document height is stable, then rect + computed type per element, + * shadow-DOM aware, cached per slug under stardust/current/measure/.json. * - * Measure a LIVE page (or a served build) the way the passes that converged did (#125): - * real Chrome, window-free (live-session launchStealthHeaded), an optional home warm-up for - * bot-managed sites (the Akamai sensor cookies must exist before a deep URL is hit — a - * recorded PDP probe 403'd without it), a SLOW-SCROLL SETTLE repeated until the document - * height is stable (lazy rails, gliders, ratings widgets hydrate on viewport entry — a - * settled live page ran 1.6–2k px taller than any single capture), then rect + computed - * type per element, shadow-DOM aware. Results are cached per slug under - * `stardust/current/measure/.json` — the egress that reaches a blocked section may - * vanish, so a measurement is evidence worth keeping. + * Library for clip-probe, content-presence and unit-geometry (every gate probe settles both sides + * with THIS routine — the instrument stays symmetric): openBrowser, openPage, visit, settle, + * measureInPage, serializeInPage, cachePath / readCache / writeCache. * - * Library (imported by clip-probe.mjs, content-presence.mjs, unit-geometry.mjs — every probe - * of the published-origin gate settles both sides with THIS routine, so the instrument stays - * symmetric): - * openBrowser(chromium, { tier }) 'stealth' (default, window-free real Chrome) | 'plain' - * openPage(browser, { width, height, locale }) - * visit(page, url, { warmup, settle, dismiss, solveWindow }) → { status, docH, passes } - * settle(page, { passes, step, dwell, emptySel, quietMs }) → { docH, passes, pendingImgs } - * measureInPage / serializeInPage in-page functions (page.evaluate) - * cachePath(slug), readCache(slug), writeCache(slug, data) - * - * CLI: - * node skills/diff/scripts/measure-live.mjs [ …] [options] - * --width viewport width (default 1440) - * --all every match per selector (capped at 60), default first match - * --serialize also dump the element deep-serialised (shadow roots expanded, - * data-r="x,y,w,h" data-t="fs/lh/fw/color[/bg]" per element) - * --warmup visit this URL first in the same context (home warm-up) - * --plain bundled Chromium instead of the stealth real-Chrome tier - * --locale Accept-Language + context locale (default en-US) - * --slug cache the result at stardust/current/measure/.json (--force to redo) - * --json also write the result to this file - * - * Requires: playwright (project devDependency — the setup step writes it into package.json, - * never `npm i --no-save`, feedback A3). Exit 0 measured, 1 error, 3 bot challenge. + * Usage: node skills/diff/scripts/measure-live.mjs [ …] [--width 1440] [--all] + * [--serialize ] [--warmup ] [--plain] [--locale en-US] [--slug ] [--force] + * [--json ] + * Exit: 0 measured, 1 error, 3 bot challenge. Requires playwright (a project devDependency). */ import { existsSync, mkdirSync, readFileSync, realpathSync, writeFileSync } from 'node:fs'; import { dirname, join } from 'node:path'; diff --git a/plugins/stardust/skills/diff/scripts/test/clip-probe.test.mjs b/plugins/stardust/skills/diff/scripts/test/clip-probe.test.mjs index d713701b3..2900ad5d7 100644 --- a/plugins/stardust/skills/diff/scripts/test/clip-probe.test.mjs +++ b/plugins/stardust/skills/diff/scripts/test/clip-probe.test.mjs @@ -3,7 +3,7 @@ // summarize groups findings by kind + clipper + parent and keeps advisory kinds apart, formatTable hides advisory // rows unless asked, verdictLine, parseArgs, --help in an empty cwd. Part (b), where playwright is importable: // a fixture page served from this process — a 238 px card with overflow:hidden whose description is cut -// mid-line and whose "View details" link sits below the box (the recorded offers defect), a line-clamped +// mid-line and whose "View details" link sits below the box (the recorded card defect), a line-clamped // paragraph (advisory only), a collapsed mega-menu (max-height 0 — must not count), an sr-only span (must not // count), a horizontal carousel track (advisory only), a "Read more" collapsible (advisory TEXT COLLAPSED) — and // a clean page (exit 0). Run: node . diff --git a/plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs b/plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs index cde25b286..ca27cb9f2 100644 --- a/plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs +++ b/plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs @@ -22,7 +22,7 @@ check('keyed: role:text keys, selector#ordinal for text-less, #2 for duplicates' const k = keyed([el('text', 'Expires', 0, 0, 10, 10), el('image', '', 0, 0, 10, 10), el('image', '', 0, 0, 10, 10), el('text', 'Expires', 0, 0, 10, 10)]); assert.deepEqual(k.map((x) => x.key), ['text:expires', 'image:img#1', 'image:img#2', 'text:expires #2']); }); -check('alignUnit: the recorded offers card — body 28 px low, details link hidden, badge paired by position, image by order', () => { +check('alignUnit: the recorded card — body 28 px low, details link hidden, badge paired by position, image by order', () => { const o = { rect: { x: 165, y: 500, w: 350, h: 260 }, elements: [el('text', '1 day left', -10, -10, 70, 70), el('image', '', 17, 76, 70, 70), el('text', 'Expires 09/26/26', 112, 12, 202, 15), el('text', 'Earn $5 W Cash rewards when you spend', 112, 27, 202, 54), el('link', 'View details', 112, 176, 91, 24), el('button', 'Clip', 12, 204, 326, 44)] }; const e = { rect: { x: 165, y: 522, w: 355, h: 238 }, elements: [el('text', '2 days left', -27, -30, 64, 64), el('image', 'coupon', 11, 46, 65, 65), el('text', 'Expires 09/26/26', 101, 16, 242, 20), el('text', 'Earn $5 W Cash rewards when yo…', 101, 55, 215, 48), el('link', 'View details', 101, 204, 91, 20, { hidden: true }), el('link', 'Clip', 12, 182, 331, 44)] }; const r = alignUnit(o, e, 4); diff --git a/plugins/stardust/skills/diff/scripts/unit-geometry.mjs b/plugins/stardust/skills/diff/scripts/unit-geometry.mjs index aef1a4b9f..d5c8d9fb4 100644 --- a/plugins/stardust/skills/diff/scripts/unit-geometry.mjs +++ b/plugins/stardust/skills/diff/scripts/unit-geometry.mjs @@ -4,34 +4,21 @@ /** * skills/diff/scripts/unit-geometry.mjs — per-unit GEOMETRY compare for repeated units (#125, D3). * - * A card, a rail item, an FAQ row: the pixel gate sees the grid of shapes, not whether the badge sits on - * the card corner or on its own row, whether the body starts 30 px too low, whether the details link is - * where the origin puts it. The category, PDP and topic workstreams of the recorded run converged only - * after MEASURING live element rects (`*-measure.mjs`); this probe makes that measurement a gate row: - * for the first N matching units on the origin and on the served page it dumps the rects of the inner - * elements (by role + text, shadow-DOM aware), aligns them, and reports per element Δx / Δy / Δw / Δh - * RELATIVE TO THE UNIT's top-left corner against a tolerance (default 4 px), plus the unit's own size - * and page position delta. Elements the served page clips away (clip-probe model) read `hidden`. + * A card, a rail item, an FAQ row: the pixel gate sees the grid of shapes, not whether the badge sits + * on the corner or the body starts 30 px low. The workstreams that converged measured live element + * rects; this makes that a gate row: for the first N matching units on both sides, the rects of the + * inner elements (role + text, shadow-DOM aware) relative to the unit's corner, aligned (key, text + * prefix, relaxed link ↔ button, images by order, leftover text by position) and reported as + * Δx / Δy / Δw / Δh against --tol, plus the unit's own size / position delta; `hidden` where the + * served page clips. Both sides settle the same way; the origin inventory caches per slug under + * stardust/current/measure/-units.json. * - * Both sides load in the same window-free real-Chrome tier and settle with measure-live's slow scroll - * (symmetric instrument). The origin inventory can be cached per slug under - * `stardust/current/measure/-units.json` (the egress that reaches a blocked section may vanish). - * - * Usage: - * node skills/diff/scripts/unit-geometry.mjs --unit "=" [options] - * --unit = repeatable; one selector when both sides share it - * --n units per selector (default 1; the first N visible matches in DOM order) - * --tol per-element tolerance on |Δx|, |Δy|, |Δw|, |Δh| (default 4) - * --width viewport width (default 1440) - * --json [] JSON on stdout or to - * --slug cache / reuse the ORIGIN inventory (--force to re-measure) - * --advisory exit 0 even with elements off / missing (report only) - * --plain | --warmup | --locale as in measure-live - * - * Exit: 0 every element within tolerance, 2 any element off / missing / hidden (or a unit missing on one - * side), 1 error, 3 bot challenge. gate-all runs it only for the units a project declares in units.json - * (block families that repeat: cards, rails, lists, FAQs). `unitInventoryInPage`, `alignUnit`, - * `formatUnits` are exported; the browser is imported lazily. + * Usage: node skills/diff/scripts/unit-geometry.mjs --unit "=" … + * [--n 1] [--tol 4] [--width 1440] [--json []] [--slug ] [--force] [--advisory] + * [--plain] [--warmup ] [--locale en-US] + * Exit: 0 within tolerance, 2 any element off / hidden / missing, 1 error, 3 bot challenge. + * gate-all runs it for the units units.json declares. `unitInventoryInPage`, `alignUnit`, + * `keyed`, `compareUnits`, `formatUnits`, `verdictOf` are exported. */ import { mkdirSync, realpathSync, writeFileSync } from 'node:fs'; import { dirname } from 'node:path'; diff --git a/plugins/stardust/skills/extract/SKILL.md b/plugins/stardust/skills/extract/SKILL.md index 6a32ca5c9..e50cc95a3 100644 --- a/plugins/stardust/skills/extract/SKILL.md +++ b/plugins/stardust/skills/extract/SKILL.md @@ -90,9 +90,8 @@ Additional checks for this sub-command: import-resolvable from the project root (probe: `node -e "import('playwright').then(()=>process.exit(0))"`); if it isn't, run `npm i -D playwright pixelmatch pngjs cheerio --legacy-peer-deps` - — as devDependencies written into `package.json`, NOT `--no-save` (#125, - feedback A3: two `--no-save` installs in one recorded run were pruned by - the next real `npm i`) — or use the Playwright MCP server, before crawling. The + (devDependencies, never `--no-save` — #125) or use the Playwright MCP + server, before crawling. The `--legacy-peer-deps` flag is required on `aem-boilerplate` targets (their pinned `eslint@8` makes a plain `npm i` exit `ERESOLVE` before playwright is even considered). Don't trust the CLI diff --git a/plugins/stardust/skills/replica/SKILL.md b/plugins/stardust/skills/replica/SKILL.md index f0a4d386c..47e2768fb 100644 --- a/plugins/stardust/skills/replica/SKILL.md +++ b/plugins/stardust/skills/replica/SKILL.md @@ -51,16 +51,11 @@ eyeballing. § Flow keys): invoking `replica` is the choice. 2. Verify Playwright is importable from the project root (extract needs it; so do the gate scripts). -3. Install the gate's deps in the project AS devDependencies — written into - `package.json`, never `--no-save`: - `npm i -D playwright pixelmatch pngjs cheerio --legacy-peer-deps`. - A `--no-save` install is PRUNED by any later real `npm i` (recorded twice in - one run: playwright vanished mid-gate when cheerio was added — feedback A3, - #125); re-probe before every gate run anyway - (`node -e "import('pixelmatch').then(()=>process.exit(0))"`). Run every - probe from the project root: ESM resolves `playwright` from the script's - own location, so an ad-hoc probe written elsewhere fails with - `ERR_MODULE_NOT_FOUND`. +3. Install the gate's deps in the project AS devDependencies, never + `--no-save`: `npm i -D playwright pixelmatch pngjs cheerio --legacy-peer-deps` + (a `--no-save` install is pruned by the next real `npm i` — recorded twice + in one run, #125). Run every probe from the project root: ESM resolves + `playwright` from the script's own location. 4. Copy scripts into the project and run them from there, not from the plugin: this skill's whole `scripts/` dir to `stardust/scripts/replica/`, the master skill's `../stardust/scripts/` @@ -422,22 +417,14 @@ had one section for the whole run). `stardust/replica/gates/-/` dir under the `pub` label (a new dir would force a fresh live capture). Only the published number counts. -- **The delivery gate is the ALL-PAGES run, four criteria, not the pixel +- **The delivery gate is the ALL-PAGES run, four criteria — not the pixel number alone** (`reference/source-fidelity-gate.md` § The all-pages - published-origin gate, #125): once the roster is deployed, - `node stardust/scripts/replica/gate-all.mjs [--only ] [--skip-existing] - [--eds-host ]` (through `run-bg.mjs`, it is a long instrument) - captures every `deployed` page on both sides, and a page is DELIVERED when - pixel % ≤ 10 AND |Δh| ≤ 5 % of the origin height AND `clip-probe` counts 0 - clipped / hidden text and controls on the served page AND - `content-presence` reports 0 MISSING / HIDDEN links and headings against the - live origin (`units.json`-declared repeated units within 4 px on top). A - recorded page passed the pixel bar at 6.7 % with all 284 of its cards - clipped and their links hidden — the pixel gate proves shapes, the two DOM - probes prove elements. Evidence: `stardust/replica/gates/all-/ - /` + `summary.{json,md}` (pixel-only and full verdicts side by side — - the calibration pair); sidecars `masks.json`, `overrides.json`, - `clip-allow.json`, `presence.json`, `units.json`, each entry documented. + published-origin gate, #125): `node stardust/scripts/replica/gate-all.mjs + [--only ] [--skip-existing] [--eds-host ]` through `run-bg.mjs`. + DELIVERED = pixel % ≤ 10 AND |Δh| ≤ 5 % AND 0 clipped text / controls + (`clip-probe`) AND 0 MISSING / HIDDEN links / headings (`content-presence`); + a recorded page passed the pixel bar with all of its cards clipped. Evidence + and sidecars: `stardust/replica/gates/all-/`. **State:** replica writes its own state under `stardust/replica/` — the inconsistency register, `progress.json` (per page type: archetype slug, diff --git a/plugins/stardust/skills/replica/reference/source-fidelity-gate.md b/plugins/stardust/skills/replica/reference/source-fidelity-gate.md index bf6d53550..48273ec82 100644 --- a/plugins/stardust/skills/replica/reference/source-fidelity-gate.md +++ b/plugins/stardust/skills/replica/reference/source-fidelity-gate.md @@ -733,73 +733,57 @@ none visible on a local harness): ## The all-pages published-origin gate (gate-all) — four criteria (#125) -**How the 2026-09 walgreens run motivated this.** The rollout gated its five archetypes and -shipped 78 sibling pages unmeasured; the owner's first feedback was that the migrated pages -"don't look formatted as the original". A project-grown runner then gated all 96 deployed pages -(stitched 1440 origin + served captures, pixel-compare) and a fix program took the roster from -3 to 67 PASS. Then `/offers/offers` PASSED that gate at **6.73 % / Δh 69 px** against -`https://www.walgreens.com/offers/offers.jsp` while every one of its 284 coupon cards was -visibly broken: the card is a fixed 238 px with `overflow: hidden`, the body starts ~30 px too -low, the description is clipped mid-glyph and the "View details" link is pushed under the Clip -button on all 284 cards — and the toolbar reads "Expiration Date" where the origin reads -"Recommended". The links ARE in the served DOM (284 in the fragments): pure CSS geometry. -Pixelmatch underweights small text inside otherwise matching shapes; the text-only number on -that page (`--text-boxes`, D4) reads **16.5 %** against the 6.7 % full-page number. "PASS" -meant "right shapes at the right places", not "every element present and legible". Element -presence and legibility are checkable — so `gate-all.mjs` checks them. - -`node stardust/scripts/replica/gate-all.mjs` (from the project root; `../scripts/gate-all.mjs` -in the plugin) runs, for every `deployed` page in `stardust/state.json`, the stitched captures -and pixel-compare of the archetype gate PLUS two DOM probes that load both pages in the same -window-free real-Chrome tier and settle them the same way (measure-live.mjs), and writes +**The recorded case.** A 96-page rollout of a pharmacy retailer gated its five archetypes and +shipped the other pages unmeasured; the owner's first feedback was that they "don't look +formatted as the original". A runner then gated every deployed page on stitched 1440 captures +and pixel-compare, and a fix program took the roster from 3 to 67 PASS. Then the coupon-listing +page PASSED at **6.7 % / Δh 69 px** while every one of its 284 cards was broken: a fixed-height +card with `overflow: hidden`, the body ~30 px too low, the description clipped mid-glyph, the +details link pushed under the primary button — and the sort control reading a different default +than the origin. The links WERE in the served DOM; pure CSS geometry. Pixelmatch underweights +small text inside otherwise matching shapes (the text-only number, `--text-boxes`, read 16.5 %). +"PASS" meant "right shapes at the right places", not "every element present and legible". +Element presence and legibility are checkable — so `gate-all.mjs` checks them. + +`node stardust/scripts/replica/gate-all.mjs` (project root; through `run-bg.mjs`) runs, for +every `deployed` page in `stardust/state.json`, the stitched captures and pixel-compare of the +archetype gate PLUS two DOM probes that load both pages in the same window-free real-Chrome tier +and settle them the same way (`measure-live.mjs`), and writes `stardust/replica/gates/all-//{origin,eds,diff}.png + pixel.json + content.json + clip.json [+ units.json]` and `summary.{json,md}`. **Verdict, per page — all four:** -1. **pixel** — overlap pixel % ≤ `--threshold` (default 10). +1. **pixel** — overlap % ≤ `--threshold` (default 10). 2. **height** — |Δh| ≤ `--height-tol` × origin height (default 5 %). The overlap crop cannot see - a render thousands of px too tall (9 pages 100–6900 px too tall passed the % alone); a - padded/union metric was tried and REJECTED — white gaps score as matches — so - `pixel-compare --pad` stays an auxiliary number and the guard is explicit. -3. **clip** — served-side `clip-probe` count ≤ `--clip-max` (default 0) plus the page's - documented allowance in `clip-allow.json`. Counted: TEXT CLIPPED (a line cut across), - TEXT HIDDEN (whole lines behind an overflow-hidden ancestor without a line-clamp), - CONTROL HIDDEN / CLIPPED (a link or button outside / cut by its clipping ancestor). - Advisory, never counted: line-clamp truncation, scrollable containers, horizontal cuts - (carousel tracks, ellipsis), collapsed menus, sr-only boxes. On the offers page: 379 - (284 "View details" anchors, 95 description lines); on the known-good pages: 0. -4. **content** — `content-presence` MISSING + HIDDEN links and h1–h3 headings = 0. Origin - and served inventories of VISIBLE headings, links (by visible text), buttons, images and - text blocks, aligned into bands by the h1–h3 sequence, plus control STATE (select values, - checked radios, `aria-selected` tabs, `aria-haspopup` triggers, result counts). "Present in - the DOM but clipped past 50 %" is HIDDEN, not present. Buttons, control state and count - deltas are reported in the row, not blocking (Clip / Clipped labels and counts are - session-variable). Session-variable regions (rails, coupon walls, reviews) are declared in - `presence.json` and compared as counts — HIDDEN still counts there, geometry is not - session-variable. When the origin cannot be probed (edge 403 from this egress, bot - challenge) the row reads `content: n/a ()` and the clip criterion still runs on the - served page — n/a never fails a page and never passes it silently. - (+ **units** — `unit-geometry` off / hidden / missing = 0 for the units `units.json` marks - `required: true`; other declared units are advisory rows. On the offers card it names the - defect in one table: every body element Δy +28…+47, the details link Δy +28 AND hidden, - the badge 70×70 at (−10,−10) vs 64×64 at (−27,−30), the card 260 → 238 px.) - -The summary records both the pixel-only verdict (1 + 2) and the full verdict per page and in -the totals — every run is calibration data for the bar. Sidecars in the gate dir, each entry -documented: `masks.json` (row masks for capture artefacts, printed on the verdict, never -reported unmasked), `overrides.json` (documented verdicts shown BESIDE the measured number, -never replacing it — "live origin drifted since capture; 9.14 % vs the gated capture"), -`clip-allow.json`, `presence.json`, `units.json`. Origin fallback chain: live stitch → -previous origin (`--recapture-origin`) → crawl fullPage screenshot (`crawl-fullpage`, -ASYMMETRIC, flagged in the row). `--eds-host` gates a code branch against the same DA -content (branch hosts serve code from the LITERAL branch name — `fix/x` → push a mirror -`fix-x`; served CSS/JS are gzip, verify with `curl --compressed`). Single-page reads: + a render thousands of px too tall; a padded/union metric was tried and rejected (white gaps + score as matches), so the guard is explicit and `pixel-compare --pad` stays auxiliary. +3. **clip** — served-side `clip-probe` count ≤ `--clip-max` (default 0) + the page's allowance + in `clip-allow.json`. Counted: a text line cut across, whole lines behind an overflow-hidden + ancestor without a line-clamp, a link or button outside or cut by its clipping ancestor. + Advisory: line-clamp, "read more" collapsibles, scrollable containers, horizontal cuts. +4. **content** — `content-presence` MISSING + HIDDEN links and h1–h3 headings = 0. Visible + headings, links (by visible text), buttons, images, text blocks per band (h1–h3 sequence + aligned by text) plus control STATE. Clipped past 50 % = HIDDEN, not present. Buttons, + control state and count deltas are reported, not blocking. Regions declared in + `presence.json` (rails, coupon walls, reviews) compare as counts — HIDDEN still counts. + Origin not probeable (edge 403, bot challenge) → `content: n/a ()`, never a fail, + never a silent pass. + (+ **units** — `unit-geometry` off / hidden / missing = 0 for `units.json` entries marked + `required: true`; others are advisory rows.) + +The pixel-only verdict (1 + 2) is recorded beside the full one per page and in the totals: every +run is calibration data. Sidecars in the gate dir, each entry documented: `masks.json` (printed +on the verdict, never reported unmasked), `overrides.json` (shown BESIDE the measured number, +never replacing it), `clip-allow.json`, `presence.json`, `units.json`. Origin fallback: live +stitch → previous origin (`--recapture-origin`) → crawl fullPage screenshot (`crawl-fullpage`, +asymmetric, flagged). `--eds-host` gates a code branch against the same content (branch hosts +serve the LITERAL branch name — `fix/x` → push a mirror `fix-x`). Read a finding with `sbs-crop.mjs origin.png eds.png out.png --y --height ` — one crop per fact. -Calibration on the walgreens roster (96 pages, 2026-09-25, same captures): see the run's -`summary.md` "Calibration" line — pixel-only PASS vs full PASS, and the failing-by-criterion -counts. Open questions recorded there: the default tolerances (clip-max 0, unit-tol 4 px, the -25 % count tolerance) and how a project declares repeated-unit block families (today: -`units.json` per page; candidate: a `repeatedUnit: true` flag on the block inventory). +Calibration on the recorded roster (96 pages, same captures): pixel-only 66 PASS → full 28 PASS; +failing by criterion pixel 30, height 6, clip 23, content 31; content n/a on 37 (edge-blocked +origins). Open: the default tolerances (`--clip-max 0`, `--unit-tol 4`, the 25 % count +tolerance), how a project declares repeated-unit block families (`units.json` per page today), +and whether buttons and control state should block once session-variable labels are modelled. ## Residual logging format diff --git a/plugins/stardust/skills/replica/scripts/gate-all.mjs b/plugins/stardust/skills/replica/scripts/gate-all.mjs index 939b2ef69..0b5318b66 100644 --- a/plugins/stardust/skills/replica/scripts/gate-all.mjs +++ b/plugins/stardust/skills/replica/scripts/gate-all.mjs @@ -3,75 +3,32 @@ /** * skills/replica/scripts/gate-all.mjs — the published-origin gate over EVERY deployed page (#125, D0). * - * Why a runner. A recorded rollout (96 pages) had pixel-gated its 5 archetypes and shipped the other 78 - * pages from the generic encoder unmeasured; the owner's first feedback was that the migrated pages "don't - * look formatted as the original". The per-page gate (gate.sh) is the right instrument for the archetype - * loop; delivery needs ONE run over the roster that captures, compares and probes every deployed page - * the same way and writes one summary. This is that run, lifted from the project that grew it. + * Per `deployed` page in stardust/state.json (url = source, liveUrl = served): stitched captures of + * both sides (stitch-shot, --settle), pixel-compare, then content-presence (D2, carries clip-probe's + * inventory for both sides, D1) and — for units.json entries — unit-geometry (D3). Writes + * //{origin,eds,diff}.png + pixel.json + content.json + clip.json [+ units.json], + * /summary.{json,md}; --only runs write /runs/-.json instead. * - * Per `deployed` page in stardust/state.json (slug, url = source, liveUrl = served, deployedPath): - * //origin.png stitch-shot of the source page (window-free real Chrome, --settle) - * //eds.png stitch-shot of the served page, same flags (symmetric instrument) - * //pixel.json pixel-compare --json (overlap %, Δh, bands; --text-boxes % when D2 ran) - * //content.json content-presence: visible headings / links / buttons / images / text per band, - * control state, both sides' clip inventories (D2 + D1) - * //clip.json the served side's clip counts + groups (from content.json, or clip-probe alone - * when the origin could not be probed) - * //units.json unit-geometry rows when units.json declares repeated units for the page (D3) - * /summary.{json,md} one row per page; --only runs write /runs/-.json instead + * VERDICT (all four): pixel % ≤ --threshold AND |Δh| ≤ --height-tol × origin height (a padded/union + * metric was tried and rejected — white gaps score as matches) AND served clipped ≤ --clip-max + + * clip-allow.json allowance AND content MISSING + HIDDEN links / headings = 0 (n/a when the origin + * could not be probed — never a fail) [+ required units within --unit-tol]. The pixel-only verdict is + * recorded beside it per page and in the totals — every run is calibration data. * - * VERDICT (all four, per page) — "pixel PASS" used to mean "right shapes at the right places", not "every - * element present and legible" (the offers example in source-fidelity-gate.md § The all-pages gate): - * 1. pixel overlap pixel % ≤ --threshold (default 10) - * 2. height |Δh| ≤ --height-tol × origin height (default 5 %) — the overlap crop cannot see a render - * thousands of px too tall; a padded/union metric was tried and REJECTED (white gaps score as - * matches — 9 pages 100–6900 px too tall passed it); pixel-compare --pad stays auxiliary - * 3. clip served-side clipped count ≤ --clip-max (default 0) + the page's clip-allow.json allowance - * 4. content content-presence MISSING + HIDDEN links / headings = 0 (control state, buttons, counts are - * reported in the row, not blocking); n/a — not a fail — when the origin could not be probed - * (+ units unit-geometry off / hidden / missing = 0 for the units units.json marks "required": true; - * other declared units are advisory rows) - * Both the pixel-only verdict (1 + 2) and the full verdict are recorded per page and totalled, so a run - * doubles as calibration data for the bar. + * Sidecars in /, each entry documented: masks.json (printed on the verdict), overrides.json + * (shown BESIDE the number, never replacing it), clip-allow.json, presence.json (session-variable + * regions), units.json (repeated units). Origin fallback: live stitch → previous origin + * (--recapture-origin) → /.png (`crawl-fullpage`, asymmetric, flagged). * - * Sidecar files in / (all optional, all documented per entry — a masked or overridden number is never - * reported as a plain one): - * masks.json { slug: { masks: ['yA:h[@yB]', …], reason } } pixel-compare --mask, printed on the verdict - * overrides.json { slug: { verdict, reason, evidence } } shown BESIDE the measured verdict, never replacing it - * clip-allow.json { slug: { max, reason } } documented clip allowance (a live page that clips) - * presence.json { slug: { variable: ['selO=selE', …], main: 'selO=selE', reason } } session-variable regions - * units.json { slug: { units: [{ origin, eds, n, required }] } } repeated-unit declarations - * - * Origin fallback chain: live stitch → (with --recapture-origin) the previous origin.png → the crawl fullPage - * screenshot /.png (instrument `crawl-fullpage`, ASYMMETRIC, flagged in the row and the - * summary — lazy rails may be placeholders there). --blocked names origins the edge denies from this - * egress (they go straight to the crawl shot unless --try-blocked). - * - * Usage: - * node skills/replica/scripts/gate-all.mjs [options] (run from the project root) - * --state stardust/state.json - * --out stardust/replica/gates/all- - * --width 1440 - * --only subset; writes runs/-.json, leaves summary.* alone - * --skip-existing keep origin.png / eds.png already in the page dir - * --recapture-eds with --skip-existing: keep origin.png, re-shoot eds.png (post-fix re-gate) - * --recapture-origin re-shoot origin.png too (the previous one is kept when the re-shoot fails) - * --eds-host gate a code branch against the same DA content (host of liveUrl replaced; - * branch hosts serve code from the LITERAL branch name — fix/x → push fix-x) - * --blocked / --try-blocked / --crawl-shots origin fallback (above) - * --origin-concurrency 2 --eds-concurrency 4 --probe-concurrency 2 - * --threshold 10 --height-tol 0.05 --clip-max 0 --unit-tol 4 - * --no-clip / --no-content / --no-probes drop criteria 3 / 4 / both (pixel-only verdict) - * --units default /units.json - * --compare-only no captures; recompute compares (+ probes unless --no-probes) from existing PNGs - * --warmup origin warm-up URL for the probes (bot-managed sites) - * --vh stitch-shot chunk height passthrough - * - * Requires: playwright, pixelmatch, pngjs (project devDependencies — the setup step writes them into - * package.json, never `npm i --no-save`). stitch-shot.mjs and pixel-compare.mjs next to this file; the diff - * skill's scripts dir alongside (../../diff/scripts or ../diff — the replica Setup copies both). - * Exit: 0 every gated page PASS (overrides count), 2 any FAIL, 1 error. `verdict` and `formatSummary` are - * exported for the contract test. + * Usage: node skills/replica/scripts/gate-all.mjs [--state f] [--out dir] [--width 1440] + * [--only ] [--skip-existing] [--recapture-eds] [--recapture-origin] [--eds-host ] + * [--blocked ] [--try-blocked] [--crawl-shots ] [--origin-concurrency 2] + * [--eds-concurrency 4] [--probe-concurrency 2] [--threshold 10] [--height-tol 0.05] + * [--clip-max 0] [--unit-tol 4] [--no-clip] [--no-content] [--no-probes] [--units ] + * [--compare-only] [--warmup ] [--vh ] + * Requires playwright, pixelmatch, pngjs (project devDependencies); stitch-shot.mjs and + * pixel-compare.mjs next to this file; the diff skill's scripts dir alongside. Exit: 0 all PASS + * (overrides count), 2 any FAIL, 1 error. `verdict`, `formatSummary`, `parseArgs` are exported. */ import { spawn } from 'node:child_process'; import { copyFileSync, existsSync, mkdirSync, readFileSync, realpathSync, writeFileSync } from 'node:fs'; diff --git a/plugins/stardust/skills/replica/scripts/pixel-compare.mjs b/plugins/stardust/skills/replica/scripts/pixel-compare.mjs index b9cfcb655..d4635dc36 100644 --- a/plugins/stardust/skills/replica/scripts/pixel-compare.mjs +++ b/plugins/stardust/skills/replica/scripts/pixel-compare.mjs @@ -47,9 +47,8 @@ * one per side under `textBoxes`); the differing-pixel * share is re-computed over those boxes only and * reported as `textPct`. Pixelmatch underweights small - * text inside otherwise matching shapes — on the - * recorded offers page 284 clipped card descriptions - * cost < 1 point of the full-page number. Not a verdict + * text inside otherwise matching shapes (a recorded page + * read 6.7 % full-page, 16.5 % text-only). Not a verdict * criterion; summary.md carries it beside the pixel %. * --timeout hard wall-clock deadline (default 120; 0 disables). * Enforced from a supervising process (the compare diff --git a/plugins/stardust/skills/replica/scripts/test/gate-all.test.mjs b/plugins/stardust/skills/replica/scripts/test/gate-all.test.mjs index 30d6eb695..d4cd0a8cd 100644 --- a/plugins/stardust/skills/replica/scripts/test/gate-all.test.mjs +++ b/plugins/stardust/skills/replica/scripts/test/gate-all.test.mjs @@ -26,7 +26,7 @@ const px = (pct, heightDelta, height = 10000) => ({ pct, heightDelta, compared: const clip = (total) => ({ counts: { total, textClipped: total, textHidden: 0, controlHidden: 0, controlClipped: 0 } }); const content = (missing, hidden, controlState = 0) => ({ totals: { missing, hidden, controlState } }); -check('the recorded offers page: pixel PASS, elements FAIL — pixelOnlyPass true, pass false, both reasons named', () => { +check('the recorded card page: pixel PASS, elements FAIL — pixelOnlyPass true, pass false, both reasons named', () => { const v = verdict({ pixel: px(6.68, 69, 29326), clip: clip(379), content: content(0, 284, 2) }); assert.equal(v.pixelOnlyPass, true); assert.equal(v.pass, false); assert.match(v.reasons.join('; '), /clipped 379/); assert.match(v.reasons.join('; '), /HIDDEN 284/); assert.match(v.reasons.join('; '), /2 control state/); diff --git a/plugins/stardust/skills/rollout/SKILL.md b/plugins/stardust/skills/rollout/SKILL.md index 404fbcd62..e0991f7c7 100644 --- a/plugins/stardust/skills/rollout/SKILL.md +++ b/plugins/stardust/skills/rollout/SKILL.md @@ -391,14 +391,9 @@ decoration ran: the runtime's `body.appear` class is set (per zero `pageerror` events, zero broken images. **Published-origin gate over the roster (#125).** `verify` proves the pages render; the -DELIVERY verdict is replica's all-pages gate: `node stardust/scripts/replica/gate-all.mjs ---skip-existing` (from the project root, through `run-bg.mjs`) — stitched captures of the -live origin and the served page per deployed page, pixel % ≤ 10 AND |Δh| ≤ 5 % AND 0 clipped -text / controls (`clip-probe`) AND 0 MISSING / HIDDEN links / headings (`content-presence`). -`summary.md` carries the pixel-only and the full verdict per page; record `delivery.gate` in -`coverage/pages.json` from it, never from the pixel number alone (a recorded page passed at -6.7 % with all 284 of its cards clipped). Rules: `../replica/reference/source-fidelity-gate.md` -§ The all-pages published-origin gate. +DELIVERY verdict is replica's `gate-all.mjs` (pixel + height + clip + content per deployed page, +`../replica/reference/source-fidelity-gate.md` § The all-pages published-origin gate). Record +`delivery.gate` from its `summary.json`, never from the pixel number alone. ### Phase E2 — Link-audit completeness From 4e79190106eae8873ebc7c2ddf1f552a5e66f4e3 Mon Sep 17 00:00:00 2001 From: Paolo Mottadelli Date: Fri, 25 Sep 2026 23:42:53 +0200 Subject: [PATCH 3/5] =?UTF-8?q?feat(stardust):=20#125=20=E2=80=94=20the=20?= =?UTF-8?q?pixel-table=20rule;=20element=20probes=20in=20every=20gate.sh?= =?UTF-8?q?=20--full=20round?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rule: every crafted prototype and every deployed page is a row in a pixel table — gate-all --stage prototype writes gates/prototypes-/summary.* (build = the served prototype, origin = the archetype round's cached live.png), the default stage writes gates/all-/. gate-evidence.mjs reads the tables as the source of record (--tables): with a table for the width the row decides pixel-gate- and a page without a row is OPEN: no table row; without any table the log rule stands and --check names the gap. The eval requires the prototype table; rollout Phase H reports both totals. gate.sh --full now runs clip-probe on the build side in every regime, content-presence live vs build in the published regime (GATE_REGIME overrides the URL heuristic) and unit-geometry when stardust/replica/units.json declares a repeated-unit family for the slug ({ family: { origin, build, n, required, pages[], templates[] } }, also read by gate-all); a clipped count, a MISSING/HIDDEN link or heading or a required unit off fails the round, and outranks a pixel PASS in gate-evidence the way the overflow assert does. Compensation: pixel-compare --pad (the rejected union metric) is removed. Co-Authored-By: Claude Fable 5.1 --- plugins/stardust/CHANGELOG.md | 17 +++- .../replica-source-fidelity/criteria.json | 2 +- .../evals/replica-source-fidelity/task.md | 2 + .../diff/scripts/test/unit-geometry.test.mjs | 3 +- .../skills/diff/scripts/unit-geometry.mjs | 51 +++++++++--- plugins/stardust/skills/replica/SKILL.md | 14 +++- .../replica/reference/recreation-procedure.md | 11 +++ .../replica/reference/source-fidelity-gate.md | 24 +++++- .../skills/replica/scripts/gate-all.mjs | 75 +++++++++++------ .../skills/replica/scripts/gate-evidence.mjs | 54 +++++++++++- .../stardust/skills/replica/scripts/gate.sh | 82 +++++++++++++++---- .../skills/replica/scripts/pixel-compare.mjs | 33 +------- .../replica/scripts/test/gate-all.test.mjs | 4 +- .../scripts/test/gate-evidence.test.mjs | 34 +++++++- .../skills/replica/scripts/test/gate.test.mjs | 37 ++++++++- plugins/stardust/skills/rollout/SKILL.md | 5 +- .../stardust/reference/scripts-index.md | 10 +-- 17 files changed, 348 insertions(+), 110 deletions(-) diff --git a/plugins/stardust/CHANGELOG.md b/plugins/stardust/CHANGELOG.md index 5905060a3..57f791ef0 100644 --- a/plugins/stardust/CHANGELOG.md +++ b/plugins/stardust/CHANGELOG.md @@ -23,8 +23,8 @@ legible". Element presence and legibility are checkable — so the gate checks t 4 px); the pixel-only verdict is recorded beside it — every run is calibration data. Sidecars, each entry documented: `masks.json`, `overrides.json` (shown beside the number, never replacing it), `clip-allow.json`, `presence.json`, `units.json`. Origin fallback live → previous → crawl shot - (asymmetric, flagged); `--eds-host` gates a code branch; a padded/union pixel metric was tried as - the height guard and rejected (white gaps score as matches) — `pixel-compare --pad` is auxiliary. + (asymmetric, flagged); `--eds-host` gates a code branch; a union (white-padded) pixel metric was + tried as the height guard and rejected (white gaps score as matches) — the guard is explicit. `replica/scripts/sbs-crop.mjs` folds three side-by-side crop helpers. - **New `diff/scripts/clip-probe.mjs` (D1)** — text lines and controls against every overflow- clipping ancestor: TEXT CLIPPED / HIDDEN, CONTROL HIDDEN / CLIPPED counted (exit 2); line-clamp, @@ -42,7 +42,18 @@ legible". Element presence and legibility are checkable — so the gate checks t Δx / Δy / Δw / Δh of the first N repeated units relative to the unit, `hidden` where clipped, origin cached per slug. Recorded card: body +28…+47 px, details link +28 and hidden, badge 70×70 at (−10,−10) vs 64×64 at (−27,−30), card 260 → 238. -- **`pixel-compare.mjs --text-boxes` (D4, auxiliary)** + merged `--pad`; `textBoxPct` exported. +- **`pixel-compare.mjs --text-boxes` (D4, auxiliary)**; `textBoxPct` exported. (`--pad`, the rejected + union metric, is not shipped.) +- **The pixel-table rule.** Every crafted prototype and every deployed page is a row: + `gate-all --stage prototype --proto-base ` writes `gates/prototypes-/summary.{json,md}` at the + end of Phase 4 (reusing each archetype's cached `live.png`), the default stage writes `gates/all-/`; + `gate-evidence.mjs` reads the tables as the source of record (`--tables`; a page without a row is + `OPEN: no table row`, `--check` names a missing table), the eval requires the prototype table, rollout + Phase H reports both totals. `gate.sh --full` runs clip-probe on the build side in every regime, + content-presence in the published regime, unit-geometry when `stardust/replica/units.json` declares a + repeated-unit family for the slug (`{ family: { origin, build, n, required, pages[], templates[] } }` + — replica recreation-procedure § Repeated-unit families); element lines fail the round and outrank a + pixel PASS in gate-evidence the way the overflow assert does. - **Upstreamed feedback.** A1 `launchStealthHeaded` is WINDOW-FREE by default (`STARDUST_HEADED_WINDOW=1` opts in; bundled-Chromium fallback with a warning). A2 Akamai's HTTP 400 escalation is a challenge marker. A3 setup steps install `playwright pixelmatch pngjs cheerio` diff --git a/plugins/stardust/evals/replica-source-fidelity/criteria.json b/plugins/stardust/evals/replica-source-fidelity/criteria.json index 70a08d6fe..ab876577f 100644 --- a/plugins/stardust/evals/replica-source-fidelity/criteria.json +++ b/plugins/stardust/evals/replica-source-fidelity/criteria.json @@ -25,7 +25,7 @@ { "name": "source_fidelity_gate_runs", "max_score": 20, - "description": "The source-fidelity gate runs all three probes (content-diff and visual-diff with --profile generic against the live URL, plus stitch-shot + pixel-compare) and its evidence (per-iteration metrics, band breakdown, height delta, diff images) is written under stardust/replica/." + "description": "The source-fidelity gate runs all three probes (content-diff and visual-diff with --profile generic against the live URL, plus stitch-shot + pixel-compare) and its evidence (per-iteration metrics, band breakdown, height delta, diff images) is written under stardust/replica/; --full rounds carry the clip-probe line, and every crafted prototype is a row in stardust/replica/gates/prototypes-/summary.json (gate-all --stage prototype) — a prototype without a row is ungated." }, { "name": "both_breakpoints_gated", diff --git a/plugins/stardust/evals/replica-source-fidelity/task.md b/plugins/stardust/evals/replica-source-fidelity/task.md index b771aa422..ed335d006 100644 --- a/plugins/stardust/evals/replica-source-fidelity/task.md +++ b/plugins/stardust/evals/replica-source-fidelity/task.md @@ -57,6 +57,8 @@ The stardust `replica` skill is invoked. It: - "Improving" the design outside the inconsistency register. - DOM-copying the source page instead of re-authoring. - Declaring fidelity without gate evidence on disk (metrics, diff artifacts). +- A crafted prototype with no row in `stardust/replica/gates/prototypes-/summary.json` + (the pixel table is the source of record; a sidecar claim alone is not evidence). - Only gating desktop (mobile is not free — the 360 pass is required). - Skipping the content-cap row — a container cap wider than 1440 is invisible to both pixel gates. - Rehosting a licensed brand font. diff --git a/plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs b/plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs index ca27cb9f2..63e4b21c3 100644 --- a/plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs +++ b/plugins/stardust/skills/diff/scripts/test/unit-geometry.test.mjs @@ -47,7 +47,8 @@ check('missing + extra when nothing pairs; verdictOf totals; formatUnits marks r const r = alignUnit(o, e, 4); assert.equal(r.summary.missing, 1); assert.equal(r.summary.extra, 1); const res = [{ sel: 'a=b', units: [{ index: 0, ...r }] }, { sel: 'x=y', units: [], error: 'no visible unit' }]; - assert.deepEqual(verdictOf(res), { units: 1, within: 0, off: 0, hidden: 0, missing: 1, errors: 1 }); + assert.deepEqual(verdictOf(res), { units: 1, within: 0, off: 0, hidden: 0, missing: 1, errors: 1, requiredOff: 1 }); + assert.equal(verdictOf([{ sel: 'a=b', required: false, units: [{ index: 0, ...r }] }]).requiredOff, 0, 'an advisory family never feeds the exit code'); const s = formatUnits(res, 4); assert.match(s, /✗ missing/); assert.match(s, /🟡 extra/); assert.match(s, /no visible unit/); }); check('compareUnits pairs units by index and reports a side without visible units', () => { diff --git a/plugins/stardust/skills/diff/scripts/unit-geometry.mjs b/plugins/stardust/skills/diff/scripts/unit-geometry.mjs index d5c8d9fb4..bbbbd7847 100644 --- a/plugins/stardust/skills/diff/scripts/unit-geometry.mjs +++ b/plugins/stardust/skills/diff/scripts/unit-geometry.mjs @@ -13,14 +13,19 @@ * served page clips. Both sides settle the same way; the origin inventory caches per slug under * stardust/current/measure/-units.json. * - * Usage: node skills/diff/scripts/unit-geometry.mjs --unit "=" … + * Repeated-unit FAMILIES are declared once per project in stardust/replica/units.json — + * { "": { "origin": "", "build": "", "n": 2, "required": true, "pages": [""], "templates": [""] } } + * — and resolved per page by `--families --slug [--template ]` (gate.sh and gate-all both do); + * a family the page does not belong to is skipped, a non-`required` family is advisory (never exit 2). + * + * Usage: node skills/diff/scripts/unit-geometry.mjs (--unit "=" … | --families --slug ) * [--n 1] [--tol 4] [--width 1440] [--json []] [--slug ] [--force] [--advisory] * [--plain] [--warmup ] [--locale en-US] - * Exit: 0 within tolerance, 2 any element off / hidden / missing, 1 error, 3 bot challenge. - * gate-all runs it for the units units.json declares. `unitInventoryInPage`, `alignUnit`, - * `keyed`, `compareUnits`, `formatUnits`, `verdictOf` are exported. + * Exit: 0 within tolerance (or nothing declared), 2 a required unit off / hidden / missing, 1 error, + * 3 bot challenge. `unitInventoryInPage`, `alignUnit`, `keyed`, `compareUnits`, `formatUnits`, + * `verdictOf`, `unitsFor` are exported. */ -import { mkdirSync, realpathSync, writeFileSync } from 'node:fs'; +import { existsSync, mkdirSync, readFileSync, realpathSync, writeFileSync } from 'node:fs'; import { dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; import { inPage } from './clip-probe.mjs'; @@ -28,8 +33,10 @@ import { openBrowser, openPage, readCache, visit, writeCache } from './measure-l const HELP = `unit-geometry — per-element Δx/Δy/Δw/Δh of the first N repeated units, origin vs served (#125 D3) -Usage: node unit-geometry.mjs --unit "=" [options] +Usage: node unit-geometry.mjs (--unit "=" … | --families --slug ) [options] --unit [=] repeatable + --families stardust/replica/units.json: { family: { origin, build, n, required, pages[], templates[] } } + --template the page's template, for family matching (read from stardust/state.json when absent) --n units per selector (default 1) --tol per-element tolerance (default 4) --width viewport width (default 1440) @@ -162,7 +169,7 @@ export function formatUnits(results, tol) { for (const res of results) { for (const u of res.units) { const { unit, summary } = u; - lines.push(`unit ${u.index + 1} of ${res.sel}: origin ${fmt(unit.origin)} vs served ${fmt(unit.eds)} → Δx ${sgn(unit.dx)} Δy ${sgn(unit.dy)} Δw ${sgn(unit.dw)} Δh ${sgn(unit.dh)}${summary.unitOff ? ' ✗ unit size' : ''}; ${summary.elements} elements: ${summary.within} within ${tol}px, ${summary.off} off, ${summary.hidden} hidden, ${summary.missing} missing, ${summary.extra} extra`); + lines.push(`unit ${u.index + 1} of ${res.family ? `${res.family} (${res.required ? 'required' : 'advisory'}) ` : ''}${res.sel}: origin ${fmt(unit.origin)} vs served ${fmt(unit.eds)} → Δx ${sgn(unit.dx)} Δy ${sgn(unit.dy)} Δw ${sgn(unit.dw)} Δh ${sgn(unit.dh)}${summary.unitOff ? ' ✗ unit size' : ''}; ${summary.elements} elements: ${summary.within} within ${tol}px, ${summary.off} off, ${summary.hidden} hidden, ${summary.missing} missing, ${summary.extra} extra`); lines.push('| element | origin x,y w×h | served x,y w×h | Δx | Δy | Δw | Δh | |', '|---|---|---|---|---|---|---|---|'); for (const r of u.rows) lines.push(`| ${r.role} "${(r.text || r.key).slice(0, 36)}"${r.byPosition ? ` ≈ "${(r.edsText || '').slice(0, 20)}" (by position)` : ''} | ${fmt(r.origin)} | ${fmt(r.eds)}${r.eds.role !== r.role ? ` (${r.eds.role})` : ''} | ${sgn(r.dx)} | ${sgn(r.dy)} | ${sgn(r.dw)} | ${sgn(r.dh)} | ${r.hidden ? '✗ hidden (clipped)' : r.off ? '✗' : '✓'} |`); for (const m of u.missing) lines.push(`| ${m.role} "${(m.text || m.key).slice(0, 36)}" | ${fmt(m)} | — | | | | | ✗ missing |`); @@ -177,10 +184,12 @@ export function formatUnits(results, tol) { export function parseArgs(argv) { const rest = argv.slice(2); if (!rest.length || rest.includes('--help') || rest.includes('-h')) { console.log(HELP); process.exit(0); } - const opts = { origin: null, eds: null, units: [], n: 1, tol: 4, width: 1440, json: false, jsonFile: null, slug: null, force: false, advisory: false, plain: false, warmup: null, locale: 'en-US' }; + const opts = { origin: null, eds: null, units: [], n: 1, tol: 4, width: 1440, json: false, jsonFile: null, slug: null, force: false, advisory: false, plain: false, warmup: null, locale: 'en-US', families: null, template: null }; for (let i = 0; i < rest.length; i += 1) { const a = rest[i]; if (a === '--unit') { const [o, e] = rest[++i].split('='); opts.units.push({ origin: o.trim(), eds: (e || o).trim() }); } + else if (a === '--families') opts.families = rest[++i]; + else if (a === '--template') opts.template = rest[++i]; else if (a === '--n') opts.n = Number(rest[++i]); else if (a === '--tol') opts.tol = Number(rest[++i]); else if (a === '--width') opts.width = Number(rest[++i]); @@ -194,7 +203,15 @@ export function parseArgs(argv) { else if (a.startsWith('--')) { console.error(`unknown flag ${a}\n\n${HELP}`); process.exit(1); } else if (!opts.origin) opts.origin = a; else if (!opts.eds) opts.eds = a; } - if (!opts.origin || !opts.eds || !opts.units.length) { console.error(`need --unit =\n\n${HELP}`); process.exit(1); } + if (opts.families) { + if (!opts.slug) { console.error(`--families needs --slug\n\n${HELP}`); process.exit(1); } + const fam = JSON.parse(readFileSync(opts.families, 'utf8')); + let { template } = opts; + if (!template && existsSync('stardust/state.json')) { const pg = (JSON.parse(readFileSync('stardust/state.json', 'utf8')).pages || []).find((p) => p.slug === opts.slug); template = pg ? pg.template : null; } + opts.units.push(...unitsFor(fam, opts.slug, template)); + opts.n = Math.max(opts.n, ...opts.units.map((u) => u.n || 1)); + } + if (!opts.origin || !opts.eds || (!opts.units.length && !opts.families)) { console.error(`need and --unit = or --families --slug \n\n${HELP}`); process.exit(1); } return opts; } @@ -211,17 +228,25 @@ export async function measureUnits(browser, url, sels, { width, locale, warmup, /** Compare two measurement results for one --unit pair. */ export function compareUnits(oInv, eInv, pair, tol) { const o = oInv.bySel[pair.origin]; const e = eInv.bySel[pair.eds]; - const res = { sel: `${pair.origin}=${pair.eds}`, originMatches: o.matches, edsMatches: e.matches, units: [] }; + const res = { sel: `${pair.origin}=${pair.eds}`, family: pair.family || null, required: pair.required !== false, originMatches: o.matches, edsMatches: e.matches, units: [] }; if (o.error || e.error) { res.error = o.error ? `origin: ${o.error}` : `served: ${e.error}`; return res; } if (!o.units.length || !e.units.length) { res.error = `no visible unit on the ${!o.units.length ? 'origin' : 'served'} side (${o.matches} / ${e.matches} DOM matches)`; return res; } for (let i = 0; i < Math.min(o.units.length, e.units.length); i += 1) res.units.push({ index: i, origin: { path: o.units[i].path }, eds: { path: e.units[i].path }, ...alignUnit(o.units[i], e.units[i], tol) }); return res; } -export const verdictOf = (results) => results.reduce((a, r) => { if (r.error) a.errors += 1; for (const u of r.units) { a.off += u.summary.off; a.hidden += u.summary.hidden; a.missing += u.summary.missing; a.within += u.summary.within; a.units += 1; } return a; }, { units: 0, within: 0, off: 0, hidden: 0, missing: 0, errors: 0 }); +// `required` counters only feed the exit code; advisory families are reported, never blocking. +export const verdictOf = (results) => results.reduce((a, r) => { if (r.error) a.errors += 1; for (const u of r.units) { a.off += u.summary.off; a.hidden += u.summary.hidden; a.missing += u.summary.missing; a.within += u.summary.within; a.units += 1; if (r.required !== false) a.requiredOff += u.summary.off + u.summary.hidden + u.summary.missing; } return a; }, { units: 0, within: 0, off: 0, hidden: 0, missing: 0, errors: 0, requiredOff: 0 }); + +/** Families a page belongs to (by slug or template) → [{ family, origin, eds, n, required }]. */ +export function unitsFor(families, slug, template = null) { + return Object.entries(families || {}).filter(([, f]) => (f.pages || []).includes(slug) || (template && (f.templates || []).includes(template))) + .map(([family, f]) => ({ family, origin: f.origin, eds: f.build || f.eds || f.origin, n: f.n || 1, required: f.required !== false })); +} async function main() { const opts = parseArgs(process.argv); + if (!opts.units.length) { console.log(`unit-geometry: none declared for ${opts.slug} in ${opts.families} → n/a`); return; } const { chromium } = await import('playwright'); const browser = await openBrowser(chromium, { tier: opts.plain ? 'plain' : 'stealth' }); let oInv; let eInv; @@ -240,10 +265,10 @@ async function main() { else { console.log(`unit-geometry @ ${opts.width}px, tol ${opts.tol}px — ${opts.origin} vs ${opts.eds}`); console.log(formatUnits(results, opts.tol)); - console.log(`Units: ${v.units} compared; elements within ${v.within}, off ${v.off}, hidden ${v.hidden}, missing ${v.missing}${v.errors ? `; ${v.errors} selector error(s)` : ''} → ${v.off + v.hidden + v.missing + v.errors ? 'FAIL' : 'PASS'}`); + console.log(`Units: ${v.units} compared; elements within ${v.within}, off ${v.off}, hidden ${v.hidden}, missing ${v.missing}${v.errors ? `; ${v.errors} selector error(s)` : ''} → ${v.requiredOff + v.errors ? 'FAIL' : v.off + v.hidden + v.missing ? 'PASS (advisory deltas)' : 'PASS'}`); if (opts.jsonFile) { mkdirSync(dirname(opts.jsonFile) || '.', { recursive: true }); writeFileSync(opts.jsonFile, JSON.stringify(out, null, 1)); console.log(`json → ${opts.jsonFile}`); } } - process.exitCode = !opts.advisory && (v.off + v.hidden + v.missing + v.errors) > 0 ? 2 : 0; + process.exitCode = !opts.advisory && (v.requiredOff + v.errors) > 0 ? 2 : 0; } function safeRealpath(p) { try { return realpathSync(p); } catch { return p; } } diff --git a/plugins/stardust/skills/replica/SKILL.md b/plugins/stardust/skills/replica/SKILL.md index 47e2768fb..790760eb6 100644 --- a/plugins/stardust/skills/replica/SKILL.md +++ b/plugins/stardust/skills/replica/SKILL.md @@ -280,6 +280,10 @@ a verdict; 0 only when all four ran and passed). - **content-cap row: `cap-probe.mjs … --against` prints `cap-probe: PASS`** (gate doc § Pass bar item 6 — every live cap held within ±20 px by kind, nothing capped only on the prototype; a ✗ names the sizing rule, no pixel iteration); +- **clip-probe: `Clipped: 0`** on the build side (a `--full` round runs it; cut or hidden + text / controls fail the round like an over-threshold pixel diff — #125); a + declared repeated-unit family (`stardust/replica/units.json`, gate doc item 7) + within 4 px; - and, outside the bar and outside the cap, the horizontal-overflow assert: `document.documentElement.scrollWidth` within 4 px of the viewport (integer rounding; `GATE_OVERFLOW_TOLERANCE`) at every breakpoint on the @@ -287,6 +291,12 @@ a verdict; 0 only when all four ran and passed). says; a `capture failed (exit 1)` round (after gate.sh's one retry) is re-queued, never counted. +**Every prototype is a row (#125).** When the archetypes pass, one run per width +writes the prototype table — `node stardust/scripts/replica/gate-all.mjs --stage +prototype --proto-base "$PROTO_BASE" --width 1440` (then 360) through `run-bg.mjs`; +it reuses each archetype's cached `live.png`. `gate-evidence.mjs` reads it as the +source of record: a prototype without a row is ungated. + **Iteration discipline: hard cap 3 iterations per breakpoint.** Each iteration's fixes come off the instruments, never off eyeballing. After 3, log the residuals in the ledger and move on — a documented 2% residual beats @@ -424,7 +434,9 @@ had one section for the whole run). DELIVERED = pixel % ≤ 10 AND |Δh| ≤ 5 % AND 0 clipped text / controls (`clip-probe`) AND 0 MISSING / HIDDEN links / headings (`content-presence`); a recorded page passed the pixel bar with all of its cards clipped. Evidence - and sidecars: `stardust/replica/gates/all-/`. + and sidecars: `stardust/replica/gates/all-/`; with the Phase 4 + `prototypes-/` table it is the pair `gate-evidence.mjs` reads (a page + without a row is ungated). **State:** replica writes its own state under `stardust/replica/` — the inconsistency register, `progress.json` (per page type: archetype slug, diff --git a/plugins/stardust/skills/replica/reference/recreation-procedure.md b/plugins/stardust/skills/replica/reference/recreation-procedure.md index 0dfeabe37..8cbf151da 100644 --- a/plugins/stardust/skills/replica/reference/recreation-procedure.md +++ b/plugins/stardust/skills/replica/reference/recreation-procedure.md @@ -70,6 +70,17 @@ the chrome crop gate (`source-fidelity-gate.md` § Pass bar, item 5) against the NEW page's live chrome, and flag any page-level compensation for back-port into the canon files so later archetypes don't re-discover it. +### Repeated-unit families (#125) + +A block that repeats a unit — cards, rail items, list rows, FAQ rows — is declared once in +`stardust/replica/units.json` when it is authored: `{ "": { "origin": "", "build": "", "n": 2, "required": true, "pages": [""], +"templates": ["