Accessibility Audit 1.1.1: fixes from the first dry submit on a real site

CHANGELOG.md

@@ -2,6 +2,16 @@
 
 The version is the same for the package `@refact/a11y` and for the plugin `accessibility-audit`. A machine gets a new copy of the plugin only when this version changes.
 
+## 1.1.1, 2026-11-01
+
+Fixes from the first dry submit on a real site, CRE Daily, on 2026-10-28.
+
+- A form that sends to its own page, for example a Gravity Forms form, is the same form on each page. Before, the engine tested the footer form of CRE Daily on 47 pages. The names of the hidden fields now tell two forms with the same visible fields apart.
+- The engine skips a form in a fixed bar or panel that is outside the screen, for example a closed slide-up bar. It also skips a form that a new render does not show yet, for example in a popup that opens 10 seconds after the load. The coverage names that form. Such forms show only after an action or a delay. Before, their buttons did not respond, and the coverage said "not tested" for 11 pages of CRE Daily.
+- The audit records the version of the package as the engine version. Before, it recorded 0.1.0. The first audit after this update is a baseline audit: an issue that disappears in it gets "resolved, rule changed", not "fixed".
+- A criterion that the processes test, for example 3.3.1 on staging, shows the process steps in its coverage (`coverage.process_units`). Before, it showed "no issue found" with 0 tested units.
+- `tests/summary.mjs` tests the engine version and the coverage of the process steps. `npm test` runs it.
+
 ## 1.1.0, 2026-10-26
 
 The dry submit: form errors are tested on production, and no data leaves the browser. See `FORMS.md`.

FORMS.md

@@ -36,7 +36,7 @@ Two other methods are not in this plan:
 
 1. On each page of the sample, at the first width, the engine finds the forms that are visible after the load.
 2. A form is a `form` element with at least one text field and a submit control. The engine skips a search form, a form with only a select or checkboxes, and a form with no submit control.
-3. The engine tests a maximum of three forms on each page. A form that has the same action, the same fields, and the same submit control as a tested form is the same form. The engine tests it one time in each audit, on the first page in a fixed order.
+3. The engine tests a maximum of three forms on each page. A form that has the same action, the same fields, the same hidden field names, and the same submit control as a tested form is the same form. A form that sends to its own page, for example a Gravity Forms form, has the same action on each page. The engine tests it one time in each audit, on the first page in a fixed order.
 4. Each form gets one or two submissions. Each submission is a state of the page, in its own render and in a new browser context.
    - Submission A: the form empty. The engine clicks the submit control.
    - Submission B: each text field filled with `a11y-dry-submit`, which is not a correct email, date, phone, or address. The engine clicks the submit control. Submission B runs only when submission A showed errors on the page.
@@ -75,6 +75,7 @@ The guard records each blocked request. The load of the page before the guard is
 | No change, and the browser marks a field as invalid | The browser shows its own message | No finding. The criteria count as tested |
 | No change, and the guard blocked a request after the click | The site checks the fields on the server only | No finding. The page gets "not tested" for 3.3.1 and 3.3.3. The coverage lists the form: its errors need part 2 |
 | No change, and no blocked request | The form needs more than one action, or does nothing | No finding. The coverage lists the form |
+| The form is not visible in a new render | The form shows only after an action or a delay, for example in a popup | No submission. The coverage lists the form. Its page can still count as tested |
 
 ### 3.4 The settings
 
@@ -91,13 +92,13 @@ The guard records each blocked request. The load of the page before the guard is
 ### 3.6 What the dry submit does not test
 
 - The errors of the server and the success message. Part 2 tests them.
-- A form that shows only after an action, for example in a dialog or a tab.
+- A form that shows only after an action, for example in a dialog, a tab, or a fixed bar that is outside the screen until it opens.
 - A form in a frame from another domain, for example an older HubSpot embed.
 - The steps after the first step of a form with more than one step.
 
 ### 3.7 The proof
 
-1. `tests/dry-submit.mjs` serves a local site whose forms send data in eleven ways. The ways are: a fetch POST, a POST form, a GET form, and a fetch GET. Also a JSONP request to the site and to another host, a beacon, and a WebSocket message. Also a form in a new window, a form in a frame, and a pixel to another host. Each page has a form with no check and a form that checks only that the fields are not empty. The server records each request. The test passes when the server gets no submission and no typed value, and when the coverage lists the forms with no check.
+1. `tests/dry-submit.mjs` serves a local site whose forms send data in eleven ways. The ways are: a fetch POST, a POST form, a GET form, and a fetch GET. Also a JSONP request to the site and to another host, a beacon, and a WebSocket message. Also a form in a new window, a form in a frame, and a pixel to another host. Each page has a form with no check and a form that checks only that the fields are not empty. The server records each request. The test passes when the server gets no submission and no typed value, and when the coverage lists the forms with no check. Since 1.1.1, two more pages have a form that sends to its own page, as Gravity Forms does, and a form in a closed fixed bar. A third form is in a popup that opens 3 seconds after the load. The engine must test the first form one time for the two pages, and skip the form in the bar and the form in the popup.
 2. The same test audits the seeded faults F59 (3.3.1) and F61 (3.3.3) with production settings and no process. The engine must find both. The base site must give no form finding.
 3. `npm test` runs the test. The seeded detection runs again for all faults.
 4. One short audit of the contact page of refact.co, the site of the company, shows the dry submit on a real form.

README.md

@@ -108,6 +108,7 @@ npm test              the type check, the formats, and the seeded site: recurren
 npm run detect        the detection of all 74 seeded faults
 npm run cycle-eval    the whole cycle on one seeded site, and the solution success rate
 npm run dry-submit    the dry submit sends nothing, and finds the form errors (also in npm test)
+npm run summary       the engine version and the coverage of the process steps (also in npm test)
 ```
 
 The seeded site is a small correct site with 74 known faults, in `tests/seeded-faults/`. The numbers of the last measurements are in `tests/seeded-faults/*.md`: 74 of 74 faults found, 29 of 29 source files, 42 of 42 solutions that fixed their issue.

formats/summary.schema.json

@@ -252,6 +252,25 @@
                   "process"
                 ],
                 "description": "What 'all units' means for this criterion."
+              },
+              "process_units": {
+                "type": "object",
+                "additionalProperties": false,
+                "required": [
+                  "tested",
+                  "all"
+                ],
+                "description": "For a criterion that the processes also test: the steps of the processes, apart from the units of the definition.",
+                "properties": {
+                  "tested": {
+                    "type": "integer",
+                    "minimum": 0
+                  },
+                  "all": {
+                    "type": "integer",
+                    "minimum": 0
+                  }
+                }
               }
             }
           },

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@refact/a11y",
-  "version": "1.1.0",
+  "version": "1.1.1",
   "private": true,
   "description": "Accessibility Audit: finds what blocks people with disabilities on a website, where each problem comes from, and how to fix it. It does not fix.",
   "type": "module",
@@ -14,7 +14,7 @@
     "build": "tsc -p tsconfig.build.json",
     "prepare": "npm run build",
     "typecheck": "tsc -p tsconfig.json",
-    "test": "npm run typecheck && npm run validate-formats && npm run seed-build && node tests/recurrence.mjs --serve && node tests/review.mjs --serve && node tests/seeded-faults/detect.mjs --serve --only F01,F05,F18,F74 && node tests/dry-submit.mjs --serve",
+    "test": "npm run typecheck && npm run validate-formats && npm run seed-build && node tests/recurrence.mjs --serve && node tests/review.mjs --serve && node tests/seeded-faults/detect.mjs --serve --only F01,F05,F18,F74 && node tests/dry-submit.mjs --serve && node tests/summary.mjs",
     "audit": "node src/cli.ts audit",
     "build-criteria": "python3 -I criteria/build.py criteria/criteria.base.json criteria/sources/axe/rule-descriptions.md criteria/sources/act/testcases.json criteria/criteria.json",
     "validate-formats": "node formats/validate.mjs",
@@ -27,7 +27,8 @@
     "regroup": "node tools/regroup.mjs",
     "review": "node tests/review.mjs",
     "cycle-eval": "node tests/cycle-eval.mjs",
-    "dry-submit": "node tests/dry-submit.mjs"
+    "dry-submit": "node tests/dry-submit.mjs",
+    "summary": "node tests/summary.mjs"
   },
   "dependencies": {
     "ajv": "^8.20.0",

plugin/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "accessibility-audit",
-  "version": "1.1.0",
+  "version": "1.1.1",
   "description": "The agent skill of Accessibility Audit: runs the WCAG 2.2 AA audit engine on a site, answers the review questions from the evidence, finds the origin and the source file of each issue, writes a specific solution, and records the decisions of a person. It never fixes the site.",
   "author": {
     "name": "Refact",

src/audit.ts

@@ -379,10 +379,11 @@ export async function runAudit(options: AuditOptions): Promise<AuditResult> {
     };
     await Promise.all(Array.from({ length: concurrency }, () => dryWorker()));
     // A sample page counts as tested for the form criteria, unless one of its forms checks on the server only or did not run.
+    // A form that is hidden in a new render shows only after an action or a delay, as a form in a closed dialog: it does not count.
     const formResult = new Map(dryRuns.map((r) => [r.form.signature, r]));
     for (const [p, forms] of formsOf) {
       if (!p.tested) continue;
-      if (forms.some((f) => { const r = formResult.get(f.signature); return !r || !r.tested || r.serverOnly; })) continue;
+      if (forms.some((f) => { const r = formResult.get(f.signature); return !r || (!r.tested && !r.hidden) || r.serverOnly; })) continue;
       for (const c of DRY_SUBMIT_CRITERIA) p.testedCriteria[first]?.add(c);
     }
 
@@ -444,6 +445,7 @@ export async function runAudit(options: AuditOptions): Promise<AuditResult> {
     const where = `the form '${r.form.name}' on ${pathOf(r.url)}`;
     if (r.serverOnly) testsNotRun.push(`dry submit: ${where} checks its fields on the server only. Its errors and its success message need a process on staging`);
     else if (r.unchanged) testsNotRun.push(`dry submit: ${where} did not change after the submission`);
+    else if (r.hidden) testsNotRun.push(`dry submit: ${where} shows only after an action or a delay, for example in a popup. The dry submit skipped it`);
     else if (!r.tested) testsNotRun.push(`dry submit: ${where} was not tested: ${r.error}`);
   }
   if (!config.interaction.processes.length) testsNotRun.push("processes: the settings have none. The success message of a form and the redundant entry test (3.3.7) run in the processes only");

src/own/dry-submit.ts

@@ -24,11 +24,20 @@ export interface FormCandidate { selector: string; submit: string; name: string;
 const TEXT = "input:not([type]), input[type=text], input[type=email], input[type=tel], input[type=url], input[type=password], textarea";
 const USABLE = `const usable = (el) => visible(el) && !el.disabled && !el.readOnly && !el.closest("[aria-hidden=true], [inert]");`;
 const SUBMIT_TYPES = "button[type=submit], button:not([type]), input[type=submit], input[type=image]";
+/** A control in a fixed bar or panel that is outside the screen. A scroll cannot bring it into view, so a click cannot reach it. */
+const UNREACHABLE = `const unreachable = (el) => {
+  let fixed = false;
+  for (let a = el; a && a !== document.documentElement; a = a.parentElement) if (getComputedStyle(a).position === "fixed") { fixed = true; break; }
+  if (!fixed) return false;
+  const r = el.getBoundingClientRect();
+  return r.bottom > innerHeight + 1 || r.top < -1 || r.right > innerWidth + 1 || r.left < -1;
+};`;
 
 /** The forms of the page: a form element with a text field and a submit control. Not a search form. */
 const FIND_FORMS = `(() => {
   ${HELPERS}
   ${USABLE}
+  ${UNREACHABLE}
   const out = [];
   for (const form of Array.from(document.forms)) {
     if (form.getAttribute("role") === "search" || form.closest("[role=search]")) continue;
@@ -41,10 +50,20 @@ const FIND_FORMS = `(() => {
     let submit = [...Array.from(form.querySelectorAll(${JSON.stringify(SUBMIT_TYPES)})), ...owned].find((b) => visible(b) && !b.disabled);
     if (!submit) submit = Array.from(form.querySelectorAll("button, [role=button], input[type=button]")).find((b) => visible(b) && !b.disabled && /send|submit|subscribe|sign ?up|sign ?in|log ?in|join|register|continue|next|request|get started|apply/i.test(nameOf(b) || b.value || ""));
     if (!submit) continue;
+    // A form in a fixed bar or panel that is outside the screen shows only after an action, as a form in a closed dialog.
+    if (unreachable(submit)) continue;
     const name = (nameOf(submit) || submit.value || "").trim().replace(/\\s+/g, " ").slice(0, 60);
-    let action = "";
-    if (form.getAttribute("action")) { try { action = new URL(form.action).pathname; } catch { action = form.getAttribute("action"); } }
-    const signature = [action, (form.getAttribute("method") || "get").toLowerCase(), all.map((e) => (e.type || e.localName) + ":" + (e.getAttribute("name") || e.id || nameOf(e))).sort().join(","), name].join("|");
+    // A form that sends to its own page, for example a Gravity Forms form, is the same form on each page.
+    // The attribute, not form.action: a field with the name "action", common in WordPress forms, hides that property.
+    let action = "self";
+    const raw = form.getAttribute("action");
+    if (raw) {
+      try { const u = new URL(raw, location.href); action = u.origin === location.origin && u.pathname.replace(/\\/$/, "") === location.pathname.replace(/\\/$/, "") ? "self" : u.pathname; }
+      catch { action = raw; }
+    }
+    // The names of the hidden fields tell two forms with the same visible fields apart, for example two lists of one newsletter.
+    const hidden = Array.from(form.querySelectorAll("input[type=hidden]")).map((e) => e.getAttribute("name") || "").filter(Boolean).sort().join(",");
+    const signature = [action, (form.getAttribute("method") || "get").toLowerCase(), all.map((e) => (e.type || e.localName) + ":" + (e.getAttribute("name") || e.id || nameOf(e))).sort().join(","), hidden, name].join("|");
     out.push({ selector: sel(form), submit: sel(submit), name: name || form.getAttribute("aria-label") || form.id || "form", signature, html: html(form).slice(0, 300), item: item(form), textFields: fields.length });
   }
   return out;
@@ -146,12 +165,13 @@ export async function guardedContext(browser: Browser, config: Config, width: nu
 }
 
 /** What a submission showed. browser: the page checked the fields. native: the browser checked them. server: the form tried to
- *  send, so the site checks on the server only. none: nothing changed. error: the engine could not do the submission. */
-export type SubmissionKind = "browser" | "native" | "server" | "none" | "error";
+ *  send, so the site checks on the server only. none: nothing changed. hidden: the form is not visible in a new render, because it
+ *  shows only after an action or a delay, for example in a popup. error: the engine could not do the submission. */
+export type SubmissionKind = "browser" | "native" | "server" | "none" | "hidden" | "error";
 
 export interface Submission { variant: "empty" | "incorrect"; kind: SubmissionKind; state: string[]; blocked: string[]; rules: string[]; reason?: string }
 
-export interface DryRun { form: FormCandidate; url: string; submissions: Submission[]; renders: number; states: number; tested: boolean; serverOnly: boolean; unchanged: boolean; error?: string }
+export interface DryRun { form: FormCandidate; url: string; submissions: Submission[]; renders: number; states: number; tested: boolean; serverOnly: boolean; unchanged: boolean; hidden: boolean; error?: string }
 
 export interface DryOptions {
   contextOptions?: Record<string, unknown>;
@@ -194,6 +214,9 @@ async function submitOnce(browser: Browser, config: Config, record: PageRecord,
     guard.values.push(DRY_VALUE);
     const present = await page.evaluate(`!!document.querySelector(${JSON.stringify(form.selector)}) && !!document.querySelector(${JSON.stringify(form.submit)})`) as boolean;
     if (!present) return { variant, kind: "error", state, blocked: [], rules: [], reason: "the form is not on the page in a new render" };
+    // A form that the page record found later, for example in a popup that opens after 10 seconds, is hidden in a new render.
+    const shown = await page.evaluate(`(() => { ${HELPERS} ${UNREACHABLE} const b = document.querySelector(${JSON.stringify(form.submit)}); return !!b && visible(b) && !unreachable(b); })()`) as boolean;
+    if (!shown) return { variant, kind: "hidden", state, blocked: [], rules: [], reason: "the form is not visible in a new render" };
     if (variant === "incorrect") {
       for (const f of await page.evaluate(TEXT_FIELDS(form.selector)) as string[]) await page.locator(f).first().fill(DRY_VALUE, { timeout: 3000 }).catch(() => {});
     }
@@ -240,7 +263,7 @@ async function submitOnce(browser: Browser, config: Config, record: PageRecord,
 
 /** The dry submit of one form: submission A empty, then submission B with incorrect values when A showed errors on the page. */
 export async function runDrySubmit(browser: Browser, config: Config, record: PageRecord, width: number, form: FormCandidate, options: DryOptions): Promise<DryRun> {
-  const run: DryRun = { form, url: record.url, submissions: [], renders: 0, states: 0, tested: false, serverOnly: false, unchanged: false };
+  const run: DryRun = { form, url: record.url, submissions: [], renders: 0, states: 0, tested: false, serverOnly: false, unchanged: false, hidden: false };
   const a = await submitOnce(browser, config, record, width, form, "empty", options);
   run.submissions.push(a); run.renders++;
   if (a.kind === "browser" && form.textFields > 0) {
@@ -250,7 +273,8 @@ export async function runDrySubmit(browser: Browser, config: Config, record: Pag
   }
   record.renders += run.renders;
   run.states = run.submissions.filter((s) => s.kind === "browser").length;
-  run.tested = a.kind !== "error";
+  run.hidden = a.kind === "hidden";
+  run.tested = a.kind !== "error" && a.kind !== "hidden";
   run.serverOnly = a.kind === "server";
   run.unchanged = a.kind === "none";
   if (a.kind === "error") run.error = a.reason;