Skip to content

Accessibility Audit 1.3.2: results across runs, refresh, and unstyled renders - #7

Merged
HosseinKarami merged 2 commits into
mainfrom
fix/1.3.2-results-across-runs
Nov 5, 2026
Merged

HosseinKarami merged 2 commits into
mainfrom
fix/1.3.2-results-across-runs

Conversation

@HosseinKarami

Copy link
Copy Markdown
Member

Summary

Version 1.3.2. Fixes from the fourth full audit of the large WordPress multisite, with 1.3.1 and a short time budget. The run lost 17 AI answers, and a refresh changed many files with no new data. The list had 10 items. The details are in CHANGELOG.md.

The results from run to run

  • A question whose pages the run did not test stays with its answers, as "not_checked": true. Before, the run deleted it. Each short check (refact-a11y check) deleted the questions of all the pages that it did not test. The report counts these questions apart, and the review tasks skip them.
  • The pages of open issues of the sample-only tests (listener, focus, operate, template criteria, states) open first and join the sample. On the multisite, this adds 10 pages to a sample of 175.
  • An own rule is "rule changed" only when its detection is newer than the engine that found the issue (src/own/versions.ts). Before, each release made every such issue "rule changed".
  • The classes of a lazy load (lazyloaded, ls-is-cached, and others) are not part of the identity or the question hash. The old IDs move to the new ones with their status, answers, and solution.

The refresh

  • It keeps replaces, the example of each question, and every part of the report. The audit writes the crawl parts of the report to runs/<run>/report-data.json, and pages.jsonl keeps the title, the reason of the sample, load_incomplete, and styles_missing.

The page loads

  • A page whose stylesheet request failed loads once more. Chromium keeps an empty sheet for a failed stylesheet, so the engine watches the network. On the multisite, one unstyled render gave 5 false target issues.

The report

  • A line for the pages that a budget left, by source and by section. The "Resolved" column counts this run only.

Test plan

  • npm test, with a new test file, across-runs (19 checks). statuses now checks the "not checked" rule with the status code, because an audit adds such a page to the sample.
  • The new test on the 1.3.1 code: 16 of its 19 checks fail there. The other 3 are a setup check and two checks that pass on both versions.
  • A refresh with this branch on a copy of the multisite results of 1.3.1: no issue file changes, no answer is lost, and no example moves. 1,060 questions get the new item hash one time. Almost all of them hold a lazy-load class.
  • A full audit of the multisite with 1.3.2.

… renders

Fixes from the fourth full audit of a large WordPress multisite, with 1.3.1
and a short time budget, which lost 17 AI answers, and whose refresh
changed many files with no new data.

- Questions: a question whose pages the run did not test stays with its
  answers as not_checked, also after a short check. Review tasks, the
  counts, and the report leave these out.
- Sample: the pages of open issues of sample-only tests open first and
  join the sample ("open issue"), so each audit checks those issues.
- Rule changed: an own rule changes only when its detection version
  (src/own/versions.ts) is newer than the engine that found the issue.
- Identity: lazy-load classes (lazyloaded, ls-is-cached, ...) are left out
  of items and question hashes; old IDs move with status and answers.
- Refresh: keeps replaces, the example of each question, and every part
  of the report (runs/<run>/report-data.json; title, sample reason,
  load_incomplete, styles_missing in pages.jsonl).
- Loading: a page whose stylesheet request failed loads once more; a
  stylesheet that stays missing is reported.
- Report: the pages that a budget left, by source and section; the
  Resolved column counts this run only.
- Tests: across-runs, in npm test. Each check of a fix fails on 1.3.1.
@HosseinKarami
HosseinKarami merged commit 7852e03 into main Nov 5, 2026
1 check passed
@HosseinKarami
HosseinKarami deleted the fix/1.3.2-results-across-runs branch November 5, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant