feat: make the elision verdict inspectable and provable per app - #1312
feat: make the elision verdict inspectable and provable per app#1312vivek7405 wants to merge 10 commits into
Conversation
…eport The elision report told an app which pages ship when they could have been elided, which is the benign over-ship direction. Nothing reported the other direction: which components were DROPPED, and on what evidence. That is the direction where a wrong verdict silently loses interactivity in production. analyzeElision now records WHY each shipping component ships, alongside every mustShip write, and returns it as componentVerdicts. analyzeAppElision projects the whole verdict (components, route modules, orphans, summary) into one sorted, app-relative, JSON-serializable object. Nothing is re-analysed: the data was already in memory and was being discarded. maskJsSet moves into a shared leaf so the framework's own differential guard and the app-facing one cannot drift apart on what the JS-loaded set even is.
…tial An app could not ask what elision dropped from its pages, and the SSR differential that proves elision changes nothing was a framework test an app could not run. `webjs elision` prints the verdict per module with the evidence behind every ship, --json emits it for a tool, and --verify boots two handlers with WEBJS_ELIDE flipped and diffs the masked SSR bytes across the app's own static route corpus. It exits non-zero on a divergence and on a corpus where nothing was compared, since a vacuous pass is the failure mode a verification command must not have. The success output states its own boundary: it proves the served bytes did not change, not post-hydration behaviour. Also corrects a claim four surfaces carried. `static interactive = true` does not rescue a component whose own registration tag is computed: the scanner requires a literal tag, so that component gets no verdict for the override to attach to.
…tests `list_elision` returns `analyzeAppElision` verbatim: that function already owns the app-level contract, so unlike list_routes there is no projector leaf and nothing to keep in sync. list_components deliberately stays a cheap lexical inventory; growing an elided flag onto it would make it build a module graph. The dev server prints one summary line per warm analysis, re-emitted after each fs.watch rebuild. Deliberately NOT a browser push: an inert route ships zero application JS, and the most useful manual check an author has is opening the network tab on that route and seeing nothing, which a dev-only boot script would corrupt on exactly the pages this feature exists to prove. The contract tests pin both documented residuals as MEASURED, which was prose until now: a computed whenDefined observer leaves its badge elided, an external stylesheet's :defined rule does too, and `static interactive = true` rescues both. A computed Class.register(tag) is recorded as a different and worse shape that the override does NOT rescue, since the scanner never sees that component. examples/blog gains forced-badge, display-only in every respect apart from the override, so the e2e probe can assert the browser really keeps its module.
…override The CLI tests drive every --verify exit path, including the two that matter most and are easy to get wrong: a corpus where nothing could be compared exits 1 rather than reporting a vacuous pass, and a real divergence is produced for real rather than stubbed (a component whose SSR output reads WEBJS_ELIDE is exactly the class of bug the differential exists to catch). The doctor tests keep the three carrier cases unchanged, which is the regression guard for rewiring that check onto the shared report, and pin the new check to pass-except-orphans plus its gate contract. The e2e probe is the first coverage anywhere above the analyser unit that static interactive = true actually keeps a module on the wire; build-stamp on the same run is the negative control.
Elision was described across six docs pages with no page owning it, and the opt-out appeared on none of them, which is exactly how the switch went undocumented. A capability with its own command, config key, env override, doctor check, and MCP tool has outgrown being a paragraph inside five topics. The skill gets the agent-facing half: how to read the evidence values, what to do with each verdict, and the two-run recipe for the behaviour half that the byte differential cannot see. Dogfooding the new doctor check turned up a real defect in the orphan scan. It read raw source, so every WebComponent subclass written inside an html template as a CODE SAMPLE counted as an unregistered component, and the repo own website reported 17 false orphans. That was tolerable as dev-console noise and is not tolerable as a doctor warning, so the scan now redacts strings and templates exactly as extractComponents already did. Both dogfood apps go from 17 and 0 false orphans to none, and a real orphan in the same tree still fires.
b770ca1 to
ef13733
Compare
The scaffold gallery-coverage gate requires every server export to be either demoed or exempted; maskJsSet and staticPageRoutes are verify-command plumbing, so they are exempt with the reason they exist at all. The elision-report fixtures move to template literals. The scanner-fuzz corpus sweep reads every file under test/elision and compares its lexical class window against a real AST, and redaction blanks a template body while keeping a plain string verbatim, so a fixture class in a plain string skews that differential.
|
Design rationale: why the report carries evidence, and why an elided row carries none The verdict data was already in memory and was being thrown away. First write wins, matching the analyser's first-match convention everywhere else. A component forced by several rules reports the rule that reached it first, which is a truthful account of why the module is on the wire even when it is not the only one. An elided row deliberately reports The one hole in that design is a future rule that adds to |
|
Decision: the orphan scan now redacts, because dogfooding turned a tolerable warning into a wrong one
I found it by running doctor over our own site: the website reported 17 orphans, every one of them a documentation code sample. Shipping a check that cries wolf on our own dogfood app is the exact failure the check's pass-except-orphans design was trying to avoid, so I fixed the scan rather than soften the check. The fix is the one This is strictly a false-positive fix, not a verdict change. Elision decides nothing differently. |
|
Measured: what Four surfaces said the override covers "a dynamically-computed tag string". I drove the analyser against real on-disk fixtures to check, and that wording conflated two different shapes with opposite outcomes. What it DOES rescue is the OBSERVER's tag, not the registration's. A shipping module writing What it does NOT rescue is a component whose OWN registration tag is computed. So the fix for that case is documentary plus visibility, not code. Invariant 3 already requires a literal tag, the docs now say so where they used to promise the override, and the shape surfaces as an |
…ile column The help renderer hardcoded the heading Config:, which is what the doctor severity gate is and what its help test pins. The elision block is a caveat about what --verify proves, not configuration, so an entry now names its own heading and doctor keeps the default. The verdict table capped every column at the same width, which is right for the tag column (a file registering five tags is the rare case) and wrong for the file column: on a freshly scaffolded app almost every module path is longer than the cap, so almost every row overflowed instead of one outlier.
vivek7405
left a comment
vivek7405
left a comment
There was a problem hiding this comment.
Went back over this one properly and it is not ready. The shape of the change is right, and the analyser side holds up: recording the evidence at each mustShip write rather than deriving it later is the correct call, and the verdict is provably unchanged against main on both dogfood apps.
The problem is that two of the three things this PR sells can currently lie. --verify never forces elision ON, so on an app with elide: false it renders the same thing twice and reports the routes identical with elision on vs off, which is a false statement about a run where elision was never on. That is the exact vacuous-pass class the zero-route exit code was added to prevent, and it is worse than the zero-route case because it prints a confident pass. The e2e probe has the same flavour of problem: its declared negative control asserts a module that is not on that page at all, so it cannot fail and the test is really a single positive assertion wearing a control.
The other two are smaller but both mislead a reader: the new docs page's meta description tells people to run a command that does not exist, and list_elision is missing from six places that enumerate the MCP tools, including the published README and the CLI's own help text.
Worth saying what is good here, because the review reads harshly: the residual contract tests are the most valuable part of the diff. Pinning what static interactive = true does and does not rescue, measured rather than asserted from the docs, is what stops the next person repeating the mistake this PR is correcting.
Three things this PR sells could lie. --verify deleted WEBJS_ELIDE for its ON side, which only falls back to webjs.elide, so on an app that opts out BOTH handlers ran with elision off and the command reported the routes identical with elision on vs off and exited 0. That is a confident pass on a run where elision was never on, worse than the zero-route vacuity the exit code already guarded. The ON side is now forced on through the override, which wins over the config key, and the run reports how many modules elision actually dropped so a trivially-true pass is visible too. The e2e probe asserted build-stamp was not downloaded on /observed as its negative control, but build-stamp is only on /, so that assertion was true whether or not elision worked and the test was one positive assertion wearing a control. /observed now renders it, and WEBJS_ELIDE=0 reds the control. The new docs page meta description named two commands that do not exist, and list_elision was missing from six surfaces that enumerate the MCP tools, including the published README and the CLI help text.
vivek7405
left a comment
vivek7405
left a comment
There was a problem hiding this comment.
Read the fix commit again with fresh eyes, tracing what each change touches across the rest of the branch. The two substantive fixes hold up: forcing the ON side really does make an opted-out app compare a real pair, and the e2e control now reds under WEBJS_ELIDE=0.
What it missed is its own blast radius on the docs. The commit set out to add list_elision everywhere the MCP tools are enumerated and got six of eight: the CLI-reference block in AGENTS.md (the same file it edited, different section) and the start-work skill still list the old four. Separately, examples/blog/AGENTS.md owns an "Elision fixtures" section that enumerates exactly these files, and it is now wrong twice over: build-stamp is documented as rendered on / only, and forced-badge, a fixture this PR adds specifically to be e2e-pinned, is not listed at all. That section is the thing a future reader consults before touching a fixture.
Two smaller ones on the new docs page: the meta description carries literal backticks, which land verbatim in the <meta> tag and the llms.txt entry rather than rendering as code, and the paragraph explaining that --verify forces the ON side was inserted after the two-run e2e code block, so it reads as describing that recipe instead of --verify.
On the exit code for a corpus where elision dropped nothing: I am leaving that at 0 deliberately, reasoning in the thread.
The previous commit added list_elision to six surfaces and missed two: the CLI-reference block in AGENTS.md, which is a different section of the same file it edited, and the start-work skill. examples/blog/AGENTS.md owns the section that enumerates the elision fixtures, which is what a future reader consults before touching one, and it was wrong twice: build-stamp is now the negative control on /observed as well as /, and forced-badge was not listed at all. It also now records the doc-comment discipline those fixtures depend on, since prose naming a tag or a whenDefined call would register as a real signal and make the tests pass vacuously. The new docs page carried literal backticks into its meta description, which is emitted verbatim into the meta tag and the llms.txt entry, and the paragraph explaining that --verify forces the ON side sat after the two-run e2e block, so it read as describing that recipe instead. The zero-drop message said the run proves nothing about elision while exiting 0, which reads as a self-contradiction. Exit 0 is right there (the question was answered, and a corpus with nothing elidable is a legitimate app), so the message now says what actually happened instead of disowning the run.
|
Resolution: the two path-level findings from this round Two of this round's findings had no line in the diff to hang off, so recording them here. The
|
vivek7405
left a comment
vivek7405
left a comment
There was a problem hiding this comment.
This round caught me codifying something that is not true. The paragraph I added to the blog's fixture inventory says the analyser reads tags and whenDefined calls out of COMMENTS, so the fixtures avoid writing them. That was true once and #179 closed it: maskComments runs before every signal scan. I drove it to be sure, with a display-only badge whose doc comment names its own tag in angle brackets AND calls whenDefined on it, and the badge is still elided. So the discipline I was promoting from an in-file aside into a normative rule guards nothing.
Worse than being wrong, it was wrong in the one place a future reader goes before touching a fixture, and it would have taught them to write around a mechanism that does not exist. Removing it rather than correcting its count, which also settles the second finding.
The third is a real seam I left open: --verify forces the override on, the plain report path reads the config, and the zero-drop message sends the author to a bare webjs elision. On the opted-out app that forcing exists FOR, that command answers elision is disabled and no verdict at all, so the hint dead-ends exactly where it is most needed.
The fixture inventory I added said the analyser reads tags and whenDefined calls out of COMMENTS, so the fixtures deliberately avoid that prose. That was true once; #179 closed it by masking comments before every signal scan. Driving a display-only badge whose doc comment names its own tag in angle brackets AND calls whenDefined on it leaves the badge elided, so the discipline guards nothing. Removing it rather than correcting its count, since a rule that protects against a closed failure mode is worse in the fixture inventory than in the aside it came from. forced-badge.ts carried the same claim and is new here, so it goes too. The zero-drop hint also dead-ended: --verify forces the override on while the plain report path reads the config, so on the opted-out app that forcing exists for, a bare `webjs elision` answers "elision is disabled" and no verdict. The hint now carries the same override the run used.
1 participant
Closes #1308
Summary
Elision is automatic and stays automatic, but until now only the benign half of its verdict was visible.
webjs doctorcould tell an app which pages and layouts ship when they could have been elided, whose worst outcome is a few wasted kilobytes. Nothing reported the other half: which component modules were DROPPED, and on what evidence. That is the half where a wrong verdict silently costs an app its interactivity in production, and an author had no way to ask what the framework removed from their page. The framework protects itself with a differential on/off test; an app had no equivalent, so every app inherited a guarantee it could not check locally.This makes the verdict inspectable and provable per app.
What changed
webjs elisionprints the per-module verdict: every component as elided or shipped, every shipped one naming the evidence that forced it (own/observed/closure/render/import/unreadable) and the module that did the forcing, every page and layout as inert / import-only / ships-whole, and any orphan class that gets no verdict at all.--jsonemits the same object, and the new MCPlist_elisiontool returns it verbatim.webjs elision --verifyboots two request handlers withWEBJS_ELIDEflipped, renders the app's own static route corpus through both, and diffs the masked SSR bytes. It is the framework's own guard pointed at an arbitrary app. It exits non-zero on a divergence AND on a corpus where nothing could be compared, because a verification command that can pass while comparing nothing is worse than no command. Its success output states its own boundary: it proves the bytes you serve did not change, not post-hydration behaviour.maskJsSethas exactly one definition now, inpackages/server/src/elision-differential.js, imported by both the framework's differential test and the CLI, so the two guards cannot drift on what "the JS-loaded set" even means.webjs doctorgrowsELISION_COMPONENTS, which passes with the elided inventory and warns only on an orphan, the one shape that is dropped with no verdict and no escape hatch. Both elision checks share ONEanalyzeAppElisioncall, so the module graph is still built once per doctor run.webjs devprints one summary line per warm analysis, re-emitted after each rebuild. Deliberately not a browser push, for the reason in the comment on the code: an inert route ships zero application JS, and a dev-only boot script would corrupt the network-tab check on exactly the pages this exists to prove.Two corrections the measurement forced.
static interactive = truedoes not rescue a component whose own registration tag is computed (four surfaces claimed it did), and the orphan scan no longer reports documentation code samples as orphans. Both have their own comments on this PR.Test plan
Every layer, all run and green.
npm test: 3978 pass, 0 fail. New:residual-contract.test.js(both residuals + the not-a-residual case, measured),elision-report.test.js(the whole contract, which had no test at all before),elision-differential.test.js, thesigil-coverageevidence guard,test/cli/elision.test.mjs, the doctor block, the MCP drift test, the orphan code-sample case.npm run test:browser: Chromium 755, Firefox 741, WebKit 755, 0 failed on each.WEBJS_E2E=1: 92 pass, 0 fail, including the newstatic interactive = truenetwork probe.node scripts/run-bun-tests.js: 297 pass, 0 genuine fail, no new DENYLIST entry.test/bun/elision-report.mjsasserts an identical verdict undernodeandbun(both run).noteShipcall reds the evidence guard and the matchingevidencecase; dropping theDOCTOR_CODESentry reds the gate case; deletingstatic interactive = truefrom the fixture reds the e2e probe.origin/main's side by side overexamples/blogandwebsiteand diffed the full elidable / inert / import-only / ships-whole sets. IDENTICAL on both. This change records why a component ships; it changes nothing about whether one does./,/docs/elision,/docs/configuration,/docs/components,/docs/data-fetching,/docs/progressive-enhancement,/ui,/ui/buttonin dist mode with no broken preloads;webjs checkclean on both apps;webjs doctorexits 0 on both.Docs
website/app/docs/elision/page.tsplus its nav entry. Elision was described across six pages with no page owning it, and the opt-out appeared on none of them.website/app/docs/configuration/page.tsgains thewebjs.elide/WEBJS_ELIDEsection, which existed on no docs page..agents/skills/webjs/):references/components.mdgains the evidence table, what the override does and does not rescue, and what to do with each verdict;references/testing.mdgains the two-run recipe;SKILL.mdgains the routing row.AGENTS.md(CLI reference, the elision bullet, the async-render paragraph),packages/server/AGENTS.md(three module-map rows plus invariant 7),packages/mcp/AGENTS.md,packages/core/src/component.d.ts.server-clock.ts. The agent skill is copied from the repo-root canonical at scaffold time, so there is no second copy to edit.README.md(no headline capability change), the editor plugins (no template grammar, snippet, or language-service surface changed), marketing copy (no positioning claim changed), a version bump (no published-package release in this PR).