diff --git a/.agents/skills/webjs/references/built-ins.md b/.agents/skills/webjs/references/built-ins.md index 4c04b6b08..4c550adca 100644 --- a/.agents/skills/webjs/references/built-ins.md +++ b/.agents/skills/webjs/references/built-ins.md @@ -8,7 +8,7 @@ Env vars, caching, rate limiting, broadcast, file storage, and the `package.json - **Caching primitives.** `cache()` with tag invalidation, HTTP `Cache-Control`, the server HTML response cache (`export const revalidate`), content-hash asset URLs, conditional GET (ETag). - **Rate limiting** (`rateLimit()` middleware) and **broadcast** (`broadcast()` over WebSockets). - **File storage.** `FileStore` / `diskStore`, safe keys, signed URLs. -- **The `"webjs"` config block.** Security headers, CSP, redirects, trailing-slash, basePath, allowed origins, client-router opt-out, ingress caps, dev/start task orchestration. +- **The `"webjs"` config block.** Security headers, CSP, redirects, trailing-slash, basePath, allowed origins, client-router opt-out, ingress caps, dev/start task orchestration, the doctor severity gate. - **Observability.** Access log, `requestId()`, the `onError` hook, `instrumentation.ts`, the build-info endpoint. Read this when wiring caching or rate limiting, storing uploads, hardening headers, or configuring redirects and observability. **Auth and sessions are a separate reference (`auth-and-sessions.md`).** Server actions, `revalidateTag` from a mutation, and the `ActionResult` envelope live in `data-and-actions.md`. @@ -93,7 +93,7 @@ html`` That emits `/public/app.css?v=` in production and gets the immutable year; the same url un-marked gets a ~1h cap and can serve stale bytes from a CDN after a deploy until something purges it. `asset()` resolves on the server; the browser has no resolver and returns the path unchanged. Call it from a PAGE, LAYOUT, or metadata route, which render only on the server. Inside a component that ships to the browser it silently costs you the caching: hydration is a full client re-render, so the bare path overwrites the hashed one and the asset downloads twice. The url stays valid either way, so this is a convention rather than a `webjs check` rule (`webjs doctor` does flag the plain form, see below). Under `webjs.basePath`, include the prefix yourself (`asset('/app/public/x.css')`): the framework base-path-prefixes only the urls it emits, so an author-written url is already yours to prefix. Two more constraints: call it INSIDE the render function, because a module-scope call is a side effect the elision analyser reads as client work and it ships the whole module; and mark only files that change with a DEPLOY, because the hash is memoized for the process lifetime, so a `public/` file rewritten in place at runtime would keep its old url while being served `immutable` for a year. Off in dev, so dev output is byte-identical. Only `public/` paths resolve; anything else (and a path that fails to resolve) is returned untouched. -Forgetting it is the one real cost of opt-in, so `webjs doctor` catches it: a page, layout, or error boundary writing a plain `` gets a WARN naming the `file:line` and the fix (#1095). It reads your source and rewrites nothing, and it stays quiet about the non-marks that are deliberate: a cross-origin sheet, a `rel="icon"`, a `rel="preload"`, and any `href=${expr}` hole. Same posture as Rails (a `stylesheet_link_tag` helper over a digest manifest) and Remix (a hashed url from the build graph, surfaced through `links()`): take the fingerprint at the point the url is PRODUCED, never by rewriting a rendered document. +Forgetting it is the one real cost of opt-in, so `webjs doctor` catches it: a page, layout, or error boundary writing a plain `` gets a WARN naming the `file:line` and the fix (#1095). It reads your source and rewrites nothing, and it stays quiet about the non-marks that are deliberate: a cross-origin sheet, a `rel="icon"`, a `rel="preload"`, and any `href=${expr}` hole. Same posture as Rails (a `stylesheet_link_tag` helper over a digest manifest) and Remix (a hashed url from the build graph, surfaced through `links()`): take the fingerprint at the point the url is PRODUCED, never by rewriting a rendered document. A warning is easy to miss, so make it fatal in the app that cares: gate `UNMARKED_ASSET_LINKS` to `error` (see the doctor severity gate below) and one `npm run doctor` step in CI stops the un-versioned url reaching a deploy. The scaffold ships exactly that. It is opt-in rather than automatic because only the author knows which urls are the REQUEST. Do NOT mark a `rel="preload"` hint whose asset is actually fetched by CSS `url()`: the preload cache is keyed on the full url, so a versioned hint can never satisfy the unversioned request the stylesheet makes, and the file is fetched twice. Mark the thing that fetches, not the hint. Every cacheable response also carries a weak `ETag`, and a repeat request with a matching `If-None-Match` gets a `304 Not Modified` with no body. Unstorable (`no-store`) and streamed responses are excluded from the ETag path. A `private` response IS validated: `private` forbids SHARED storage, not validation, and the ETag hashes that response's own body, so two users with different bodies get different ETags and neither can match the other's, while two users with identical bodies are asking about identical bytes, where a 304 discloses nothing (#1140). That is what keeps the client router's partial responses cheap on a page that opted into caching; a default `no-store` page has nothing to validate either way. Dev is byte-faithful (no hashing). @@ -145,7 +145,7 @@ setFileStore(diskStore({ dir: '/var/data/uploads', baseUrl: '/files' })); ## The `"webjs"` config block (package.json) -All keys are optional; a malformed entry is dropped at boot with a warning, never crashing the pipeline. +All keys are optional, and a malformed entry in a key the SERVER reads is dropped at boot with a warning, never crashing the pipeline. The one exception is `doctor.gate`, which is read by the `webjs doctor` CLI rather than the server and rejects a bad entry outright (see the doctor severity gate below): a gate whose typo was quietly ignored would leave CI un-gated while looking gated, which is the one thing that mechanism cannot afford. ### Security headers @@ -210,6 +210,23 @@ An over-limit body responds `413` without buffering the whole payload. `before` runs to completion first (a non-zero exit aborts the boot). `parallel` (dev only) runs long-lived watchers alongside the server and tears them down on exit. `watch` (dev only) adds extra live-reload directories outside the app tree. +### Doctor severity gate + +`webjs doctor` reports project health, and by default only a broken toolchain fails the exit. `--strict` makes EVERY warning fatal, which is unusable in CI, because four checks are environment-shaped: `GIT_HOOK` wants a local pre-commit hook a runner has no reason to have, `ENV_DRIFT` compares against a `.env` CI does not carry, `VENDOR_PIN` fetches the network, and `FRAMEWORK_RESOLVE` depends on the environment. So per-check severity is CONFIG, keyed by the stable code every result carries. + +```jsonc +{ "webjs": { + "doctor": { "gate": { + "UNMARKED_ASSET_LINKS": "error", // fail the exit on this one + "ELISION_CARRIERS": "off" // silence it entirely, even under --strict + } } +} } +``` + +Three levels, the same scale ESLint uses: `error` fails the exit, `warn` reports without failing, `off` silences the check, meaning its finding is not printed and it cannot fail the exit (it still appears on the checklist as `[off]` and in the summary's silenced count, so a silenced check is never invisible, and `--json` still carries the whole result). A code with no entry keeps its default (`error` for a hard toolchain failure, `warn` otherwise), so an app that declares nothing behaves exactly as before. Read the codes off `webjs doctor --json`, where every result carries its `code` and its effective `severity`. + +Two guarantees worth knowing. A result that could not check (a network or toolchain outage) is capped at `warn` and can never be escalated, so a jspm or npm outage cannot red your CI. And a malformed gate exits 1 naming the offender rather than being ignored, so a typo cannot silently un-gate the build. That covers an unknown code, a bad severity, a wrong shape (a non-object `doctor` or `gate`), and a misspelled sibling of `gate` such as `gates`, since every one of those would otherwise leave the build un-gated while the `package.json` looks gated. Under `--json` the offenders come back as a `configErrors` array alongside an empty `results`, each entry a `{ kind }` of `malformed` / `unknown-key` / `unknown-code` / `bad-severity`. Wire it up with one workflow step, `npm run doctor`, and change what is fatal in `package.json` rather than in the workflow. + ## Observability Wired at the single response funnel, covering pages, routes, actions, and assets uniformly. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c5cb50d5a..71167f120 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -39,6 +39,22 @@ jobs: ( cd "$app" && node "$GITHUB_WORKSPACE/packages/cli/bin/webjs.js" check ) echo "::endgroup::" done + # Project health on the same four apps (#1257). WHICH findings are fatal + # is each app's own call, declared in its package.json `webjs.doctor.gate` + # rather than here, so a local `webjs doctor` and this step agree. Today + # website + examples/blog gate UNMARKED_ASSET_LINKS to error. Two checks + # red with no gate entry at all, NODE_VERSION and TSCONFIG_ERASABLE, + # since either would 500 an app at runtime; everything else stays a warn + # and cannot red this job. Deliberately NOT --strict: + # the git-hook, env-drift, vendor-pin, and framework-resolve checks are + # environment-shaped and would fail a perfectly healthy runner. + - name: webjs doctor (blog, website, docs host, ui host) + run: | + for app in examples/blog website docs packages/ui/packages/website; do + echo "::group::webjs doctor $app" + ( cd "$app" && node "$GITHUB_WORKSPACE/packages/cli/bin/webjs.js" doctor ) + echo "::endgroup::" + done - name: Framework runtime packages are buildless (no .ts source) run: | # Invariant: packages/{core,server,cli} and packages/editors/* are diff --git a/AGENTS.md b/AGENTS.md index 795c4a198..11ba7368e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -85,7 +85,7 @@ Every code change MUST include, automatically: 1. **Tests, every applicable layer (not just unit).** Ship the tests that prove the change across EVERY layer it touches: **unit** (`packages/*/test/**`, `test/**`, including the counterfactual that fails when reverted), **browser** (`*/test/**/browser/*` via `npm run test:browser`, for hydration / DOM / slots / client router / custom-element upgrade), **e2e** (`test/e2e/*.test.mjs` via `WEBJS_E2E=1`, including network probes / navigation / streaming), and **smoke** (`test/examples/*/smoke/*`). A unit test is NECESSARY BUT NOT SUFFICIENT for any client-router / component / browser-facing change (the headline behaviour is a browser/e2e assertion). **Bun parity is part of the task, not an afterthought:** WebJs runs on Node 24+ AND Bun (#508), so a change to a runtime-sensitive surface (the serializer, the node:http vs `Bun.serve` listener + request path, SSR / action / CSRF dispatch, streams, `node:crypto`, the TS stripper, auth / session / cors) MUST be proven on Bun (`node scripts/run-bun-tests.js` + the touched `test/bun/*.mjs` under `bun`) AND ship an added/updated `test/bun/.mjs` cross-runtime assertion. `npm test` does NOT run browser, e2e, or Bun; run them yourself and report the result. Never report work done with failing or missing tests. See `references/testing.md`. Enforced by `.claude/hooks/require-tests-with-src.sh` (the scaffold variant WARNS unless `WEBJS_TEST_GATE=block`) and `.claude/hooks/require-bun-parity-with-runtime-src.sh` (BLOCKS a commit that stages runtime-sensitive source with no `test/bun/**` test; escape hatch `WEBJS_BUN_VERIFIED=1`). 2. **Documentation, part of the definition of done (not optional).** A task is NOT done until EVERY doc surface its change touches is in sync: `AGENTS.md` + the skill at `.agents/skills/webjs/` (SKILL.md + references/) for new API surface, `CONVENTIONS.md` (and per-package `AGENTS.md`) for new conventions, the docs site (`website/app/docs/`), the marketing `website/`, the scaffold templates (`packages/cli/templates/` per-agent rule files), and `README.md` for a headline capability. Updating `AGENTS.md` alone reproduces the #488 gap (docs site left stale). Invoke the `webjs-doc-sync` skill to sync every applicable surface. Enforced by `.claude/hooks/require-docs-with-src.sh`, which BLOCKS a commit that stages public `packages/*/src` source with no doc surface alongside it (a genuinely internal refactor / CI / release / perf change with no behaviour change bypasses with `WEBJS_NO_DOC_GATE=1`). 3. **Scaffold + skill sync (when a feature changes what apps should do).** The scaffold `webjs create` emits is a gallery index home + a root layout + db wiring, a densely-commented feature gallery (`packages/cli/templates/gallery/**`, single-concept demos under `app/features/` plus the `app/examples/todo` app, shipped in every UI template) and the api backend-features showcase (`packages/cli/lib/api-gallery.js`), plus the one cross-agent skill at `packages/cli/templates/.agents/skills/webjs/` (SKILL.md + references). So when a WebJs feature is added or changed, ask: does the generator (`packages/cli/lib/{create,api-gallery}.js`), a gallery demo (`packages/cli/templates/gallery/`), or the agent skill (`.agents/skills/webjs/SKILL.md` + its `references/`) need to move so a freshly scaffolded app and the skill teach the new reality? Verify by generating an app and running `generate + boot + webjs check` (the generators emit strings, so an escaping bug only shows in a freshly generated app). See `framework-dev.md`. -4. **Convention validation.** Run `webjs check` and fix violations. +4. **Convention validation.** Run `webjs check` and fix violations. Run `webjs doctor` too when you touched an in-repo app (`examples/blog`, `website`, `docs`, `packages/ui/packages/website`): the required `conventions` CI job runs it over all four, and it fails on a hard toolchain check or on whatever that app's `webjs.doctor.gate` marks `error` (today `UNMARKED_ASSET_LINKS` in `website` and `examples/blog`), so a clean `webjs check` alone is not enough to predict that job (#1257). ### Git workflow (mandatory) @@ -506,7 +506,7 @@ webjs test [--server] [--browser] [--watch] webjs check [--rules] [--json] # correctness validator (report-only, no autofix); --json for an agent loop webjs routes [--json] [--table] [--no-headers] # print the route table (path / owner file / methods, #975). Default tree; --json is byte-identical to the MCP list_routes tool; --no-headers drops the --table header for piping webjs mcp # read-only MCP: routes, actions (RPC hashes), components, check, ui kit -webjs doctor [--json] [--strict] # project-health checklist (incl. a framework-resolve check that warns when @webjsdev/core can't be resolved from the app dir, the fresh-worktree-without-node_modules trap #954; a page/layout elision advisory; a warning when a route module writes a `` without `asset()`, #1095); non-zero exit on a hard fail. --json emits `{ results, summary }` (results is the DoctorResult[], each carrying a stable code); --strict also fails the exit on warnings (#975) +webjs doctor [--json] [--strict] # project-health checklist (incl. a framework-resolve check that warns when @webjsdev/core can't be resolved from the app dir, the fresh-worktree-without-node_modules trap #954; a page/layout elision advisory; a warning when a route module writes a `` without `asset()`, #1095); non-zero exit on a hard fail OR on a check the app gated `error`. --json emits `{ results, summary }` (results is the DoctorResult[], each carrying a stable code + its effective severity; summary counts pass/warn/fail/off), plus a third `configErrors` key on the one path where a rejected `webjs.doctor` config stops any check running; --strict additionally fails on every REMAINING warning (#975). Per-check severity is CONFIG, not a flag: `webjs.doctor.gate` maps a code to `off` / `warn` / `error` so CI gates a chosen subset (#1257) webjs types # generate .webjs/routes.d.ts (typed Route union + per-route params, #258) webjs version # print the installed @webjsdev/cli version (also: webjs --version / -v, #975) webjs help [command] # full usage banner, or per-command usage + Options + Examples (e.g. webjs help routes, #975). Flag forms: webjs --help / -h (banner), webjs --help / -h (that command). typecheck/db/ui --help forward to their wrapped tool; an unknown topic exits 1 @@ -524,7 +524,7 @@ webjs vendor pin|unpin|list|audit|outdated|update [--from PROVIDER] # importma ## Environment, server config, caching, observability - **Env vars.** `process.env.X` reads are server-only; `WEBJS_PUBLIC_`-prefixed names are exposed in the browser via an inline `