Wait for complete Cloudflare asset propagation during verification

docs/custom-domain-plan.md

@@ -27,3 +27,9 @@ The remaining issues are bounded:
 - Report real deployment evidence separately from local tests. Cloud/C# Containers and the future Hello API are a subsequent worktree, after this delivery step is verified.
 
 References: [Workers subdomain configuration](https://developers.cloudflare.com/workers/configuration/routing/workers-dev/), [Web Analytics and no-transform](https://developers.cloudflare.com/web-analytics/get-started/).
+
+## Targeted live correction
+
+Main run `35032047709` deployed the custom-domain configuration, but its immediate live check saw a response without `no-transform` after the new identity was already available. Without another deployment, every public file then returned the expected headers and all twelve real browser tests passed. Identity readiness alone therefore cannot establish that every edge response is ready.
+
+The correction is limited to delivery verification: wait at most two minutes for the complete read-only file/header/404 check to converge, with a shared abort deadline and per-request limits. Preserve exact content checks, report the last failing path, never redeploy automatically, and still fail permanently incorrect content/configuration. Browser assertions remain mandatory and are not automatically retried.

docs/deploying.md

@@ -25,7 +25,7 @@ The token stays in CI. The browser never receives Cloudflare management credenti
 3. Chromium, Firefox and WebKit test that candidate served through local Wrangler, including accessibility, no-JavaScript behavior, local greeting, CSP/cache behavior and missing-path 404s.
 4. `Verify` requires all applicable checks. On a `push` to `main`, the deployment job downloads the exact candidate artifact by its ID and verifies source/hashes. It never rebuilds it. A skipped PR-only dependency review cannot skip deployment.
 5. Missing account/token configuration is an explicit failure. The job checks that the source is still current main and that `arcforges.com` belongs to `arcforges-web` in production, serializes deployment, and deploys with Wrangler. Checking the mapping only needs the existing Workers Scripts permission.
-6. Bounded, read-only HTTPS polling waits for the expected `https://arcforges.com/__build.json`; then every public candidate file is compared by hash, security/cache headers are checked, and missing routes must return the real 404. HTML requests explicitly accept HTML. The existing Chromium, Firefox and WebKit suite then runs against the real domain to check browser-visible edge behavior. A failed check never automatically creates another deployment.
+6. Bounded, read-only HTTPS polling waits for the expected `https://arcforges.com/__build.json`; then every public candidate file is compared by hash, security/cache headers are checked, and missing routes must return the real 404. A shared two-minute deadline permits those responses to converge after the identity becomes available; permanent differences fail with the last diagnostic. HTML requests explicitly accept HTML. The existing Chromium, Firefox and WebKit suite then runs against the real domain to check browser-visible edge behavior. A failed check never automatically creates another deployment.
 7. Only successful file and browser verification creates a GitHub prerelease containing the original candidate archive and deployment evidence. Versions are `0.1.0-ci.<run_number>.<run_attempt>`; reruns are distinct. These are preview releases, not an assertion that the complete product is implemented.
 
 PR, schedule and manually dispatched workflows validate but do not deploy. Main pushes deploy automatically once the credential is present. No second Cloudflare Git integration is required; enabling one would create an independent deployment path that bypasses this candidate process.

tests/unit/delivery.test.ts

@@ -5,7 +5,12 @@ import { tmpdir } from "node:os";
 import { join } from "node:path";
 import { expect, test } from "vitest";
 import { contentSecurityPolicy, releaseVersion, seal, verify } from "../../tooling/project.ts";
-import { deploymentUrl, requireCustomDomain, waitForIdentity } from "../../tooling/cloudflare.ts";
+import {
+  deploymentUrl,
+  requireCustomDomain,
+  waitForDelivery,
+  waitForIdentity,
+} from "../../tooling/cloudflare.ts";
 test("CSP authorizes exact prerendered scripts without unsafe inline/eval", () => {
   const script = "console.log('hello')";
   const csp = contentSecurityPolicy([`<script>${script}</script><script src='/x.js'></script>`]);
@@ -77,3 +82,24 @@ test("deployment requires the custom domain to belong to this production Worker"
   ])
     expect(() => requireCustomDomain(domains)).toThrow("Attach arcforges.com");
 });
+
+test("asset verification waits for headers as well as identity and fails boundedly", async () => {
+  let reads = 0;
+  await waitForDelivery(
+    async (signal) => {
+      expect(signal.aborted).toBe(false);
+      reads++;
+      if (reads === 1) throw new Error("Missing no-transform: /404.css");
+    },
+    { timeoutMs: 1000, intervalMs: 0 },
+  );
+  expect(reads).toBe(2);
+  await expect(
+    waitForDelivery(
+      async () => {
+        throw new Error("Deployed bytes differ: /hello/");
+      },
+      { timeoutMs: 10, intervalMs: 0 },
+    ),
+  ).rejects.toThrow("no redeployment was attempted. Error: Deployed bytes differ: /hello/");
+});

tooling/cloudflare.ts

@@ -3,6 +3,7 @@ import assert from "node:assert/strict";
 import { readFile } from "node:fs/promises";
 import { join, resolve } from "node:path";
 import { fileURLToPath } from "node:url";
+import { setTimeout as delay } from "node:timers/promises";
 import { candidate, digest, json, npm, root, run, save, verify } from "./project.ts";
 
 const statePath = join(root, "artifacts/deployment.json");
@@ -58,6 +59,30 @@ export async function waitForIdentity(
     "Deployed assets did not reach the expected source/version within the read-only polling window. No redeployment was attempted.",
   );
 }
+export async function waitForDelivery(
+  probe: (signal: AbortSignal) => Promise<void>,
+  options: { timeoutMs?: number; intervalMs?: number } = {},
+) {
+  const signal = AbortSignal.timeout(options.timeoutMs ?? 120000);
+  let lastError: unknown;
+  while (!signal.aborted) {
+    try {
+      await probe(signal);
+      signal.throwIfAborted();
+      return;
+    } catch (error) {
+      lastError = error;
+    }
+    if (!signal.aborted) {
+      try {
+        await delay(options.intervalMs ?? 2000, undefined, { signal });
+      } catch (error) {
+        if (!signal.aborted) throw error;
+      }
+    }
+  }
+  throw new Error(`Public assets did not converge; no redeployment was attempted. ${lastError}`);
+}
 async function deploy() {
   const manifest = await verify();
   assert(!manifest.dirty, "Commit the source before deployment");
@@ -139,43 +164,52 @@ async function smoke() {
   checkUrl(state.url);
   console.log("Waiting for the published static candidate using read-only HTTPS requests.");
   await waitForIdentity(state.url, state);
-  for (const [path, hash] of Object.entries(manifest.files)) {
-    if (!path.startsWith("assets/") || path === "assets/_headers" || path === "assets/404.html")
-      continue;
-    let route = path.slice("assets".length);
-    if (route.endsWith("/index.html")) route = route.slice(0, -"index.html".length);
-    const response = await fetch(`${state.url}${route}`, {
-      headers: { Accept: route.endsWith("/") ? "text/html" : "*/*" },
-      redirect: "error",
-      signal: AbortSignal.timeout(15000),
-    });
-    assert(response.ok, `Deployed ${route} returned ${response.status}`);
-    assert.equal(
-      digest(new Uint8Array(await response.arrayBuffer())),
-      hash,
-      `Deployed bytes differ: ${route}`,
-    );
-    assert.equal(response.headers.get("x-content-type-options"), "nosniff");
-    assert(response.headers.get("cache-control")?.includes("no-transform"));
-    if (route.endsWith("/"))
+  await waitForDelivery(async (signal) => {
+    for (const [path, hash] of Object.entries(manifest.files)) {
+      if (!path.startsWith("assets/") || path === "assets/_headers" || path === "assets/404.html")
+        continue;
+      let route = path.slice("assets".length);
+      if (route.endsWith("/index.html")) route = route.slice(0, -"index.html".length);
+      const response = await fetch(`${state.url}${route}`, {
+        headers: { Accept: route.endsWith("/") ? "text/html" : "*/*" },
+        redirect: "error",
+        signal: AbortSignal.any([signal, AbortSignal.timeout(15000)]),
+      });
+      assert(response.ok, `Deployed ${route} returned ${response.status}`);
+      assert.equal(
+        digest(new Uint8Array(await response.arrayBuffer())),
+        hash,
+        `Deployed bytes differ: ${route}`,
+      );
+      assert.equal(
+        response.headers.get("x-content-type-options"),
+        "nosniff",
+        `Missing nosniff: ${route}`,
+      );
       assert(
-        response.headers.get("content-security-policy")?.includes("script-src 'self' 'sha256-"),
-        "Missing generated CSP",
+        response.headers.get("cache-control")?.includes("no-transform"),
+        `Missing no-transform: ${route}; received ${response.headers.get("cache-control")}`,
       );
-    if (route.startsWith("/assets/"))
-      assert(response.headers.get("cache-control")?.includes("immutable"));
-  }
-  for (const path of ["/api/missing", "/assets/missing.js", "/not-a-page"]) {
-    const response = await fetch(state.url + path, {
-      signal: AbortSignal.timeout(15000),
-      redirect: "error",
-    });
-    assert.equal(response.status, 404, `Unexpected fallback at ${path}`);
-    assert.equal(
-      digest(new Uint8Array(await response.arrayBuffer())),
-      digest(await readFile(join(candidate, "assets/404.html"))),
-    );
-  }
+      if (route.endsWith("/"))
+        assert(
+          response.headers.get("content-security-policy")?.includes("script-src 'self' 'sha256-"),
+          "Missing generated CSP",
+        );
+      if (route.startsWith("/assets/"))
+        assert(response.headers.get("cache-control")?.includes("immutable"));
+    }
+    for (const path of ["/api/missing", "/assets/missing.js", "/not-a-page"]) {
+      const response = await fetch(state.url + path, {
+        signal: AbortSignal.any([signal, AbortSignal.timeout(15000)]),
+        redirect: "error",
+      });
+      assert.equal(response.status, 404, `Unexpected fallback at ${path}`);
+      assert.equal(
+        digest(new Uint8Array(await response.arrayBuffer())),
+        digest(await readFile(join(candidate, "assets/404.html"))),
+      );
+    }
+  });
   console.log("Verifying the real domain in Chromium, Firefox and WebKit.");
   process.stdout.write(
     npm(["exec", "--no", "--", "playwright", "test", "--config", "playwright.live.config.ts"]),