Add signed preserving USB application updates

docs/development/firmware-upgrades.md

@@ -109,7 +109,8 @@ crashes/restarts before confirmation are handled by the rollback-enabled bootloa
 Confirmation does not establish audible quality or long-term runtime stability.
 
 The signed request remains durable through boot-slot selection and startup.
-A cut before selection retains the queue for a fresh safe-window download.
+A cut before selection retains a Wi-Fi queue for a fresh safe-window download.
+USB requests instead require an explicit owner decision and never become network downloads.
 A selected candidate waits for restart without erasing or selecting another slot.
 Startup reports rollback only when the requested inactive image is marked invalid
 or aborted by OTA metadata; the running requested version reports success after
@@ -156,3 +157,8 @@ successful slot switching, power/network interruption, failed startup, queue
 cancellation/restoration and concurrent networking/audio heap/fragmentation.
 Synthetic CI audio must never be installed or played. Publication and physical
 installation remain separate approvals.
+
+The additive [USB application-update protocol](usb-firmware-updates.md) reuses
+these trust, partition and startup contracts. It waits for owner power handoff
+after selection. The companion website keeps public USB writes disabled until
+new transport-specific physical qualification and a capable release are available.

docs/development/usb-firmware-updates.md

@@ -0,0 +1,107 @@
+# Preserving USB application updates
+
+This additive development protocol streams a signed application into the existing
+inactive 2 MiB OTA slot. It does not write the bootloader, partition table,
+settings/credentials, prayer-consumption history or 3.5 MiB shared audio. The
+public website keeps USB updating disabled until transport-specific physical
+qualification and publication of a capable release. Legacy released firmware
+continues to support credential recovery; it needs an initial Wi-Fi or maintainer
+update before using this extension.
+
+The [development validation report](usb-update-validation-2026-10-08.md) records
+automated results, matched OTA measurements and the outstanding physical limits.
+
+## Trust and ownership
+
+The firmware owns signature validation, hardware/layout/settings/audio-format
+compatibility, stable newer-version checks, the 1.5 MiB application budget, image
+hash and embedded version validation, inactive-slot selection, startup confirmation
+and rollback. It reuses the [existing upgrade contracts](firmware-upgrades.md).
+BEGIN requires a confirmed running application, compatible rollback bootloader,
+healthy services/storage, no playback and a safe prayer window. A missing clock
+permits transfer only while automatic playback is blocked and services are healthy.
+No firmware-health or rollback requirement is relaxed for Atom-only USB power.
+
+USB and Wi-Fi update ownership are exclusive. A pending credential join/scan blocks
+BEGIN; active USB requests block other USB credential/scan mutations and authenticated
+HTTP mutations except authenticated `POST /api/stop` with an empty JSON object.
+Stop remains available during transfer, interruption and power handoff. It stops
+playback without changing USB ownership or the durable update request; ordinary
+host, origin, JSON and authentication checks still apply. Read-only status and
+identity remain available. No USB transfer
+automatically restarts, retries chunks or becomes a Wi-Fi download.
+
+## Wire contract
+
+Use existing OpenAthan extension framing at 115200 baud. RPC request type 3 and
+response type 4 retain their field encoding. Error type 2 retains codes 1 invalid,
+2 unsupported and 255 busy/storage/uncertain selection.
+
+| Command | Fields | Successful response |
+| --- | --- | --- |
+| INFO `0x10` | none | `1`, installed version, source commit, `supported`/`unsupported`, state, `confirmed`/`pending`, result, queued version or empty, received application bytes |
+| BEGIN `0x11` | descriptor bytes, 1–8192 | nonzero token as 16 lowercase hexadecimal digits |
+| VERIFY `0x12` | token | `accepted`, verified offered version |
+| FINISH `0x13` | token | `verified`, selected version |
+| ABORT `0x14` | token | `aborted` |
+| ABORT `0x14` | `interrupted` | `aborted`, only for a rebooted incomplete USB request with no selection marker |
+
+DATA type 5 payload is a 13-byte header plus 1–242 bytes: little-endian uint64
+token, uint8 kind (0 descriptor, 1 application), little-endian uint32 exact byte
+offset. ACK type 6 returns the same token/kind with the next offset and no data.
+Out-of-order, duplicate, stale-token and oversized chunks are rejected, not replayed.
+No flash erase occurs before VERIFY validates the complete signed descriptor.
+Application chunks are copied into bounded internal stack RAM for flash writes;
+the descriptor scratch allocation is released after verification or abort.
+
+One browser owner permits one outstanding RPC or chunk. Every mutation is sent
+once. Missing/malformed acknowledgements or disconnects block subsequent writes;
+read-only reconciliation remains allowed. FINISH errors and failed post-FINISH
+readback require uncertainty handling even if a device error was received, because
+selection may already have happened. A transfer idle for more than 40 seconds
+aborts before selection. Once selected, a timeout cannot erase or cancel it.
+
+## Durable request and power handoff
+
+Existing Wi-Fi requests remain schema 1 with `queue` and `expected`. USB requests
+use schema 2 with `source: "usb"`, the signed `queue` and `expected`; no source is
+inferred from old records. The request is committed before erase, and the expected
+version before boot selection. Only confirmed capable firmware creates USB records.
+If interrupted before selection, reconciliation reports `usb_interrupted` and
+waits for explicit USB discard. If the exact candidate is selected, it reports
+`awaiting_power` and never reboots automatically. A selection error retains the
+journal and reports `usb_selection_uncertain` until authoritative reconciliation.
+Unreadable boot selection, missing image identification after a completed handoff,
+or contradictory metadata also retain this uncertain state; they never authorize
+another transfer or journal clearing. Recovered reads return a selected matching
+candidate to `awaiting_power`, including a request whose `expected` marker is empty.
+A successful read selecting the confirmed running application is required before
+reporting an interrupted transfer. An absent inactive-slot OTA entry is normal after
+staging and cannot establish rejection by itself. `ABORT interrupted` requires
+confirmed startup health and repeats the selection check under the updater lock
+before clearing an interrupted request. Rollback requires an identified requested
+image with a successfully read `INVALID` or `ABORTED` state. A newer running
+version supersedes an old USB request only when selection proves the candidate
+is unselected; version comparison alone cannot clear a pending selection.
+
+After FINISH, the browser reports **written and verified**, closes USB and directs
+the owner to unplug Atom USB and power only the Pyramid bottom port. Startup
+success is a separate observation on the device page. Existing 30-second health
+confirmation and 90-second failed-startup rollback apply; no network or valid clock
+is required for confirmation. The running requested version or an actually rejected
+candidate clears bookkeeping through the existing success/rollback reconciliation.
+Settings/history/audio formats and the previous application remain compatible.
+
+## Required qualification
+
+Host and browser tests establish software contracts, not physical preservation,
+power behavior or memory headroom. Use the provisioning hardware runbook for fresh
+preflight, current-state readback and private backups. Retain source/artifact hashes
+and test actual Chrome and Edge USB ownership; Atom-only transfer and bottom-only
+startup; success and failed-startup rollback; interruptions before descriptor
+verification, during writes and around selection; wrong signatures/images/tokens/
+offsets; competing Wi-Fi/settings activity; and before/after settings, credentials,
+consumption history and audio evidence. Measure runtime heap/fragmentation and
+prayer/audio responsiveness during transfer separately from binary size. Do not
+install synthetic CI recordings. Keep public activation and firmware publication
+separately reviewed after these results.

docs/development/usb-update-validation-2026-10-08.md

@@ -0,0 +1,176 @@
+# USB application-update development validation, 2026-10-08
+
+This report covers the additive USB updater and device-local status presentation
+in the `codex/usb-updates` change based on `310b978`. These are development builds,
+not release candidates or physical acceptance. The companion website retains
+`usbUpdateEnabled: false` until new qualification and a capable release.
+
+## Automated behavior
+
+- CMake/CTest: all 16 host tests passed.
+- Required Python runner: all 103 tests passed without skips, using the resolved
+  ArduinoJson headers and built audio/settings adapters.
+- Production updater runtime checks passed with UBSan for existing version
+  reconciliation, the new USB sender/receiver contract, ordinary isolated-build
+  rejection and qualification startup success/failure.
+- USB cases cover bad signatures before erase, exact token/offset and duplicate
+  rejection, conflicting Wi-Fi updates, offline-clock transfer with healthy
+  services, unhealthy-service refusal, hash mismatch, idle abort, a power cut
+  between durable handoff and selection, startup success/rollback, and selection
+  errors both before and after metadata changed. USB interruption never starts
+  an HTTP download; verified selection never automatically reboots.
+- Chromium and WebKit device UI cases cover USB transfer/handoff messages and
+  blocked network update/cancel controls. Browser endpoints are simulated with
+  the production Digest verifier.
+- Companion website validation covers actual published v0.4.0 signed descriptor
+  and OTA import, signature/image corruption rejection, session ownership,
+  ambiguous finalization, disabled public writes, whole-flow desktop/mobile
+  browser behavior and production simulator exclusion. It consumes artifacts
+  without compiling firmware.
+
+## Initial USB implementation measurements
+
+Baseline and candidate use identical configuration, compile-only synthetic audio,
+qualification trust material and pinned Python 3.13.0, ESPHome 2026.9.0, ESP-IDF
+5.5.5 and Xtensa compiler 14.2.0_20260121. The startup-failure variant uses
+qualification version v0.0.3 and its existing health-failure switch. Each build
+passed `check_feasibility.py`, including image/partition and dependency checks.
+No synthetic recording was installed or played.
+
+| Variant | Baseline OTA bytes | Candidate OTA bytes | Delta | Remaining 1.5 MiB budget | Free 2 MiB slot bytes |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Reference | 1,265,760 | 1,273,472 | +7,712 | 299,392 | 823,680 |
+| Provisioning isolated | 1,267,552 | 1,274,848 | +7,296 | 298,016 | 822,304 |
+| Upgrade qualification | 1,272,928 | 1,280,400 | +7,472 | 292,464 | 816,752 |
+| Startup failure | 1,272,928 | 1,280,400 | +7,472 | 292,464 | 816,752 |
+
+Candidate static RAM is 115,395 bytes for reference/provisioning and 115,467 bytes
+for qualification variants. These figures do not establish runtime heap,
+fragmentation or PSRAM headroom. Both application slots, the 3.5 MiB shared audio
+partition and bootloader allowlist remain unchanged.
+
+Initial candidate OTA SHA-256 identities from these measured development builds:
+
+| Variant | SHA-256 |
+| --- | --- |
+| Reference | `964b5517d4be6ad32bc0d3e9c0ce8a2b62f308d51f71f0b7e2d83941cb7b2fa2` |
+| Provisioning | `5f871b4ecff9b7c6d0b8b53159e1b0567ef30c06fc5af124c668654965e5636f` |
+| Qualification | `5721e64e734ec62ce6687525dbf4c290a7f17b227d8ece49126b1ffdd24d264d` |
+| Startup failure | `c53b8f67f36023a36919471fcc0541f78180809629a6d63dc50c491acc94625e` |
+
+## USB reconciliation correction
+
+The correction compares exact initial PR head `4143747` with the shared USB
+selection classifier and fresh interrupted-discard guard. Both stages used the
+same pinned toolchain, configurations, synthetic audio, qualification trust and
+`development` build identity described above. All four before/after builds passed
+capacity, dependency, partition and image-consistency checks.
+
+The regression failed against the original updater: unreadable selection became
+`usb_interrupted` even though the candidate remained selected. Corrected UBSan
+runtime checks cover failed boot/description/state reads, absent state entries,
+mismatched versions and unexpected partitions after successful or ambiguous
+FINISH; repeated loops/restart; requests with an already-empty handoff marker;
+read recovery; changed selection at discard time; incomplete unselected images;
+true rejection, healthy/unhealthy startup and superseded USB requests whose
+older candidate is still selected. Uncertain or selected requests
+retain their journal and USB ownership without writes, new selection, reboot or
+HTTP requests. A positive current-slot selection permits explicit incomplete
+transfer discard, including absent inactive metadata. Existing Wi-Fi, ordinary
+isolated and qualification profiles also pass.
+
+CMake/CTest passed all 16 tests; the complete required Python suite passed all 103
+without skips. The resolved SDK request/header regressions passed. Focused
+Chromium/WebKit checks passed both USB-status cases, including uncertainty
+keeping network actions disabled and cancellation hidden. The complete device UI
+rerun passed all 466 tests without skips.
+
+| Variant | Before OTA bytes | Corrected OTA bytes | Delta | Remaining 1.5 MiB budget | Free 2 MiB slot bytes |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Reference | 1,273,472 | 1,274,144 | +672 | 298,720 | 823,008 |
+| Provisioning isolated | 1,274,848 | 1,275,568 | +720 | 297,296 | 821,584 |
+| Upgrade qualification | 1,280,400 | 1,281,184 | +784 | 291,680 | 815,968 |
+| Startup failure | 1,280,400 | 1,281,184 | +784 | 291,680 | 815,968 |
+
+Static RAM remains 115,395 bytes for reference/provisioning and 115,467 bytes for
+qualification variants. The correction introduces no new dependency or persistent
+allocation. Runtime heap/fragmentation and PSRAM remain physically unqualified.
+Both application slots and shared audio retain their existing layout.
+
+Corrected development-build OTA identities:
+
+| Variant | SHA-256 |
+| --- | --- |
+| Reference | `5e371b03e527b76b6b74a162c7b71e6d53db0fc27c0f1497fbb5934578b8895b` |
+| Provisioning isolated | `663f9a09d770c0bc90f6118e4498e856dde60749f98af6264c0350078b4baa2c` |
+| Upgrade qualification | `c2c19a64f5c300c730b9500cbddcced06aaf98938f4781e3953e9edfcc3dc627` |
+| Startup failure | `2bfc1d9601d9699a0f392c611f0feeb4058dd097ed28f8106625a86e1ba89c89` |
+
+## Playback Stop correction
+
+USB ownership now permits only the exact authenticated `POST /api/stop` endpoint
+alongside existing reads. It still applies maintenance, host, method, origin,
+JSON content-type, Digest authentication and rate-limit checks before dispatching
+to the unchanged local API. Other mutations remain blocked throughout transfer,
+interruption and power handoff.
+
+The new UBSan HTTP regression compiles the production handler, host validator
+and helpers unchanged with the real Digest verifier and USB updater. Platform
+I/O is deterministic and local API dispatch is observed through a test adapter.
+It fails against `39c3d14`: a partial application transfer leaves a durable
+request after simulated restart, and authenticated Stop returns 409. The
+correction returns 200 in `usb_interrupted`, `usb_descriptor`, `usb_receiving`,
+`usb_selection_uncertain` and `awaiting_power`. It preserves USB ownership,
+the journal, NVS commit count, OTA writes/erases, boot selection and reboot count.
+Other writes and similar Stop URLs remain blocked; unauthenticated, invalid
+origin/host/content-type/method, replayed and rate-limited requests cannot stop
+playback.
+
+Production and isolated local API tests use the real scheduler to start a
+scheduled prayer, reject malformed/non-object/nonempty Stop payloads, then verify
+valid Stop returns `playing: false` without changing saved settings or any
+persisted settings/history records. CMake/CTest passed all 16 tests; the required
+Python suite passed all 103 without skips. All eleven updater/HTTP runtime
+profiles passed with UBSan, and the resolved SDK request/header regression passed.
+
+The complete device UI suite passed all 486 Chromium/WebKit cases without skips
+using Node 24.19.0 and locked Playwright 1.62.1. Twenty new cases verify Stop in
+Today and Settings across all five USB states, authoritative stopped feedback,
+unchanged device settings/update state and blocked network update controls.
+Browser endpoints remain simulated with the production Digest verifier.
+
+Matched builds compare `39c3d14` with the Stop correction using the same pinned
+toolchain, configurations, synthetic audio, qualification trust and `development`
+build identity described above. All eight builds passed capacity, dependency,
+partition and image-consistency checks.
+
+| Variant | Before OTA bytes | Corrected OTA bytes | Delta | Remaining 1.5 MiB budget | Free 2 MiB slot bytes |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Reference | 1,274,144 | 1,274,176 | +32 | 298,688 | 822,976 |
+| Provisioning isolated | 1,275,568 | 1,275,616 | +48 | 297,248 | 821,536 |
+| Upgrade qualification | 1,281,184 | 1,281,216 | +32 | 291,648 | 815,936 |
+| Startup failure | 1,281,184 | 1,281,216 | +32 | 291,648 | 815,936 |
+
+Static RAM remains 115,395 bytes for reference/provisioning and 115,467 bytes for
+qualification variants. No new dependency, persistent allocation or partition
+change is introduced. These figures do not establish runtime heap or PSRAM
+headroom, and no hardware was accessed or changed.
+
+Stop-corrected development-build OTA identities:
+
+| Variant | SHA-256 |
+| --- | --- |
+| Reference | `4068373c9713a7879792b48f143ce33aaff3a12e1312136db5cb7ea5d1410b40` |
+| Provisioning isolated | `02cc91ae6e4001dd6d1420b4e95b5437dd7297d2d387592cd31b435d1b8a02ef` |
+| Upgrade qualification | `2c03cdc00e3e33b8326e69e37070e12e0db12bd008870f394c061937a607995a` |
+| Startup failure | `63c95565f3a36c713ee7e493363b7c2c6a3715aa921e386c7ee690ee9c1bc188` |
+
+## Physical and runtime limits
+
+No hardware was accessed or changed. Atom-only USB health/transfer, bottom-power
+startup, actual credential/settings/history/audio preservation, interruption
+recovery, failed-startup rollback and concurrent audio/network heap/fragmentation
+remain untested for this transport. Existing Wi-Fi qualification cannot substitute
+for these results. Follow the [USB qualification contract](usb-firmware-updates.md)
+and current hardware runbooks before activation. Firmware publication and public
+USB enablement require separate review and approval.