[FIX] Reach the escrow route the contract serves, and retry a 401 once on the typed path - #434
Open
justin13888 wants to merge 11 commits into
Open
justin13888 wants to merge 11 commits into
justin13888 wants to merge 11 commits into
Conversation
… client
`capsule_sdk::recovery` built `{api_root}/backup/escrow` from a `const` and sent it
with hand-written `reqwest` calls. The committed Kynos document serves
`GET`/`PUT /v1/auth/escrow`, so every networked recovery flow — enroll, the
stale-cache refresh, and the guided re-wrap's escrow replace — failed against a real
server while S-D12 read `done`.
The route is not fixed by editing the constant. Both operations are
`application/octet-stream` in each direction, which is a media type spargen lowers, so
both are already generated and neither is narrowed out in `build.rs`; `AGENTS.md`
requires that everything which parses or serializes is generated, the byte-serving
endpoints included. `RecoveryClient` now holds one `AuthenticatedClient` and
orchestrates `fetch_escrow`/`store_escrow`, so the path is a function of the document
and cannot drift again.
What the move changes:
- `RecoveryClient::new` is fallible (`RecoveryError::InvalidBaseUrl`) — the generated
client parses its base once at construction rather than per call. The two FFI
callers each grow a `?`.
- `RecoveryError` drops `Body(reqwest::Error)` and `Auth(AuthError)`, which nothing can
construct once the reqwest path is gone, and gains `Transport`, `Unauthorized`,
`Malformed` and `InvalidBaseUrl`, plus an `error_code()` returning the stable
`error.escrow.*`/`error.auth.*` code a client localizes.
- A refused credential keeps its auth identity across the FFI boundary:
`Unauthorized` maps to `FfiError::Auth`, where a failed refresh used to arrive as
`RecoveryError::Auth`. A request-construction failure maps there too — these
operations take no parameters and their base URL is already parsed, so the only way
either fails before a byte leaves is the bearer provider, and a dead session must
reach a caller as one rather than as a transport blip.
Both in-repo mocks were answering whichever path they were handed, which is why the
wrong route survived. They now route on `/v1/auth/escrow` and answer `501` elsewhere,
so a route regression fails loudly instead of reading as "no escrow stored", and their
refusals carry real RFC 9457 bodies because a generated operation decodes them.
The proof the old tests could not give is a new case in
`capsule-server/tests/sdk_client.rs`: the SDK stores and fetches a real wrap over a
socket against the assembled router, and asserts the bytes come back byte-identical
and still open under the recovery secret.
Refs #408
Deploying capsule with
|
| Latest commit: |
da86f04
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://48906b8e.capsule-22k.pages.dev |
| Branch Preview URL: | https://fix-sdk-escrow-route-and-401.capsule-22k.pages.dev |
…oute Three defects an adversarial read of the previous commit turned up. **An unreachable server was reported as an expired session.** reqwest builds every failure of the request it executes with `error::request(..)`, so `is_request()` is true for connection-refused, DNS and TLS failures, and spargen's taxonomy files all of them under `RequestConstruction` next to the genuine pre-flight ones. Mapping that class to `Unauthorized` therefore told an offline device to sign in again — the one remedy that cannot work without a network. The source discriminates them instead: a bearer provider that could not mint a token is boxed as the generated runtime's own `AuthError`, and nothing else on this path is. A closed port is now `Transport`, with a test that binds a socket, drops it, and points the client at the address. **`error_code()` guessed where it should have read.** A `400` reported `error.escrow.malformed` even when the server said otherwise; a `413` reported it too, so a client localizing the code would tell a user their recovery blob was corrupt when it was merely too large; and the `500`'s `error.escrow.unavailable` — the one code that route bothers to set — was thrown away into a transport string. `Malformed` now carries the server's own code, `413` carries `error.request.too_large`, and `500` is its own `Unavailable` variant. The mocks stop inventing `error.*` strings that exist in no catalog and use `capsule_i18n::error_codes` throughout. `FfiError::Escrow` gains the `code` the enum's own doc already promised every variant carries, so `error.escrow.not_stored` reaches a native client — the distinction between "set up a recovery key" and "we could not read the one you have". **The route was pinned by accident.** The socket test relied on a wrong path producing something other than `NotEnrolled`, which held only because Kynos's unmatched-route `404` carries no code and therefore fails to parse. It now asserts the route directly through the fixture's in-process client: what the SDK stored is read back at `/v1/auth/escrow`, and a rotation seeded at that path is what the SDK fetches next. A client on any other path satisfies neither. Relatedly, an uncoded `404` is deliberately *not* read as `NotEnrolled` any more. Reading every `404` as "this account has escrowed nothing" is precisely what let a wrong route look like an empty escrow for a whole slice; an intermediary answering `404 text/html` is a broken path, not an enrollment state. Refs #408
The generated client had only the proactive half of the refresh contract: the token provider refreshes when the stored token is within its skew of expiry, before the request leaves. That cannot cover a token the server stops honouring early — a revocation mid-flight, or a clock the two ends disagree about — and the hand-written clients closed that race years ago while the typed path did not. `RefreshOn401` is an `rest::HttpBackend` wrapping `ReqwestBackend`, installed by `AuthenticatedClient::build_client` through `Client::with_backend`. On a `401` it refreshes once through a new `pub(crate) Session::refresh_rejected` and replays the request once. It touches no generated code and covers every generated operation at once, so there is no per-call retry loop to keep in step and nothing to redo when the document is re-sourced. Why the transport seam and not spargen's `Middleware`: `Next::run` takes `self` by value, and `Next` is neither `Clone` nor constructible outside the generated runtime, so a middleware physically cannot send twice. `RetryBackend` is the precedent this follows, including its rule that a request whose `try_clone()` is `None` — a one-shot streaming body — is executed once and never replayed. `Session::refresh_rejected` wraps `ensure_refreshed(RefreshTrigger::Rejected(stale))` rather than reusing `Session::refresh`, because `refresh` re-reads the *current* token and would refresh again on top of a concurrent rotation, spending a single-use refresh token the server had already closed. Passing the exact token the server refused is what lets the existing single-flight gate coalesce. Exactly once, and by construction: the replay is straight-line code, not a loop with a counter. Four properties are pinned as unit tests — one refresh and one replay carrying the rotated token; a persistent `401` surfaced after exactly two upstream requests; a request with no bearer never retried; and a refresh that itself fails surfacing the **server's** `401` rather than a synthesized transport error, so the typed `Status401` mapping still fires and the caller reads the `error.*` code that separates an expired token from an unreadable revocation ledger. Over a socket, `a_token_the_server_stopped_honouring_is_refreshed_and_the_call_replayed` reproduces the race against the real router: the server validates `exp` against its injected clock, so advancing the fixture past `ACCESS_TOKEN_TTL` revokes the access token for real while the refresh token lives, and the client is handed the same pair with a far-future deadline. The pre-flight half cannot fire, so the call only succeeds through the reactive layer. Both new tests were confirmed to fail with the backend uninstalled. `reqwest_client()` becomes one shared client for the process. It owns a connection pool, and the FFI's escrow verbs build a fresh `AuthenticatedClient` per call because the API root is a per-call argument — so a per-client transport meant a fresh TLS handshake for every escrow read. Nothing here is configured per instance, so there is nothing to vary. Refs #408
`POST /v1/albums/{album_id}/upgrade` had no client. It is one of the four
`application/cbor` operations `build.rs` narrows out of the generated client — spargen
0.4's `classify_media` does not know that media type — and it was the only one of the
four with nothing hand-written behind it, so the SDK could not start the ceremony at
all.
`capsule_sdk::upgrade::UpgradeClient::begin` posts the signed intent **verbatim**. The
bytes are the canonical CBOR `capsule_core::crypto::upgrade` signed, and the server
verifies that signature against the proposing device's DSK in the account's published
directory; re-encoding them here would detach them from the signature and the failure
would look like a forged proposal.
Every refusal keeps its own identity and the code the *server* stamped, because these
are the refusals an admin reads: `409 error.album.upgrade_in_flight` carries the live
`intent_id`, `403 error.album.upgrade_proposer` means the signing device is not
published, and a client that flattened either into "malformed" would have someone
re-signing intents forever. The `413` body backstop carries no problem body at all, so
its code is the client's — `error.request.too_large`, not the intent-malformed code.
The phase decodes into typed ids and a `jiff::Timestamp`, so a caller compares instants:
the deadline is the one field in this ceremony where a string comparison would be a
correctness bug rather than an inconvenience. An unparseable deadline is a malformed
response, never a silent `None`, which would tell a client the ceremony never expires.
`GET` and `DELETE` on the same path are plain JSON and *are* generated; the module doc
says so and deliberately does not duplicate them.
Proven over a socket in `the_sdk_proposes_an_album_upgrade_over_a_socket`, which is the
only shape that can prove anything here: the directory is anchored, the album
provisioned, and the intent signed with the same `capsule-core` types the server
verifies with, so what the test asserts is that the bytes the SDK put on the wire are
the bytes that verify. A mock answering `200` would have proven only that the client can
post.
Refs #408
Five standing falsehoods in `capsule-sdk`'s own documentation, and the two `SLICES.md` rows this issue moves. - The document is **OpenAPI 3.2** and has been since Kynos was pinned with `openapi_as(SpecVersion::V3_2)`. `lib.rs` said 3.1 twice and `build.rs` once. - `mise run openapi` does not exist. The tasks are `openapi-kynos` and `openapi-check-kynos`. - `build.rs` said `capsule_sdk::directory` hand-writes two of the four `application/cbor` operations and "the other two have no client yet". One of those two had a client all along (`verify::StorageVerifyClient::fetch_receipt`) and the other now does (`capsule_sdk::upgrade`), so all four are named, with the one upstream change that retires all four. - The sync feed is not gRPC. `lib.rs` said `sync` stays hand-written because its protocol is too stateful for codegen, which is true of `upload` and false of `sync`: `S-D28` made the feed `GET /v1/sync`, a generated operation, and what is hand-written is the cursor and anti-rewind state machine over it. `ffi/tests.rs` still called `sync_pull` gRPC, and `FfiError`'s doc still offered foreign apps a "bare HTTP/gRPC status" to avoid. `SLICES.md`: `S-D12` records the route defect and its closure, and carries the escrow store response as an owed item pointing at #442. `S-D17` flips to `MIXED | done` — the Area corrects because the layer is live code in this workspace that does not re-scope, even though the client under it is regenerated — with the backend, the rejected `Middleware` alternative, and the socket case named, plus the reason `capsule_sdk::sync` keeps its own loop. Refs #408
Three review findings on this branch.
**The client no longer invents an `error.*` code.** A body-less `413` carries no problem
body, so there is no code to carry — and both hand-written clients were filling that gap
with `error.request.too_large`. Every other code either module reports is the one the
*server* stamped; a code minted on this side asserts that the server said something it
did not, and a client localizing it reads the SDK's guess as the server's judgement. Both
sites now report `code: None` with the English detail, and the variant already carries
the actionable half ("these bytes will not do, do not resend them"). The upgrade test
that asserted the minted code now asserts its absence.
**The auth/transport split has a test on both sides.** `RequestConstruction` carries two
completely different events — a bearer the session could not mint, and a connection that
never opened — and only the boxed source separates them. The transport side was pinned;
the auth side was not, so a spargen change to how a provider failure is boxed would have
silently demoted every expired refresh token to `Transport`, and the FFI would tell a
user to retry where it must tell them to sign in again.
`a_session_that_cannot_mint_a_bearer_is_an_auth_failure` drives a session whose stored
token is past expiry against a mock that serves no `/refresh`, on both the read and the
write path. Confirmed to fail with the downcast arm disabled.
**`reqwest_client()`'s doc stops overclaiming.** Sharing one client for the process does
not remove every per-construction client: `Client::with_backend` still builds its own
default `reqwest::Client` internally, one per `AuthenticatedClient`. That one only
assembles requests — every byte is executed through the backend, and so through the
shared client — so it opens no connection and costs one throwaway allocation. The comment
now says so, and names the `with_client_and_backend` constructor spargen would need to
remove even that.
Refs #408
…route-and-401-retry-408
5 tasks
`UploadBundle` described everything a push needs except the one blob that makes a push visible. It carried the ciphertext, the sealed metadata blob and every derivative, and not the asset's **envelope object** — the `provenance` blob of provenance.md § Physical Storage, which the server stores verbatim, serves back unchanged on the feed, and takes its per-asset chain head from. `provenance_blob` is the canonical CBOR of the chain's head `ProvenanceRecord`, encoded once here so the SDK ladder and the FFI cannot disagree about it. The encoding is forced, not chosen: the server's chain head is the SHA-256 of these bytes while a client's next `prior_provenance_hash` is `record_hash()` — the digest of the canonical *record* — so the bare manifest is the one encoding under which no lifecycle op could ever chain onto a pushed asset. The manifest travels inside it, canonically encoded and therefore byte-identical to its own signed bytes, so "the signed bytes are the served bytes" holds through the wrapper. Unlike `metadata_blob_hash` it is unconditional: a managed asset always has a chain head, even when its head action binds no metadata blob. **capsule-core freeze accounting (#399).** One additive public field on `UploadBundle`. The type is constructed in exactly one place — `Workspace::upload_bundle` — so no caller breaks; the `!` marks it as a struct-literal break for any out-of-tree constructor there is not one of. No method, module or path changes. Refs #464
`bundle_blobs` shipped the sealed metadata blob (T0), every derivative (T1) and the original (T2), and stopped. It never uploaded a `provenance` blob — so an asset pushed by `capsule push`, by `push_bundle`, or by an app driving `FfiWorkspace::upload_blobs` went to the server and stayed invisible: - the server publishes an asset to other devices only once it holds **both** index-tier roles (`upload::visibility::INDEX_TIER_ROLES` — provenance *and* metadata), so the asset never reached anybody's feed, including the pusher's own; - the server's per-asset chain head is the SHA-256 of the provenance blob's bytes, so the row had no chain head and no lifecycle op could ever chain onto it. Both failures are silent. Every blob uploaded, every session finalized, nothing errored. T0 is two blobs now, provenance first: it is the blob the server reads `prior_provenance_hash`, `action` and `retention_until` out of, so the asset's own claims arrive before the bytes they describe. Its content address is derived from the bytes rather than read off a field, because it *is* `record_hash()` by definition — no field could carry it and no signature covers it, since the chain is what signs. The metadata rung stays conditional (a head action binding none contributes none); the provenance rung is unconditional, because a managed asset always has a chain head. `FfiWorkspace::upload_blobs` inherits the fix — it projects `bundle_blobs` — and gains the doc line an app needs: the index tier is two blobs and both must be pushed. It also gains `provenance_head`, the bytes the feed carries for an asset, alongside the existing `signed_manifest`, which is the manifest alone and is now documented as *not* what `apply_sync_entry` takes. Refs #464
`apply_remote_entry` decoded a feed entry's `manifest_cbor` as an `AssetManifest`. The feed serves the **provenance blob's** bytes unchanged (`routes/sync.rs`), and that blob is the canonical CBOR of a `ProvenanceRecord` — the only encoding whose digest is `record_hash()` and therefore the only one under which a lifecycle op can chain. A record wraps the manifest with its chain position, so the decode failed on every correctly pushed asset: a receiving device quarantined it as malformed. The two defects were exactly complementary. The SDK never uploaded the record, and core could not have read it if it had. Neither side's tests could see the other's half, so both passed. What lands with the decode: - the record's `prior_provenance_hash` is checked against the manifest's own before either is used. provenance.md § Chained, Append-Only Structure calls the pair "a checked invariant, not trusted redundancy", and only the manifest's copy is signed — so a divergence is refused rather than resolved in either direction. Until now the chain walker was the only checker and the wire path had none, which is why `mirrors_manifest` is `pub(crate)` rather than private. - a record whose `asset_id` is not its manifest's `file_id` is refused: it is a splice of two assets' history, and the manifest inside it still verifies, so nothing downstream would have caught it. The module doc now states what the feed carries and cites the three provenance.md sections that make the encoding forced rather than chosen, including why the wrapper does not re-author the manifest: canonical CBOR is deterministic, so the manifest sub-map is byte-identical to its own signed bytes and `verify_asset` still verifies over exactly what the signatures covered. Fixtures now take their wire bytes straight off `UploadBundle::provenance_blob`, so they cannot drift from what the ladder actually uploads, and three cases pin the new behaviour: the blob hashes to the chain head, a divergent mirror quarantines, and a spliced `asset_id` quarantines. The FFI's sync-apply tests move to `provenance_head` for the same reason, and `SyncPage`'s `manifest_cbor` doc stops calling those bytes an `AssetManifest`. Refs #465
…ifies The case neither side could write alone. A real `capsule_core::Workspace` seals an asset, the SDK's ladder pushes it to this router over a socket, and the feed is pulled back and applied — which is the only shape in which the three facts meet: - the publish gate is the **server's** (`upload::visibility`), so only a real server can show that an asset without its provenance rung is never published; - the chain head is the server's too, computed over the blob's stored bytes; - the decode is the **client's**, and it only sees what the server actually served. Both defects were confirmed to fail this test with their fixes backed out: with the provenance rung removed from the ladder the push still succeeds and the feed comes back empty of the asset, and with the decode reverted to `AssetManifest` the served entry is quarantined as `MalformedManifest`. The library's manifests are signed on the wall clock and the fixture's starts at the Unix epoch, so the case walks the server's clock up to the manifest's own instant rather than loosening the timestamp window — the window is a real check, and what is under test is the ladder. `tempfile` joins the dev-dependencies for the throwaway library. A hand-built bundle would have agreed with a wrong ladder, which is the whole reason the input is a real workspace's own output. Refs #464, #465
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
capsule-sdk's recovery client called a route the contract does not serve, and threesmaller gaps rode along with it. This lands the route fix on the transport
AGENTS.mdmandates, the reactive
401layerS-D17asks for, a client for the fourthapplication/cboroperation, and truthful crate docs.Summary
The defect.
capsule-sdk/src/recovery/mod.rsbuilt{api_root}/backup/escrowfrom aconstand sent it with hand-writtenreqwestcalls. The committed Kynos document servesGET/PUT /v1/auth/escrow. Every networked recovery flow — enroll, the stale-cacherefresh, the guided re-wrap's escrow replace — failed against a real server while
S-D12read
done. It survived because the path was a string constant no gate checks and themodule's own mock answered whichever path it was handed.
The fix is not the constant. Both escrow operations are
application/octet-streamineach direction, a media type spargen lowers, so both are already generated and neither is
narrowed out in
build.rs.RecoveryClientnow holds oneAuthenticatedClientandorchestrates
fetch_escrow/store_escrow; the path is a function of the document andcannot drift again.
Alongside it:
S-D17—RefreshOn401, anrest::HttpBackendwrappingReqwestBackend, installedby
AuthenticatedClient. Every generated operation now recovers from a401with onesingle-flight refresh and exactly one replay. No generated code is touched.
capsule_sdk::upgrade— a client forPOST /v1/albums/{album_id}/upgrade, the fourthapplication/cboroperation, sending the signed intent verbatim.OpenAPI 3.1→3.2,mise run openapi→mise run openapi-kynos, thehand-written CBOR client count, and the gRPC-sync-feed claims in
client.rs,lib.rs,auth.rsandffi.rs(sync.rsstill carries four such lines and is outside this lane'smanifest — see Unresolved review notes).
Both in-repo escrow mocks now route on
/v1/auth/escrowand answer501elsewhere, so aroute regression fails loudly instead of reading as "no escrow stored" — and the socket test
asserts the route from both ends through the fixture's in-process client rather than relying
on how the router happens to render a
404.Six commits plus a base merge, each coherent on its own and each revertable alone:
1a2656549d208416fb5f5S-D17—RefreshOn4019143e74capsule_sdk::upgrade17f17f67c71817732967fchore/freeze-capsule-core-api-399atf508bf1a(clean; no conflicts, no overlapping files)f0548e1UploadBundlecarries the provenance rung's bytes9a85e7b40d08a5sync_applydecodes the feed's bytes as the provenance record they areda86f04The reopen: two defects the E2E lane found on this surface
capsule_sdk::pushandcapsule_core::lifecycle::sync_applywere broken in exactlycomplementary ways, which is why neither side's tests could see it:
never the
provenanceblob. The server publishes an asset only once it holds bothindex-tier roles, and its per-asset chain head is the SHA-256 of that blob's bytes. So a
CLI/SDK-pushed asset was invisible on the feed to every device including the pusher's, and
no lifecycle op could chain onto it. Both failures are silent: every blob uploaded, every
session finalized, nothing errored.
apply_remote_entrydecoded the feed'smanifest_cboras anAssetManifest.The feed serves the provenance blob's bytes unchanged, and that blob is the canonical CBOR
of a
ProvenanceRecord. So a correctly pushed asset was quarantined as malformed by everyreceiving device.
The encoding is forced rather than chosen: the server's chain head is the digest of those
bytes while a client's next
prior_provenance_hashisrecord_hash()— the digest of thecanonical record. Any other encoding makes the two different numbers. The manifest is not
re-authored by the wrapper: canonical CBOR is deterministic, so the manifest sub-map is
byte-identical to its own signed bytes and provenance.md's "the signed bytes are the served
bytes" holds through it.
Validation
Every command below was run inside the worktree on the final head
732967f9— after themerge of
chore/freeze-capsule-core-api-399atf508bf1a— and its result observed.mise run check-rustis reported as its own fifteen steps because the aggregate task waskilled twice by the machine (exit 137/143) under parallel-lane load, never by a check failing;
each step was then run and observed individually.
cargo nextest run -p capsule-sdkcargo nextest run -p capsule-sdk --features fficargo nextest run -p capsule-server --test sdk_clientmise run format-check-rustmise run lint-check-rustmise run doc-check-rust--document-private-itemsgate)mise run i18n-checkmise run i18n-guardmise run openapi-check-kynosmise run architecture-checkmise run license-checkmise run translate-readme-checkmise run build-rustmise run build-check-wasmmise run build-ffimise run lint-check-ffimise run gen-bindingstarget/bindingsand are not committed, soFfiError::Escrow's newcodemember changes no tracked filemise run verify-examplesmise run test-rustcapsule-core --features ffi729/729,capsule-sdk --features ffi177/177mise run check-docs-truthmise run check-mdDeliberate negative controls. Three of the new tests were confirmed to fail with the
change backed out, so none passes vacuously:
Client::with_backendreverted towith_client,client::tests::a_401_is_refreshed_once_and_the_call_replayedanda_token_the_server_stopped_honouring_is_refreshed_and_the_call_replayedboth fail with theserver's
401;rest::AuthErrordowncast arm disabled,a_session_that_cannot_mint_a_bearer_is_an_auth_failurefails — the case exists preciselybecause that arm had no test;
store_escrowagainst/backup/escrowanswers an undocumented404.The reopen's gates, on
da86f045Run in the foreground with
CARGO_TARGET_DIR=/var/tmp/capsule-lane-408/target(the worktree'sown
target/was removed by the orchestrator with the scratch volume at 100%).cargo nextest run -p capsule-sdkcargo nextest run -p capsule-core lifecyclecargo nextest run -p capsule-server --test sdk_clientmise run format-check-rustmise run lint-check-rustmise run doc-check-rustmise run i18n-checkmise run i18n-guardmise run openapi-check-kynosmise run architecture-checkmise run license-checkmise run translate-readme-checkmise run build-rustmise run build-check-wasmmise run build-ffimise run lint-check-ffimise run gen-bindingsmise run verify-examplesmise run test-rustcapsule-core --features ffi732/732,capsule-sdk --features ffi178/178mise run check-docs-truthmise run check-mdgen-bindingscould not be run as the task.mise-tasks/gen-bindingshardcodestarget/debug/...andtarget/bindings, so it cannot see a build made under an overriddenCARGO_TARGET_DIR, and rebuilding into the worktree needs ~9 GB on a volume with 19 GB freeand other lanes on it. Symlinking
targetis forbidden. Its content was therefore runverbatim against the same inputs — both
cargo run -p <crate> --features ffi-bindgen --bin uniffi-bindgen -- generateinvocations for both namespaces in both languages, plus everyrequiresymbol the task asserts — and passed, withprovenanceHeadadded to the assertionlist. The task itself is reported
unavailablein this environment and is green on CI, whichdoes not override
CARGO_TARGET_DIR.Deliberate negative controls for the reopen. Both fixes were confirmed necessary by
backing each out and re-running
a_pushed_asset_reaches_the_feed_and_the_entry_decodes_and_verifies:bundle_blobs, the push still succeeds and the feedcomes back with the asset absent — the silent failure, reproduced;
sync_apply's decode reverted toAssetManifest, the served entry isQuarantined(MalformedManifest("missing fieldcore")).CI on this branch (last observed at
17f17f64, pre-merge): every required check green(
requiredSUCCESS), including Rust fmt+clippy+build, Rust tests, all four cross builds,docs-truth, markdown and commit lint. One job fails — Build Capsule.apk + :core JVM smoke —
and it is pre-existing: the same job fails on PR #426, this branch's base, and nothing
here touches Kotlin, Android or Gradle.
Risks and rollout
Client-only.
openapi.json,capsule-server/src/**andcapsule-core/**are untouched, sothe wire contract is unchanged and
openapi-check-kynosstays byte-identical.RecoveryClient::newbecoming fallible is a source break inside this crate only(
capsule-sdkispublish = false); both call sites are inffi.rs.RecoveryErrorlosesBodyandAuthand gains five variants. Nothing outsidecapsule-sdkmatches on it.FfiError::Escrowgains acode: Option<String>member, which changes the uniffibinding shape for Swift and Kotlin (decision 6). The generated bindings are emitted to
target/bindingsand are not committed, so no tracked file changes andmise run gen-bindingspasses; the only in-repo consumer matchesEscrow { .. }and is unaffected.A native app that constructed the variant positionally would need the extra field — none
does today, and the change is what lets a client tell "set up a recovery key" from "we could
not read the one you have".
AuthenticatedClient's observable behaviour changes: every generated operation now replaysonce on a
401. The operations it serves carry in-memory bodies; the one streamed shaperides the hand-written
upload.rs, andtry_clone() == Noneshort-circuits it regardless.UploadBundlegains a public field andFfiWorkspacegainsprovenance_head— bothadditive; see decisions 11 and 12 and the freeze accounting under them.
an asset pushed before it is still invisible, and re-running a push uploads the missing rung
(the server answers
duplicate_blobfor what it already holds, which resolves as a merge),so recovery is a re-push with no migration.
sync_applydecodes anAssetManifest. It quarantines the entry rather than mis-applying it, and today no buildever produced an entry it could apply — the rung was never uploaded, so nothing was ever
published. There is no window in which this change breaks something that worked.
git revertper commit; no persisted data and no deployed behaviour.Related Issues
Closes #408
Filed by this lane for work it deliberately did not do:
spargen: classify_media has no application/cbor, forcing four hand-written clients in capsule-sdk. The upstream change that retirescapsule_sdk::directory,capsule_sdk::upgrade,verify::fetch_receiptand the fourOmitRules inbuild.rs.sdk: fetch_receipt decodes JSON where the contract serves application/cbor. Thesame defect class and the same cause as this issue, excluded by decision 1 below.
sdk: store_escrow discards stored_at/replaced, so the stale-cache rule has nothing to read. Carried on theS-D12row as its owed remainder.Also fixed here, filed by the E2E lane (#463) against this surface:
sync_applydecoded the feed's bytes as anAssetManifest.Contributor Checklist
Decisions taken
Issue 408 - sdk: recovery calls a route the contract does not serve, and three smaller gaps
Plan: v1 (planned against f433d91; executed on the head of lane #399's branch)
Branch: fix/sdk-escrow-route-and-401-retry-408
Base: the head of lane #399's branch (chore/freeze-capsule-core-api-399), stacked; the PR targets that branch until it merges
Worktree: /var/mnt/scratch/golem/dev/Capsulsaurus/Capsule.worktrees/Capsule-fix-sdk-escrow-route-and-401-retry-408
Cause: capsule-sdk/src/recovery/mod.rs:41 hand-writes a reqwest path to
{api_root}/backup/escrow; the contract serves GET/PUT /v1/auth/escrow, and those operations are already generated (octet-stream is a media type spargen lowers). The path survived because S-D28 re-sourced the document from Kynos and nobody re-checked the hand-written client.Touches: capsule-sdk/src/{recovery/mod.rs,client.rs,auth.rs,upgrade.rs (new),lib.rs,ffi.rs}, capsule-sdk/build.rs, capsule-server/tests/sdk_client.rs (append-only), SLICES.md (rows/blocks S-D12 and S-D17 ONLY)
Will not: touch Cargo.toml default-members (#399 owns it), capsule-server/openapi.json, capsule-server/src/, capsule-sdk/src/sync.rs, capsule-sdk/src/verify.rs, capsule-core/
Lane: serialised behind #399
Settled: capsule-sdk joins default-members — settled by #399. Base branch = head of PR #418 → this lane stacks on #399's head.
Decisions taken.
Deliverable boundary - all four issue bullets in one lane, minus the receipt-decode defect.
Taken: Ship the escrow fix, S-D17, the upgrade client, the doc corrections and the filed spargen issue as five slices on one branch; the in-process server harness at capsule-server/tests/sdk_client.rs:45-129 already exists.
Rejected: Also fix capsule-sdk/src/verify.rs:305-313, which decodes GET /v1/upload/{id}/receipt as JSON while the handler returns Binary (capsule-server/src/routes/receipts.rs:91) - not in sdk: recovery calls a route the contract does not serve, and three smaller gaps #408, changes the release-gate path that verify/tests.rs:263-304 mocks as JSON, and needs an attestation-key fixture over a socket: a second issue's worth of test work.
Reverses: git revert the branch; the receipt defect is untouched either way.
Filed: the lane files "sdk: fetch_receipt decodes JSON where the contract serves application/cbor".
Escrow transport - move to the generated client rather than fix the path string.
Taken: Replace recovery/mod.rs's hand-written reqwest calls with AuthenticatedClient::{fetch_escrow, store_escrow}; AGENTS.md requires generated parsing, the generated methods exist, and the route cannot drift again.
Rejected: Change ESCROW_PATH to "v1/auth/escrow" and keep the reqwest path - keeps a second parser the rule forbids and a route string no gate checks.
Reverses: restore recovery/mod.rs from the parent commit and set ESCROW_PATH to "v1/auth/escrow".
S-D17 seam - an HttpBackend wrapper, not the Middleware trait.
Taken: RefreshOn401 as rest::HttpBackend wrapping ReqwestBackend, installed via Client::with_backend in client.rs::build_client, replaying with Request::try_clone() and refreshing through a new pub(crate) Session::refresh_rejected. No generated code touched; every generated operation is covered.
Rejected: spargen's Middleware trait - Next::run consumes self and Next is not Clone/constructible (rest_client.rs:1250-1275), so a middleware cannot send twice. Also rejected: copying sync.rs:472-499's per-call loop into callers - the duplication S-D17 exists to remove.
Reverses: revert client.rs to Client::with_client and delete refresh_rejected.
sync.rs's bespoke 401 loop - leave it.
Taken: SyncConsumer keeps its own refresh-and-retry; it has a static-token mode (SyncAuth::Static, sync.rs:460) with no session to refresh and interleaves 401 with the shared retry engine's transient class.
Rejected: Route SyncConsumer through AuthenticatedClient - AuthenticatedClient::new takes a Session unconditionally (client.rs:60); adopting it means dropping static-token mode or widening AuthenticatedClient, both larger than this issue.
Reverses: construct SyncConsumer's client with with_backend and delete the 401 arm of the pull loop once AuthenticatedClient grows a static-token constructor.
Decisions taken inside the manifest, during delivery (same shape; numbering continues the record).
An uncoded 404 on the escrow fetch is a wire failure, never "no escrow".
Taken: A
404whose body is not a parseableCodedProblemmaps toRecoveryError::Transport, never toNotEnrolled. Only the server's own codedbody proves an unenrolled account; an intermediary's
404must not read as"enroll first". Pinned by
an_uncoded_404_is_not_read_as_an_empty_escrow.Rejected: Fall back to
NotEnrolledon any404, which is what the hand-written client did.Evidence: that reading is precisely the defect class this issue exists to close -
it let
backup/escrowlook like an account that had escrowed nothing for a wholeslice. An intermediary answering
404 text/htmlis a broken path, not anenrollment state.
Reverses: map a bare
404toNotEnrolledinwire_error'sDecodearm - and reintroducethe blindness.
FfiError::Escrow gains a
codefield.Taken: Add
code: Option<String>, populated fromRecoveryError::error_code(), so Swiftand Kotlin can switch on
error.escrow.*-not_stored("set up a recovery key")against
unavailable("we could not read the one you have"). The enum's own docalready claimed every variant carries the catalog code, and
Escrowwas the onethat did not. This is a uniffi binding shape change; see Risks and rollout.
Rejected: Fold the code into the
messagestring. Evidence: foreign clients cannot switchon a substring, which is the entire reason the
{ error, code }contract exists.Also rejected: leave it and file an issue - this lane is the one that introduced
error_code(), and shipping an accessor no boundary can read is a half-done fix.Reverses: drop the field and the three
code:initializers.A body-less 413 carries no code - the client never mints one.
Taken: Both hand-written clients report
code: Nonewith an English detail on413(
recovery/mod.rs,upgrade.rs). The transport backstop sends no problem body, sothere is no code; every other code either module reports is the one the server
stamped, and a code minted on this side asserts the server said something it did
not. The typed variant already carries the actionable half ("these bytes will not
do, do not resend them").
Rejected: Mint
error.request.too_largeclient-side, which is what the first version ofthis branch did. Evidence: a client localizing it would read the SDK's guess as
the server's judgement, and it is the only code in either module with no server
behind it. Also rejected: stamp the code server-side in the
413backstop -capsule-serveris outside this lane's manifest; recorded here as a note for thelane that owns
problem.rs/limits.rs.Reverses: restore the two
REQUEST_TOO_LARGEarms and their assertions.Manifest widened by one file - capsule-sdk/src/ffi/tests.rs.
Taken: Repoint the ffi module's own test mock from
/api/backup/escrowto/api/v1/auth/escrow, and give itsPUTa JSONStoreEscrowResponseand its emptyGETan RFC 9457 body. The record's Touches line namescapsule-sdk/src/ffi.rs;the
ffimodule's tests live in the siblingffi/tests.rs, and the plan did notforesee that that mock hardcodes the route. Without it
ffi::tests::ffi_escrow_and_device_directory_round_tripfails against its ownmock, so slice 1 cannot be delivered coherently at all. Recorded here rather than
taken silently, per the manifest rule.
Rejected: Stop and return "needs re-plan". Evidence: the file is the test submodule of a
file already in the manifest, the change is a route string and two response bodies
in a test double, and no other lane touches
capsule-sdk/src/ffi/**- freezing alane over a mock's path constant costs the whole deliverable and buys nothing.
Reverses: revert the
ffi/tests.rshunk; the test then fails, which is the honest signal.RecoveryError::Auth deleted, and RequestConstruction split by its source.
Taken: Drop the
Auth(#[from] AuthError)variant - once the reqwest path is gone nothingcan construct it - and instead discriminate spargen's
RequestConstructionclassby downcasting its source to the generated runtime's
AuthError, which only thebearer provider produces. A dead session therefore still reaches a caller as
Unauthorized->FfiError::Auth, preserving the re-auth signal the hand-writtenpath gave, while a refused connection stays
Transport. Both sides are pinned:a_session_that_cannot_mint_a_bearer_is_an_auth_failureandan_unreachable_endpoint_is_a_transport_failure_not_an_auth_one.Rejected: (a) Keep
Authas an unconstructible variant - a promise no code path can keep, ina public error enum. (b) Map the whole
RequestConstructionclass toUnauthorized- the first attempt, refuted by the adversarial read: reqwest builds every failure
of the request it executes with
error::request(..), sois_request()is true forconnection-refused, DNS and TLS, and the mapping told an offline device to sign in
again.
Reverses: restore the variant and map
RequestConstructiontoTransportunconditionally;the cost is that a dead session reads as a network blip at the FFI boundary.
One shared reqwest client for the process.
Taken: Make
client::reqwest_client()aOnceLockreturning clones. The FFI's escrowverbs build a fresh
AuthenticatedClientper call because the API root is aper-call argument, and a
reqwest::Clientowns a connection pool - so before this,every escrow read paid a fresh TLS handshake where the old code rode the session's
shared client. Nothing here is configured per instance, so there is nothing to
vary. The residual cost is named in the code:
Client::with_backendstill buildsits own request-assembly client per construction, which opens no connection.
Rejected: Cache an
AuthenticatedClientonFfiSession. Evidence: the base URL is aper-call argument, so the cache needs a key and an eviction rule - larger than the
regression it fixes, and outside this issue.
Reverses: inline the builder again; correctness is unaffected either way.
The push ladder ships the provenance rung, first in T0 (sdk: the push ladder never uploads the provenance blob, so SDK/CLI pushes never reach the feed #464).
Taken:
bundle_blobsemits aBlobRole::Provenanceblob atUploadTier::Indexahead ofthe metadata blob; its bytes are
UploadBundle::provenance_blob— the canonicalCBOR of the chain head
ProvenanceRecord, encoded once incapsule-coreso theSDK and the FFI cannot disagree — and its envelope is
envelope_for(bundle, hash)like every other rung. Its content address is derived by hashing the bytes rather
than read off a field, because it is
record_hash()by definition: no fieldcould carry it and no signature covers it, since the chain is what signs.
Provenance leads T0 because it is the blob the server reads
prior_provenance_hash,actionandretention_untilout of, so the asset's ownclaims arrive before the bytes they describe. The rung is unconditional, unlike
the metadata rung: a managed asset always has a chain head.
Rejected: (a) Leave it to callers, as the E2E harness does today
(
capsule-e2e/src/push.rs::push_provenance). Evidence: every caller would have toknow the encoding whose digest the server's chain head is, which is exactly the
knowledge a ladder exists to hold once; the harness only carries it because the
SDK did not. (b) Add a second
Vec<u8>and a stored hash toUploadBundle.Evidence: the digest is a pure function of the bytes and
record_hash()isdefined as exactly that, so a stored copy is a derivable value on a frozen type —
metadata_blob_hashisOption<Hash32>because it comes from the signedmanifest, which is not this case.
Reverses: drop the first
blobs.pushinbundle_blobsand the field; the push thensilently stops publishing again, which
a_pushed_asset_reaches_the_feed_and_the_entry_decodes_and_verifiescatches.
capsule-core freeze accounting (core: freeze the capsule-core public API and remove the dead surface #399). One additive public field,
UploadBundle::provenance_blob: Vec<u8>.UploadBundleis constructed in exactly oneplace —
Workspace::upload_bundle— so no in-tree caller breaks; the commit is marked!for any out-of-tree struct-literal constructor. No method, module or path changes, and the
rustdoc gate (
doc-check-rust, now--document-private-items) is green.sync_applydecodes the feed's bytes as aProvenanceRecord(core: sync_apply decodes feed manifest bytes as AssetManifest, but the feed serves provenance-record bytes #465).Taken:
apply_remote_entrydecodesentry.manifest_cboras aProvenanceRecord, checksmirrors_manifest()and thatrecord.asset_id == record.manifest.core.file_id,then verifies
record.manifestthrough theverify_assetchokepoint exactly asbefore. Cited: provenance.md § Physical Storage (the server's chain is "the
append-only sequence of envelope objects … served back unchanged"), § Chained,
Append-Only Structure (the two
prior_provenance_hashcopies are "a checkedinvariant, not trusted redundancy"), and § Asset Manifest ("the signed bytes
are the served bytes", which holds through the wrapper because canonical CBOR is
deterministic).
mirrors_manifestbecomespub(crate): the chain walker was itsonly checker and the wire path had none.
Rejected: (a) Change the server to serve a re-projected manifest. Evidence: § Asset
Manifest forbids exactly that — "no server-side surface … may re-encode a manifest
from the PostgreSQL projection", because the projection holds no signature.
(b) Accept either encoding. Evidence: it would leave two wire shapes for one
field, and the chain-head arithmetic still only works for one of them.
(c) Resolve a mirror divergence by preferring the manifest's copy. Evidence: only
the manifest's is signed, so preferring either is trusting a value no signature
covers; the doc says reject.
Reverses: decode
AssetManifestagain; every correctly pushed asset is then quarantined,which the socket case catches.
Manifest widenings taken for decisions 11 and 12
Both were named by the reopen brief; the rest follow from them and are recorded here rather
than taken silently.
capsule-core/src/lifecycle/upload.rscapsule-core/src/lifecycle/sync_apply.rsbundle.provenance_blobso they cannot drift from the laddercapsule-core/src/crypto/provenance/record.rsmirrors_manifestbecomespub(crate)so the wire path can run the invariant the doc calls checked. Duplicating the comparison insync_applywas the alternative, and a second copy of an invariant is how the two copies divergecapsule-sdk/src/push.rs,push/tests.rscapsule-sdk/src/ffi/workspace.rs,ffi/tests.rsupload_blobsprojectsbundle_blobs, so the FFI push path is fixed by the same change;provenance_headis added additively besidesigned_manifestrather than repurposing it, and the tests move to itcapsule-sdk/src/sync.rssync.rs's bespoke 401 loop (decision 4) — that is about behaviour. Leavingmanifest_cbor's doc saying "the signedAssetManifest", on the field whose meaning this change corrects, is the defect class the whole PR is aboutcapsule-server/Cargo.toml,Cargo.locktempfileas a dev-dependency for the socket case's throwaway library. No new dependency domain (capsule-sdkalready dev-depends on it), dev-only, so nodependencies.mdrow is owedcapsule-server/src/**andopenapi.jsonare still untouched: every server behaviour thisdepends on — the publish gate, the chain head, the served bytes — was already correct.
Unresolved review notes
Raised against this diff before committing (a focused sub-agent read the first commit
adversarially) and in the lane review of
17f17f64. Everything actionable was fixed in49d2084and7c71817; what remains is outside the lane manifest and is listed here ratherthan widened into silently.
capsule-core/src/lifecycle/backup.rs:29still documentsPUT /backup/escrow. Thesame stale route, in a doc comment.
capsule-core/**is explicitly out of this lane'smanifest (core: freeze the capsule-core public API and remove the dead surface #399 owns that tree), so it is untouched. One-line fix for whoever holds it next.
SLICES.md:3384lists the surface asGET/PUT /v1/auth/backup/escrow. Also wrong —the route is
/v1/auth/escrow. That row is neitherS-D12norS-D17, and this lane mayedit only those two, so it is untouched.
capsule-sdk/src/sync.rs:213,:258,:430still describe the feed in gRPC terms(
:11names it historically, which is fine). Pre-existing, andsync.rsis named in therecord's Will not list — decision 4 keeps its bespoke
401loop — so its prose was leftalone. The issue's doc-truth bullet named
client.rs, which is fixed, along withlib.rs,auth.rsandffi.rs.413backstop sends noerror.*code at all. Decision 7 stops the client frominventing one, which leaves the gap where it belongs: on the server. Stamping
error.request.too_largeincapsule-server's body-limit backstop(
problem.rs/limits.rs) is the real fix, andcapsule-server/src/**is outside thislane's manifest. Noted for the lane that owns it; no issue filed, per this lane's brief.
Client::with_backendbuilds one throwawayreqwest::ClientperAuthenticatedClient.It only assembles requests — execution goes through the shared client below it — so it opens
no connection and costs one allocation. Removing even that needs a
with_client_and_backendconstructor spargen does not expose. That is generator work of the same kind as the
application/cborgap and would land alongside spargen: classify_media has no application/cbor, forcing four hand-written clients in capsule-sdk #440; spargen: classify_media has no application/cbor, forcing four hand-written clients in capsule-sdk #440 was not edited to say so.push_provenance.capsule-e2e/src/push.rs(PR [TEST] Land the bounded E2E cases in a capsule-e2e crate #463) uploads the rung itself and says in its module doc that it does so because the SDK
does not. With sdk: the push ladder never uploads the provenance blob, so SDK/CLI pushes never reach the feed #464 fixed the harness's own rung is now a duplicate upload — harmless (the
server answers
duplicate_blob, which resolves as a merge) but no longer load-bearing.Removing it belongs to that lane's file, not this one.
RecoveryError,UpgradeErrorandFfiErrorare not#[non_exhaustive]. This changeadds variants and a field to all three. Nothing outside
capsule-sdkmatches on any of them(
capsule-sdkispublish = false, and the only in-repo consumer ofFfiError::Escrowmatches
{ .. }), so nothing breaks — but a future variant would be a source break again.Marking them
non_exhaustiveis itself a break and was not taken unilaterally.