From de059102c1fb3e126b71e4fae721ddfa23c43636 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 22:56:11 +0300 Subject: [PATCH 001/100] docs: open the audit #67 decision ledger and index it --- CLAUDE.md | 1 + docs/README.md | 1 + docs/audit-67-decisions.md | 71 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 73 insertions(+) create mode 100644 docs/audit-67-decisions.md diff --git a/CLAUDE.md b/CLAUDE.md index 3f8319e..c1de24c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -230,6 +230,7 @@ until it was dissolved on 2026-09-04. | [`docs/work/mvp/2026-09-04-open-items-dissolution.md`](docs/work/mvp/2026-09-04-open-items-dissolution.md) | **Archive of record, not a register.** The dissolved open-items register, moved here whole on 2026-09-04 with every open question in it decided. Nothing is appended to it; its item IDs stay reserved and still resolve, because they are cited from source. | no — archive | | `docs/first-release.md` | **Live.** Release-readiness: what the release path already does, and the blockers that must clear before a first publish. Was the `NFR-16` row of the dissolved deferral register. | yes | | `docs/deviations.md` | **Register.** The as-built audit of §10, and where a deviation found outside a phase lands. | yes | +| `docs/audit-67-decisions.md` | **Ledger.** Decisions taken during the audit #67 remediation run, and the release-machinery work it deferred. | yes | | `docs/assets/` | Vendored wordmark SVGs the root `README.md` renders. | yes | **Frozen means a maintenance tool refuses to write there**, not merely that you should not. The diff --git a/docs/README.md b/docs/README.md index 659726f..6b9507d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -16,6 +16,7 @@ is the index; the rule is that nothing in `docs/` is unowned, and nothing is wri | [the dissolved open-items register](./work/mvp/2026-09-04-open-items-dissolution.md) | **Archive of record.** Everything the register held when it was dissolved on 2026-09-04, with every open question in it decided. Item IDs stay reserved and still resolve | — | No — nothing is appended | | [`first-release.md`](./first-release.md) | Release readiness: what the release path already does, the confirmed mechanics, and the blockers that must clear before a first publish. Was the `NFR-16` row of `deferred-items.md`, which was dissolved on 2026-09-04 — its five still-live rows are archived under *Live deferrals* in [`work/mvp/2026-09-04-register-retirement-purge.md`](./work/mvp/2026-09-04-register-retirement-purge.md) | The maintainer | Yes — edited as blockers clear | | [`deviations.md`](./deviations.md) | The as-built audit of §10, and the landing point for a deviation found outside a phase | An audit or review | Yes — appends | +| [`audit-67-decisions.md`](./audit-67-decisions.md) | Decision ledger for the audit #67 remediation run: cross-task decisions, rejected alternatives, and the release-machinery work deferred from it | The remediation supervisor | Yes — appends | | [`assets/`](./assets/) | Vendored wordmark SVGs the root `README.md` renders | Copied from `dexpace/morphic` | Yes | ## Frozen means frozen diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md new file mode 100644 index 0000000..652c016 --- /dev/null +++ b/docs/audit-67-decisions.md @@ -0,0 +1,71 @@ +# Audit #67 remediation — decision ledger + +Supervisor-owned record for the remediation run of the 2026-09-04 audit +([umbrella #67](https://github.com/dexpace/nodejs-sdk/issues/67), subtasks #68–#82). One entry per +cross-task decision: which issue raised it, what was decided, the alternatives rejected, and which later +issues it constrains. Deferred work is listed at the end so the release pass can recover it. Umbrella +branch: `audit/remediation-67`, base `mvp`. Task branches: `audit/67/-`. + +A decision that departs from the spec text is also a dated row in +[`deviations.md`](./deviations.md) under "Deviations recorded outside a phase"; this file records the +*choice*, that file records the *deviation*. + +## Ground rules fixed before wave 1 + +### D0 — Where a remediation deviation is written (raised by #68, #69, #71, #72, #74, #75) +Several subtask issues say "record the reading in the Phase Nx ledger section". Those sections live in +`docs/work/mvp/`, which CLAUDE.md declares a dated record that is never retro-edited. **Decision:** every +deviation or reading this run records goes to `docs/deviations.md` under "Deviations recorded outside a +phase", dated, with `file:line` evidence and "Found by: audit #67 / #". Phase ledger sections are +not touched. *Rejected:* appending to phase ledgers (retro-edits a dated record; §10 is frozen so it is +not an option either). *Constrains:* every later subtask. + +### D1 — Release machinery is out of scope (raised by the run's own brief) +No changesets, no version bumps, no `docs/first-release.md` edits. Each PR lists what it skipped under +"Deferred — release machinery"; the consolidated list is at the end of this file. + +### D2 — Wave overlap adjustments +- Wave 1 (#68, #69) both edit the "Deviations recorded outside a phase" table in `docs/deviations.md` and + `packages/core/src/context/instrumentation.ts`. Kept concurrent: #68 edits existing rows/lines and the + `tracerFactory` TSDoc; #69 only *appends* rows at the end of the table and changes the single + `activeSpan` line. The supervisor resolves the adjacent-hunk conflict at merge. +- Wave 3 becomes #72, #74, #75 (all M3); #73 moves to wave 4 with #76, #77. Reason: #72 and #73 both + reshape `packages/core/src/retry/engine.ts` (`withTrail` vs. the per-attempt re-send of the template), + and #73's layering choice is easier to make on top of #72's landed trail shape. +- Wave 6 splits: #81 first, then #82. Both edit `undici-transport.ts` and add rows to + `packages/transport-conformance/src/run-suite.ts` + `fixtures.ts`; #82's "make undici match" clause + depends on #81's drop-set rewrite. + +## Decisions taken for #69 (M1) — the maintainer-decision items + +### D3 — CTX-15: fix, not ledger +`noopInstrumentationBundle.activeSpan` becomes `NOOP_SPAN`. One line plus a test. #80 lists the same item; +it is done here, #80 skips it. *Rejected:* ledger row (the fix is smaller than the row). + +### D4 — PIPE-37: ledger the gap, do not implement in M1 +No `PRE_REDIRECT` status-mapping pipeline step exists; `statusMappingStep` is a `ResponseStep`. Recorded +as a row in `deviations.md` naming the gap, the Phase 4→5 hand-off that dropped it, and the petstore +spike finding 2 as the same work. Implementing it is real pipeline work with public surface, outside a +docs-only milestone, and opening a tracking issue is a remote action this run is not authorised to take — +the maintainer opens it if wanted. *Constrains:* none of #70–#82 depends on it. + +### D5 — REDIR-3: keep the current-hop-method reading, pin it, ledger it +Spec text says "original request method". The port evaluates eligibility against the method of the +request being redirected at *this* hop. The two differ only after an opted-in 303 rewrote POST→GET and a +later 301/302 arrives: the port follows it (GET is in the default set), the literal reading would refuse +it. **Decision:** keep the port's reading — the rewritten GET is idempotent and body-less, the 303 rewrite +is opt-in, and refusing would make `allow303` half-useful — pin it with a test (`allow303: true`, POST, +303 then 301) and record the reading as a `deviations.md` row. *Rejected:* switching to the literal reading +(behaviour change inside a docs milestone; stricter without a safety gain). + +### D6 — PAGE-19: WHATWG relative resolution is the intended reading +`; rel=next` resolves against the page URL under WHATWG rules, so it is a followable relative +reference, not an unparseable one. A target that fails `new URL(target, base)` still ends the stream. +Pin with a test and ledger the reading. *Rejected:* adding an ad-hoc "looks unparseable" heuristic. + +### D7 — HTTP-46, IO-13, BODY-9, BODY-34, IO-38, transport `reasonPhrase`: ledger rows +All six are recorded as rows (evidence + reading). `reasonPhrase` sits beside §10 item 13; #82 reads it as +already done and does not re-ledger it. + +## Deferred — release machinery (recoverable list) +_(appended per PR)_ From ae46aedc2d0b573eb59a9c16c801a2380ca33573 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:12:46 +0300 Subject: [PATCH 002/100] docs(serde): stop claiming a top-level wire null can never decode `DecodeTarget.admitsNull` (`packages/core/src/seams/serde.ts:126-145`) is the caller's opt-in that `T` includes `null`, and `jsonSerde()` honours it at `packages/codec-json/src/json-serde.ts:170`. Two `@public` TSDoc blocks still said the rejection "cannot be conditional" and that `tristate(inner)` is a field combinator only -- the text that predates the flag, shipped in both packages' `.d.ts`. Both now describe the default and the opt-in. Pinned already by `json-serde.test.ts:581,591`; no behaviour change. Part of audit #67 / #68 (SERDE-5, SERDE-13). --- packages/codec-json/src/json-serde.ts | 14 ++++++++------ packages/core/src/seams/serde.ts | 16 +++++++++------- 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/packages/codec-json/src/json-serde.ts b/packages/codec-json/src/json-serde.ts index 140390d..cd7086d 100644 --- a/packages/codec-json/src/json-serde.ts +++ b/packages/codec-json/src/json-serde.ts @@ -264,12 +264,14 @@ function makeDeserializer(): Deserializer { * `JSON.stringify(v, tristateReplacer)` gets the nested and array-element behaviour but not the * top-level degradation — see {@link tristateReplacer}. * - * **A top-level wire `null` never decodes (SERDE-13).** Every decode target is treated as non-null: - * a schema value carries no nullability this codec could read, so the rejection is unconditional and - * runs *before* the schema. A `200` whose entire body is the literal `null` therefore raises a - * `DeserializationError`, and `tristate(inner)` is a field combinator for use inside - * `tristateObject`, not a top-level decode target. Deliberate: checking after the schema would let a - * permissive schema such as `{parse: (i) => i}` return that `null` as a non-null `T`. + * **A top-level wire `null` decodes only into a target that admits one (SERDE-13).** A schema value + * carries no nullability this codec could read, so the rejection is unconditional *by default* and + * runs *before* the schema: a `200` whose entire body is the literal `null` raises a + * `DeserializationError`. Setting `admitsNull: true` on the `DecodeTarget` is the caller stating what + * the schema value cannot — that `T` includes `null` — and skips the check, which is the one case + * where `tristate(inner)` serves as a top-level target rather than a field combinator for use inside + * `tristateObject`. Checking after the schema instead would let a permissive schema such as + * `{parse: (i) => i}` return that `null` as a non-null `T`. * * @param options - opt out of the Tristate wiring; everything else is fixed by the format. * @returns a frozen, stateless bundle safe to share across concurrent operations (SERDE-29). diff --git a/packages/core/src/seams/serde.ts b/packages/core/src/seams/serde.ts index bb0c404..0e26146 100644 --- a/packages/core/src/seams/serde.ts +++ b/packages/core/src/seams/serde.ts @@ -162,13 +162,15 @@ export interface DecodeTarget { * streams bytes straight into {@link Deserializer.deserializeFrom} and never holds a parsed value to * inspect, and core owning a parser would violate SEAM-1. * - * **Every decode target is treated as non-null.** An implementation sees a schema *value*, which - * carries no nullability it could read, so the check above cannot be conditional — it rejects a - * top-level wire `null` unconditionally. A legitimately nullable top-level target is therefore - * outside this contract: a `200` whose whole body is the literal `null` does not decode, and - * `tristate(inner)` is a *field* combinator rather than a top-level decode target. This is - * deliberate — the alternative lets a permissive schema such as `{parse: (i) => i}` launder a wire - * `null` into a non-null `T`, which is the heap pollution SERDE-5 and SERDE-13 exist to prevent. + * **A decode target is treated as non-null unless the caller says otherwise.** An implementation + * sees a schema *value*, which carries no nullability it could read, so the check above cannot be + * derived from the schema — it rejects a top-level wire `null` unconditionally *by default*. The one + * way to admit one is {@link DecodeTarget.admitsNull}, the caller stating what the schema value + * cannot: that `T` includes `null`. With it set the check is skipped, the `null` reaches the schema, + * which is free to reject it, and `tristate(inner)` can serve as a top-level target rather than only + * a *field* combinator. Off by default, and deliberately so — the alternative lets a permissive + * schema such as `{parse: (i) => i}` launder a wire `null` into a non-null `T`, which is the heap + * pollution SERDE-5 and SERDE-13 exist to prevent. * * **One spelling, both layers.** Every decode entry point takes the schema and its diagnostic label * bundled as a {@link DecodeTarget}: the SPI here, and `decodeResponse` / `decodeSuccessResponse` From 217fe70cb66d6dbc53d713d2e813f4689e4e8814 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:12:57 +0300 Subject: [PATCH 003/100] docs(core): correct six TSDoc blocks that state the opposite of the code Each was true when written and is not now: - `context/instrumentation.ts` `tracerFactory` said it returns "the started span". It returns a **tracer**; both consumers narrow it and call `startSpan()` themselves (`pipeline/runtime.ts:52-57`, `observability/logging-step.ts:263-269`). - `io/index.ts` said `isIoError`, `AllocationLimitError`, `ClosedResourceError` and `SourceContractViolationError` "remain internal". `1f48926` put all four on the barrel (`core/src/index.ts:39-47`) and in `core.api.md`. - `http/request-conditions.ts` said reconciling HTTP-48's obs-text permission against HTTP-18 is "left to a later phase". Item 15 of `sdk-design-nodejs/10` decided it: the strict outbound path stays, no relaxed emit path. - `body/request-body-logging.ts` ended mid-sentence before its `@internal` tag, which has been true since `e3ba885`. Completed with what `snapshot()` actually guarantees (`io/byte-queue.ts:96`). - `retry/retry-step.ts` said `RequestOptionsBuilder.maxRetries` "rejects only a negative value ... it admits Infinity, NaN, and fractions". `http/request-options.ts:212-219` rejects all three, pinned by that file's `maxRetries validation (HTTP-35)` block. The `invariant` stays, and now says why. - `pagination/strategies.ts` said the template "never changes across the walk". `pageNumberStrategy` returns the next one and `paginator.ts:213` installs it; `strategy.ts:10-15` already had it right. Doc-only; no signature moved, so no `etc/*.api.md` changed. Part of audit #67 / #68 (CTX-14, HTTP-18, HTTP-35, BODY-19, PAGE-17, TRANSPORT-20). --- packages/core/src/body/request-body-logging.ts | 4 ++++ packages/core/src/context/instrumentation.ts | 11 ++++++++--- packages/core/src/http/request-conditions.ts | 8 ++++++-- packages/core/src/io/index.ts | 17 ++++++++++------- packages/core/src/pagination/strategies.ts | 10 +++++++--- packages/core/src/retry/retry-step.ts | 17 +++++++++++------ 6 files changed, 46 insertions(+), 21 deletions(-) diff --git a/packages/core/src/body/request-body-logging.ts b/packages/core/src/body/request-body-logging.ts index 28779a8..c52165d 100644 --- a/packages/core/src/body/request-body-logging.ts +++ b/packages/core/src/body/request-body-logging.ts @@ -73,6 +73,10 @@ function tappedSink(state: TapState): WritableStream { * Mirrors up to tapCapBytes of each writeTo call into an internal tap while forwarding the full, * untruncated payload to the primary sink (BODY-17). The tap clears at the start of every write so a * retry against a replayable delegate does not accumulate stale bytes (BODY-18). No handle onto the tap's + * buffer escapes: `snapshot()` returns a fresh, independent copy of the current contents (BODY-19, + * `../io/byte-queue.ts:96`), so a caller holding one cannot observe or disturb a later write. Each + * `materialize()` wraps its own `ByteQueue` for the same reason (BODY-21). + * * @internal Consumed by the LOGGING pillar step (OBS-36). */ export function withRequestLogging( diff --git a/packages/core/src/context/instrumentation.ts b/packages/core/src/context/instrumentation.ts index 4bd4d15..dd1838b 100644 --- a/packages/core/src/context/instrumentation.ts +++ b/packages/core/src/context/instrumentation.ts @@ -36,13 +36,18 @@ export interface InstrumentationBundle { */ readonly activeSpan: unknown; /** - * Starts a child span for `operationName`. A no-op returning `undefined` when tracing is disabled. + * Returns the tracer to open `operationName`'s span from — a **tracer**, not a started span. Every + * consumer in this package narrows the result and calls `startSpan()` on it itself + * (`packages/core/src/pipeline/runtime.ts:52-57`, `observability/logging-step.ts:263-269`), and + * `createInstrumentationBundle` supplies a `(operationName: string) => Tracer` + * (`observability/tracing.ts:212,223`). A no-op returning `undefined` when tracing is disabled; + * both consumers substitute `NOOP_TRACER` for that `undefined`. * * PROVISIONAL: the return type is `unknown` pending Phase 7a's tracing adapter — see this * interface's own note. * - * @param operationName - the operation to name the child span after. - * @returns the started span, or `undefined` when tracing is disabled. + * @param operationName - the operation whose span the returned tracer will be asked to start. + * @returns the tracer for that operation, or `undefined` when tracing is disabled. */ readonly tracerFactory: (operationName: string) => unknown; } diff --git a/packages/core/src/http/request-conditions.ts b/packages/core/src/http/request-conditions.ts index 2af7ec9..a973e88 100644 --- a/packages/core/src/http/request-conditions.ts +++ b/packages/core/src/http/request-conditions.ts @@ -118,8 +118,12 @@ export class RequestConditions { * untouched. * * Emission goes through the strict outbound header path, which rejects obs-text. An ETag whose - * opaque tag carries obs-text is legal per HTTP-48 but cannot be emitted here; reconciling that - * against HTTP-18 is left to a later phase rather than guessed at now. + * opaque tag carries obs-text is legal per HTTP-48 but cannot be emitted here, so a server-issued + * one does not round-trip. That is decided, not open: HTTP-18 is MUST-level and reinforced by + * XCUT-18's splitting defense, HTTP-48's permission is SHOULD-level, and no relaxed emit path is + * added — see item 15 of + * `docs/sdk-design-nodejs/10-deliberate-deviations-from-the-reference-contract.md` and its + * as-built audit in `docs/deviations.md`. * * @param headers - the headers to derive from; not modified. * @returns a new {@link Headers} carrying the preconditions. diff --git a/packages/core/src/io/index.ts b/packages/core/src/io/index.ts index c943288..e099471 100644 --- a/packages/core/src/io/index.ts +++ b/packages/core/src/io/index.ts @@ -7,13 +7,16 @@ // api-extractor surface so a later phase can promote them deliberately (styleguide 10.3), or not at // all: 3b shaped BODY-1's write-to-sink around the platform's WritableStream rather than BufferedSink. // -// The ERROR leaves are a different case, and this comment claimed otherwise until 2026-09-02. Phase -// 8a promoted `IoError` and `TransportFailureError` to the public barrel (`index.ts:34`) because -// TRANSPORT-20 makes the subtyping a requirement and `retry/classify.ts`'s cause-walk is -// load-bearing on it, and Phase 9's U9 pass promoted `EndOfStreamError` — it was the subject of four -// `@throws` tags on public symbols with no class a caller could catch. `isIoError`, -// `AllocationLimitError`, `ClosedResourceError` and `SourceContractViolationError` remain internal -// (docs/work/mvp/2026-09-04-open-items-dissolution.md H8). +// The ERROR leaves are a different case, and this comment has twice claimed otherwise. Phase 8a +// promoted `IoError` and `TransportFailureError` to the public barrel because TRANSPORT-20 makes the +// subtyping a requirement and `retry/classify.ts`'s cause-walk is load-bearing on it, and Phase 9's +// U9 pass promoted `EndOfStreamError` — it was the subject of four `@throws` tags on public symbols +// with no class a caller could catch. `1f48926` finished the set: `isIoError`, +// `AllocationLimitError`, `ClosedResourceError` and `SourceContractViolationError` are exported as +// well, so every error symbol re-exported below is also on `packages/core/src/index.ts:39-47` and in +// `packages/core/etc/core.api.md`. Nothing in this file's error block is internal any more +// (docs/work/mvp/2026-09-04-open-items-dissolution.md H8, whose remaining sub-item was the category +// catch `isIoError` now provides). export {BufferedSink} from './buffered-sink.js'; export {BufferedSource} from './buffered-source.js'; export {ByteQueue, copyBytes} from './byte-queue.js'; diff --git a/packages/core/src/pagination/strategies.ts b/packages/core/src/pagination/strategies.ts index 165ce12..d2f0ecf 100644 --- a/packages/core/src/pagination/strategies.ts +++ b/packages/core/src/pagination/strategies.ts @@ -54,9 +54,13 @@ export function cursorStrategy(init: { * An empty items list ends the stream **before** any arithmetic runs — defensive against servers that keep * returning an empty page past the end instead of signalling termination, which would otherwise walk forever. * - * The current page comes from the *executed* request's query, not the template's, because the template never - * changes across the walk. An absent, empty, or non-numeric value falls back to `startPage`; `startPage: 0` - * supports 0-based servers. + * The current page comes from the *executed* request's query (`response.request.url`), not the template's. + * The template is not a fixed page-1 request -- it advances with the walk, since this function returns the + * next one as `nextRequest` and the engine makes that the following hop's template + * (`paginator.ts:165,213`; the contract is on `PaginationStrategy.parse` in `strategy.ts:10-15`). It is the + * *pre-flight* request for this hop, so it is the response's own request that reflects a redirect or any + * rewrite a step applied on the way out, and that is the page number worth incrementing. An absent, empty, + * or non-numeric value falls back to `startPage`; `startPage: 0` supports 0-based servers. * * @public */ diff --git a/packages/core/src/retry/retry-step.ts b/packages/core/src/retry/retry-step.ts index 09b5534..da146c0 100644 --- a/packages/core/src/retry/retry-step.ts +++ b/packages/core/src/retry/retry-step.ts @@ -65,12 +65,17 @@ function attemptVia(fork: () => Next): RetryDispatch { * RETRY-41/HTTP-35: the per-call `RequestOptions.maxRetries` override wins over the configured budget * when present. The option counts retries; `maxAttempts` counts total sends, hence the `+ 1`. * - * The value IS revalidated here. `RequestOptionsBuilder.maxRetries` rejects only a negative value, - * which is strictly weaker than the `Number.isFinite(...) && >= 1` guard `retrySettings()` applies - * to the configured budget -- it admits `Infinity`, `NaN`, and fractions. Any of the first two - * reaching `maxAttempts` makes the engine's `attempt >= maxAttempts` gate permanently false and the - * retry loop unbounded, so the per-call route must not be the one path into the engine that skips - * the check the configured route enforces. + * The value IS revalidated here, and the two guards now agree. `RequestOptionsBuilder.maxRetries` + * rejects a negative, fractional or non-finite value at the call site that supplied it + * (`../http/request-options.ts:212-219`, pinned by `request-options.test.ts`'s + * `maxRetries validation (HTTP-35)` block), so this `invariant` is the engine asserting its own + * precondition rather than the only thing enforcing it -- it should be unreachable, and tripping it + * means the builder's guard was weakened. That was not true when this comment was first written: the + * builder then rejected only a negative value, which was strictly weaker than the + * `Number.isFinite(...) && >= 1` guard `retrySettings()` applies to the configured budget. The + * assertion stays either way, because `Infinity` or `NaN` reaching `maxAttempts` makes the engine's + * `attempt >= maxAttempts` gate permanently false and the retry loop unbounded, and the per-call + * route must not be the one path into the engine that skips the check the configured route enforces. * * The derived object is frozen: a spread of a frozen source is NOT itself frozen, and RETRY-42 * requires every policy component to be immutable after construction, not merely typed `readonly`. From 1fddb2137107c5024bbb4f61552571d17207f202 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:13:03 +0300 Subject: [PATCH 004/100] docs(core): name RequestBodyNotAllowedError on buildRequest, and pin it `buildRequest` assembles through `Request.Builder.build()`, so a descriptor pairing a body with GET/HEAD/TRACE/CONNECT throws `RequestBodyNotAllowedError` (`http/request.ts:221`). Its `@throws` list named only `OperationAssemblyError` and `UrlConstructionError`, so the class was unreachable from the shipped `.d.ts` for a caller trying to catch it. No test pinned the path either. Added one; confirmed it fails when the assertion is inverted to `.not.toThrow()`. Part of audit #67 / #68 (HTTP-7, SEAM-26). --- packages/core/src/seams/operation.test.ts | 22 ++++++++++++++++++++-- packages/core/src/seams/operation.ts | 3 +++ 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/packages/core/src/seams/operation.test.ts b/packages/core/src/seams/operation.test.ts index 0217b72..3927e4e 100644 --- a/packages/core/src/seams/operation.test.ts +++ b/packages/core/src/seams/operation.test.ts @@ -1,7 +1,8 @@ // SPDX-License-Identifier: MIT // packages/core/src/seams/operation.test.ts // Exercises: SEAM-26 (the four projections default to empty), SEAM-27 (buildRequest's encoding and base-URL -// composition rules), reusing HTTP-29's encodeRfc3986Component for path-segment encoding. +// composition rules), HTTP-7 (a body projected onto a body-forbidding method fails assembly), reusing +// HTTP-29's encodeRfc3986Component for path-segment encoding. import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import { @@ -9,7 +10,10 @@ import { OperationAssemblyError, type OperationDescriptor, } from './operation.js'; -import {UrlConstructionError} from '../http/errors.js'; +import { + RequestBodyNotAllowedError, + UrlConstructionError, +} from '../http/errors.js'; import {QueryParams} from '../http/query-params.js'; import {Headers} from '../http/headers.js'; @@ -113,6 +117,20 @@ describe('operation headers and body projections are threaded through', () => { expect(request.headers.get('x-trace')).toBe('abc'); expect(request.body).toBe(body); }); + + // HTTP-7: assembly ends at `Request.Builder.build()`, so the builder's method/body legality check is + // buildRequest's. Pinned because the throw reaches a caller through `buildRequest` and was absent from + // its `@throws` list until audit #67 / #68 added it -- an undocumented, unpinned throw path is exactly + // the kind that a later refactor swallows. + test('a body on a body-forbidding method throws RequestBodyNotAllowedError', () => { + expect(() => + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/pets', + body: stringBody('Fido'), + }), + ).toThrow(RequestBodyNotAllowedError); + }); }); describe('SEAM-27: dot-segment path-param values are rejected, not silently normalized away', () => { diff --git a/packages/core/src/seams/operation.ts b/packages/core/src/seams/operation.ts index 260c36e..b6213f2 100644 --- a/packages/core/src/seams/operation.ts +++ b/packages/core/src/seams/operation.ts @@ -162,6 +162,9 @@ function composeQuery( * or a supplied value is a dot segment (`.`/`..`) — fix the descriptor; no request was assembled. * @throws {@link UrlConstructionError} when `baseUrl` is malformed, non-absolute, or carries a * fragment — supply a clean absolute base URL. + * @throws {@link RequestBodyNotAllowedError} when the descriptor pairs a body with GET, HEAD, TRACE + * or CONNECT (HTTP-7). Assembly ends at `Request.Builder.build()`, so that builder's validation is + * this function's validation. * * @public */ From 8d777c9a163384aee11b0d83418a3749e0237496 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:13:09 +0300 Subject: [PATCH 005/100] test(transport-conformance): stop claiming TRANSPORT-22 the suite never drives The file header listed `TRANSPORT-20..27` and `registerFailureRows`'s describe was named `TRANSPORT-4/5/6/20/22`, but no row in it forces an adaptation throw -- its four rows are a dead port and three timeout cases. Forcing one needs a hook into the native response, so each adapter asserts it against its own (`transport-fetch/src/fetch-transport.test.ts:118`, `transport-undici/src/undici-transport.test.ts:503`). Both header and describe now say so. Comment-only; the suite runs the same rows. Part of audit #67 / #68 (TRANSPORT-22). --- packages/transport-conformance/src/run-suite.ts | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 316110a..b6eadfa 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -1,10 +1,13 @@ // SPDX-License-Identifier: MIT // packages/transport-conformance/src/run-suite.ts // The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot -// drift. Exercises: TRANSPORT-1..9, TRANSPORT-15..17, TRANSPORT-20..27, TRANSPORT-29, SEAM-12, -// SEAM-16, SEAM-30, NFR-15. TRANSPORT-10..14 are asserted at their source in @dexpace/transport-shared; -// TRANSPORT-18/28's collapses are Deviation Ledger rows; TRANSPORT-30's full flow is -// transport-undici's challenge-handler.test.ts. +// drift. Exercises: TRANSPORT-1..9, TRANSPORT-15..17, TRANSPORT-20..21, TRANSPORT-23..27, +// TRANSPORT-29, SEAM-12, SEAM-16, SEAM-30, NFR-15. TRANSPORT-10..14 are asserted at their source in +// @dexpace/transport-shared; TRANSPORT-18/28's collapses are Deviation Ledger rows; TRANSPORT-30's +// full flow is transport-undici's challenge-handler.test.ts. TRANSPORT-22 is NOT driven from here -- +// forcing an adaptation throw needs a per-transport hook into the native response, so each adapter +// asserts it against its own (transport-fetch's fetch-transport.test.ts:118, transport-undici's +// undici-transport.test.ts:503). import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; import { getBuildInfo, @@ -311,7 +314,7 @@ function registerProducerRows(ctx: SuiteContext): void { } function registerFailureRows(ctx: SuiteContext): void { - describe('TRANSPORT-4/5/6/20/22: failure classification and socket release', () => { + describe('TRANSPORT-4/5/6/20: failure classification and per-call timeouts', () => { test('a dead port surfaces the retryable TransportFailureError', async () => { await withTransport(ctx.makeTransport, async transport => { const request = Request.newBuilder().url('http://127.0.0.1:1').build(); From 0c4c94ae2301bba4a47254f9dbeb730fedfe7a57 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:13:22 +0300 Subject: [PATCH 006/100] docs: re-anchor deviations.md items 3, 5, 15, 17 and the OBS-29 row Every `file:line` in the sections this audit touches now points at the text it cites, verified line by line against the worktree: - **Item 5** quoted `deserialize(data, schema, typeName?)` at `serde.ts:145` and the `Serde` interface at `:182`. The signature took its witness in a `DecodeTarget` on 2026-09-04; it is `:194`, `Serde` is `:241`, `deserializeFrom` is `:221`, and `DecodeTarget` is `:122-124`. The codec anchors move from `json-serde.ts:236,242` to `:244,250`. The deviation is unchanged -- a schema value still stands in for a reflected type token. - **Item 17**'s hierarchy lines were four to six off (`29,49,65,80` -> `29,51,67,83`; `isIoError` `102` -> `108`; `TransportFailureError` `126` -> `132`). Its "cause-walk returns retryable for any `IoError`" reads as covering all five I/O classes and does not: `classify.ts:73` tests `instanceof IoError`, which the flat leaves fail. Recorded as an anchor correction with the rule left to #78, which decides `instanceof` vs. `isIoError` and rewrites the rationale. - **The OBS-29 row** said "the 1:1 binding is NOT met" and cited only the per-attempt span in `logging-step.ts`. `pipeline/runtime.ts:33-48,154-157,171-175` opens one span per logical operation outside every pillar, and `send`'s own `@remarks` states the binding. Marked IN PROGRESS for #80, which owns what a caller can reach. - **Items 3 and 15** are re-anchored because this branch's own source edits shifted them (`retry-step.ts:137` -> `:142`, `request-conditions.ts:129-142` -> `:133-146`). Two counts deleted rather than corrected, per CLAUDE.md: the section intro's "Four rows as of 2026-09-02" above a five-row table, and the `invariant()` cell's "fifteen" for a `pipeline/` that ships three. The cell now carries the command to re-derive them instead. Part of audit #67 / #68 (SERDE-5, SERDE-13, OBS-29, TRANSPORT-20, HTTP-18). --- docs/deviations.md | 52 +++++++++++++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 17 deletions(-) diff --git a/docs/deviations.md b/docs/deviations.md index 7ffe3d6..b4a293b 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -134,7 +134,7 @@ non-bridge clause survives and is enforced as an ordinary obligation on `send()` ## 3. Two retry stacks collapse into one, with the total-timeout budget explicitly opt-in **Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:354`) — with exactly two thin -callers: the pillar step (`packages/core/src/retry/retry-step.ts:137`) and the dispatch adapter +callers: the pillar step (`packages/core/src/retry/retry-step.ts:142`) and the dispatch adapter (`packages/core/src/retry/retry-dispatch.ts:53`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | undefined` and undefined by default (`packages/core/src/retry/settings.ts:27`), pinned by a test named for `RETRY-28` (`packages/core/src/retry/settings.test.ts:20`). @@ -172,11 +172,16 @@ language-level ceiling. ## 5. Schema-as-witness replaces reflective generic-type capture -**Verified.** `Serde` is not generic in `T` (`packages/core/src/seams/serde.ts:182`); the witness is a -decode-time parameter, `deserialize(data, schema: Schema, typeName?)` -(`packages/core/src/seams/serde.ts:145`). The codec-configuration knobs are absent and documented as -absent — `packages/codec-json/src/json-serde.ts:236,242` explain that `SERDE-21`/`22` have no coercion -setting because there is no coercing codec, and `SERDE-23`'s unknown-field policy belongs to the schema. +**Verified.** `Serde` is not generic in `T` (`packages/core/src/seams/serde.ts:241`); the witness is a +decode-time parameter, `deserialize(data: Uint8Array, target: DecodeTarget): T` +(`packages/core/src/seams/serde.ts:194`, and `deserializeFrom` at `:221` takes the same target), +where `DecodeTarget` bundles the `Schema` witness with its optional diagnostic label +(`:122-124`). The witness moved into that object on 2026-09-04 — the SPI took it positionally as +`(data, schema, typeName?)` until then, which is the signature this row quoted; the deviation is +unchanged by the reshaping, since a schema *value* is still what stands in for a reflected type token. +The codec-configuration knobs are absent and documented as absent — +`packages/codec-json/src/json-serde.ts:244,250` explain that `SERDE-23`'s unknown-field policy belongs +to the schema and that `SERDE-21`/`22` have no coercion setting because there is no coercing codec. `packages/codec-json/src/conformance.test.ts:8` states outright that no code implements `SERDE-21` or `SERDE-22`. @@ -379,7 +384,7 @@ produce that evidence; it unblocks at first release and not before. ## 15. A server-issued ETag containing obs-text does not round-trip **Verified.** `RequestConditions.applyTo` writes every entity tag through the outbound `Headers` builder's -`set` (`packages/core/src/http/request-conditions.ts:129-142`), which enforces `HTTP-18`'s HTAB + printable +`set` (`packages/core/src/http/request-conditions.ts:133-146`), which enforces `HTTP-18`'s HTAB + printable ASCII 0x20–0x7E rule (`packages/core/src/http/ascii-validation.ts:16`). The inbound path is separately laxer and permits obs-text, exactly as `HTTP-19` requires (`packages/core/src/http/ascii-validation.ts:41`, `packages/core/src/http/headers.ts:246,262`). @@ -412,21 +417,34 @@ single-subscription behavior is forced by HTTP itself, since a consumed response **Verified.** `IoError extends DexpaceError` (`packages/core/src/io/errors.ts:13`); the four I/O leaves — `EndOfStreamError`, `SourceContractViolationError`, `ClosedResourceError`, `AllocationLimitError` — each -extend `DexpaceError` **directly** (lines 29, 49, 65, 80) and are grouped by the `isIoError` predicate -(line 102) rather than by a middle tier. `TransportFailureError extends IoError` (line 126) is the single +extend `DexpaceError` **directly** (lines 29, 51, 67, 83) and are grouped by the `isIoError` predicate +(line 108) rather than by a middle tier. `TransportFailureError extends IoError` (line 132) is the single three-level branch. **Why it cannot be corrected.** `TRANSPORT-20` requires `TransportFailureError` to *be* an `IoError` — the subtyping is the requirement, not an artifact of modelling. It is also load-bearing: `classify.ts`'s -cause-walk returns retryable for any `IoError`, so the `extends` is what makes a no-response transport -failure retryable with zero edits to the retry layer. A flat sibling would have to be enumerated by hand in -the retry classifier, and again for every transport added later — trading one level of depth for an -open-ended maintenance obligation that the styleguide's own rule exists to prevent. Held at exactly three; -a fourth level is not sanctioned by this entry. +cause-walk tests `current instanceof IoError` (`packages/core/src/retry/classify.ts:73`), so the `extends` +is what makes a no-response transport failure retryable with zero edits to the retry layer. A flat sibling +would have to be enumerated by hand in the retry classifier, and again for every transport added later — +trading one level of depth for an open-ended maintenance obligation that the styleguide's own rule exists +to prevent. Held at exactly three; a fourth level is not sanctioned by this entry. + +> **Anchor correction 2026-09-04 (audit #67 / #68); the rule itself is not decided here.** The line +> numbers above were stale and are re-derived. The substantive point this paragraph used to make — "the +> cause-walk returns retryable for any `IoError`" — reads as covering all five I/O classes, and it does +> not: because the tree is flat, `instanceof IoError` at `classify.ts:73` matches `IoError` itself and +> `TransportFailureError` only. `EndOfStreamError`, `SourceContractViolationError`, +> `ClosedResourceError` and `AllocationLimitError` are **not** retryable through that branch. Whether +> they should be — and therefore whether the classifier tests `instanceof IoError` or the `isIoError` +> predicate — is decided by audit subtask #78, and this row's rationale is rewritten there. Nothing in +> the deviation itself (the three-level branch, and why it stays) turns on that answer. ## Deviations recorded outside a phase -Four rows as of 2026-09-02, all from the `docs/work/mvp/2026-09-04-open-items-dissolution.md` register audit (that file's Section V). +Every row here was found by a pass over shipped code rather than produced by a phase. The +`docs/work/mvp/2026-09-04-open-items-dissolution.md` register audit (that file's Section V) opened the +section on 2026-09-02; later reviews and audits append to it. Deliberately uncounted — a stated total is a +number that goes wrong on the next append, which is that file's U10. A row here has no owning phase — it was found by a review, an audit, or a maintenance pass over shipped code. It is recorded on the day it is found rather than waiting for `docs/sdk-design-nodejs/10-…`, which is in a @@ -437,8 +455,8 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended |---|---|---|---|---| | **`HTTP-11`'s range classifications are on `Status` only, not mirrored onto `Response`.** The spec places them on both: `docs/product-spec/04-core-http-domain-model.md:23` reads "Status MUST classify by range … **and a response MUST expose these derived from its status**", and appendix C (`appendix-c-consolidated-normative-requirement-index.md:47`) restates it as "Response MUST expose these same classifications derived from its status." The port ships them once, on `Status`, reachable as `response.status.isSuccess`. Six delegating getters on `Response` would duplicate surface that cannot drift, since there would be one implementation behind both — but the letter of the requirement does name the response, so the reading is recorded rather than assumed | the dissolved register's A3, register audit | 2026-09-02 | `packages/core/src/http/status.ts` carries all six; `packages/core/src/http/response.ts` carries `status` and no classification of its own | not yet in §10 | | **`REDIR-20`'s "fully override" is read as scoped to code/method eligibility, not to the safety mechanics that follow it.** A configured redirect predicate replaces the built-in follow decision; it does **not** bypass userinfo stripping, credential hygiene, the downgrade guard, body replayability, or loop/cap detection. Those are stated as unconditional MUSTs elsewhere in the same chapter and are not "should this kind of redirect be followed" policy — a caller predicate opting to follow a 307 with a single-use body still cannot make that body re-sendable. Genuinely ambiguous wording, decided one way and now recorded as decided | the dissolved register's G4, register audit | 2026-09-02 | `packages/core/src/redirect/decide.ts` consults `settings.predicate` at the eligibility gate and runs every later guard unconditionally; pinned by "the predicate does NOT bypass the safety mechanics" in `decide.test.ts`. Phase 9's conformance sweep was to re-confirm this and closed without doing so | not yet in §10 | -| **Span-shaped tracing carries `OBS-29`'s ordered lifecycle, but not its 1:1 tracer-to-operation binding.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`. The **ordering half is satisfied** under different names: a span is started once (`observability/logging-step.ts:427`) and ended once on exactly one of two paths — `span.end()` for success (`:398`), or `span.recordException(error)` then `span.end()` for failure (`:414-415`) — mutually exclusive, once each, in order. **The 1:1 binding is NOT met**, and this row says so plainly rather than implying the requirement is covered: `PIPE-2` fixes the `LOGGING` pillar step *inside* the `RETRY` and `REDIRECT` pipelines, so a span is opened per transmission attempt and per redirect hop, not once per logical operation (the dissolved register's L4 records the same). The per-attempt and retries-exhausted events therefore have no operation-scoped tracer to attach to. Appendix C's own entry (`appendix-c-consolidated-normative-requirement-index.md:509`) anticipates the gap: "This is a documented emission contract; pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced." The wiring stays open as the dissolved register's V2 | the dissolved register's L1/V2, register audit | 2026-09-02 | `packages/core/src/observability/tracing.ts:40-42` (`Tracer`, one method); `observability/logging-step.ts:398,414-415,427` (the lifecycle); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | -| **`invariant()` density is not a target, project-wide.** `docs/knowledge/harvested/assertions.md:6-7` sets a 2-per-function module average. The port's position: a module gains an `invariant()` when it has an internal precondition worth asserting, and `recovery/`, `http/`, `seams/` and `generated/` have none — measured 2026-09-02, all four at zero. Adding assertions to reach an average would assert nothing. `recovery/` is the sharp case: an `invariant()` inside `ResponseRecoveryChain.apply()` throws, and `RECOV-8` forbids `apply()` from throwing, so a density rule would push that module toward a shape the specification forbids | the dissolved register's F3/H6 and the deferral register's *Assertion-density rule applied project-wide* row (retired to [the purge note](./work/mvp/2026-09-04-register-retirement-purge.md)), register audit | 2026-09-02 | `packages/core/src/recovery/`, `http/`, `seams/`: zero `invariant(` calls. `pipeline/` ships fifteen and `context/` three, so the rule is applied where it earns its place | not yet in §10 | +| **`OBS-29` is carried by spans rather than by the named tracer callbacks. IN PROGRESS — see [#80](https://github.com/dexpace/nodejs-sdk/issues/80) for the final disposition.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`. The **ordering half is satisfied** under different names, at both levels: `Runtime.send` opens one span before the drive and ends it on exactly one of two paths — `span.end()` for success, or `span.recordException(error)` then `span.end()` for failure (`pipeline/runtime.ts:193,198-199`) — and the per-attempt spans the LOGGING pillar step opens follow the same shape (`observability/logging-step.ts:427`, then `:398` or `:414-415`). **The 1:1-binding half was recorded here on 2026-09-02 as NOT met, and that reading is out of date.** `Runtime.send` is the one place that runs exactly once per logical operation, and it is where the operation span is opened, outside every pillar, so a retry's second attempt and a redirect's second hop stay inside the same span (`pipeline/runtime.ts:33-48,171-175`; `send`'s own `@remarks` at `:154-157` states the binding). `PIPE-2` still fixes the LOGGING step *inside* the RETRY and REDIRECT pipelines, so its spans remain per transmission — they are now children of the operation span rather than the only spans there are. What stays open is what a *caller* can reach: appendix C's entry (`appendix-c-consolidated-normative-requirement-index.md:509`) says "pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced", and the operation span is not reachable from the public API. #80 decides that and rewrites this row; anchors re-derived by audit #67 / #68 | the dissolved register's L1/V2, register audit; anchors re-derived by audit #67 / #68 | 2026-09-02, re-anchored 2026-09-04 | `packages/core/src/observability/tracing.ts:40-42` (`Tracer`, one method); `packages/core/src/pipeline/runtime.ts:33-48,154-157,171-175,193,198-199` (the operation span and its 1:1 binding); `observability/logging-step.ts:398,414-415,427` (the per-attempt span); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | +| **`invariant()` density is not a target, project-wide.** `docs/knowledge/harvested/assertions.md:6-7` sets a 2-per-function module average. The port's position: a module gains an `invariant()` when it has an internal precondition worth asserting, and `recovery/`, `http/`, `seams/` and `generated/` have none — measured 2026-09-02, all four at zero. Adding assertions to reach an average would assert nothing. `recovery/` is the sharp case: an `invariant()` inside `ResponseRecoveryChain.apply()` throws, and `RECOV-8` forbids `apply()` from throwing, so a density rule would push that module toward a shape the specification forbids | the dissolved register's F3/H6 and the deferral register's *Assertion-density rule applied project-wide* row (retired to [the purge note](./work/mvp/2026-09-04-register-retirement-purge.md)), register audit | 2026-09-02 | `packages/core/src/recovery/`, `http/`, `seams/`: zero `invariant(` calls. `pipeline/` and `context/` both carry them, so the rule is applied where it earns its place. Counted qualitatively on purpose — the two figures this cell used to state were wrong by the time anyone read them; re-derive with `grep -rn 'invariant(' packages/core/src/ --include='*.ts' \| grep -v '\.test\.'` | not yet in §10 | | **`PIPE-40` and `REDIR-22` contradict each other on the non-replayable-body path, and the port implements `REDIR-22`.** Two MUSTs name the same trigger and prescribe opposite dispositions. `docs/product-spec/08-execution-pipelines.md:20` (`PIPE-40`): "on paths that abandon a re-drive (redirect cycle, **non-replayable body**, budget exhausted) the in-flight response MUST be returned unclosed." `docs/product-spec/10-redirect-handling.md:22` (`REDIR-22`): "if building the follow-up throws (**non-replayable body**, downgrade rejection) the current response MUST be closed before the error propagates." The port closes, then throws, on three grounds: `REDIR-6` independently fixes the control flow ("the operation MUST fail with a clear error naming replayability"), so the path throws and a response never *returned* cannot be "returned unclosed"; specific governs general, since §10 of the spec owns the redirect step's lifecycle; and closing is the safer reading, because the alternative leaks a body on an error path with no caller holding a reference to close it. `PIPE-40`'s other two named paths do genuinely return, and both return unclosed as it requires. **The erratum this needs is proposed below and is not applied here**, because `docs/product-spec/` is frozen and correcting a normative sentence is the specification owner's act, not a maintenance one | the dissolved register's G1, Phase 5b design's Deviation Ledger | 2026-09-04 | `packages/core/src/redirect/redirect-step.ts` closes before throwing, with the reasoning asserted inline in `redirect-step.test.ts`; nothing in the code waits on the erratum | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) From c983c0cc862678a7f022dc5a6c28cc6e829788a7 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:13:31 +0300 Subject: [PATCH 007/100] docs: make three as-built guides match the functions they document - `write-a-response-handler.md` said `decodeSuccessResponse` delegates to `toHttpError` "otherwise", and that `HttpStatusError` carries "the status, the headers and a bounded body preview". `serde/response-handlers.ts:189-201` routes 4xx/5xx there and closes-then-raises a `DeserializationError` for every other non-2xx; `body/http-status-error.ts` has `status`, `body()` and `preview()` and no headers accessor. - `write-a-paging-strategy.md` twice described `template` as "the request the walk started from". It is the request that fetched this page and it advances (`pagination/paginator.ts:165,213`, `strategy.ts:10-15`) -- the same wrong reading `strategies.ts` carried. - `auth.md` named the source of `perCall` and `client` but not `operation`. It is `RequestOptions.operationAuth` (`auth/auth-step.ts:252,756`, `auth/resolve.ts:19`), which has had a source only since `1f48926`. `probe.mjs` and `check-fences.mjs` both pass. The credential-shape example in `auth.md` is left for #71. Part of audit #67 / #68 (SERDE-28, PAGE-5, PAGE-17, AUTH-4). --- docs/sdk-documentation/auth.md | 8 +++++-- .../write-a-paging-strategy.md | 21 ++++++++++++++----- .../write-a-response-handler.md | 11 +++++++--- 3 files changed, 30 insertions(+), 10 deletions(-) diff --git a/docs/sdk-documentation/auth.md b/docs/sdk-documentation/auth.md index 8f17c2b..2b5b32d 100644 --- a/docs/sdk-documentation/auth.md +++ b/docs/sdk-documentation/auth.md @@ -16,8 +16,12 @@ interface AuthTiers { The most specific tier that is set wins (`AUTH-4`). `perCall` comes from `RequestOptions.newBuilder().auth(descriptor)`, which is how one call opts out of, or into, something -different from the client default. Only the *selection* is tiered; whether a selected requirement can -be satisfied at all is `AUTH-6`, and failing it is an `AuthResolutionError`. +different from the client default. **`operation` comes from `RequestOptions.operationAuth`**, set with +`RequestOptions.newBuilder().operationAuth(descriptor)` — both per-call slots ride on the same +`RequestOptions`, and the AUTH pillar step reads them into the two tiers +(`packages/core/src/auth/auth-step.ts:252,756`; `packages/core/src/auth/resolve.ts:19` names the mapping). `client` is the +configured default. Only the *selection* is tiered; whether a selected requirement can be satisfied at +all is `AUTH-6`, and failing it is an `AuthResolutionError`. An `AuthDescriptor` is a list of `AuthRequirement`s, each a scheme plus optional scopes and parameters. Both are built by factory, never as an object literal: diff --git a/docs/sdk-documentation/write-a-paging-strategy.md b/docs/sdk-documentation/write-a-paging-strategy.md index d2bc52c..40d1700 100644 --- a/docs/sdk-documentation/write-a-paging-strategy.md +++ b/docs/sdk-documentation/write-a-paging-strategy.md @@ -13,9 +13,18 @@ interface PageInfo { } ``` -Given the page that just arrived and the request template the walk started from, produce this page's -items and the request that fetches the next one. `undefined` for `nextRequest` is how a walk ends — -there is no separate "done" flag to keep consistent with it. +Given the page that just arrived and the request that fetched it, produce this page's items and the +request that fetches the next one. `undefined` for `nextRequest` is how a walk ends — there is no +separate "done" flag to keep consistent with it. + +**`template` is not the request the walk started from.** The glossary calls it "the original request +template", but the engine passes the request it sent for *this* page and then makes your `nextRequest` +the following hop's template — it advances with the walk +(`packages/core/src/pagination/paginator.ts:165,213`, and the contract on +`PaginationStrategy.parse` at `strategy.ts:10-15`). Read the parameter as "the request to derive the +next one from". Where you specifically want the URL the response actually came from — after a redirect +or a step's rewrite — use `response.request.url`, which is what `pageNumberStrategy` reads its current +page number from. ## Three ship already @@ -101,8 +110,10 @@ protects — single use, nothing retained — is what the async signature preser [`docs/work/mvp/2026-09-04-register-retirement-purge.md`](../work/mvp/2026-09-04-register-retirement-purge.md), where the dissolved deferral register's rows went — precisely so an async signature does not later read as an oversight. Every shipped strategy's `extract` above is `async` for the same reason. -**2. Build `nextRequest` from the template, not from the response.** The template carries the headers, -auth tier and options the walk was started with. A next request built from scratch loses all of them. +**2. Build `nextRequest` from the template, not from scratch.** The template carries the headers, auth +tier and options the walk was started with, and it is the previous hop's request rather than page one's, +so deriving from it accumulates the walk's state instead of re-deriving it. A next request built from +scratch loses all of that. ```typescript const next = template diff --git a/docs/sdk-documentation/write-a-response-handler.md b/docs/sdk-documentation/write-a-response-handler.md index ed46630..da5d364 100644 --- a/docs/sdk-documentation/write-a-response-handler.md +++ b/docs/sdk-documentation/write-a-response-handler.md @@ -41,9 +41,14 @@ DOM global under a `lib` that includes `"DOM"`, and the two are unrelated types `Response` has `status: Status`, `close()` and `request`, and no `json()`. The mistake typechecks until you try to pass one. -`decodeSuccessResponse` delegates to `decodeResponse` on a 2xx and to `toHttpError` otherwise, so a -`4xx` becomes an `HttpStatusError` carrying the status, the headers and a bounded body preview -(`SERDE-28`). +`decodeSuccessResponse` delegates to `decodeResponse` on a 2xx and to `toHttpError` on a **4xx or 5xx**, +so a `404` becomes an `HttpStatusError` carrying `status`, a replayable `body()` and a non-consuming +`preview()` — a bounded in-memory copy at the 1 MiB `HTTP-52`/`BODY-30` cap, taken before the response +was closed. There is no headers accessor on it; read anything else you need off the response before you +hand it over. Any *other* non-2xx — a `1xx`, or an unfollowed `3xx` such as a `304` — is neither decoded +nor mapped: the response is closed and a `DeserializationError` is raised whose message leads with the +status code and which carries `ETag` and `Location` as fields, so conditional and redirect context +survives the close (`SERDE-28`). ## What the shipped handlers guarantee, and what you must reproduce From 66ee76ddc0edba5fdc4f4dffd34190b8e9ba4288 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:14:55 +0300 Subject: [PATCH 008/100] fix: give CTX-15's no-op instrumentation bundle a no-op span (#69) `noopInstrumentationBundle.activeSpan` was `undefined`. CTX-15 asks for "a no-op span" beside the no-op tracer factory, and `createInstrumentationBundle` has always used `NOOP_SPAN` for the enabled bundle, so the two constructors of one interface disagreed. Phase 4a shipped the gap deliberately and ledgered it as partial: no `Span` type existed then. `Span`/`NOOP_SPAN` landed in Phase 7b, so the reason expired. The one-line value change needs a module split to land. `observability/ tracing.ts` imports `InstrumentationBundle` for `createInstrumentationBundle`'s return type, so importing `NOOP_SPAN` back into `context/instrumentation.ts` closes a cycle -- and `verify:import-cycles` counts type-only edges on purpose. Its failure message prescribes the fix taken here: the inert declarations (`SpanContext`, `Span`, `Tracer`, `NOOP_SPAN`, `NOOP_TRACER`) move to a leaf module `observability/span.ts` that imports nothing, and `tracing.ts` re-exports all five. No import path and no line of `etc/core.api.md` changes; `bun run api` is unchanged. `span.ts`'s module header is line comments that never write the internal-marker JSDoc tag out. `stripInternal` (on, from gts) tests it by substring-scanning every leading comment range of a declaration, line comments included, so a header merely mentioning the tag deleted `SpanContext` from the emitted `.d.ts` with no `tsc` diagnostic -- the build failed one package later on an unresolved name inside core's own `dist/`. Decided as D3 of the audit #67 decision ledger: fix, not a deviations.md row. --- .../core/src/context/instrumentation.test.ts | 14 +-- packages/core/src/context/instrumentation.ts | 6 +- packages/core/src/observability/span.ts | 88 +++++++++++++++++++ packages/core/src/observability/tracing.ts | 73 ++------------- 4 files changed, 109 insertions(+), 72 deletions(-) create mode 100644 packages/core/src/observability/span.ts diff --git a/packages/core/src/context/instrumentation.test.ts b/packages/core/src/context/instrumentation.test.ts index acd5f3f..d647d4d 100644 --- a/packages/core/src/context/instrumentation.test.ts +++ b/packages/core/src/context/instrumentation.test.ts @@ -3,6 +3,7 @@ // Exercises: CTX-14 (bundle shape), CTX-15 (no-op default: invalid sentinels, isValid/isRemote false, // no-op span/tracer factory), CTX-20 (tracer factory safe to invoke concurrently, emits nothing) import {describe, expect, test} from 'bun:test'; +import {NOOP_SPAN} from '../observability/span.js'; import {noopInstrumentationBundle} from './instrumentation.js'; describe('noopInstrumentationBundle (CTX-15)', () => { @@ -26,11 +27,14 @@ describe('noopInstrumentationBundle (CTX-15)', () => { expect(noopInstrumentationBundle.isRemote).toBe(false); }); - // CTX-15 says "a no-op span". With `activeSpan` typed `unknown` until a real tracing adapter lands - // (Phase 7), there is no Span shape to build a no-op instance of, so absence is the encoding. Logged as a - // partial deviation in the design's Deviation Ledger -- revisit when the adapter defines Span. - test('has no active span', () => { - expect(noopInstrumentationBundle.activeSpan).toBeUndefined(); + // CTX-15 says "a no-op span", and it means an object: the requirement lists the no-op span beside the + // no-op tracer factory, and `createInstrumentationBundle` has always used `NOOP_SPAN` for the ENABLED + // bundle (`observability/tracing.ts`). Phase 4a shipped `undefined` here because no `Span` type existed + // yet and recorded the gap as a partial deviation; `Span`/`NOOP_SPAN` landed in Phase 7b, so the reason + // expired and the two bundles no longer disagree. Identity, not shape: a caller narrowing `unknown` may + // compare against the exported singleton. + test('carries the no-op span singleton, not undefined', () => { + expect(noopInstrumentationBundle.activeSpan).toBe(NOOP_SPAN); }); test('tracerFactory emits nothing and is safe to invoke repeatedly (CTX-20)', () => { diff --git a/packages/core/src/context/instrumentation.ts b/packages/core/src/context/instrumentation.ts index 4bd4d15..9f239dc 100644 --- a/packages/core/src/context/instrumentation.ts +++ b/packages/core/src/context/instrumentation.ts @@ -1,5 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/context/instrumentation.ts +import {NOOP_SPAN} from '../observability/span.js'; /** * Correlation/instrumentation bundle every execution context carries (CTX-14), reachable from a @@ -30,7 +31,8 @@ export interface InstrumentationBundle { /** Whether the trace context was propagated in from a caller rather than started locally. */ readonly isRemote: boolean; /** - * The span this call runs inside, or `undefined` when tracing is disabled. + * The span this call runs inside. Never absent: the disabled-tracing default carries the inert + * `NOOP_SPAN` singleton, which is what CTX-15's "a no-op span" asks for. * * PROVISIONAL: typed `unknown` pending Phase 7a's tracing adapter — see this interface's own note. */ @@ -62,6 +64,6 @@ export const noopInstrumentationBundle: InstrumentationBundle = Object.freeze({ traceIdEncoding: 'none', isValid: false, isRemote: false, - activeSpan: undefined, + activeSpan: NOOP_SPAN, tracerFactory: () => undefined, }); diff --git a/packages/core/src/observability/span.ts b/packages/core/src/observability/span.ts new file mode 100644 index 0000000..9b45f33 --- /dev/null +++ b/packages/core/src/observability/span.ts @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/observability/span.ts + +// The inert half of the tracing surface: the span/tracer shapes (OBS-21, OBS-23) and the two no-op +// singletons (OBS-25) that need nothing else in the package to exist. +// +// This module exists to break an import cycle, not to introduce a layer. `tracing.ts` needs +// `InstrumentationBundle` from `context/instrumentation.ts` to type `createInstrumentationBundle`, and +// `context/instrumentation.ts` needs `NOOP_SPAN` for CTX-15's disabled-tracing default. Holding both in +// `tracing.ts` closes a cycle that `bun run verify:import-cycles` rejects -- type-only edges count there, +// deliberately -- and that gate's own message prescribes this fix: move the shared declaration into a +// module both sides can import. So this file imports nothing, and `tracing.ts` re-exports every name +// below, which is why no existing import path changed. +// +// Line comments, and deliberately none of them writes out the internal-marker JSDoc tag. Two traps stack +// at the top of a module: a leading `/** */` block binds to the first declaration below it rather than to +// the file, and gts turns on `stripInternal`, whose test is a plain substring scan of EVERY leading comment +// range of a declaration -- line comments included, doc block or not. So a module header that merely +// MENTIONS that tag deletes `SpanContext` from the emitted `.d.ts` while `tsc` stays silent; the failure +// surfaces one package later, as an unresolved name inside core's own `dist/`. Measured twice on the way +// to this wording. If this note is ever reworded, rebuild and check `dist/observability/span.d.ts` still +// declares all four names. + +/** + * Contextual metadata identifying a trace and span in distributed tracing (OBS-23, OBS-26). + * + * @public + */ +export interface SpanContext { + readonly traceId: string; + readonly spanId: string; + readonly traceFlags?: number | undefined; + readonly traceState?: string | undefined; +} + +/** + * A structural subset of `@opentelemetry/api`'s own `Span` shape (OBS-21, OBS-23). + * + * @public + */ +export interface Span { + readonly isRecording: boolean; + setAttribute(key: string, value: unknown): this; + recordException(error: unknown): this; + end(): void; + spanContext?(): SpanContext | undefined; +} + +/** + * Tracing facade interface for creating spans. + * + * @public + */ +export interface Tracer { + startSpan(name: string): Span; +} + +/** + * Inert no-op {@link Span} singleton (OBS-21, OBS-25). + * + * @public + */ +export const NOOP_SPAN: Span = Object.freeze({ + isRecording: false, + setAttribute(): Span { + return NOOP_SPAN; + }, + recordException(): Span { + return NOOP_SPAN; + }, + end(): void { + return; + }, + spanContext(): SpanContext { + return {traceId: '0'.repeat(32), spanId: '0'.repeat(16)}; + }, +}); + +/** + * Inert no-op {@link Tracer} singleton (OBS-25). + * + * @public + */ +export const NOOP_TRACER: Tracer = Object.freeze({ + startSpan(): Span { + return NOOP_SPAN; + }, +}); diff --git a/packages/core/src/observability/tracing.ts b/packages/core/src/observability/tracing.ts index 408198e..c0d9bcb 100644 --- a/packages/core/src/observability/tracing.ts +++ b/packages/core/src/observability/tracing.ts @@ -5,73 +5,16 @@ import { createAsyncScopedStore, pushDiagnosticFields, } from './diagnostic-context.js'; +import {NOOP_SPAN, NOOP_TRACER} from './span.js'; +import type {Span, Tracer} from './span.js'; import type {InstrumentationBundle} from '../context/instrumentation.js'; -/** - * Contextual metadata identifying a trace and span in distributed tracing (OBS-23, OBS-26). - * - * @public - */ -export interface SpanContext { - readonly traceId: string; - readonly spanId: string; - readonly traceFlags?: number | undefined; - readonly traceState?: string | undefined; -} - -/** - * A structural subset of `@opentelemetry/api`'s own `Span` shape (OBS-21, OBS-23). - * - * @public - */ -export interface Span { - readonly isRecording: boolean; - setAttribute(key: string, value: unknown): this; - recordException(error: unknown): this; - end(): void; - spanContext?(): SpanContext | undefined; -} - -/** - * Tracing facade interface for creating spans. - * - * @public - */ -export interface Tracer { - startSpan(name: string): Span; -} - -/** - * Inert no-op {@link Span} singleton (OBS-21, OBS-25). - * - * @public - */ -export const NOOP_SPAN: Span = Object.freeze({ - isRecording: false, - setAttribute(): Span { - return NOOP_SPAN; - }, - recordException(): Span { - return NOOP_SPAN; - }, - end(): void { - return; - }, - spanContext(): SpanContext { - return {traceId: '0'.repeat(32), spanId: '0'.repeat(16)}; - }, -}); - -/** - * Inert no-op {@link Tracer} singleton (OBS-25). - * - * @public - */ -export const NOOP_TRACER: Tracer = Object.freeze({ - startSpan(): Span { - return NOOP_SPAN; - }, -}); +// The span/tracer shapes and the two inert singletons live in `span.js` so that +// `context/instrumentation.ts` can reach `NOOP_SPAN` for CTX-15's disabled-tracing default without +// closing an import cycle with this module's `InstrumentationBundle` type edge. They are re-exported +// verbatim here, which is this module's published surface and every existing import path. +export type {Span, SpanContext, Tracer} from './span.js'; +export {NOOP_SPAN, NOOP_TRACER} from './span.js'; function randomHex(byteLength: number): string { const bytes = new Uint8Array(byteLength); From 2287649788b3969174ae32591b6adde28ff7eef1 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:15:04 +0300 Subject: [PATCH 009/100] test: pin REDIR-3's current-hop reading and PAGE-19's relative-reference reading (#69) Two places where the port reads a MUST differently from the spec's literal text, kept deliberately and now held by a test rather than by nobody. Both readings land as rows in docs/deviations.md in the next commit. REDIR-3 says eligibility is measured against the ORIGINAL request method; the port measures it against the CURRENT hop's (`decide.ts:241` into `codes.ts:69`). The readings diverge on exactly one chain: an opted-in 303 rewrites POST to GET, and a following 301 is then eligible under the default {GET, HEAD} set. `decide.test.ts` now drives that chain end to end. Mutation- checked: forcing `isEligibleByCode` back to the seed method turns the new case red with `return-current`. PAGE-19's conformance note offers `; rel=next` as an example of "stream ends, no exception". Under WHATWG URL -- the only RFC 3986 resolver available without a runtime dependency (SEAM-1) -- a supplied base makes that a valid relative path reference, so the port follows it to `/repo/not%20a%20url`. The requirement's normative sentence is about a target that CANNOT resolve, and that half was already pinned; this adds the half the fixture disagrees with. Mutation-checked the same way. Both files' `Exercises:` headers name the new readings and point at docs/deviations.md. --- .../core/src/pagination/strategies.test.ts | 31 +++++++++++- packages/core/src/redirect/decide.test.ts | 49 ++++++++++++++++++- 2 files changed, 78 insertions(+), 2 deletions(-) diff --git a/packages/core/src/pagination/strategies.test.ts b/packages/core/src/pagination/strategies.test.ts index 264602c..513f522 100644 --- a/packages/core/src/pagination/strategies.test.ts +++ b/packages/core/src/pagination/strategies.test.ts @@ -3,7 +3,9 @@ // Exercises: PAGE-16 (cursor: single body read, null OR empty ends, configurable parameter), PAGE-17 // (page-number: empty items ends, start-page fallback on absent/empty/garbage, configurable name and start), // PAGE-18/19/20 (link header: rel=next, RFC 3986 reference resolution, query-only reference preserves the path, -// unresolvable target ends the stream without throwing). +// unresolvable target ends the stream without throwing, and the spec's own `` conformance fixture +// resolving as a relative reference instead -- recorded as a deliberate reading in docs/deviations.md under +// "Deviations recorded outside a phase" (2026-09-04, audit #67 / #69)). import {expect, test} from 'bun:test'; import type {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; @@ -208,6 +210,33 @@ test('an unresolvable target ends the stream rather than throwing (PAGE-19)', as expect(info.items).toEqual(['a']); }); +test("the spec's `` fixture is a RELATIVE reference, so it is followed (PAGE-19)", async () => { + // PAGE-19's conformance note (`docs/product-spec/12-pagination.md:52`) gives `; rel=next` as an + // example of "stream ends, no exception". Under WHATWG URL — the resolver `strategies.ts` uses, and the + // only one available without a runtime dependency (SEAM-1) — a base makes that string a perfectly valid + // relative path reference: it resolves to `/repo/not%20a%20url`. The requirement's own normative sentence + // is "a target that CANNOT RESOLVE into a valid URL", and this one resolves, so the port follows it. Only + // the illustrative fixture disagrees; the test below keeps the end-of-stream half honest with a target that + // genuinely fails to resolve. Recorded in `docs/deviations.md`, "Deviations recorded outside a phase" — + // rejected alternative: an ad-hoc "looks unparseable" heuristic in front of the resolver. + const strategy = linkHeaderStrategy({ + extract: () => Promise.resolve(['a']), + }); + const info = await strategy.parse( + response({ + url: 'https://api.test/repo/issues?page=1', + headers: {link: ['; rel="next"']}, + }), + template('https://api.test/repo/issues?page=1'), + ); + expect( + () => new URL('not a url', 'https://api.test/repo/issues'), + ).not.toThrow(); + expect(info.nextRequest?.url.href).toBe( + 'https://api.test/repo/not%20a%20url', + ); +}); + test('the fixture above really is unparseable — the guard is not vacuous (PAGE-19)', () => { expect(() => new URL('http://[', 'https://api.test/items')).toThrow(); // And the near-miss that does NOT throw, pinned so nobody "simplifies" the fixture back to it later. diff --git a/packages/core/src/redirect/decide.test.ts b/packages/core/src/redirect/decide.test.ts index 14a84e9..1a90f60 100644 --- a/packages/core/src/redirect/decide.test.ts +++ b/packages/core/src/redirect/decide.test.ts @@ -15,7 +15,9 @@ // replayability gate; 303 exempt), REDIR-7 (Authorization always stripped), REDIR-9/REDIR-10 (Cookie and // Proxy-Authorization stripped only cross-origin), REDIR-11 (the marker set only on a cross-origin hop), // REDIR-5 (the 303 GET rebuild drops the body and every Content-* header), REDIR-3/REDIR-4 (a followed -// method-preserving redirect keeps the original method). +// method-preserving redirect keeps the original method), and REDIR-3's eligibility reference point -- the +// CURRENT hop's method rather than the spec's literal "original request method", recorded as a deliberate +// reading in docs/deviations.md under "Deviations recorded outside a phase" (2026-09-04, audit #67 / #69). import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import type {Body} from '../body/body.js'; @@ -753,6 +755,51 @@ describe('header construction -- the 303 rebuild and method preservation', () => }); }); +describe('REDIR-3 measures eligibility against the CURRENT hop, not the original request', () => { + // The spec says "the ORIGINAL request method" (`docs/product-spec/10-redirect-handling.md:8`); the port + // reads `currentRequest.method` (`decide.ts:241` into `codes.ts:69`). The two readings agree on every + // chain except one, and this is it: an opted-in 303 rewrites POST to GET (REDIR-5), and a 301 arriving on + // that rewritten hop is followed under the DEFAULT {GET, HEAD} set, where the literal reading would refuse + // it because the request that started the chain was a POST. Kept deliberately -- the rewritten GET is + // idempotent and carries no body, so the literal reading buys no safety, and refusing the hop would make + // `allow303` half-useful. Recorded in `docs/deviations.md`, "Deviations recorded outside a phase". + test('a 303-rewritten GET makes a following 301 eligible under the default method set (REDIR-3/REDIR-5)', () => { + const original = aRequest({ + method: 'POST', + url: 'https://example.com/a', + body: stringBody('x'), + }); + const first = decide( + aResponse(303, 'https://example.com/b'), + contextFor(original), + redirectSettings({allow303: true}), + ); + expect(first.kind).toBe('follow'); + if (first.kind !== 'follow') return; + expect(first.nextRequest.method).toBe('GET'); + + // Nothing about the second hop is opted into: the default set is {GET, HEAD} and excludes the method + // the chain started with, which is what makes the two readings disagree here rather than coincide. + const settings = redirectSettings(); + expect(settings.allowedMethods.has('POST')).toBe(false); + expect(original.method).toBe('POST'); + + const second = decide( + aResponse(301, 'https://example.com/c'), + contextFor(first.nextRequest, { + visited: new Set([original.url.href, first.nextRequest.url.href]), + redirectsFollowed: 1, + }), + settings, + ); + expect(second.kind).toBe('follow'); + if (second.kind === 'follow') { + expect(second.nextRequest.url.href).toBe('https://example.com/c'); + expect(second.nextRequest.method).toBe('GET'); + } + }); +}); + describe('loop detection survives URL normalization', () => { // `visited` keys on `URL.href`, which WHATWG normalizes -- so a server cannot spin the loop past the // cap by varying only the case of the scheme/host or by writing the scheme's default port out. Both From e341d3d12a9f4ebd20c40c19ad84a51be762f776 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:15:18 +0300 Subject: [PATCH 010/100] docs: record the deviations the 2026-09-04 audit found outside a phase (#69) `deviations.md`'s scope paragraph said its collection section was empty and that the 2026-08-31 sweep found no unrecorded deviation. Both were true of that sweep and neither is true now: that sweep read the registers, and the 2026-09-04 code audit (#67) read the shipped code. It found MUST-level narrowings and undecided readings that lived only in a phase design's ledger, only in a test comment, or nowhere. The paragraph is rewritten to say so, and deliberately states no count -- a count in a collection point is wrong on the next append. Nine rows appended, each with verified `file:line` evidence: - PIPE-37 -- the outermost pre-redirect status-mapping step was never built. `statusMappingStep` is a `ResponseStep`, not a staged pipeline `Step`; Phase 4's checklist handed the wiring to Phase 5 and Phase 5 shipped without it. Ledgered, not implemented: it is public pipeline surface, and the petstore spike's finding 2 wants the same work from the other side. (D4) - REDIR-3 -- eligibility reads the current hop's method, not the original's. (D5) - PAGE-19 -- the spec's own `` fixture resolves as a relative reference here and is followed; the normative clause is satisfied. (D6) - HTTP-46 -- `Request.equals` compares the body by identity: reading a single-use body to compare it would make equality destructive. - IO-13 -- write-side encodings are UTF-8 and ISO-8859-1 only. - BODY-9 -- `StreamBody` is always single-use; the SHOULD's condition is unmeetable on `ReadableStream`. - BODY-34 -- the shared preview cap covers the two logging tees, not `toHttpError`, whose cap HTTP-52 fixes. - IO-38 -- cross-thread close visibility has no subject on this platform; recorded as not applicable rather than as satisfied. - transport `reasonPhrase` -- fetch sets it, undici has no value to read. Sits beside item 13's `Protocol.HTTP_1_1` gap, which does not name it. (D7 for the last six.) Also re-anchors the OBS-29 row's `Tracer` citation, which the previous commit moved from `observability/tracing.ts` to `observability/span.ts`. Phase records under docs/work/ keep their pre-move paths, as that tree is never retro-edited. --- docs/deviations.md | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/docs/deviations.md b/docs/deviations.md index 7ffe3d6..1dad3ad 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -14,9 +14,16 @@ the claim. 2. **The collection point.** A deviation found outside a phase, by a review or a maintenance pass, with no phase ledger to write to and no permission to write to §10. It is appended under "Deviations recorded outside a phase" at the end of this file, dated, and folded into §10 the next time §10 - is deliberately amended. That section is empty today: the 2026-08-31 restructure swept the non-frozen tree - and found no unrecorded deviation, only three unrecorded *deferrals* (the dissolved register's U2) and six - mis-numbered register citations (the dissolved register's U6). + is deliberately amended. **That section is no longer empty, and the paragraph that once said it was is + this one.** The 2026-08-31 restructure swept the non-frozen tree and found no unrecorded deviation there, + only three unrecorded *deferrals* (the dissolved register's U2) and six mis-numbered register citations + (the dissolved register's U6) — a clean result, and one that held only because that sweep read the + *registers*. The 2026-09-02 register audit and the 2026-09-04 code audit + ([#67](https://github.com/dexpace/nodejs-sdk/issues/67)) both read something else and both filled the + section: the second went through the shipped **code** and found MUST-level narrowings and undecided + readings that lived only in a phase spec, only in a test comment, or nowhere at all. Read the table below + for what is in it — this paragraph deliberately does not count the rows, because a stated count is the + one thing in a collection point that goes wrong on the next append. **This file is the audit and the mutable collection point; §10 is the ledger and it is frozen.** (The cross-reference was added 2026-08-30, when nothing in the repo linked the two and the numbering they share had @@ -437,9 +444,18 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended |---|---|---|---|---| | **`HTTP-11`'s range classifications are on `Status` only, not mirrored onto `Response`.** The spec places them on both: `docs/product-spec/04-core-http-domain-model.md:23` reads "Status MUST classify by range … **and a response MUST expose these derived from its status**", and appendix C (`appendix-c-consolidated-normative-requirement-index.md:47`) restates it as "Response MUST expose these same classifications derived from its status." The port ships them once, on `Status`, reachable as `response.status.isSuccess`. Six delegating getters on `Response` would duplicate surface that cannot drift, since there would be one implementation behind both — but the letter of the requirement does name the response, so the reading is recorded rather than assumed | the dissolved register's A3, register audit | 2026-09-02 | `packages/core/src/http/status.ts` carries all six; `packages/core/src/http/response.ts` carries `status` and no classification of its own | not yet in §10 | | **`REDIR-20`'s "fully override" is read as scoped to code/method eligibility, not to the safety mechanics that follow it.** A configured redirect predicate replaces the built-in follow decision; it does **not** bypass userinfo stripping, credential hygiene, the downgrade guard, body replayability, or loop/cap detection. Those are stated as unconditional MUSTs elsewhere in the same chapter and are not "should this kind of redirect be followed" policy — a caller predicate opting to follow a 307 with a single-use body still cannot make that body re-sendable. Genuinely ambiguous wording, decided one way and now recorded as decided | the dissolved register's G4, register audit | 2026-09-02 | `packages/core/src/redirect/decide.ts` consults `settings.predicate` at the eligibility gate and runs every later guard unconditionally; pinned by "the predicate does NOT bypass the safety mechanics" in `decide.test.ts`. Phase 9's conformance sweep was to re-confirm this and closed without doing so | not yet in §10 | -| **Span-shaped tracing carries `OBS-29`'s ordered lifecycle, but not its 1:1 tracer-to-operation binding.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`. The **ordering half is satisfied** under different names: a span is started once (`observability/logging-step.ts:427`) and ended once on exactly one of two paths — `span.end()` for success (`:398`), or `span.recordException(error)` then `span.end()` for failure (`:414-415`) — mutually exclusive, once each, in order. **The 1:1 binding is NOT met**, and this row says so plainly rather than implying the requirement is covered: `PIPE-2` fixes the `LOGGING` pillar step *inside* the `RETRY` and `REDIRECT` pipelines, so a span is opened per transmission attempt and per redirect hop, not once per logical operation (the dissolved register's L4 records the same). The per-attempt and retries-exhausted events therefore have no operation-scoped tracer to attach to. Appendix C's own entry (`appendix-c-consolidated-normative-requirement-index.md:509`) anticipates the gap: "This is a documented emission contract; pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced." The wiring stays open as the dissolved register's V2 | the dissolved register's L1/V2, register audit | 2026-09-02 | `packages/core/src/observability/tracing.ts:40-42` (`Tracer`, one method); `observability/logging-step.ts:398,414-415,427` (the lifecycle); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | +| **Span-shaped tracing carries `OBS-29`'s ordered lifecycle, but not its 1:1 tracer-to-operation binding.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`. The **ordering half is satisfied** under different names: a span is started once (`observability/logging-step.ts:427`) and ended once on exactly one of two paths — `span.end()` for success (`:398`), or `span.recordException(error)` then `span.end()` for failure (`:414-415`) — mutually exclusive, once each, in order. **The 1:1 binding is NOT met**, and this row says so plainly rather than implying the requirement is covered: `PIPE-2` fixes the `LOGGING` pillar step *inside* the `RETRY` and `REDIRECT` pipelines, so a span is opened per transmission attempt and per redirect hop, not once per logical operation (the dissolved register's L4 records the same). The per-attempt and retries-exhausted events therefore have no operation-scoped tracer to attach to. Appendix C's own entry (`appendix-c-consolidated-normative-requirement-index.md:509`) anticipates the gap: "This is a documented emission contract; pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced." The wiring stays open as the dissolved register's V2 | the dissolved register's L1/V2, register audit | 2026-09-02 | `packages/core/src/observability/span.ts:54-56` (`Tracer`, one method — declared in `tracing.ts` until 2026-09-04, when audit #67 / #69 moved the inert tracing declarations into their own module to break an import cycle; `tracing.ts` re-exports them, so the public path is unchanged); `observability/logging-step.ts:398,414-415,427` (the lifecycle); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | | **`invariant()` density is not a target, project-wide.** `docs/knowledge/harvested/assertions.md:6-7` sets a 2-per-function module average. The port's position: a module gains an `invariant()` when it has an internal precondition worth asserting, and `recovery/`, `http/`, `seams/` and `generated/` have none — measured 2026-09-02, all four at zero. Adding assertions to reach an average would assert nothing. `recovery/` is the sharp case: an `invariant()` inside `ResponseRecoveryChain.apply()` throws, and `RECOV-8` forbids `apply()` from throwing, so a density rule would push that module toward a shape the specification forbids | the dissolved register's F3/H6 and the deferral register's *Assertion-density rule applied project-wide* row (retired to [the purge note](./work/mvp/2026-09-04-register-retirement-purge.md)), register audit | 2026-09-02 | `packages/core/src/recovery/`, `http/`, `seams/`: zero `invariant(` calls. `pipeline/` ships fifteen and `context/` three, so the rule is applied where it earns its place | not yet in §10 | | **`PIPE-40` and `REDIR-22` contradict each other on the non-replayable-body path, and the port implements `REDIR-22`.** Two MUSTs name the same trigger and prescribe opposite dispositions. `docs/product-spec/08-execution-pipelines.md:20` (`PIPE-40`): "on paths that abandon a re-drive (redirect cycle, **non-replayable body**, budget exhausted) the in-flight response MUST be returned unclosed." `docs/product-spec/10-redirect-handling.md:22` (`REDIR-22`): "if building the follow-up throws (**non-replayable body**, downgrade rejection) the current response MUST be closed before the error propagates." The port closes, then throws, on three grounds: `REDIR-6` independently fixes the control flow ("the operation MUST fail with a clear error naming replayability"), so the path throws and a response never *returned* cannot be "returned unclosed"; specific governs general, since §10 of the spec owns the redirect step's lifecycle; and closing is the safer reading, because the alternative leaks a body on an error path with no caller holding a reference to close it. `PIPE-40`'s other two named paths do genuinely return, and both return unclosed as it requires. **The erratum this needs is proposed below and is not applied here**, because `docs/product-spec/` is frozen and correcting a normative sentence is the specification owner's act, not a maintenance one | the dissolved register's G1, Phase 5b design's Deviation Ledger | 2026-09-04 | `packages/core/src/redirect/redirect-step.ts` closes before throwing, with the reasoning asserted inline in `redirect-step.test.ts`; nothing in the code waits on the erratum | not yet in §10 | +| **`PIPE-37`'s outermost pre-redirect status-mapping step was never built, and until this row nothing recorded that.** `PIPE-37` (MUST) requires a step whose correctness depends on the single terminal response — status-to-typed-error mapping is its own worked example — to occupy the outermost pre-redirect slot, so it runs outside both the redirect and the retry loop. The port ships the *mapping*, but as `statusMappingStep`, a `ResponseStep` on the response-recovery chain (RECOV-15/RECOV-16), not as a pipeline `Step` carrying `stage: 'PRE_REDIRECT'`. The slot itself exists and is installable — `config/clientIdentityStep` occupies it today — so this is a wiring gap, not a missing mechanism. It has an owner in writing and the owner never took it: Phase 4's checklist marked the row ⏳ and said "the obligation lands on whichever phase wires 4b's `statusMappingStep` into a real pipeline — **Phase 5**", and Phase 5 shipped without it, with no deferral carrying the hand-off forward. That is why a code audit found it and no checklist did. **Ledgered rather than implemented.** Installing a `PRE_REDIRECT` mapping step is public pipeline surface with a real behaviour change behind it, and the petstore spike arrives at the same work from the other side: its finding 2 wants a declarative `StatusErrorMap` applied *at* the `toHttpError` call site rather than wrapped around it, so a mapped error class can see a decoded payload instead of raw bytes. Whoever does one does both | audit #67 / #69 | 2026-09-04 | `packages/core/src/recovery/status-mapping.ts:26` (`statusMappingStep(response: Response): Promise` — a `ResponseStep`, with no `StepDescriptor` and no stage); `packages/core/src/pipeline/stage.ts:39` (`PRE_REDIRECT` heads `STAGE_ORDER`) and `packages/core/src/config/client-identity-step.ts:124` (the one shipped step that occupies it); `docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md:141` (the dropped hand-off); `docs/product-spec/08-execution-pipelines.md:26` (PIPE-37 shares that line with PIPE-25/36/38); `examples/petstore/FINDINGS.md:96-97` | not yet in §10 | +| **`REDIR-3`'s eligibility test reads the CURRENT hop's method, where the spec says the ORIGINAL request's.** `isEligibleByCode` is handed `currentRequest.method`, the method of the request being redirected at this hop. The two readings agree on every chain but one: an opted-in 303 rewrites POST to GET (REDIR-5), and a 301 or 302 arriving on that rewritten hop is then followed under the default {GET, HEAD} set, where the literal reading would refuse it because the request that started the chain was a POST. **Kept, deliberately.** The rewritten GET is idempotent and carries no body — REDIR-5 dropped it — so the literal reading buys no wire safety here, only a refusal; and refusing would make `allow303` half-useful, opting into the rewrite but not into anything the rewritten request can reach. Every other guard on the hop is unaffected, since eligibility is step 3 of eight and the userinfo strip, downgrade guard, replayability gate, loop detection and hop cap all run after it regardless. Switching to the literal reading stays a narrow, mechanical change — thread the seed request's method through `RedirectContext` — if a later reading of §10 wants it | audit #67 / #69 | 2026-09-04 | `packages/core/src/redirect/decide.ts:241` passes `currentRequest.method` into `packages/core/src/redirect/codes.ts:69`'s `eligibility.allowedMethods.has(method)`; `docs/product-spec/10-redirect-handling.md:8` is the "ORIGINAL request method" wording. Pinned by "a 303-rewritten GET makes a following 301 eligible under the default method set" in `packages/core/src/redirect/decide.test.ts`, which goes red the moment the reference point moves | not yet in §10 | +| **`PAGE-19`'s own conformance fixture `; rel=next` does not end the stream on this platform — it is followed.** The requirement's normative sentence is "a target that cannot resolve into a valid URL MUST be treated as end-of-stream", and its illustrative conformance note offers `` as an instance. Under WHATWG `URL` — the resolver `strategies.ts` uses, and the only RFC 3986 resolver available without a runtime dependency (SEAM-1) — a supplied base makes that string a perfectly ordinary relative *path* reference: it resolves to `/repo/not%20a%20url`. It resolves, so the port follows it. **The normative half is satisfied exactly as written**: a target that genuinely fails `new URL(target, base)` returns the page with no next request and throws nothing. Only the fixture disagrees, and it disagrees because it was written against a resolver that rejects a space. **Rejected:** an ad-hoc "looks unparseable" heuristic in front of the resolver, which would have to guess at strings RFC 3986 defines, and would make the followable-relative-reference clause of the same requirement wrong instead | audit #67 / #69 | 2026-09-04 | `packages/core/src/pagination/strategies.ts:115-120` (the `new URL(target, response.request.url)` resolve, with the `catch` returning `pageInfo(items)`); `docs/product-spec/12-pagination.md:52` is the conformance note. Both halves pinned in `packages/core/src/pagination/strategies.test.ts` — "the spec's `` fixture is a RELATIVE reference, so it is followed" and "an unresolvable target ends the stream rather than throwing" | not yet in §10 | +| **`Request.equals` compares the body by reference identity, where `HTTP-46` says by value.** `HTTP-46` requires equality to compare "method, headers, and body by value", with only the URL singled out for textual comparison. The port compares method, `url.href` and headers by value as required, and the body with `===`. **Why it stays.** `Body` is a lifecycle object, not a value: a `StreamBody` is single-use (BODY-9) and reading its bytes to compare them consumes it, which would make `equals` destructive — an equality operator that empties its operands is worse than one that under-reports. The variants that *could* compare cheaply (`ByteArrayBody`, `StringBody`) would give a comparison whose cost and semantics depend on which variant a caller happened to build, which is the drift `HTTP-1`'s value-model rules exist to prevent. Reference identity is sound in the safe direction — it never reports two different bodies equal — and it is what `Request.equals`'s own TSDoc has always said it does. Archived as blocked in the dissolution record and in neither §10 nor the register; recorded here so it is somewhere a reader will look | audit #67 / #69 | 2026-09-04 | `packages/core/src/http/request.ts:137` (`this.#body === other.#body`, beside the by-value method, `url.href` and `headers.equals` comparisons on 134-136); `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:82` | not yet in §10 | +| **`IO-13`'s "symmetric write-side encodings" ship for UTF-8 and ISO-8859-1 only; every other charset throws on write.** The read side stays fully general through `TextDecoder`. The write side does not, because `TextEncoder` is UTF-8-only — there is no `TextEncoder('iso-8859-1')` — and SEAM-1 forbids adding an encoding library to `@dexpace/core`. ISO-8859-1 is therefore hand-rolled as the direct code-point-to-byte map, which is also what lets the decode side round-trip it: WHATWG maps the label `iso-8859-1` onto windows-1252, so delegating would break the symmetry `IO-13` is about. Any other label raises `IoError` naming the charset rather than silently re-encoding as UTF-8 and corrupting the bytes on the wire. `IO-13`'s own conformance note names ISO-8859-1 as the non-UTF-8 case, so the two shipped encodings are the two the requirement exercises. **Recorded in the Phase 3a design's ledger since 2026-07-24 and nowhere else** — it never reached §10 or this file, which is the gap this row closes | audit #67 / #69 | 2026-09-04 | `packages/core/src/io/text-codec.ts:32-51` (`encodeText`, and the `unsupported write charset` throw at :36-39); `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md:406` | not yet in §10 | +| **`BODY-9`'s mark/reset replay path for a stream-backed body is not built: `StreamBody` is always single-use.** `BODY-9` is a SHOULD, and it is conditional — replayable "when and only when the stream supports mark/reset". Node's `ReadableStream` has no generic mark/reset to support, so the condition is never met and the SHOULD's own fallback ("otherwise it MUST be single-use") is the branch that applies. `StreamBody.replayable` is a hardcoded `false`, which every consumer already reads: `decide.ts` fails a non-303 redirect carrying one (REDIR-6), and the retry pillar refuses to re-send it. A caller who wants replay materializes first or uses `byteArrayBody`. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/stream-body.ts:24` (`readonly replayable = false`); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:441` | not yet in §10 | +| **`BODY-34`'s one shared preview cap covers the two logging tees only, not `toHttpError`'s error-body capture.** Read literally as "every in-memory capture in the package", `BODY-34` would put the request-side tee, the response-side drain and `toHttpError`'s error buffer behind one configurable value. They cannot share one: `HTTP-52` **fixes** the error-body cap at 1 MiB, and a spec-fixed constant cannot also be the configurable shared setting. The two capture sites `BODY-34` actually names — the request-side tee-capture and the response-side drain-on-first-access — do share one cap, so the requirement's own enumeration is satisfied; only the wider reading is not. `http-status-error.ts` states the split at the constant rather than leaving it to be inferred. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/http-status-error.ts:17-19` (`ERROR_BODY_CAP_BYTES`, with the "Deliberately NOT BODY-34's shared preview cap" note); `packages/core/src/body/response-body-logging.ts:47` and `packages/core/src/body/response-body-logging.ts:105-115` (the shared `cap` and the bounded drain that honours it); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:447` | not yet in §10 | +| **`IO-38`'s cross-thread close visibility has no subject on this platform and is recorded as not applicable, not as satisfied.** The requirement presupposes that a source or buffer instance can reach a second thread, so that closing it there invalidates a slice being read here. None can. Class instances are not structured-cloneable at all — `postMessage`/`structuredClone` preserve neither prototypes nor `#private` fields, so a `ByteQueue` or `BufferedSource` sent to a worker arrives as a plain object with no methods and no close state to observe. `BufferedSource` is doubly excluded: it holds a `ReadableStreamDefaultReader`, which is neither cloneable nor transferable. A raw `ArrayBuffer` *can* be transferred, but it carries no close state and derives no slices, so the hazard has no subject there either. This row exists because "not applicable" and "done" look identical in a checklist and are not the same claim. **Recorded in the Phase 3a design's ledger since 2026-07-24 and nowhere else** | audit #67 / #69 | 2026-09-04 | `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md:401`; `packages/core/src/io/buffered-source.ts:47` (the reader it holds); no `worker_threads`, `postMessage` or `structuredClone` call exists anywhere in `packages/core/src/` | not yet in §10 | +| **The two shipped transports disagree on `HTTP-6`'s optional reason phrase: `transport-fetch` sets it, `transport-undici` leaves it `undefined`.** `HTTP-6` (MUST) has a response carry "an optional reason phrase", and the domain model provides the slot. `fetch-transport` fills it from the WHATWG `Response.statusText`, normalizing the empty string to `undefined`. `undici-transport` does not call `.reasonPhrase()` at all, because undici's `ResponseData` carries `statusCode` and no phrase — HTTP/2 has no reason phrase and undici's parser does not surface HTTP/1.1's. **Platform-shaped, and it sits beside §10 item 13's `Protocol.HTTP_1_1` gap rather than under it.** Item 13 already ledgers the negotiated-version gap in both adapters for the same reason — the value is not readable — but it does not name the reason phrase, so this row does. The field is optional in the requirement, and no shipped step or conformance row reads it, so the disagreement is observable to a caller and to nothing else. Recorded rather than papered over: synthesizing a phrase from the status code in the undici adapter would report a value the server never sent | audit #67 / #69 | 2026-09-04 | `packages/transport-fetch/src/fetch-transport.ts:180` (`.reasonPhrase(...)` fed from the WHATWG `statusText`, with the empty string normalized to `undefined`) against `packages/transport-undici/src/undici-transport.ts:342-350` (the builder chain, with no `.reasonPhrase` call); `docs/product-spec/04-core-http-domain-model.md:13`; the neighbouring gap is item 13 of the audit above | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) From fe3e4a095d84e7fa30e74fceb63a6229ba805767 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:17:44 +0300 Subject: [PATCH 011/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20record=20t?= =?UTF-8?q?he=20#69=20outcome,=20the=20span.ts=20split=20and=20the=20strip?= =?UTF-8?q?Internal=20trap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index 652c016..675097b 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -42,6 +42,20 @@ No changesets, no version bumps, no `docs/first-release.md` edits. Each PR lists `noopInstrumentationBundle.activeSpan` becomes `NOOP_SPAN`. One line plus a test. #80 lists the same item; it is done here, #80 skips it. *Rejected:* ledger row (the fix is smaller than the row). +**D3 outcome (2026-09-04, #69, PR #84).** The size estimate was wrong: importing `NOOP_SPAN` into +`context/instrumentation.ts` closed an import cycle with `observability/tracing.ts` (which imports +`InstrumentationBundle`), and `verify:import-cycles` counts type-only edges. Fixed as that gate prescribes: +`SpanContext`, `Span`, `Tracer`, `NOOP_SPAN`, `NOOP_TRACER` moved to a new leaf module +`packages/core/src/observability/span.ts`; `tracing.ts` re-exports them, so no import path and no API +report changed. The decision itself stands. *Constrains #80:* a `file:line` citation into `tracing.ts` for +those five names is now stale; CTX-15 is done, skip it. + +**Trap for every later subtask.** gts turns on `stripInternal`, and TypeScript tests it by substring-scanning +every leading comment of a declaration, line comments included. A module header that merely *mentions* the +`@internal` tag deletes the first exported declaration from the emitted `.d.ts` with no `tsc` diagnostic; +the failure surfaces one package later as an unresolved name in core's own `dist/`. Do not write that tag in +a file-level comment. + ### D4 — PIPE-37: ledger the gap, do not implement in M1 No `PRE_REDIRECT` status-mapping pipeline step exists; `statusMappingStep` is a `ResponseStep`. Recorded as a row in `deviations.md` naming the gap, the Phase 4→5 hand-off that dropped it, and the petstore @@ -68,4 +82,6 @@ All six are recorded as rows (evidence + reading). `reasonPhrase` sits beside § already done and does not re-ledger it. ## Deferred — release machinery (recoverable list) -_(appended per PR)_ +| Issue / PR | Deferred item | +|---|---| +| #69 / PR #84 | patch changeset for `@dexpace/core`: `noopInstrumentationBundle.activeSpan` changed from `undefined` to `NOOP_SPAN` (documented default of a `@public` interface) | From 73c64d028a84bd0e11c8a6cedff06c38c5384fd8 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:19:01 +0300 Subject: [PATCH 012/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20record=20t?= =?UTF-8?q?he=20#68=20outcome=20and=20its=20round-2=20scope?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index 675097b..ff84d14 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -56,6 +56,15 @@ every leading comment of a declaration, line comments included. A module header the failure surfaces one package later as an unresolved name in core's own `dist/`. Do not write that tag in a file-level comment. +**#68 outcome (2026-09-04, PR #83).** Round 1 corrected the TSDoc and guides; round 2 requested because +five `docs/deviations.md` anchors it found stale (items 2, 3, 4, 8, 11) were left unfixed as "outside the +partition" — they are the issue's own acceptance criterion. The false "nothing consumes either yet" claim at +`context/instrumentation.ts:8-13` was assigned to #68 in round 2. *Constrains #78:* item 17 now states the +cause-walk matches `IoError` and `TransportFailureError` only, and names #78 as the decider. *Constrains +#80:* the OBS-29 row is marked in progress with the 1:1 binding recorded as met at `pipeline/runtime.ts`; +the open part is caller-reachability of the operation span. *Constrains #71:* `auth.md`'s credential-shape +example is untouched and is #71's. + ### D4 — PIPE-37: ledger the gap, do not implement in M1 No `PRE_REDIRECT` status-mapping pipeline step exists; `statusMappingStep` is a `ResponseStep`. Recorded as a row in `deviations.md` naming the gap, the Phase 4→5 hand-off that dropped it, and the petstore @@ -84,4 +93,6 @@ already done and does not re-ledger it. ## Deferred — release machinery (recoverable list) | Issue / PR | Deferred item | |---|---| +| #68 / PR #83 | patch notes for `@dexpace/core` and `@dexpace/codec-json`: shipped `.d.ts` prose changed for `Deserializer`, `jsonSerde()`, `InstrumentationBundle.tracerFactory`, `buildRequest`, `RequestConditions.applyTo` | +| #68 / PR #83 | `docs/first-release.md:117` claims `serde.ts:99,170` cite `H15` — `H15` appears nowhere in `serde.ts`; `:159` puts `deserializeFrom` at `:162` and `serializeTo` at `:96` (actual `:221`, `:104`). File suspended under D1 | | #69 / PR #84 | patch changeset for `@dexpace/core`: `noopInstrumentationBundle.activeSpan` changed from `undefined` to `NOOP_SPAN` (documented default of a `@public` interface) | From 70c1a3e80b0fa72f1379f4d54c96e3ebd7be15fd Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:23:57 +0300 Subject: [PATCH 013/100] docs: re-anchor the last five stale citations in deviations.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes the file against this task's own criterion — every `file:line` points at the text it cites. Each replacement was pinned with `sed -n` before it was written, and the whole file re-swept afterwards. | Item | Was | Now | |---|---|---| | 1 | `context.ts:104,125,149` — `:104` unrelated prose, `:125` a signature close | `:112` is the `Symbol()` mint; `:128`/`:149` are the two per-flavor key defaults | | 3 | `engine.ts:354`, inside `runWithRetry`'s TSDoc | `:358`, the declaration | | 4 | `configuration.ts:72`, which is `.emit();` | `:100`, `export interface Configuration` | | 8 | `"sideEffects": false` at `:20`/`:21`/`:21` — all three a bare `}` | `packages/core/package.json:25`, both transports at `:26` | | 12 | `auth-step.ts:387-390`, the HTTPS-guard paragraph | the marker read at `:395` and the two branches it gates, `:401` and `:786`, plus `OutboundPlan.crossOrigin` at `:375` for why the answer survives the dispatch | Item 4's "**61 interfaces** against 58 classes" is deleted rather than corrected, per CLAUDE.md — the real figures had drifted to 65 and 71, and the sentence never needed a number to make its point. Part of audit #67 / #68, round 2. --- docs/deviations.md | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/docs/deviations.md b/docs/deviations.md index b4a293b..52e4a8b 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -96,8 +96,9 @@ Items **7**, **11**, and the `NFR-12` half of **14** are absent from this list o **Verified.** `Transport.send()` is `Promise`-only, one method satisfying `SEAM-11` and `SEAM-16` at once (`packages/core/src/seams/transport.ts:49`). `ContextStore` holds a plain `Map` (`packages/core/src/context/store.ts:24`) with a fresh `Symbol()` per call -(`packages/core/src/context/context.ts:104,125,149`). `Next` is `(request?) => Promise` with no sync -twin (`packages/core/src/pipeline/step.ts:23`). `wrapCancellation` degenerates to `failure(error)` and says so +(`packages/core/src/context/context.ts:112`, defaulted per flavor at `:128` and `:149`). `Next` is +`(request?) => Promise` with no sync twin (`packages/core/src/pipeline/step.ts:23`). +`wrapCancellation` degenerates to `failure(error)` and says so (`packages/core/src/recovery/cancellation.ts:31`). `ASYNC-18` holds: SSE parses `retryMs` (`packages/core/src/sse/parser.ts:131`) but never acts on it — reconnection is caller-owned, so no adapter schedules a delay outside the retry engine. @@ -133,7 +134,7 @@ non-bridge clause survives and is enforced as an ordinary obligation on `send()` ## 3. Two retry stacks collapse into one, with the total-timeout budget explicitly opt-in -**Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:354`) — with exactly two thin +**Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:358`) — with exactly two thin callers: the pillar step (`packages/core/src/retry/retry-step.ts:142`) and the dispatch adapter (`packages/core/src/retry/retry-dispatch.ts:53`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | undefined` and undefined by default (`packages/core/src/retry/settings.ts:27`), pinned by a test named for @@ -160,9 +161,9 @@ language-level ceiling. > **Corrected in the ledger 2026-08-29 — the text was wrong, the deviation is not.** Item 4's stated mitigation — "exporting only > concrete classes, never bare structural interfaces, from each package's public entry point" — is false as -> written. `packages/core/etc/core.api.md` exports **61 interfaces** against 58 classes, and at least one is -> a builder-built, validated, frozen type: `Configuration` is `export interface Configuration` -> (`packages/core/src/config/configuration.ts:72`) returned from `ConfigurationBuilder.build()`, and +> written. `packages/core/etc/core.api.md` exports interfaces and classes in comparable numbers, and at least +> one interface is a builder-built, validated, frozen type: `Configuration` is `export interface Configuration` +> (`packages/core/src/config/configuration.ts:100`) returned from `ConfigurationBuilder.build()`, and > `setGlobalConfiguration()` / `resolveProxyOptions()` accept any hand-rolled object of that shape. Most > other exported interfaces are seams (`Transport`, `Serde`, `Logger`) or options records, where structural > typing is the point. The *deviation* is real and uncorrectable; the *mitigation sentence* overstated what @@ -258,8 +259,8 @@ configuration file would be empty by construction. The structurally equivalent J `Object.defineProperty` statement** — a top-level side effect — in four files across three packages that all declare `"sideEffects": false` (`packages/core/src/sse/stream.ts:209`, `packages/core/src/pagination/page.ts:114`, `packages/transport-fetch/src/fetch-transport.ts:314`, -`packages/transport-undici/src/undici-transport.ts:566`; the manifest field at `packages/core/package.json:20`, -`packages/transport-fetch/package.json:21`, `packages/transport-undici/package.json:21`). That field entitles a +`packages/transport-undici/src/undici-transport.ts:566`; the manifest field at `packages/core/package.json:25`, +`packages/transport-fetch/package.json:26`, `packages/transport-undici/package.json:26`). That field entitles a bundler to drop a module whose exports go unused, and nothing forbids a future one from also dropping a top-level statement it judges inert — which would silently un-install disposal in the shipped artifact while every type still checks, the same failure shape this guard already exists for. `fixture-app.ts`'s @@ -279,10 +280,13 @@ bundle from 16,671 to 17,689 bytes against a 24 KiB budget (`packages/shrink-tes ## 12. The redirect/auth cross-origin marker is a real header, not a `WeakSet` **Verified.** `CROSS_ORIGIN_MARKER_HEADER` is set, cleared, and tested per hop -(`packages/core/src/redirect/cross-origin.ts:80,93,104,118`). The auth step reads it before deciding whether -to react to a challenge, not just whether to stamp (`packages/core/src/auth/auth-step.ts:387-390`). An -independent `POST_AUTH` backstop strips it even in a pipeline with no auth step -(`packages/core/src/redirect/strip-marker-step.ts`), satisfying `REDIR-11(c)`'s porter caveat. +(`packages/core/src/redirect/cross-origin.ts:80,93,104,118`). The auth step reads it once on the outbound +pass (`packages/core/src/auth/auth-step.ts:395`) and gates both branches on the answer — preemptive stamping +at `:401`, and whether to react to a challenge at all at `:786`, not merely whether to stamp. The answer +rides across the dispatch on `OutboundPlan.crossOrigin` (`:375`, and `:370-372` for why), because the marker +itself is cleared from the request before it reaches the wire. An independent `POST_AUTH` backstop strips it +even in a pipeline with no auth step (`packages/core/src/redirect/strip-marker-step.ts`), satisfying +`REDIR-11(c)`'s porter caveat. **Why it cannot be corrected.** The reference's in-process marker was tried and withdrawn during Phase 5b's own drafting: retry's attempt-stamping sits between redirect and auth and produces a **fresh `Request` From bd396b6526f7d06b348e7832b0bb3084ea43a3ec Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:24:06 +0300 Subject: [PATCH 014/100] docs(core): InstrumentationBundle no longer claims nothing consumes its two unknowns MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `packages/core/src/context/instrumentation.ts:8-13` justified `activeSpan` and `tracerFactory` being typed `unknown` with "nothing in the package consumes either yet, pending Phase 7a". Phase 7a landed in `bd37a08` and shipped `Span` and `Tracer` in `observability/tracing.ts`; it did not narrow these two. Measured rather than assumed, and the two members differ: - `tracerFactory` IS consumed — `pipeline/runtime.ts:52` and `observability/logging-step.ts:263`, both reaching a `Tracer` through a cast. - `activeSpan` is filled by `createInstrumentationBundle` (`observability/tracing.ts:222`) and read back by nothing in the package, so half the original sentence was true and half was not. The note now states the real reason they stay `unknown`: narrowing a published member widens what a caller may pass and narrows what they receive, which is a version-bump decision. `tracerFactory`'s own PROVISIONAL line stops citing a phase that has shipped and defers to that note. The `activeSpan` member TSDoc and `noopInstrumentationBundle` are deliberately untouched — #69 owns both. Part of audit #67 / #68, round 2 (CTX-14, CTX-15, OBS-29). --- packages/core/src/context/instrumentation.ts | 22 +++++++++++++------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/packages/core/src/context/instrumentation.ts b/packages/core/src/context/instrumentation.ts index dd1838b..8fc1365 100644 --- a/packages/core/src/context/instrumentation.ts +++ b/packages/core/src/context/instrumentation.ts @@ -5,12 +5,18 @@ * Correlation/instrumentation bundle every execution context carries (CTX-14), reachable from a * custom step as `ctx.context.instrumentation`. * - * **Two members are provisional.** `activeSpan` and `tracerFactory` are typed `unknown` rather than a - * Span/Tracer interface because nothing in the package consumes either yet, and the real tracing - * adapter — Phase 7a — owns their eventual shape. They are published as `unknown` deliberately, not - * accidentally: narrowing them later to a concrete Span/Tracer type is a widening of what a caller - * may pass and a narrowing of what they receive, so code that reads either today should treat it as - * opaque and re-check when 7a lands. Every other member below is stable. + * **Two members are still typed `unknown`, and the reason this note used to give is no longer true.** + * `activeSpan` and `tracerFactory` are `unknown` rather than a `Span`/`Tracer`. The original reason — + * "nothing in the package consumes either yet, pending Phase 7a" — expired when Phase 7a landed + * (`bd37a08`) and shipped `Span` and `Tracer` in `observability/tracing.ts`. That phase did not narrow + * these two: `tracerFactory` is consumed, by `pipeline/runtime.ts:52` and + * `observability/logging-step.ts:263`, both of which reach a `Tracer` through a cast; + * `createInstrumentationBundle` fills `activeSpan` (`observability/tracing.ts:222`), and nothing in + * this package reads it back. + * + * They stay `unknown` because narrowing a published member is a breaking change rather than a + * maintenance edit — it widens what a caller may pass and narrows what they receive — so it belongs + * to a deliberate version bump. Treat either as opaque until then. Every other member below is stable. * * @public */ @@ -43,8 +49,8 @@ export interface InstrumentationBundle { * (`observability/tracing.ts:212,223`). A no-op returning `undefined` when tracing is disabled; * both consumers substitute `NOOP_TRACER` for that `undefined`. * - * PROVISIONAL: the return type is `unknown` pending Phase 7a's tracing adapter — see this - * interface's own note. + * PROVISIONAL: the return type is `unknown`, and narrowing it is a version-bump decision — see this + * interface's own note for why Phase 7a landing did not settle it. * * @param operationName - the operation whose span the returned tracer will be asked to start. * @returns the tracer for that operation, or `undefined` when tracing is disabled. From e99f9d3d814d94dc9552215ef5dd2b205f2883b2 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:44:32 +0300 Subject: [PATCH 015/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20wave=201?= =?UTF-8?q?=20landed;=20D8/D9=20for=20wave=202?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index ff84d14..4300f50 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -90,6 +90,34 @@ Pin with a test and ledger the reading. *Rejected:* adding an ad-hoc "looks unpa All six are recorded as rows (evidence + reading). `reasonPhrase` sits beside §10 item 13; #82 reads it as already done and does not re-ledger it. +## Wave 1 — landed 2026-09-04 +PR #83 (#68) and PR #84 (#69) merged into the umbrella at `840f355`. One conflict (the OBS-29 row of +`deviations.md`): kept #68's "in progress, see #80" text and carried #69's moved `span.ts` citation. Merged +tree preflighted before the merge in a throwaway worktree (byte-identical result): all 20 steps passed. + +## Decisions taken for wave 2 (M2) + +### D8 — #70: redact inside the error messages, and keep the raw URLs on the error properties +`SchemeDowngradeError` and `NonReplayableBodyError` build their messages from `redactUrl(url)`; `fromUrl` / +`toUrl` stay raw for program use. Reason: the message is what every logger, `cause` chain and consumer +`console.error` renders, so redacting at the source protects paths this SDK does not own, not only +`http.redirect.rejected`. If `emitRejected` can also carry the redacted URL fields the other redirect events +carry, add them — but the message fix is the required one. *Rejected as sole fix:* logging `error.name` plus +fields and dropping the message (leaves the raw message reachable through `cause` on the thrown error). +*Constrains:* none. + +### D9 — #71: credential classes, and "once guarded, always guarded" for the replay +- `BasicCredential` and `DigestCredential` become classes with `#password`, `toString` and the + `nodejs.util.inspect.custom` override, following whatever shape `auth/credential.ts` already uses for + `ApiKeyCredential` / `BearerToken` (class plus `createX()` factory if that is the pattern there). Public + shape change, free before the first version bump; `api:local` on core. +- Replay guard: if the original request required HTTPS (the step guarded it), `requireHttps` runs on the + replacement request unconditionally, regardless of which header names it carries. *Rejected:* building the + set of credential-carrying header names from configuration (misses a `challengeHook` that invents a + header). AUTH-8 names bearer, API-key and name-key only; the wider reading is a `deviations.md` row (D0). +- `docs/sdk-documentation/auth.md`'s credential-shape example is rewritten here (#68 left it). +*Constrains #74:* it edits `auth-step.ts` next wave on top of this; the guard site moves. + ## Deferred — release machinery (recoverable list) | Issue / PR | Deferred item | |---|---| From f03a847f806d0833fd604aaa387a1f0499564d52 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:56:58 +0300 Subject: [PATCH 016/100] fix: build the redirect error messages from redactUrl (#70) `SchemeDowngradeError` and `NonReplayableBodyError` interpolated their raw `from`/`to` URLs into their messages. Every other redirect event sends its URLs through `redactUrl()`, but `http.redirect.rejected` hands the decision error to `LogEvent.cause()`, which renders it as `name: message` (`observability/logger.ts:74`) -- so userinfo and query-string tokens reached the log record in clear text, past a field cap of 8192 that truncated nothing. Redacted at construction rather than at the log site (ledger D8): the message is what every logger, `cause` chain and consumer `console.error` renders, and this SDK owns none of those. `redactUrl` is total under OBS-15, so a constructor can call it unguarded; an input that never parsed yields `[malformed url]`, which is the safe reading of "unparseable" rather than "carries no secret". `targetUrl` / `fromUrl` / `toUrl` stay RAW -- program code reads the property, log and human output read the message. That split is now stated in each class's TSDoc. OBS-11, OBS-12, OBS-15, REDIR-28, XCUT-19(a)/(b). --- packages/core/src/redirect/errors.test.ts | 67 +++++++++++++++++++++++ packages/core/src/redirect/errors.ts | 41 ++++++++++++-- 2 files changed, 103 insertions(+), 5 deletions(-) diff --git a/packages/core/src/redirect/errors.test.ts b/packages/core/src/redirect/errors.test.ts index b9442b6..67f6ec1 100644 --- a/packages/core/src/redirect/errors.test.ts +++ b/packages/core/src/redirect/errors.test.ts @@ -4,6 +4,11 @@ // corrupting or truncating the re-send), REDIR-15 (an HTTPS->HTTP hop is rejected with a clear error by // default). Both are operational failures a caller can legitimately hit mid-redirect, so both are typed // error leaves rather than `invariant()` programmer-error assertions. +// Also: OBS-11 (userinfo is always redacted to `***:***@`), OBS-12 (query values are `***` unless +// allow-listed), OBS-15 (redaction is total -- an unparseable input yields the sentinel, never a throw) +// and XCUT-19(a)/(b) as they apply to the ERROR MESSAGE rather than to a log field. The message is what +// every logger, `cause` chain and consumer `console.error` renders, so it is redacted at construction; +// `targetUrl` / `fromUrl` / `toUrl` stay raw for program use. import {describe, expect, test} from 'bun:test'; import {DexpaceError} from '../http/errors.js'; import {NonReplayableBodyError, SchemeDowngradeError} from './errors.js'; @@ -31,6 +36,36 @@ describe('NonReplayableBodyError', () => { new NonReplayableBodyError('https://example.com/next', {cause}).cause, ).toBe(cause); }); + + test('redacts userinfo and non-allow-listed query values in the message', () => { + const error = new NonReplayableBodyError( + 'https://alice:hunter2@example.com/next?access_token=SUPERSECRET&api-version=2', + ); + + expect(error.message).toContain('***:***@'); + expect(error.message).toContain('access_token=***'); + // OBS-12's default allow-list is exactly {api-version}, and the message inherits it whole. + expect(error.message).toContain('api-version=2'); + expect(error.message).not.toContain('alice'); + expect(error.message).not.toContain('hunter2'); + expect(error.message).not.toContain('SUPERSECRET'); + }); + + test('keeps the RAW target on the field even when the message is redacted', () => { + const raw = + 'https://alice:hunter2@example.com/next?access_token=SUPERSECRET'; + expect(new NonReplayableBodyError(raw).targetUrl).toBe(raw); + }); + + test('degrades an unparseable target to the sentinel rather than throwing (OBS-15)', () => { + // A target that never parsed cannot be redacted, and OBS-15 makes that total: the sentinel, not + // the raw string, because "unparseable" is not the same as "carries no secret". + const error = new NonReplayableBodyError('not a url?token=SUPERSECRET'); + + expect(error.message).toContain('[malformed url]'); + expect(error.message).not.toContain('SUPERSECRET'); + expect(error.targetUrl).toBe('not a url?token=SUPERSECRET'); + }); }); describe('SchemeDowngradeError', () => { @@ -59,4 +94,36 @@ describe('SchemeDowngradeError', () => { const error = new SchemeDowngradeError('https://a', 'http://b', {cause}); expect(error.cause).toBe(cause); }); + + test('redacts userinfo and non-allow-listed query values on BOTH sides of the message', () => { + const error = new SchemeDowngradeError( + 'https://alice:hunter2@example.com/start?access_token=SUPERSECRET', + 'http://example.com/next?code=ALSOSECRET', + ); + + expect(error.message).toContain('***:***@'); + expect(error.message).toContain('access_token=***'); + expect(error.message).toContain('code=***'); + expect(error.message).not.toContain('alice'); + expect(error.message).not.toContain('hunter2'); + expect(error.message).not.toContain('SUPERSECRET'); + expect(error.message).not.toContain('ALSOSECRET'); + }); + + test('keeps BOTH raw URLs on the fields even when the message is redacted', () => { + const from = 'https://alice:hunter2@example.com/start?access_token=SECRET'; + const to = 'http://example.com/next?code=ALSOSECRET'; + const error = new SchemeDowngradeError(from, to); + + expect(error.fromUrl).toBe(from); + expect(error.toUrl).toBe(to); + }); + + test('degrades an unparseable side to the sentinel rather than throwing (OBS-15)', () => { + const error = new SchemeDowngradeError('::::', 'http://example.com/next'); + + expect(error.message).toContain('[malformed url]'); + expect(error.message).toContain('http://example.com/next'); + expect(error.fromUrl).toBe('::::'); + }); }); diff --git a/packages/core/src/redirect/errors.ts b/packages/core/src/redirect/errors.ts index 16ec10b..b893f3a 100644 --- a/packages/core/src/redirect/errors.ts +++ b/packages/core/src/redirect/errors.ts @@ -1,6 +1,18 @@ // SPDX-License-Identifier: MIT // packages/core/src/redirect/errors.ts import {DexpaceError} from '../http/errors.js'; +import {redactUrl} from '../observability/redaction.js'; + +// Both messages below interpolate `redactUrl(...)`, never the raw URL -- XCUT-19(a)/(b) and +// OBS-11/OBS-12 applied to the MESSAGE rather than to a log field. An error message is a public API: +// it travels into every logger, every `cause` chain, and every consumer's own `console.error`, and +// this SDK owns none of those. `http.redirect.rejected` in particular hands the decision error to +// `LogEvent.cause()`, which renders it as `name: message` (`observability/logger.ts`), so a raw +// `from`/`to` URL here put userinfo and query-string tokens into the log record in clear text however +// carefully the surrounding fields were redacted. Redacting at construction is the only placement that +// also covers the paths this SDK does not own. OBS-15 makes the call safe from a constructor: an input +// that will not parse yields `[malformed url]` rather than throwing. The raw value stays on the +// error's own property, which is what program code reads. /** * REDIR-6: a method-preserving redirect (301/302/307/308) re-sends the original body, so the body must @@ -8,15 +20,23 @@ import {DexpaceError} from '../http/errors.js'; * already-consumed single-use body -- this one is a fail-fast gate evaluated BEFORE any write is * attempted, and names replayability specifically as the requirement demands. * + * @remarks + * The `message` names the target in REDACTED form (OBS-11/OBS-12): userinfo as `***:***@` and every + * non-allow-listed query value as `***`. Read {@link NonReplayableBodyError.targetUrl} for the raw URL. + * * @public */ export class NonReplayableBodyError extends DexpaceError { /** - * The redirect target that would have received the re-send. + * The redirect target that would have received the re-send, **raw and unredacted**. * * Carried as a field, not only interpolated into the message, per * `docs/knowledge/harvested/error-handling.md` -- so it survives serialization and reaches a structured log * without anyone parsing the message back apart. Phase 7b's rejection event reads it directly. + * + * This is the raw URL; the `message` carries the redacted form. Program code that needs the real + * target reads this property; anything that renders to a human or to a log backend reads the + * message. */ readonly targetUrl: string; @@ -26,7 +46,7 @@ export class NonReplayableBodyError extends DexpaceError { */ constructor(targetUrl: string, options?: ErrorOptions) { super( - `cannot follow redirect to '${targetUrl}': request body is not replayable`, + `cannot follow redirect to '${redactUrl(targetUrl)}': request body is not replayable`, options, ); this.targetUrl = targetUrl; @@ -37,12 +57,23 @@ export class NonReplayableBodyError extends DexpaceError { * REDIR-15: an HTTPS-to-HTTP hop, rejected unless `RedirectSettings.allowSchemeDowngrade` is set. * Evaluated per hop transition, so an HTTPS-to-HTTP-to-HTTPS chain flags only the hop that downgraded. * + * @remarks + * The `message` names both URLs in REDACTED form (OBS-11/OBS-12): userinfo as `***:***@` and every + * non-allow-listed query value as `***`. Read {@link SchemeDowngradeError.fromUrl} and + * {@link SchemeDowngradeError.toUrl} for the raw URLs. + * * @public */ export class SchemeDowngradeError extends DexpaceError { - /** The current hop's request URL -- the HTTPS side of the rejected transition. */ + /** + * The current hop's request URL -- the HTTPS side of the rejected transition, **raw and + * unredacted**. The `message` carries the redacted form. + */ readonly fromUrl: string; - /** The resolved redirect target -- the HTTP side of the rejected transition. */ + /** + * The resolved redirect target -- the HTTP side of the rejected transition, **raw and unredacted**. + * The `message` carries the redacted form. + */ readonly toUrl: string; /** @@ -52,7 +83,7 @@ export class SchemeDowngradeError extends DexpaceError { */ constructor(fromUrl: string, toUrl: string, options?: ErrorOptions) { super( - `redirect from '${fromUrl}' to '${toUrl}' would downgrade HTTPS to HTTP`, + `redirect from '${redactUrl(fromUrl)}' to '${redactUrl(toUrl)}' would downgrade HTTPS to HTTP`, options, ); this.fromUrl = fromUrl; From e5ea278f8f4c942a412a0e52635b99c35992c188 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:57:03 +0300 Subject: [PATCH 017/100] feat: carry a redacted url.full on http.redirect.rejected (#70) The rejection record named no URL at all, so the only hop a reader could recover from it was the raw one interpolated into the cause's message. It now carries `url.full` through `redactUrl()` -- the same field name and the same policy `http.redirect.hop`, `loopDetected` and `downgradePermitted` already use -- on both the `fail` and the `return-current` branch. Defence in depth over the message fix, not a replacement for it: this covers a future error class on this path that forgets to redact, while the message fix covers the `cause` chain this step cannot reach. REDIR-28, OBS-11, OBS-12, XCUT-19. --- .../core/src/redirect/redirect-step.test.ts | 143 ++++++++++++++++++ packages/core/src/redirect/redirect-step.ts | 19 ++- 2 files changed, 158 insertions(+), 4 deletions(-) diff --git a/packages/core/src/redirect/redirect-step.test.ts b/packages/core/src/redirect/redirect-step.test.ts index 3608ed9..9fb52cd 100644 --- a/packages/core/src/redirect/redirect-step.test.ts +++ b/packages/core/src/redirect/redirect-step.test.ts @@ -9,6 +9,11 @@ // closes the current response and propagates SchemeDowngradeError), and the cancellation check (an // an abort DURING a hop returns the current response open rather than issuing a further hop, while a // signal already aborted at entry is refused by the cursor before the step runs -- see V15). +// +// Also: REDIR-28 / XCUT-19 / OBS-11 / OBS-12 for the `http.redirect.rejected` record specifically. The +// other three redirect events already route their URL fields through `redactUrl()`; the rejection +// event carried the decision error's MESSAGE, which interpolated the raw `from`/`to` URLs, so a +// userinfo password or a query-string token reached the log record in clear text. import {describe, expect, test} from 'bun:test'; import { createRequestContext, @@ -28,6 +33,7 @@ import type {SuppressedErrorLike} from '../suppress.js'; import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; import {NonReplayableBodyError, SchemeDowngradeError} from './errors.js'; import {REDIRECT_STEP_TYPE, redirectStep} from './redirect-step.js'; +import type {RedirectSettings} from './settings.js'; const SEED = Request.newBuilder().url('https://example.com/start').build(); const CANCEL_FAILURE = new Error('cancel exploded'); @@ -587,3 +593,140 @@ describe('Phase 7b retrofit: redirect rejection logging', () => { } }); }); + +// --- REDIR-28 / XCUT-19: the rejection record --------------------------------------------------- + +/** A seed carrying both things OBS-11 and OBS-12 name: userinfo, and a non-allow-listed query value. */ +const SECRET_SEED = + 'https://alice:hunter2@example.com/start?access_token=SUPERSECRET'; +/** What `redactUrl()` makes of {@link SECRET_SEED}; the shape every URL field on this path must have. */ +const REDACTED_SEED = 'https://***:***@example.com/start?access_token=***'; + +/** + * Drives one redirect step to settlement over `transport`, capturing every record it emits, and + * restores the shipped no-op logger before returning. Rejections are swallowed: each caller here is + * asserting on the LOG, and the thrown error's own redaction is `errors.test.ts`'s row. + */ +async function recordsFromRun( + seed: Request, + transport: FakeTransport, + overrides?: Partial, +): Promise[]> { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + try { + await rejectionOf( + new Cursor({ + steps: [redirectStep(overrides)], + transport, + request: seed, + context: aRequestContext(seed), + }).advance(), + ); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + return events; +} + +/** The one `http.redirect.rejected` record, rendered field-by-field as a logger backend sees it. */ +function rejection( + events: Map[], +): Map | undefined { + const rejected = events.filter( + e => e.get('event') === 'http.redirect.rejected', + ); + expect(rejected).toHaveLength(1); + return rejected[0]; +} + +/** Asserts no field of `record` carries any of `secrets` in clear text -- XCUT-19's actual claim. */ +function expectNoSecret( + record: Map | undefined, + secrets: readonly string[], +): void { + for (const field of record?.values() ?? []) { + for (const secret of secrets) { + expect(String(field)).not.toContain(secret); + } + } +} + +const SECRETS = ['alice', 'hunter2', 'SUPERSECRET', 'ALSOSECRET'] as const; + +describe('REDIR-28/XCUT-19: the rejection record never carries a raw URL', () => { + test('redacts the downgrade rejection cause and its url.full field', async () => { + const hop = countingResponse(301); + const records = await recordsFromRun( + Request.newBuilder().url(SECRET_SEED).build(), + new FakeTransport([ + withLocation(hop.response, 'http://example.com/next?code=ALSOSECRET'), + ]), + ); + + const record = rejection(records); + // The logger renders a cause as `name: message` (observability/logger.ts), so the MESSAGE is the + // log field. The 8192-byte field cap truncates nothing here, and would not have helped if it did. + expect(String(record?.get('cause'))).toStartWith('SchemeDowngradeError: '); + expect(String(record?.get('cause'))).toContain('***:***@'); + expect(String(record?.get('cause'))).toContain('access_token=***'); + expect(String(record?.get('cause'))).toContain('code=***'); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); + + test('redacts the non-replayable-body rejection cause and its url.full field', async () => { + const oneShot = streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ); + const hop = countingResponse(307); + const records = await recordsFromRun( + Request.newBuilder() + .method('POST') + .url(SECRET_SEED) + .body(oneShot) + .build(), + new FakeTransport([ + withLocation(hop.response, 'https://example.com/next?code=ALSOSECRET'), + ]), + {allowedMethods: new Set(['GET', 'HEAD', 'POST'])}, + ); + + const record = rejection(records); + expect(String(record?.get('cause'))).toStartWith( + 'NonReplayableBodyError: ', + ); + expect(String(record?.get('cause'))).toContain('code=***'); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); + + test('carries url.full on a rejection with no error, matching the other events', async () => { + // The `return-current` rejections -- loop detected, hop cap, malformed Location -- emit the same + // event with no cause. They still name the hop, and it is still redacted. The malformed-Location + // one is the sharpest case: REDIR-28 lets the sibling `malformedLocation` event log the header + // RAW, so this record is the only redacted URL anywhere on that path. + const hop = countingResponse(302); + const records = await recordsFromRun( + Request.newBuilder().url(SECRET_SEED).build(), + new FakeTransport([withLocation(hop.response, 'javascript:alert(1)')]), + ); + + const record = rejection(records); + expect(record?.has('cause')).toBe(false); + expect(record?.get('url.full')).toBe(REDACTED_SEED); + expectNoSecret(record, SECRETS); + }); +}); diff --git a/packages/core/src/redirect/redirect-step.ts b/packages/core/src/redirect/redirect-step.ts index e3951d3..c324d3f 100644 --- a/packages/core/src/redirect/redirect-step.ts +++ b/packages/core/src/redirect/redirect-step.ts @@ -88,11 +88,22 @@ function emitStopReason(stop: { } } -function emitRejected(error?: unknown): void { +/** + * REDIR-28's rejection event. Carries `url.full` -- the hop the rejection is about -- through + * `redactUrl()`, the same field and the same policy `http.redirect.hop`, `loopDetected` and + * `malformedLocation` already use; before that this record named no URL at all, and the only URL a + * reader could recover was the raw one interpolated into the cause's message. + * + * The cause itself is safe to attach because `redirect/errors.ts` now builds both message texts from + * `redactUrl()` output. That is deliberately fixed at the ERROR rather than dropped here: `cause` is + * also what a caller's own `console.error` renders, and this step cannot redact that one. + */ +function emitRejected(request: Request, error?: unknown): void { try { const event = getGlobalLogger() .atLevel('warning') - .event('http.redirect.rejected'); + .event('http.redirect.rejected') + .field('url.full', redactUrl(request.url)); if (error !== undefined) { event.cause(error); } @@ -216,7 +227,7 @@ export function redirectStep( if (decision.kind === 'return-current') { if (response.status.isRedirect) { - emitRejected(); + emitRejected(request); emitStopReason({ reason: decision.reason, request, @@ -227,7 +238,7 @@ export function redirectStep( } if (decision.kind === 'fail') { const releaseError = await releaseQuietly(response); - emitRejected(decision.error); + emitRejected(request, decision.error); throw withReleaseFailure(decision.error, releaseError); } if (signal?.aborted === true) return response; From 86091f456bde7e2495bdad8ce64ba9c490f6a142 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:57:12 +0300 Subject: [PATCH 018/100] test: prove the rejected-redirect path leaks no secret end to end (#70) One XCUT-19 row over the real two-origin fixture and the composed retry+redirect+auth+logging pipeline: a one-shot body makes a 307 unfollowable, so `decide()` fails with the redirect target in the message, and the target carries a credential-shaped query value the test itself supplies. Asserts the captured `http.redirect.rejected` record shows `access_token=***`, and that no field of ANY record the whole pipeline emitted carries the secret in clear text -- which is XCUT-19's actual claim, not just this one event's. Needs a Location with a secret in it, which no existing fixture route serves, so `/redirect-secret-target` joins `server.ts`; it echoes the caller's own `?secret=` back into the Location so the test owns the string it then greps for. XCUT-19(a)/(b), OBS-11, OBS-12, REDIR-28. --- tests/conformance/xcut/fixtures/server.ts | 10 +++ .../security-by-default.conformance.test.ts | 62 +++++++++++++++++++ 2 files changed, 72 insertions(+) diff --git a/tests/conformance/xcut/fixtures/server.ts b/tests/conformance/xcut/fixtures/server.ts index 2ac2dfa..754cb65 100644 --- a/tests/conformance/xcut/fixtures/server.ts +++ b/tests/conformance/xcut/fixtures/server.ts @@ -95,6 +95,16 @@ function route( res.writeHead(302, {location: `${crossOrigin}/echo`}); res.end(); return; + case '/redirect-secret-target': { + // A method-preserving 307 whose Location carries a credential-shaped query value, echoed back + // from the caller's own `?secret=` so the test owns the string it then greps the log for. + // XCUT-19's rejected-redirect row needs the secret in the redirect TARGET specifically: that is + // the URL `NonReplayableBodyError` names, and the seed URL alone never reaches that message. + const secret = url.searchParams.get('secret') ?? 'secret'; + res.writeHead(307, {location: `/echo?access_token=${secret}`}); + res.end(); + return; + } case '/fail-500': res.writeHead(500, {'content-type': 'text/plain'}); res.end('server error'); diff --git a/tests/conformance/xcut/security-by-default.conformance.test.ts b/tests/conformance/xcut/security-by-default.conformance.test.ts index dd32f93..a317647 100644 --- a/tests/conformance/xcut/security-by-default.conformance.test.ts +++ b/tests/conformance/xcut/security-by-default.conformance.test.ts @@ -3,6 +3,8 @@ // Exercises: XCUT-17 (redirect credential hygiene -- Authorization stripped before EVERY re-issue, // origin-scoped credentials additionally stripped cross-origin), XCUT-16 (no credential is ever // stamped over a non-HTTPS transport, and the refusal lands BEFORE any token fetch). +// Exercises: XCUT-19 (default-deny log redaction, clause (a) userinfo and clause (b) query values) on +// the rejected-redirect path, with OBS-11, OBS-12 and REDIR-28 as the requirements it lands under. // // These run over a real two-origin socket pair through the composed retry+redirect+auth+logging // pipeline. 5b's own tests decide the hop in isolation against constructed inputs; this is the first @@ -23,10 +25,16 @@ import { createAuthDescriptor, createAuthRequirement, createBearerToken, + createLogger, Headers, + NonReplayableBodyError, + NOOP_LOGGER, PlaintextCredentialError, Request, + setGlobalLogger, + streamBody, type AuthStepSettings, + type Method, } from '@dexpace/core'; import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; @@ -106,6 +114,60 @@ describe('XCUT-17: origin-scoped credentials are additionally stripped cross-ori }); }); +describe('XCUT-19: a rejected redirect logs no raw URL (OBS-11, OBS-12, REDIR-28)', () => { + const SECRET = 'SUPERSECRETTOKEN'; + + test('redacts the redirect target inside the rejection cause', async () => { + // A one-shot body makes the 307 unfollowable, so `decide()` fails with the target interpolated + // into the error message -- the one field on this path that `redactUrl` did not already cover. + const seed = Request.newBuilder() + .method('POST') + .url(`${server.url}/redirect-secret-target?secret=${SECRET}`) + .body( + streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ), + ) + .build(); + const pipeline = buildComposedPipeline({ + redirect: {allowedMethods: new Set(['GET', 'HEAD', 'POST'])}, + }); + const records: Map[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + records.push(new Map(fields)); + }), + ); + + try { + const rejected = await rejectionOf(pipeline.runtime.send(seed)); + + expect(rejected).toBeInstanceOf(NonReplayableBodyError); + const rejections = records.filter( + r => r.get('event') === 'http.redirect.rejected', + ); + expect(rejections).toHaveLength(1); + expect(String(rejections[0]?.get('cause'))).toContain('access_token=***'); + // Nothing the whole composed pipeline emitted -- not the redirect events, not the + // request/response pair around them -- carries the secret in clear text. + for (const record of records) { + for (const field of record.values()) { + expect(String(field)).not.toContain(SECRET); + } + } + } finally { + setGlobalLogger(NOOP_LOGGER); + await pipeline.close(); + } + }); +}); + describe('XCUT-16: a credential is never stamped over a non-HTTPS transport', () => { test('refuses a bearer credential over http:// before fetching the token', async () => { let providerInvocations = 0; From b92fb7224559a7b5c3a60514a9835fe34cf612ce Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:57:12 +0300 Subject: [PATCH 019/100] docs: state the redacted-message / raw-property split in the errors guide (#70) `errors.md` listed both redirect errors in its table and said nothing about which of their URL surfaces is safe to log. A consumer reading `error.message` into their own logger now knows it is already redacted, and that `fromUrl`/`toUrl`/`targetUrl` are the raw values for program use. --- docs/sdk-documentation/errors.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/sdk-documentation/errors.md b/docs/sdk-documentation/errors.md index 3be0a84..b9985fa 100644 --- a/docs/sdk-documentation/errors.md +++ b/docs/sdk-documentation/errors.md @@ -162,6 +162,24 @@ Eight classes that a `@throws` tag named but no package exported were promoted t | `CursorAlreadyAdvancedError` | a step reusing an already-invoked `next()`/`fork()` continuation | | `EndOfStreamError` | a `BufferedSource` read that required more bytes than the source delivered | +**The two redirect errors carry redacted messages and raw properties, and the split is deliberate.** +`SchemeDowngradeError.message` and `NonReplayableBodyError.message` name their URLs in the redacted +form `OBS-11`/`OBS-12` define — userinfo as `***:***@`, every non-allow-listed query value as `***` — +because a message is rendered by every logger, every `cause` chain and every consumer `console.error`, +and `http.redirect.rejected` hands the error straight to `LogEvent.cause()`. The unredacted URLs stay +on `fromUrl` / `toUrl` / `targetUrl`, which is where program code reads them: + +```ts +try { + await client.send(request); +} catch (error) { + if (error instanceof SchemeDowngradeError) { + console.error(error.message); // https://***:***@api.example.com/v1?token=*** + retargetTo(error.toUrl); // the real URL, for code rather than for a log + } +} +``` + Two more joined them on the same date, both from `XCUT-8`'s "never fabricate a successful exception": | Error | Raised by | From 9798d221d2b39b0b41c422fa7a040c8fd9cc92c0 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:58:39 +0300 Subject: [PATCH 020/100] feat: redact the Basic and Digest passwords behind credential classes (#71) `BasicCredential` and `DigestCredential` shipped as structural interfaces with a public `readonly password: string`, so `util.inspect` of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}` and `JSON.stringify` serialized both passwords. Both are now classes in `auth/credential.ts` on exactly the pattern `ApiKeyCredential`/`NameKeyCredential`/`BearerToken` already use: a `#password` field, a redacted `toString()`, the `nodejs.util.inspect.custom` hook, and an in-package `credentialPassword()` friend hook as the sole read path -- the mirror of `credentialKey()`, and `buildHandlers` is its only caller. AUTH-8's redaction clause enumerates bearer, API key and name key only; reading it as covering every credential type is a deliberate widening, ledgered separately. Validation is deliberately NOT duplicated onto the classes: AUTH-14's non-empty-whitespace-permitted rule and AUTH-16's acceptable-set rule stay single-sourced in `basicHandler()`/`digestHandler()`, which `authStep()` builds at construction, so a blank password still fails synchronously from that factory. Public shape change: `{username, password}` object literals no longer type-check. Free before the first version bump. --- packages/core/etc/core.api.md | 14 +- packages/core/src/auth/auth-step.test.ts | 37 ++--- packages/core/src/auth/auth-step.ts | 57 +++----- packages/core/src/auth/credential.test.ts | 161 ++++++++++++++++++++- packages/core/src/auth/credential.ts | 168 ++++++++++++++++++++++ packages/core/src/index.ts | 14 +- scripts/verify-consumer-types.mjs | 12 +- tests/node-conformance/auth.test.mjs | 3 +- 8 files changed, 390 insertions(+), 76 deletions(-) diff --git a/packages/core/etc/core.api.md b/packages/core/etc/core.api.md index 4809b58..3431dbb 100644 --- a/packages/core/etc/core.api.md +++ b/packages/core/etc/core.api.md @@ -105,8 +105,10 @@ export interface BackoffSettings { } // @public -export interface BasicCredential { - readonly password: string; +export class BasicCredential { + [INSPECT](): string; + constructor(username: string, password: string); + toString(): string; readonly username: string; } @@ -345,9 +347,11 @@ export class DexpaceError extends Error { export type DigestAlgorithm = 'MD5' | 'MD5-sess' | 'SHA-256' | 'SHA-256-sess'; // @public -export interface DigestCredential { - readonly algorithmPreference?: readonly DigestAlgorithm[] | undefined; - readonly password: string; +export class DigestCredential { + [INSPECT](): string; + constructor(username: string, password: string, algorithmPreference?: readonly DigestAlgorithm[]); + readonly algorithmPreference: readonly DigestAlgorithm[] | undefined; + toString(): string; readonly username: string; } diff --git a/packages/core/src/auth/auth-step.test.ts b/packages/core/src/auth/auth-step.test.ts index 6c428d3..bd70a33 100644 --- a/packages/core/src/auth/auth-step.test.ts +++ b/packages/core/src/auth/auth-step.test.ts @@ -41,7 +41,12 @@ import { availableSchemesOf, type AuthCredentialSet, } from './auth-step.js'; -import {createBearerToken, ApiKeyCredential} from './credential.js'; +import { + createBearerToken, + ApiKeyCredential, + BasicCredential, + DigestCredential, +} from './credential.js'; import {createAuthDescriptor} from './descriptor.js'; import {AuthResolutionError, PlaintextCredentialError} from './errors.js'; import {createAuthRequirement} from './requirement.js'; @@ -236,8 +241,8 @@ describe('availableSchemesOf (AUTH-5)', () => { test('maps each configured credential to its scheme', () => { const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, - digest: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), + digest: new DigestCredential('u', 'p'), bearer: {provider: () => Promise.resolve(createBearerToken('t'))}, apiKey: {credential: new ApiKeyCredential('k')}, }; @@ -379,7 +384,7 @@ describe('authStep: the HTTPS guard and tier resolution (AUTH-6/AUTH-28)', () => test('BASIC/DIGEST never stamp preemptively -- the outbound request carries no Authorization', async () => { const transport = new FakeTransport([countingResponse(200).response]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -437,7 +442,7 @@ describe('authStep: the cross-origin marker (AUTH-29)', () => { countingResponse(200).response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -464,7 +469,7 @@ describe('authStep: challenge detection (AUTH-25/AUTH-33)', () => { success.response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -492,7 +497,7 @@ describe('authStep: challenge detection (AUTH-25/AUTH-33)', () => { countingResponse(200).response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -508,7 +513,7 @@ describe('authStep: challenge detection, negative cases (AUTH-33)', () => { const the401 = countingResponse(401); const transport = new FakeTransport([the401.response]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -541,7 +546,7 @@ describe('authStep: the challenge replay (AUTH-30/AUTH-31)', () => { success.response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -566,7 +571,7 @@ describe('authStep: the challenge replay (AUTH-30/AUTH-31)', () => { ); const transport = new FakeTransport([first.response, second.response]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -590,7 +595,7 @@ describe('authStep: answering a Digest challenge (AUTH-15..AUTH-22)', () => { countingResponse(200).response, ]); const credentials: AuthCredentialSet = { - digest: {username: 'u', password: 'p'}, + digest: new DigestCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('DIGEST')}); @@ -617,7 +622,7 @@ describe('authStep: the replayability gate (AUTH-31)', () => { countingResponse(200).response, ]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -641,7 +646,7 @@ describe('authStep: the replayability gate (AUTH-31)', () => { ); const transport = new FakeTransport([challenged.response]); const credentials: AuthCredentialSet = { - basic: {username: 'u', password: 'p'}, + basic: new BasicCredential('u', 'p'), }; const descriptor = authStep({credentials, tiers: tiersFor('BASIC')}); @@ -749,7 +754,7 @@ describe('authStep: hook override and non-reactive schemes (AUTH-30)', () => { ]); let hookInvoked = false; const descriptor = authStep({ - credentials: {basic: {username: 'u', password: 'p'}}, + credentials: {basic: new BasicCredential('u', 'p')}, tiers: tiersFor('BASIC'), challengeHook: (_response, request) => { hookInvoked = true; @@ -1137,7 +1142,7 @@ describe('authStep: answering an unrecognized scheme through challengeHook', () countingResponse(200).response, ]); const descriptor = authStep({ - credentials: {basic: {username: 'u', password: 'p'}}, + credentials: {basic: new BasicCredential('u', 'p')}, tiers: tiersFor('BASIC'), challengeHook: (_response, request) => Promise.resolve( @@ -1459,7 +1464,7 @@ describe('authStep: a challenge this client cannot echo (AUTH-21/AUTH-22)', () = ); const transport = new FakeTransport([challenge.response]); const descriptor = authStep({ - credentials: {digest: {username: 'u', password: 'p'}}, + credentials: {digest: new DigestCredential('u', 'p')}, tiers: tiersFor('DIGEST'), }); diff --git a/packages/core/src/auth/auth-step.ts b/packages/core/src/auth/auth-step.ts index 540a9d3..b06a552 100644 --- a/packages/core/src/auth/auth-step.ts +++ b/packages/core/src/auth/auth-step.ts @@ -15,48 +15,21 @@ import {BearerTokenCache} from './bearer-cache.js'; import {parseChallenges} from './challenge.js'; import type {Challenge, ChallengeHandler} from './challenge.js'; import {composingHandler, type ComposingHandler} from './composing-handler.js'; -import type { - ApiKeyCredential, - NameKeyCredential, - TokenProvider, +import { + credentialPassword, + type ApiKeyCredential, + type BasicCredential, + type DigestCredential, + type NameKeyCredential, + type TokenProvider, } from './credential.js'; import type {AuthDescriptor} from './descriptor.js'; import {digestHandler} from './digest.js'; -import type {DigestAlgorithm} from './digest.js'; import {PlaintextCredentialError} from './errors.js'; import {resolveAuthRequirement, type AuthTiers} from './resolve.js'; import type {AuthScheme} from './scheme.js'; import {stampStaticKey} from './static-key.js'; -/** - * Username and password for the `BASIC` scheme. - * - * @public - */ -export interface BasicCredential { - /** The user id (AUTH-14: non-empty; whitespace permitted). */ - readonly username: string; - /** The password (AUTH-14: non-empty; whitespace permitted). */ - readonly password: string; -} - -/** - * Username, password, and algorithm preference for the `DIGEST` scheme. - * - * @public - */ -export interface DigestCredential { - /** The user id. Must not be blank. */ - readonly username: string; - /** The password. Must not be blank. */ - readonly password: string; - /** - * Preferred-first order, and also the acceptable set (AUTH-16). Omitted means strongest-first over - * all four supported algorithms. - */ - readonly algorithmPreference?: readonly DigestAlgorithm[] | undefined; -} - /** * The token source and refresh margin for the `OAUTH2` scheme. * @@ -145,16 +118,24 @@ function buildHandlers( credentials: AuthCredentialSet, ): readonly ChallengeHandler[] { const handlers: ChallengeHandler[] = []; + // `credentialPassword()`, not a public `.password` property: AUTH-8's secret stays off the + // published surface and this function is the one sanctioned reader, exactly as `stampStaticKey` + // is for `credentialKey()`. if (credentials.digest !== undefined) { handlers.push( - digestHandler(credentials.digest.username, credentials.digest.password, { - algorithmPreference: credentials.digest.algorithmPreference, - }), + digestHandler( + credentials.digest.username, + credentialPassword(credentials.digest), + {algorithmPreference: credentials.digest.algorithmPreference}, + ), ); } if (credentials.basic !== undefined) { handlers.push( - basicHandler(credentials.basic.username, credentials.basic.password), + basicHandler( + credentials.basic.username, + credentialPassword(credentials.basic), + ), ); } return handlers; diff --git a/packages/core/src/auth/credential.test.ts b/packages/core/src/auth/credential.test.ts index 23f76c9..f15189e 100644 --- a/packages/core/src/auth/credential.test.ts +++ b/packages/core/src/auth/credential.test.ts @@ -1,20 +1,33 @@ // SPDX-License-Identifier: MIT // packages/core/src/auth/credential.test.ts // Exercises: AUTH-8 (BearerToken is value-equal; ApiKeyCredential/NameKeyCredential are -// reference-equal via bare `===`, no equals() override; ALL THREE redact their secret in every -// string/diagnostic form and none is reachable through JSON.stringify/Object.keys), AUTH-9 (blank +// reference-equal via bare `===`, no equals() override; ALL FIVE credential types -- bearer, API key, +// name key, Basic and Digest -- redact their secret in every string/diagnostic form and none is +// reachable through JSON.stringify/Object.keys, including when a whole AuthCredentialSet is handed to +// `util.inspect`), AUTH-9 (blank // rejected as a programmer error, and every type is nominal so the validation cannot be routed // around), AUTH-10 (expiry math: undefined never locally expires; expired iff nowMs + marginMs > -// expiresAt), AUTH-26 (`credentialKey()` is the sole read path for a static key's secret). +// expiresAt), AUTH-26 (`credentialKey()` is the sole read path for a static key's secret), +// AUTH-14/AUTH-16 (the Basic and Digest credentials carry the username and the algorithm preference +// as non-secret fields, which AUTH-8 permits to stay visible). import {describe, expect, test} from 'bun:test'; +// The ONE `node:` import in this package's tests, and it is the point of the AuthCredentialSet rows +// below: the reported leak was `util.inspect(credentials)` printing `password: 'hunter2'`, so the row +// that proves it closed has to drive the real `util.inspect`, not the hook it happens to call. +import {inspect} from 'node:util'; import {InvariantViolation} from '../invariant.js'; +import type {AuthCredentialSet} from './auth-step.js'; +import type {DigestAlgorithm} from './digest.js'; import { ApiKeyCredential, + BasicCredential, BearerToken, + DigestCredential, NameKeyCredential, bearerTokensEqual, createBearerToken, credentialKey, + credentialPassword, isBearerTokenExpired, } from './credential.js'; @@ -23,7 +36,12 @@ import { const INSPECT = Symbol.for('nodejs.util.inspect.custom'); function inspectOf( - value: ApiKeyCredential | BearerToken | NameKeyCredential, + value: + | ApiKeyCredential + | BasicCredential + | BearerToken + | DigestCredential + | NameKeyCredential, ): string { const hooks = value as unknown as Record string) | undefined>; const hook = hooks[INSPECT]; @@ -191,3 +209,138 @@ describe('NameKeyCredential (AUTH-8)', () => { expect(serialized).not.toContain('super-secret'); }); }); + +describe('BasicCredential (AUTH-8/AUTH-14)', () => { + test('two instances with identical fields are NOT equal -- reference identity only', () => { + expect( + new BasicCredential('u', 'p') === new BasicCredential('u', 'p'), + ).toBe(false); + }); + + test('toString and inspect redact the password but keep the username', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(credential.toString()).toContain('alice'); + expect(credential.toString()).not.toContain('super-secret'); + expect(String(credential)).not.toContain('super-secret'); + expect(inspectOf(credential)).not.toContain('super-secret'); + }); + + test('JSON.stringify reaches the username but never the password', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(JSON.stringify(credential)).toContain('alice'); + expect(JSON.stringify(credential)).not.toContain('super-secret'); + expect(Object.keys(credential)).toEqual(['username']); + }); + + test('the password is reachable ONLY through the internal credentialPassword() hook', () => { + const credential = new BasicCredential('alice', 'super-secret'); + expect(credentialPassword(credential)).toBe('super-secret'); + // No public `password` property: that is exactly what the structural interface this class + // replaced put on the published surface (AUTH-8). + expect('password' in credential).toBe(false); + }); + + test('is frozen', () => { + expect(Object.isFrozen(new BasicCredential('u', 'p'))).toBe(true); + }); +}); + +describe('DigestCredential (AUTH-8/AUTH-16)', () => { + test('two instances with identical fields are NOT equal', () => { + expect( + new DigestCredential('u', 'p') === new DigestCredential('u', 'p'), + ).toBe(false); + }); + + test('toString and inspect redact the password but keep username and preference', () => { + const credential = new DigestCredential('bob', 'super-secret', ['SHA-256']); + expect(credential.toString()).toContain('bob'); + expect(credential.toString()).toContain('SHA-256'); + expect(credential.toString()).not.toContain('super-secret'); + expect(String(credential)).not.toContain('super-secret'); + expect(inspectOf(credential)).not.toContain('super-secret'); + }); + + test('JSON.stringify reaches the username but never the password', () => { + const credential = new DigestCredential('bob', 'super-secret'); + expect(JSON.stringify(credential)).toContain('bob'); + expect(JSON.stringify(credential)).not.toContain('super-secret'); + expect(Object.keys(credential)).toEqual([ + 'username', + 'algorithmPreference', + ]); + }); + + test('the password is reachable ONLY through the internal credentialPassword() hook', () => { + const credential = new DigestCredential('bob', 'super-secret'); + expect(credentialPassword(credential)).toBe('super-secret'); + expect('password' in credential).toBe(false); + }); + + test('an omitted algorithmPreference stays undefined -- the handler owns the default', () => { + // `digestHandler` applies AUTH-16's strongest-first default. Materializing it here would put a + // second copy of that list on the public surface, free to drift from the one that is used. + expect( + new DigestCredential('bob', 'p').algorithmPreference, + ).toBeUndefined(); + }); + + test('the algorithm preference is copied and frozen, not aliased (HTTP-3)', () => { + const supplied: DigestAlgorithm[] = ['SHA-256', 'MD5']; + const credential = new DigestCredential('bob', 'p', supplied); + supplied.push('MD5-sess'); + + expect(credential.algorithmPreference).toEqual(['SHA-256', 'MD5']); + expect(Object.isFrozen(credential.algorithmPreference)).toBe(true); + }); + + test('is frozen', () => { + expect(Object.isFrozen(new DigestCredential('u', 'p'))).toBe(true); + }); +}); + +describe('a whole AuthCredentialSet is diagnostic-safe (AUTH-8)', () => { + /** Every scheme's material at once -- the shape a caller hands `authStep()`. */ + function everyCredential(): AuthCredentialSet { + return { + basic: new BasicCredential('alice', 'basic-hunter2'), + digest: new DigestCredential('bob', 'digest-hunter2', ['SHA-256']), + apiKey: { + credential: new NameKeyCredential('x-api-key', 'api-key-hunter2'), + headerName: 'X-Api-Key', + }, + bearer: { + provider: () => + Promise.resolve(createBearerToken('bearer-hunter2', 1000)), + }, + }; + } + + test('util.inspect prints no secret from any of the four schemes', () => { + const rendered = inspect(everyCredential(), {depth: null}); + + expect(rendered).not.toContain('basic-hunter2'); + expect(rendered).not.toContain('digest-hunter2'); + expect(rendered).not.toContain('api-key-hunter2'); + // The non-secret fields AUTH-8 permits to stay visible are still there, which is what makes the + // rendering worth printing at all. + expect(rendered).toContain('alice'); + expect(rendered).toContain('bob'); + expect(rendered).toContain('x-api-key'); + }); + + test('util.inspect honours the inspect hook on a BearerToken too', () => { + // `inspectOf` proves the hook exists; this proves `util.inspect` actually calls it. + expect(inspect(createBearerToken('bearer-hunter2', 1000))).not.toContain( + 'bearer-hunter2', + ); + }); + + test('JSON.stringify serializes no secret from any of the four schemes', () => { + const serialized = JSON.stringify(everyCredential()); + + expect(serialized).not.toContain('basic-hunter2'); + expect(serialized).not.toContain('digest-hunter2'); + expect(serialized).not.toContain('api-key-hunter2'); + }); +}); diff --git a/packages/core/src/auth/credential.ts b/packages/core/src/auth/credential.ts index 1fb9403..93d95df 100644 --- a/packages/core/src/auth/credential.ts +++ b/packages/core/src/auth/credential.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/core/src/auth/credential.ts import {invariant} from '../invariant.js'; +import type {DigestAlgorithm} from './digest.js'; // No `as unique symbol` cast: TypeScript rejects `unique symbol` in a type assertion (TS1335). A // `const` initialized directly by a `Symbol.for()` call already gets the `unique symbol` type, which @@ -283,6 +284,173 @@ export class NameKeyCredential { } } +/** + * The friend-class hooks for the two password credentials' secrets, and the reason those two are + * classes at all. `auth-step.ts`'s `buildHandlers` -- the ONLY sanctioned reader -- reaches them + * through {@link credentialPassword}. + * + * They shipped as structural interfaces with a public `readonly password: string` until 2026-09-04, + * which put a live password on the published `.d.ts` and, worse, on the object graph: `util.inspect` + * of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and + * `JSON.stringify` serialized it. AUTH-8's redaction clause names bearer, API-key and name-key + * explicitly; reading it as covering EVERY credential type is a deliberate widening, recorded in + * `docs/deviations.md` (found by audit #67 / #71). + */ +let readBasicPassword: (credential: BasicCredential) => string; +let readDigestPassword: (credential: DigestCredential) => string; + +/** + * The in-package read hook for a Basic or Digest credential's password. + * + * Exported (still internal-only, absent from the package barrel) because the one caller -- + * `buildHandlers` in `auth-step.ts` -- lives in another module and TypeScript has no friend-class + * visibility to express that with. The same shape {@link credentialKey} uses for the static keys. + * + * @param credential - the credential whose password is about to reach a handler. + * @returns the raw password. + * + * @internal + */ +export function credentialPassword( + credential: BasicCredential | DigestCredential, +): string { + return credential instanceof BasicCredential + ? readBasicPassword(credential) + : readDigestPassword(credential); +} + +/** + * Username and password for the `BASIC` scheme (AUTH-8, AUTH-14). + * + * `#password`, not `readonly password`, for the runtime-privacy reason {@link ApiKeyCredential} + * states at length: `private` is erased and a plain property is reachable through + * `credential['password']`, `Object.keys`, `JSON.stringify` and a default `util.inspect`. The + * redacted `toString`/inspect pair below is what those paths get instead. `username` is non-secret, + * which AUTH-8 explicitly permits to stay visible. + * + * Reference equality, like the two key credentials and for the same reason: there is deliberately no + * `equals` member anywhere in this module. + * + * **Validation is not repeated here.** AUTH-14's rule -- non-empty, whitespace permitted, which is + * deliberately laxer than `.trim().length > 0` -- lives in `basicHandler()`, and `authStep()` builds + * a handler for every configured credential at construction, so a blank password still fails + * synchronously from that factory. Restating the rule here would put a second copy of it one edit + * away from disagreeing with the copy that is actually applied to the wire. + * + * @public + */ +export class BasicCredential { + /** The user id (AUTH-14: non-empty; whitespace permitted). Non-secret, so visible. */ + readonly username: string; + readonly #password: string; + + /** + * @param username - the user id. + * @param password - the password. Never readable back off the instance. + */ + constructor(username: string, password: string) { + this.username = username; + this.#password = password; + Object.freeze(this); + } + + static { + readBasicPassword = credential => credential.#password; + } + + /** + * AUTH-8's redacted string form: the username survives, the password does not. + * + * @returns the representation with the password masked. + */ + toString(): string { + return `BasicCredential{username=${this.username}, password=***}`; + } + + /** + * The same redaction for `console.log`/`util.inspect`, which do not route object arguments through + * `toString`. See {@link ApiKeyCredential} for why both hooks are needed. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + +/** + * Username, password, and algorithm preference for the `DIGEST` scheme (AUTH-8, AUTH-16). + * + * `#password` for the same runtime-privacy reason as {@link BasicCredential}. `username` and + * `algorithmPreference` are non-secret and stay visible. + * + * **Validation is not repeated here**, for the reason {@link BasicCredential} states: AUTH-16's + * acceptable-set rule and the blank/header-safety checks live in `digestHandler()`, which + * `authStep()` builds at construction. + * + * @public + */ +export class DigestCredential { + /** The user id. Non-secret, so visible. */ + readonly username: string; + /** + * Preferred-first order, and also the acceptable set (AUTH-16). `undefined` means strongest-first + * over all four supported algorithms -- the default is applied by `digestHandler()`, not + * materialized here, so there is only ever one copy of that list. + */ + readonly algorithmPreference: readonly DigestAlgorithm[] | undefined; + readonly #password: string; + + /** + * @param username - the user id. + * @param password - the password. Never readable back off the instance. + * @param algorithmPreference - preferred-first acceptable algorithms; copied and frozen, so a + * caller mutating the array afterwards cannot change what this credential accepts (HTTP-3's + * no-aliasing rule, applied to the one collection this type holds). + */ + constructor( + username: string, + password: string, + algorithmPreference?: readonly DigestAlgorithm[], + ) { + this.username = username; + this.#password = password; + this.algorithmPreference = + algorithmPreference === undefined + ? undefined + : Object.freeze([...algorithmPreference]); + Object.freeze(this); + } + + static { + readDigestPassword = credential => credential.#password; + } + + /** + * AUTH-8's redacted string form: the username and the algorithm preference survive, the password + * does not. + * + * @returns the representation with the password masked. + */ + toString(): string { + const preference = + this.algorithmPreference === undefined + ? 'default' + : this.algorithmPreference.join('|'); + return `DigestCredential{username=${this.username}, password=***, algorithmPreference=${preference}}`; + } + + /** + * The same redaction for `console.log`/`util.inspect`. See {@link ApiKeyCredential} for why both + * hooks are needed. + * + * @returns the redacted representation. + */ + [INSPECT](): string { + return this.toString(); + } +} + /** * AUTH-11: an async token source. A plain function type, no class. * diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index fc7ae83..eff7f03 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -165,20 +165,20 @@ export type { ApiKeyCredentialConfig, AuthCredentialSet, AuthStepSettings, - BasicCredential, BearerCredential, ChallengeHook, - DigestCredential, } from './auth/auth-step.js'; export type {AuthTiers} from './auth/resolve.js'; export type {AuthScheme} from './auth/scheme.js'; export type {DigestAlgorithm} from './auth/digest.js'; // Factories, not bare interfaces: AUTH-3 validates and freezes inside `createAuthDescriptor`, and -// `ApiKeyCredential`/`NameKeyCredential`/`BearerToken` are NOMINAL -- they carry a `#` field, so no -// caller-side object literal is assignable and the AUTH-9 validation in each factory cannot be routed -// around. Without these, API_KEY and OAUTH2 auth are unreachable from outside the package. -// `BearerToken` is a VALUE export, not a type-only one: it is a class, and `TokenProvider` returns it. +// every credential type is NOMINAL -- each carries a `#` field, so no caller-side object literal is +// assignable and the AUTH-9 validation in each factory cannot be routed around. Without these, +// API_KEY, OAUTH2, BASIC and DIGEST auth are unreachable from outside the package. +// All five are VALUE exports, not type-only ones: they are classes, `TokenProvider` returns a +// `BearerToken`, and `BasicCredential`/`DigestCredential` became classes on 2026-09-04 so AUTH-8's +// redaction covers their passwords too (audit #67 / #71). export type {AuthDescriptor} from './auth/descriptor.js'; export {createAuthDescriptor} from './auth/descriptor.js'; export type {AuthRequirement} from './auth/requirement.js'; @@ -189,7 +189,9 @@ export { export type {TokenProvider} from './auth/credential.js'; export { ApiKeyCredential, + BasicCredential, BearerToken, + DigestCredential, NameKeyCredential, bearerTokensEqual, createBearerToken, diff --git a/scripts/verify-consumer-types.mjs b/scripts/verify-consumer-types.mjs index 9a874e2..48a10ed 100644 --- a/scripts/verify-consumer-types.mjs +++ b/scripts/verify-consumer-types.mjs @@ -141,7 +141,7 @@ import { type AuthStepSettings, type AuthTiers, type BackoffSettings, - type BasicCredential, + BasicCredential, type BearerCredential, BearerToken, bearerTokensEqual, @@ -159,7 +159,7 @@ import { type DeserializationErrorOptions, type Deserializer, type DigestAlgorithm, - type DigestCredential, + DigestCredential, type DispatchContext, type ExchangeContext, type ExecutionContext, @@ -333,8 +333,8 @@ export function assemble(transport: Transport): Runtime { const settings: AuthStepSettings = { credentials: { apiKey: {credential: new ApiKeyCredential('k'), prefix: 'ApiKey'}, - basic: {username: 'u', password: 'p'}, - digest: {username: 'u', password: 'p', algorithmPreference: ['SHA-256' satisfies DigestAlgorithm]}, + basic: new BasicCredential('u', 'p'), + digest: new DigestCredential('u', 'p', ['SHA-256' satisfies DigestAlgorithm]), bearer: {provider, marginMs: 5_000}, }, tiers: { @@ -379,8 +379,8 @@ export function narrow(error: unknown): string | undefined { if (error instanceof AuthResolutionError) return error.requiredSchemes?.[0]; return undefined; } -export function credentialSet(set: AuthCredentialSet): BasicCredential | undefined { - return set.basic; +export function credentialSet(set: AuthCredentialSet): string | undefined { + return set.basic?.username; } export function bearerCredential(c: BearerCredential): TokenProvider { return c.provider; diff --git a/tests/node-conformance/auth.test.mjs b/tests/node-conformance/auth.test.mjs index bd48a3b..e29c28c 100644 --- a/tests/node-conformance/auth.test.mjs +++ b/tests/node-conformance/auth.test.mjs @@ -39,6 +39,7 @@ import assert from 'node:assert/strict'; import {getEventListeners} from 'node:events'; import {describe, it} from 'node:test'; import { + BasicCredential, CancellationError, Request, authStep, @@ -216,7 +217,7 @@ describe('challenge response lifecycle over Node Web Streams (AUTH-30/AUTH-31/AU const tiers = { client: createAuthDescriptor([createAuthRequirement('BASIC')]), }; - const credentials = {basic: {username: 'u', password: 'p'}}; + const credentials = {basic: new BasicCredential('u', 'p')}; it('closes the original 401 before re-driving, and leaves the replacement response open', async () => { const challenged = challengeResponse( From 0c06d74f0c1eae73f0a1eca89e3ce2b4125625e7 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:58:50 +0300 Subject: [PATCH 021/100] fix: guard a challenge replay whenever the outbound hop was guarded (#71) `guardReplayScheme` tested two header NAMES -- `Authorization` and `Proxy-Authorization`. `ApiKeyCredentialConfig.headerName` lets this very step stamp any header the caller names, so a `challengeHook` answering a 401 with `X-Api-Key: SECRET` over a downgraded `http://` URL was dispatched in clear text with no `PlaintextCredentialError`. `OutboundPlan` now carries `guarded` -- whether the outbound pass ran `requireHttps` on this hop -- and the replay inherits it: once guarded, always guarded, without inspecting a single header name. Deriving the set of credential-carrying names from configuration was rejected; a hook may invent a carrier this step has never been told about, so no enumeration can be complete, whereas "this hop is credentialed" is a fact the outbound pass already decided. The header test survives as a second arm: a `NO_AUTH` hop is never guarded outbound, and a hook answering its challenge with an `Authorization` header is attaching a credential all the same. XCUT-16's carve-out is preserved -- a genuinely credential-free re-issue over any scheme still proceeds. --- packages/core/src/auth/auth-step.test.ts | 84 +++++++++++++- packages/core/src/auth/auth-step.ts | 74 +++++++++--- .../security-by-default.conformance.test.ts | 107 ++++++++++++++++++ 3 files changed, 249 insertions(+), 16 deletions(-) diff --git a/packages/core/src/auth/auth-step.test.ts b/packages/core/src/auth/auth-step.test.ts index bd70a33..ca4dd71 100644 --- a/packages/core/src/auth/auth-step.test.ts +++ b/packages/core/src/auth/auth-step.test.ts @@ -4,7 +4,9 @@ // lands before any token fetch or header write, and it applies only on the credential-attaching path -- // a marker-suppressed cross-origin re-issue may proceed credential-free over any scheme), // AUTH-27 (exactly one AUTH-stage descriptor, pinned to the pillar), AUTH-28 (HTTPS guard, -// NO_AUTH exempt, re-applied on the replay path), AUTH-29 (the cross-origin marker skips the guard and +// NO_AUTH exempt, re-applied on the replay path -- unconditionally once the outbound pass guarded the +// hop, whatever header names the replacement carries, and additionally on an Authorization or +// Proxy-Authorization the hook adds to an unguarded NO_AUTH hop), AUTH-29 (the cross-origin marker skips the guard and // stamping, is cleared from the outbound headers, and suppresses the challenge reaction too -- so the // credential cannot re-enter via the 401), AUTH-25 (a 407 is answered from Proxy-Authenticate into // Proxy-Authorization), AUTH-30 (401 + WWW-Authenticate invokes the hook; a replacement re-drives @@ -1010,7 +1012,87 @@ describe('authStep: the replay HTTPS guard (AUTH-28)', () => { expect(transport.sendCount).toBe(1); // the replacement never reached the wire expect(challenged.cancelCount()).toBe(1); // and the 401 was closed before the throw, not leaked }); +}); + +describe('authStep: once guarded, the replay stays guarded (AUTH-28)', () => { + test('a replacement carrying a NON-standard credential header over plaintext is refused (AUTH-28)', async () => { + // The reported hole (audit #67 / #71): the guard tested two header NAMES, and + // `ApiKeyCredentialConfig.headerName` lets this very step stamp any header it is told to. A hook + // that downgrades the URL and answers with `X-Api-Key` therefore went out in clear text, with no + // `PlaintextCredentialError`. The rule is now "the outbound pass guarded this hop, so the replay + // is guarded too", which does not depend on reading header names at all. + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: { + apiKey: { + credential: new ApiKeyCredential('secret'), + headerName: 'X-Api-Key', + }, + }, + tiers: tiersFor('API_KEY'), // the outbound pass DID guard: the seed is https + challengeHook: (_response, request) => + Promise.resolve( + request + .newBuilder() + .url('http://example.com/a') + .headers( + request.headers.newBuilder().set('X-Api-Key', 'SECRET').build(), + ) + .build(), + ), + }); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect(transport.sendCount).toBe(1); // the replacement never reached the wire + expect(challenged.cancelCount()).toBe(1); // and the 401 was closed before the throw + }); + + test('a header-free replacement over plaintext is refused too, once the hop was guarded (AUTH-28)', async () => { + // The other half of the same rule, and the reason it is stated as "once guarded, always guarded" + // rather than as a wider header list: a hook is free to invent a credential carrier this step has + // never heard of, so no enumeration of names can be complete. The scheme that made the outbound + // pass credentialed is what the replay inherits. + const challenged = challengeResponse( + 401, + 'WWW-Authenticate', + 'Basic realm="x"', + ); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: { + apiKey: {credential: new ApiKeyCredential('secret')}, + }, + tiers: tiersFor('API_KEY'), + challengeHook: (_response, request) => + Promise.resolve( + Request.newBuilder() + .url('http://example.com/b') + .method(request.method) + .build(), + ), + }); + + const error = await rejectionOf(runThrough(descriptor, transport)); + + expect(error).toBeInstanceOf(PlaintextCredentialError); + expect(transport.sendCount).toBe(1); + }); +}); +describe('authStep: an unguarded NO_AUTH hop (AUTH-28/AUTH-29)', () => { test('a credential-free replacement over plaintext is NOT blocked by the replay guard (AUTH-28/AUTH-29)', async () => { const challenged = challengeResponse( 401, diff --git a/packages/core/src/auth/auth-step.ts b/packages/core/src/auth/auth-step.ts index b06a552..3097161 100644 --- a/packages/core/src/auth/auth-step.ts +++ b/packages/core/src/auth/auth-step.ts @@ -354,6 +354,12 @@ async function preemptiveStamp( */ interface OutboundPlan { readonly crossOrigin: boolean; + /** + * Whether the outbound pass ran {@link requireHttps} on this hop -- i.e. whether this is one of + * AUTH-28's "paths where a credential will be attached". Carried past the dispatch because the + * replay inherits it: see {@link guardReplayScheme}. + */ + readonly guarded: boolean; readonly outbound: Request; } @@ -379,9 +385,14 @@ async function planOutbound( .headers(clearCrossOriginMarker(seedRequest.headers)) .build(); - if (crossOrigin) return {crossOrigin, outbound: cleared}; - if (context.scheme !== 'NO_AUTH') requireHttps(cleared.url, context.scheme); - return {crossOrigin, outbound: await preemptiveStamp(cleared, context)}; + if (crossOrigin) return {crossOrigin, guarded: false, outbound: cleared}; + const guarded = context.scheme !== 'NO_AUTH'; + if (guarded) requireHttps(cleared.url, context.scheme); + return { + crossOrigin, + guarded, + outbound: await preemptiveStamp(cleared, context), + }; } interface ChallengeSelection { @@ -518,24 +529,45 @@ async function defaultChallengeHook( } } +/** What {@link guardReplayScheme} needs. Bundled to stay inside `max-params`. */ +interface ReplayGuardInput { + readonly replacement: Request; + readonly response: Response; + readonly scheme: AuthScheme; + /** {@link OutboundPlan.guarded} for this hop. */ + readonly outboundGuarded: boolean; +} + /** * AUTH-28 on the REPLAY path. The outbound guard is not sufficient here: it is skipped entirely for * `NO_AUTH`, and nothing constrains a caller-supplied hook to preserve the request URL. A replay - * carrying a credential header is by definition "a path where a credential will be attached", and - * AUTH-28 says ANY such path. + * carrying a credential is by definition "a path where a credential will be attached", and AUTH-28 + * says ANY such path. + * + * **Once guarded, always guarded.** When the outbound pass ran the guard, so does the replay -- + * unconditionally, without inspecting a single header name. The rule used to be "the replacement + * carries `Authorization` or `Proxy-Authorization`", and that missed the case this step creates + * itself: `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, so a hook + * answering a 401 with `X-Api-Key: SECRET` over a downgraded `http://` URL went out in clear text + * with no `PlaintextCredentialError` (audit #67 / #71). Deriving the credential-carrying names from + * configuration instead was considered and rejected: a `challengeHook` may invent a carrier this step + * has never been told about, so no enumeration can be complete, whereas "this hop is credentialed" + * is a fact the outbound pass already decided. + * + * The header test survives as a SECOND arm rather than being replaced, because it still reaches + * somewhere the first cannot: a `NO_AUTH` hop is never guarded outbound, and a hook that answers its + * challenge with an `Authorization` header is attaching a credential all the same. * * The challenge response is closed before the throw, for the same reason AUTH-32 closes it on a hook * throw: this is past the point where the caller still owns it, so propagating unclosed leaks the body. */ -async function guardReplayScheme( - replacement: Request, - response: Response, - scheme: AuthScheme, -): Promise { - const carriesCredential = +async function guardReplayScheme(input: ReplayGuardInput): Promise { + const {replacement, response, scheme, outboundGuarded} = input; + const attachesCredential = + outboundGuarded || replacement.headers.has('Authorization') || replacement.headers.has('Proxy-Authorization'); - if (!carriesCredential) return; + if (!attachesCredential) return; try { requireHttps(replacement.url, scheme); } catch (error) { @@ -579,6 +611,8 @@ async function runHook( interface ChallengeDrive { readonly response: Response; readonly outbound: Request; + /** {@link OutboundPlan.guarded}, carried in for {@link guardReplayScheme}. */ + readonly outboundGuarded: boolean; readonly fork: () => (request?: Request) => Promise; readonly settings: AuthStepSettings; readonly hookContext: DefaultHookContext; @@ -594,7 +628,8 @@ interface ChallengeDrive { * failure. */ async function handleChallenge(drive: ChallengeDrive): Promise { - const {response, outbound, fork, settings, hookContext} = drive; + const {response, outbound, outboundGuarded, fork, settings, hookContext} = + drive; const selection = pickChallengeHeader(response); // AUTH-33: no matching challenge header -> unchanged, and the hook is never consulted. @@ -641,7 +676,13 @@ async function handleChallenge(drive: ChallengeDrive): Promise { // a second wire send on a request nobody is waiting for is the one thing that must not happen. if (isAborted(hookContext.signal)) return response; - await guardReplayScheme(replacement, response, hookContext.scheme); // AUTH-28 + // AUTH-28 + await guardReplayScheme({ + replacement, + response, + scheme: hookContext.scheme, + outboundGuarded, + }); await response.close(); // AUTH-30: the original is closed before the replacement is driven. // AUTH-30: exactly once, through a FRESH chain copy, with no further challenge handling on it. @@ -678,6 +719,8 @@ async function handleChallenge(drive: ChallengeDrive): Promise { * @returns the descriptor to install in a pipeline's AUTH slot. * @throws PlaintextCredentialError — as a rejected promise — when the resolved scheme would attach a * credential over a non-HTTPS URL (AUTH-28), on the outbound pass and again on a challenge replay. + * A replay whose hop was guarded outbound is guarded again whatever URL and headers the hook chose, + * so a hook that downgrades the scheme fails here rather than on the wire. * Recover by fixing the endpoint's scheme; retrying will not help. * @throws AuthResolutionError — as a rejected promise — when the selected tier lists no scheme with a * matching configured credential (AUTH-6; AUTH-4 governs only WHICH tier is selected), or when the @@ -750,7 +793,7 @@ export function authStep(settings: AuthStepSettings): StepDescriptor { signal, }; - const {crossOrigin, outbound} = await planOutbound( + const {crossOrigin, guarded, outbound} = await planOutbound( seedRequest, stampContext, ); @@ -769,6 +812,7 @@ export function authStep(settings: AuthStepSettings): StepDescriptor { return handleChallenge({ response, outbound, + outboundGuarded: guarded, fork, settings, hookContext: {...stampContext, composing}, diff --git a/tests/conformance/xcut/security-by-default.conformance.test.ts b/tests/conformance/xcut/security-by-default.conformance.test.ts index dd32f93..977d1a1 100644 --- a/tests/conformance/xcut/security-by-default.conformance.test.ts +++ b/tests/conformance/xcut/security-by-default.conformance.test.ts @@ -18,15 +18,24 @@ // XCUT-19 default-deny redaction-> packages/core/src/observability/redaction.test.ts // XCUT-20 observability never throws -> packages/core/src/observability/logging-step.test.ts // XCUT-21 CSPRNG cnonce -> packages/core/src/auth/digest.test.ts (AUTH-20) +// +// Also exercises: XCUT-16/AUTH-28 on the CHALLENGE-REPLAY path -- a hop the outbound pass guarded +// stays guarded, so a `challengeHook` that answers a 401 by downgrading to `http://` is refused +// whatever header it carries the credential in (audit #67 / #71). import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; import { createAuthDescriptor, createAuthRequirement, createBearerToken, Headers, + NameKeyCredential, PlaintextCredentialError, + Protocol, Request, + Response, + Status, type AuthStepSettings, + type Transport, } from '@dexpace/core'; import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; @@ -159,3 +168,101 @@ describe('XCUT-16: a credential is never stamped over a non-HTTPS transport', () await pipeline.close(); }); }); + +/** + * A transport that answers everything with the same 401 challenge, so the replay path can be driven + * without a TLS fixture. `XCUT-16`'s replay clause needs an outbound hop that is HTTPS — the guard + * cannot have run otherwise — and the plaintext fixture server above cannot provide one. The stubbed + * transport is the same device `error-taxonomy.conformance.test.ts` uses for the inputs a live socket + * cannot produce; everything above the transport is still the real composed pipeline. + */ +class ChallengingTransport implements Transport { + send(request: Request): Promise { + return Promise.resolve( + Response.newBuilder() + .request(request) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(401)) + .headers( + Headers.newBuilder() + .setInbound('WWW-Authenticate', 'Basic realm="x"') + .build(), + ) + .build(), + ); + } + + async close(): Promise { + // Nothing to release: this transport never opens anything. + } +} + +describe('XCUT-16: a guarded hop stays guarded across a challenge replay', () => { + /** `X-Api-Key`, not `Authorization`: the header this step is configured to stamp. */ + function apiKeyAuth(replacement: (request: Request) => Request): { + auth: AuthStepSettings; + transport: Transport; + } { + return { + auth: { + credentials: { + apiKey: { + credential: new NameKeyCredential('x-api-key', 'SECRET'), + headerName: 'X-Api-Key', + }, + }, + tiers: { + operation: createAuthDescriptor([createAuthRequirement('API_KEY')]), + }, + challengeHook: (_response, request) => + Promise.resolve(replacement(request)), + }, + transport: new ChallengingTransport(), + }; + } + + test('refuses a replacement that downgrades to http:// and carries the key in X-Api-Key', async () => { + // The reported hole: the replay guard tested two header NAMES, and neither of them is the one + // `ApiKeyCredentialConfig.headerName` told this step to stamp. The credential went out in clear + // text with the whole suite green. + const {auth, transport} = apiKeyAuth(request => + request + .newBuilder() + .url('http://example.com/echo') + .headers( + request.headers.newBuilder().set('X-Api-Key', 'SECRET').build(), + ) + .build(), + ); + const pipeline = buildComposedPipeline({auth, transport}); + + const pending = pipeline.runtime.send( + Request.newBuilder().url('https://example.com/echo').build(), + ); + + expect(await rejectionOf(pending)).toBeInstanceOf(PlaintextCredentialError); + // One dispatch: the guarded outbound pass. The replay never reached the transport. + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); + + test('refuses a downgraded replacement even with no credential header on it at all', async () => { + // The rule is "this hop was guarded", not "this replacement looks credentialed" — a hook is free + // to invent a carrier no enumeration of header names would know to look for. + const {auth, transport} = apiKeyAuth(request => + Request.newBuilder() + .url('http://example.com/echo') + .method(request.method) + .build(), + ); + const pipeline = buildComposedPipeline({auth, transport}); + + const pending = pipeline.runtime.send( + Request.newBuilder().url('https://example.com/echo').build(), + ); + + expect(await rejectionOf(pending)).toBeInstanceOf(PlaintextCredentialError); + expect(pipeline.dispatches()).toBe(1); + await pipeline.close(); + }); +}); From d5acd9ed094ec6e22c64df5848bf273855dd7dc5 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:58:56 +0300 Subject: [PATCH 022/100] docs: rewrite auth.md's credential shape and ledger the two AUTH-8/XCUT-16 readings (#71) `auth.md`'s `AuthCredentialSet` sketch still showed `{username, password}` object literals directly above the claim that a credential cannot leak into a log line by accident, which was the false half of the pair the audit found. It now shows the five classes, states that there is no `password` property to read back and why, and documents the "once guarded, always guarded" replay rule under the HTTPS-guard heading. Two rows appended to `deviations.md` under "Deviations recorded outside a phase": AUTH-8's redaction clause read as covering every credential type rather than the three it enumerates, and XCUT-16's replay guard keyed on whether the hop was guarded rather than on whether the replacement looks credentialed -- the second is knowingly wider than the requirement's letter. --- docs/deviations.md | 2 ++ docs/sdk-documentation/auth.md | 41 ++++++++++++++++++++++++++++------ 2 files changed, 36 insertions(+), 7 deletions(-) diff --git a/docs/deviations.md b/docs/deviations.md index b8ee6b2..885fe34 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -478,6 +478,8 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **`BODY-34`'s one shared preview cap covers the two logging tees only, not `toHttpError`'s error-body capture.** Read literally as "every in-memory capture in the package", `BODY-34` would put the request-side tee, the response-side drain and `toHttpError`'s error buffer behind one configurable value. They cannot share one: `HTTP-52` **fixes** the error-body cap at 1 MiB, and a spec-fixed constant cannot also be the configurable shared setting. The two capture sites `BODY-34` actually names — the request-side tee-capture and the response-side drain-on-first-access — do share one cap, so the requirement's own enumeration is satisfied; only the wider reading is not. `http-status-error.ts` states the split at the constant rather than leaving it to be inferred. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/http-status-error.ts:17-19` (`ERROR_BODY_CAP_BYTES`, with the "Deliberately NOT BODY-34's shared preview cap" note); `packages/core/src/body/response-body-logging.ts:47` and `packages/core/src/body/response-body-logging.ts:105-115` (the shared `cap` and the bounded drain that honours it); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:447` | not yet in §10 | | **`IO-38`'s cross-thread close visibility has no subject on this platform and is recorded as not applicable, not as satisfied.** The requirement presupposes that a source or buffer instance can reach a second thread, so that closing it there invalidates a slice being read here. None can. Class instances are not structured-cloneable at all — `postMessage`/`structuredClone` preserve neither prototypes nor `#private` fields, so a `ByteQueue` or `BufferedSource` sent to a worker arrives as a plain object with no methods and no close state to observe. `BufferedSource` is doubly excluded: it holds a `ReadableStreamDefaultReader`, which is neither cloneable nor transferable. A raw `ArrayBuffer` *can* be transferred, but it carries no close state and derives no slices, so the hazard has no subject there either. This row exists because "not applicable" and "done" look identical in a checklist and are not the same claim. **Recorded in the Phase 3a design's ledger since 2026-07-24 and nowhere else** | audit #67 / #69 | 2026-09-04 | `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md:401`; `packages/core/src/io/buffered-source.ts:47` (the reader it holds); no `worker_threads`, `postMessage` or `structuredClone` call exists anywhere in `packages/core/src/` | not yet in §10 | | **The two shipped transports disagree on `HTTP-6`'s optional reason phrase: `transport-fetch` sets it, `transport-undici` leaves it `undefined`.** `HTTP-6` (MUST) has a response carry "an optional reason phrase", and the domain model provides the slot. `fetch-transport` fills it from the WHATWG `Response.statusText`, normalizing the empty string to `undefined`. `undici-transport` does not call `.reasonPhrase()` at all, because undici's `ResponseData` carries `statusCode` and no phrase — HTTP/2 has no reason phrase and undici's parser does not surface HTTP/1.1's. **Platform-shaped, and it sits beside §10 item 13's `Protocol.HTTP_1_1` gap rather than under it.** Item 13 already ledgers the negotiated-version gap in both adapters for the same reason — the value is not readable — but it does not name the reason phrase, so this row does. The field is optional in the requirement, and no shipped step or conformance row reads it, so the disagreement is observable to a caller and to nothing else. Recorded rather than papered over: synthesizing a phrase from the status code in the undici adapter would report a value the server never sent | audit #67 / #69 | 2026-09-04 | `packages/transport-fetch/src/fetch-transport.ts:180` (`.reasonPhrase(...)` fed from the WHATWG `statusText`, with the empty string normalized to `undefined`) against `packages/transport-undici/src/undici-transport.ts:342-350` (the builder chain, with no `.reasonPhrase` call); `docs/product-spec/04-core-http-domain-model.md:13`; the neighbouring gap is item 13 of the audit above | not yet in §10 | +| **`AUTH-8`'s redaction clause is read as covering EVERY credential type, not the three it enumerates.** The requirement's own list is "API key, name-key secret, bearer token", and it was implemented to the letter: `ApiKeyCredential`, `NameKeyCredential` and `BearerToken` shipped as classes with a `#private` secret, a redacted `toString()` and the `nodejs.util.inspect.custom` hook, while the Basic and Digest credentials shipped as structural interfaces with a public `readonly password: string`. That is a leak the same requirement's first four words forbid: `util.inspect` of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and `JSON.stringify` serialized both passwords. **Widened, deliberately.** Both are now classes on the same pattern, with the password reachable only through the in-package `credentialPassword()` friend hook. The cost is a public-shape change — `{username, password}` object literals no longer type-check — taken now because it is free before the first version bump. Validation is deliberately NOT duplicated onto the classes: AUTH-14's non-empty-whitespace-permitted rule and AUTH-16's acceptable-set rule stay single-sourced in `basicHandler()`/`digestHandler()`, which `authStep()` builds at construction, so a blank password still fails synchronously from that factory | audit #67 / #71 | 2026-09-04 | `docs/product-spec/11-authentication.md:12` is AUTH-8's three-type enumeration. `packages/core/src/auth/credential.ts:342` (`BasicCredential`) and `:393` (`DigestCredential`), with the friend hooks at `:299-300` and `credentialPassword()` at `:314`; the sole reader is `buildHandlers` in `packages/core/src/auth/auth-step.ts:131-152`. Pinned by "a whole AuthCredentialSet is diagnostic-safe (AUTH-8)" in `packages/core/src/auth/credential.test.ts`, which drives the real `util.inspect` rather than the hook it calls | not yet in §10 | +| **`XCUT-16`'s replay guard is keyed on whether the hop was guarded, not on whether the replacement looks credentialed.** `XCUT-16` and `AUTH-28` say the guard applies "on any path where a credential will be attached", and carve out "a deliberately credential-free re-issue MAY proceed over any scheme". Deciding which of the two a challenge replacement is cannot be done by reading header names: the step's own `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, and a `challengeHook` may invent a carrier this step has never been told about. The port therefore reads "a credential will be attached" as a property of the HOP — if the outbound pass ran the HTTPS guard, so does the replay, whatever URL and headers the hook chose. **Strictly wider than the requirement's letter**, and knowingly so: it refuses a downgraded replacement that carries no credential at all, on a hop that is credentialed. The carve-out is preserved where it is observable — a `NO_AUTH` hop is never guarded outbound, and its replay is guarded only when the replacement carries `Authorization` or `Proxy-Authorization`, which is the previous rule kept as a second arm. *Rejected:* deriving the credential-carrying header names from configuration, which misses the hook-invented carrier and is the shape that let the reported leak through | audit #67 / #71 | 2026-09-04 | `docs/product-spec/19-cross-cutting-invariants-and-policies.md:44` is the requirement and its carve-out. `packages/core/src/auth/auth-step.ts:389` sets `OutboundPlan.guarded`; `:564-575` is `guardReplayScheme` and its two arms. Pinned by "a replacement carrying a NON-standard credential header over plaintext is refused" and "a header-free replacement over plaintext is refused too" in `packages/core/src/auth/auth-step.test.ts`, and by the "XCUT-16: a guarded hop stays guarded across a challenge replay" block in `tests/conformance/xcut/security-by-default.conformance.test.ts` | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) diff --git a/docs/sdk-documentation/auth.md b/docs/sdk-documentation/auth.md index 2b5b32d..bc997e1 100644 --- a/docs/sdk-documentation/auth.md +++ b/docs/sdk-documentation/auth.md @@ -43,8 +43,8 @@ The factories validate and freeze (`AUTH-3`). A descriptor containing a `NO_AUTH ```typescript interface AuthCredentialSet { readonly bearer?: {provider: TokenProvider; marginMs?: number}; - readonly basic?: {username: string; password: string}; - readonly digest?: {username: string; password: string; algorithmPreference?: DigestAlgorithm[]}; + readonly basic?: BasicCredential; + readonly digest?: DigestCredential; readonly apiKey?: {credential: ApiKeyCredential | NameKeyCredential; headerName?: string; prefix?: string}; } ``` @@ -53,18 +53,37 @@ The five schemes are `OAUTH2`, `API_KEY`, `BASIC`, `DIGEST` and `NO_AUTH`. A req scheme; the credential set supplies the material for it. A requirement with no matching credential is an `AuthResolutionError` at send time, not a silent unauthenticated request. -**`ApiKeyCredential`, `NameKeyCredential` and `BearerToken` are nominal, not structural.** Each carries -a `#private` field, so no caller-side object literal is assignable to them and the validation in each -factory cannot be routed around. They also override `toString()` and Node's inspect symbol, so a -credential cannot leak into a log line or a stack trace by accident. +**Every credential type is nominal, not structural.** `ApiKeyCredential`, `NameKeyCredential`, +`BearerToken`, `BasicCredential` and `DigestCredential` each carry a `#private` field, so no +caller-side object literal is assignable to them and the validation behind each cannot be routed +around. All five override `toString()` and Node's inspect symbol, so a credential cannot leak into a +log line or a stack trace by accident. ```typescript -import {ApiKeyCredential, createBearerToken} from '@dexpace/core'; +import { + ApiKeyCredential, + BasicCredential, + createBearerToken, + DigestCredential, +} from '@dexpace/core'; + +const credentials = { + basic: new BasicCredential('alice', 'super-secret'), + digest: new DigestCredential('bob', 'super-secret', ['SHA-256', 'MD5']), + apiKey: {credential: new ApiKeyCredential('super-secret'), headerName: 'X-Api-Key'}, +}; String(new ApiKeyCredential('super-secret')); // 'ApiKeyCredential{key=***}' String(createBearerToken('t', 1)); // 'BearerToken{token=***, expiresAt=1}' — the expiry survives +String(credentials.basic); // 'BasicCredential{username=alice, password=***}' ``` +There is no `password` property to read back, by design: `BasicCredential` and `DigestCredential` +shipped as plain `{username, password}` records until 2026-09-04, and a plain property is reachable +through `credential['password']`, `Object.keys`, `JSON.stringify` and a default `util.inspect` — so +`util.inspect(credentials)` printed the password in clear beside `ApiKeyCredential{key=***}`. The +username and the Digest algorithm preference stay visible; `AUTH-8` permits non-secret fields to. + The inspect symbol matters as much as `toString`: `console.log(credential)` and `util.inspect` do not route an object argument through `toString`, so both are overridden (`AUTH-8`). @@ -106,6 +125,14 @@ request is dispatched (`AUTH-28`). There is no option to disable it. `NO_AUTH` n is why an auth-less `standardResilience()` installs a `NO_AUTH`-only step rather than no step at all — the pillar slot stays filled and the behaviour stays uniform. +**Once guarded, always guarded.** A `challengeHook` may return any request it likes, including one +whose URL has been downgraded to `http://`. When the outbound pass ran the guard, the replay is +guarded too — unconditionally, without inspecting a single header name, because +`ApiKeyCredentialConfig.headerName` lets this step stamp a header no fixed list would contain. On a +`NO_AUTH` hop, which is never guarded outbound, a replacement carrying `Authorization` or +`Proxy-Authorization` still trips the guard. A genuinely credential-free re-issue over `http://` is +what `XCUT-16` explicitly permits, and it still proceeds. + ## Challenges ```typescript From 60174dc6b98259f92f78ce36955b2d22dad46a18 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Fri, 4 Sep 2026 23:59:05 +0300 Subject: [PATCH 023/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20record=20t?= =?UTF-8?q?he=20#70=20outcome=20and=20the=20api-extractor=20link=20trap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index 4300f50..0b3f66e 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -118,9 +118,21 @@ fields and dropping the message (leaves the raw message reachable through `cause - `docs/sdk-documentation/auth.md`'s credential-shape example is rewritten here (#68 left it). *Constrains #74:* it edits `auth-step.ts` next wave on top of this; the guard site moves. +**D8 outcome (2026-09-04, #70, PR #85).** Both messages built from `redactUrl()`; raw URLs stay on +`targetUrl` / `fromUrl` / `toUrl`; `http.redirect.rejected` gained `url.full` (redacted) like the sibling +events. A non-URL string handed to either public constructor now renders `[malformed url]` in the message +(OBS-15 totality read as the safe default; pinned). New fixture route `/redirect-secret-target` in +`tests/conformance/xcut/fixtures/server.ts` (307 to `/echo?access_token=`), reusable. +*Constrains #74:* build any URL-naming message from `redactUrl()` at the constructor. *Constrains #72, #78:* +`docs/sdk-documentation/errors.md` redirect section was edited here. + +**Trap for later subtasks (api-extractor).** `{@link SomeError.message}` does not resolve (`message` is +inherited from `Error`; `ae-unresolved-link`). Write it as backticked prose. + ## Deferred — release machinery (recoverable list) | Issue / PR | Deferred item | |---|---| | #68 / PR #83 | patch notes for `@dexpace/core` and `@dexpace/codec-json`: shipped `.d.ts` prose changed for `Deserializer`, `jsonSerde()`, `InstrumentationBundle.tracerFactory`, `buildRequest`, `RequestConditions.applyTo` | | #68 / PR #83 | `docs/first-release.md:117` claims `serde.ts:99,170` cite `H15` — `H15` appears nowhere in `serde.ts`; `:159` puts `deserializeFrom` at `:162` and `serializeTo` at `:96` (actual `:221`, `:104`). File suspended under D1 | +| #70 / PR #85 | patch changeset for `@dexpace/core`: redirect error messages now carry redacted URLs (and `[malformed url]` for unparseable input); `http.redirect.rejected` gained `url.full` | | #69 / PR #84 | patch changeset for `@dexpace/core`: `noopInstrumentationBundle.activeSpan` changed from `undefined` to `NOOP_SPAN` (documented default of a `@public` interface) | From fc33e90f2b9690e04e99b949350e38bcda369f57 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 00:03:26 +0300 Subject: [PATCH 024/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20record=20t?= =?UTF-8?q?he=20#71=20outcome?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index 0b3f66e..268d438 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -126,6 +126,20 @@ events. A non-URL string handed to either public constructor now renders `[malfo *Constrains #74:* build any URL-naming message from `redactUrl()` at the constructor. *Constrains #72, #78:* `docs/sdk-documentation/errors.md` redirect section was edited here. +**D9 outcome (2026-09-05, #71, PR #86).** `BasicCredential` / `DigestCredential` are classes in +`auth/credential.ts` with `#password`, read inside the package through an `@internal` `credentialPassword()` +friend hook; `DigestCredential` takes `algorithmPreference` as a third positional. No construction-time +validation on the classes — AUTH-14/AUTH-16 stay single-sourced in `basicHandler()` / `digestHandler()`, +which `authStep()` builds at construction, so a blank password still fails there. Replay guard keys on +`OutboundPlan.guarded` ("once guarded, always guarded"); `guardReplayScheme` now takes a `ReplayGuardInput` +bundle. Two `deviations.md` rows: AUTH-8 widened to every credential type; XCUT-16 guard deliberately wider +than its letter. `scripts/verify-consumer-types.mjs`'s fixture changed because no structural +`BasicCredential` shape exists any more. *Constrains #74:* `auth-step.ts` conflict surface is the +`./credential.js` import block, `buildHandlers`, `OutboundPlan`/`planOutbound`, `guardReplayScheme`, +`ChallengeDrive`; `credential.ts` now has a type-only import of `./digest.js`, so a new edge from +`digest.ts` back into `credential.ts` closes a cycle. A stubbed `ChallengingTransport` exists in +`security-by-default.conformance.test.ts` for clauses needing an `https://` hop. + **Trap for later subtasks (api-extractor).** `{@link SomeError.message}` does not resolve (`message` is inherited from `Error`; `ae-unresolved-link`). Write it as backticked prose. @@ -135,4 +149,5 @@ inherited from `Error`; `ae-unresolved-link`). Write it as backticked prose. | #68 / PR #83 | patch notes for `@dexpace/core` and `@dexpace/codec-json`: shipped `.d.ts` prose changed for `Deserializer`, `jsonSerde()`, `InstrumentationBundle.tracerFactory`, `buildRequest`, `RequestConditions.applyTo` | | #68 / PR #83 | `docs/first-release.md:117` claims `serde.ts:99,170` cite `H15` — `H15` appears nowhere in `serde.ts`; `:159` puts `deserializeFrom` at `:162` and `serializeTo` at `:96` (actual `:221`, `:104`). File suspended under D1 | | #70 / PR #85 | patch changeset for `@dexpace/core`: redirect error messages now carry redacted URLs (and `[malformed url]` for unparseable input); `http.redirect.rejected` gained `url.full` | +| #71 / PR #86 | minor changeset for `@dexpace/core`: `BasicCredential`/`DigestCredential` become classes (breaking for object-literal callers); patch note for `authStep`'s `@throws PlaintextCredentialError` prose; `docs/first-release.md` untouched though this is its "free before the first bump" class | | #69 / PR #84 | patch changeset for `@dexpace/core`: `noopInstrumentationBundle.activeSpan` changed from `undefined` to `NOOP_SPAN` (documented default of a `@public` interface) | From 4487660b2bf4b40b8d003f3b9e9cea1ec6e21da8 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 00:09:53 +0300 Subject: [PATCH 025/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20wave=202?= =?UTF-8?q?=20landed,=20run=20paused=20before=20wave=203?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index 268d438..a09f9d5 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -95,6 +95,12 @@ PR #83 (#68) and PR #84 (#69) merged into the umbrella at `840f355`. One conflic `deviations.md`): kept #68's "in progress, see #80" text and carried #69's moved `span.ts` citation. Merged tree preflighted before the merge in a throwaway worktree (byte-identical result): all 20 steps passed. +## Wave 2 — landed 2026-09-05 +PR #85 (#70) and PR #86 (#71) merged into the umbrella. One conflict (the import list of +`tests/conformance/xcut/security-by-default.conformance.test.ts`), unioned. Merged tree preflighted in a +throwaway worktree before the merge (byte-identical result): all 20 steps passed. Run paused here by the +maintainer; wave 3 (#72, #74, #75) not yet cut. + ## Decisions taken for wave 2 (M2) ### D8 — #70: redact inside the error messages, and keep the raw URLs on the error properties From cc27e81262368ab9c6c0629ef1d6d7685f069706 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:07:02 +0300 Subject: [PATCH 026/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20pre-take?= =?UTF-8?q?=20wave=203=20decisions=20D10=E2=80=93D12=20and=20the=20file=20?= =?UTF-8?q?partition?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 86 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index a09f9d5..157f3b6 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -149,6 +149,92 @@ than its letter. `scripts/verify-consumer-types.mjs`'s fixture changed because n **Trap for later subtasks (api-extractor).** `{@link SomeError.message}` does not resolve (`message` is inherited from `Error`; `ae-unresolved-link`). Write it as backticked prose. +## Decisions taken for wave 3 (M3) — pre-taken 2026-09-05, before dispatch + +### D10 — #72: the final typed error is surfaced as-is; the trail rides in a side table, read through `retryAttempts()` +The surfaced error of `retryStep` / `dispatchWithRetry` is the final attempt's own error, class untouched: +`instanceof TransportFailureError` holds for `maxAttempts` 1 and 3 alike, and an abort during backoff surfaces +the `CancellationError` that `abortToSdkError` built (`retry/engine.ts:385`), which `withTrail` at `:386` was +undoing. Earlier attempts' errors are reachable through a new `@public` accessor exported from core, +`retryAttempts(error: unknown): readonly unknown[]` — oldest first, the surfaced instance itself excluded +(RETRY-34's skip-self clause), `[]` for an error that carries no trail — backed by a module-private `WeakMap` +that the engine writes once per terminal failure. **Not a deviation, a correction:** RETRY-34 says the prior +failures are "attached to the surfaced exception as suppressed", which is Java's `addSuppressed` — the +surfaced exception stays what it is and grows a list. Wrapping it in `SuppressedError` made the surfaced +*type* a function of how many attempts ran, which is what XCUT-1's "assert the surfaced error is the +cancellation type" clause catches. No `deviations.md` row. `suppress()` stays for its RECOV-12 job. +*Rejected:* an own property (`attempts` / `errors` / `suppressed`) defined on the surfaced error — a foreign +error may be frozen or non-extensible, so `defineProperty` in the engine's failure path can itself throw; a +primitive thrown value cannot carry one at all; and `.suppressed` already means "the one secondary" on +`SuppressedErrorLike`. *Rejected:* a `RetryExhaustedError` wrapper (hides `CancellationError`, the row XCUT-1 +is about). *Rejected:* threading the trail through `cause` (`cause` is already the raw abort reason at +`:383`, and it means "why", not "before"). A primitive surfaced value is passed through unchanged with no +trail entry rather than wrapped. Update the `retryStep` TSDoc, `retry-dispatch.ts:45`'s `@throws` prose, +`docs/sdk-documentation/pipelines.md`'s retry section, the "suppressed trail" wording at +`docs/sdk-documentation/errors.md:188`, and `write-a-response-handler.md` so the RECOV-12 wrapper is documented +as the *only* place a `SuppressedError` is built. `api:local` on core. *Constrains #78:* the classify +cause-walk sees the typed error directly now, never through a `SuppressedError.error` hop. *Constrains #73:* +the trail accessor is the shape it layers on. + +### D11 — #74: parse every challenge header; emit `cnonce` for `-sess` regardless of `qop`; empty `realm`/`nonce` are unsatisfiable +- **Repeated `WWW-Authenticate` / `Proxy-Authenticate`.** `pickChallengeHeader` reads `headers.getAll(name)` + and parses each value with `parseChallenges`, concatenating the lists in wire order — parse-each rather + than comma-join, so a malformed later value cannot poison the parse of an earlier one. `rank` selects across + the concatenation. Conformance row in `packages/transport-conformance` (`run-suite.ts` + `fixtures.ts`): a + fixture route sending two `WWW-Authenticate` headers, asserting the *parsed challenge list* is identical + through both transports — `getAll` legitimately returns one comma-joined entry through fetch and two entries + through undici, and the list after parsing is the only thing the transport is answerable for. Fix the + `Set-Cookie`-only comment at `undici-transport.ts:334`; touch nothing else in that file (#81 owns it). +- **`-sess` without `qop`: emit `cnonce`.** RFC 7616 §3.4 says of `cnonce` "This parameter MUST be used by all + implementations", and §3.4.2 folds it into A1 for every `-sess` algorithm; a `-sess` response without it is + unverifiable by construction, which is what the port sends today (`digest.ts:337-343` hashes a cnonce the + header at `:386-387` omits). `nc` and `qop` stay conditional on a negotiated `qop`. AUTH-22's "emit + cnonce/nc/qop only when qop is negotiated" is RFC 2617's RFC 2069-compatibility form, which predates + `-sess`. Departure from AUTH-22's letter: one `deviations.md` row (D0). *Rejected:* declining the challenge — + it turns every `-sess`-without-`qop` server into a guaranteed 401 for no security gain, and the value is + already computed. `computeDigestResponse` vector for `MD5-sess` with no `qop`. +- **Empty `realm` or `nonce`** is unsatisfiable: `parseDigestChallenge` requires non-empty strings, the + challenge is declined and the next one tried (AUTH-25's "return no header when it cannot satisfy any"). No + row; AUTH-12's verbatim storage is unchanged, the check sits at selection. +- No new core exports. `api:local` on core only if a `@public` TSDoc changes. No changeset (D1). +*Constraints inherited:* D9 (the `auth-step.ts` surface #71 reshaped: `buildHandlers`, `OutboundPlan`, +`planOutbound`, `guardReplayScheme`, `ChallengeDrive`; `credential.ts` imports `./digest.js` type-only, so no +edge from `digest.ts` back into `credential.ts`); D8 (any URL-naming message is built from `redactUrl()`). + +### D12 — #75: keep the ownership transfer, and ledger it +`sseEvents$` / `typedSse$` keep passing `() => stream.close()` as `fromAsyncIterable`'s `release`. The +issue's "spec-faithful one-line change" is neither: (1) `SseStream` self-releases on **any** iterator +termination by SSE-30's own design — `#iterate`'s `finally` runs `#releaseQuietly()` when the runtime calls +`return()` (`packages/core/src/sse/stream.ts:136-138`), and `fromAsyncIterable` must call `iterator.return()` +exactly once (ASYNC-6), so the socket closes with or without the callback; a `for await` with `break` closes it +the same way. ASYNC-21's "MUST NOT close the caller-owned source" presumes a source whose iterator return does +not release, which this port's `SseStream` deliberately is not. (2) The release-*before*-`return()` ordering +is what settles an in-flight pull on unsubscribe (`packages/rx/src/from-async-iterable.ts:44-48`): an async +generator's `return()` queues behind a suspended `next()`, so dropping the callback would leave an unsubscribe +during a stalled read pending until the server sends a byte. Pagination passes no release because its pulls +are bounded HTTP exchanges; SSE's are not. Removing the callback would change only the failure channel and the +ordering, not whether the source closes — and it would reintroduce the hang. **Recorded as a deviation** from +ASYNC-21's non-closing clause: one `deviations.md` row (D0) naming `sse.ts:35,57-59`, `from-async-iterable.ts:103-108`, +the two reasons above, and the Phase 8b checklist gist at +`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67` that dropped the clause (a +dated record; not retro-edited). Tests in `packages/rx/src/sse.test.ts`: early unsubscribe, source error, and +end-of-source each close the underlying resource **exactly once** (count the resource's `close`, not the +facade's — `SseStream.close()` is idempotent by SSE-28, so the facade count proves nothing); plus the +unsubscribe-during-suspended-pull case settles. TSDoc on both functions states the transfer outright +("subscribing hands the stream to the adapter; do not call `close()` yourself, and do not iterate it +afterwards"), and `packages/rx/README.md` says the same beside its `for await` guidance. `api:local` on rx. +*Rejected:* dropping `release` (above). *Rejected:* a caller-facing `{ownership}` option (two behaviours to +document for a case with one correct answer). No changeset (D1). + +### Wave 3 partition +| Task | Owns | Shared, append-only | +|---|---|---| +| #72 | `packages/core/src/retry/**`, the retry exports in `packages/core/src/index.ts`, `packages/core/etc/core.api.md` (retry names), `docs/sdk-documentation/pipelines.md` retry section, `errors.md:188`, `write-a-response-handler.md`, `tests/conformance/xcut/retry-safety.*` and `cancellation-and-timeout.*` | — | +| #74 | `packages/core/src/auth/{digest,auth-step,challenge}.ts` and tests, `undici-transport.ts:334` comment only, `packages/transport-conformance/src/{run-suite,fixtures}.ts`, `docs/sdk-documentation/auth.md` | `docs/deviations.md` (one row at table end) | +| #75 | `packages/rx/**` | `docs/deviations.md` (one row at table end) | +Known merge seams: the `deviations.md` table tail (#74 + #75), and `core.api.md` if #74 changes a `@public` +TSDoc beside #72's new export. Supervisor resolves both, as in waves 1 and 2. + ## Deferred — release machinery (recoverable list) | Issue / PR | Deferred item | |---|---| From 034a20c452896c7de763344592e9f915d495f252 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:13:03 +0300 Subject: [PATCH 027/100] fix(auth): read every value of the challenge header, parsed per value pickChallengeHeader called headers.get(), which returns the FIRST value only. A 401 whose first WWW-Authenticate line offers an algorithm this client does not support and whose second offers one it does ended with no Authorization at all, while the identical pair comma-joined into one value authenticated -- so whether a server's RFC 7616 3.3 algorithm-discovery challenge worked depended on which transport delivered it (@dexpace/transport-fetch joins repeated values, @dexpace/transport-undici keeps them apart; RFC 9110 5.3 permits both). It now reads getAll() and parses each value on its own, concatenating the challenge lists in wire order. Parse-each rather than comma-join because parseChallenges' AUTH-13 recovery is bounded by the string it is handed: an unterminated quoted string terminates at end-of-input, so joining lets a malformed earlier value swallow a satisfiable later one. Audit #67 / #74. AUTH-12, AUTH-13, AUTH-16, AUTH-25. --- packages/core/src/auth/auth-step.test.ts | 103 ++++++++++++++++++++++- packages/core/src/auth/auth-step.ts | 40 +++++++-- 2 files changed, 133 insertions(+), 10 deletions(-) diff --git a/packages/core/src/auth/auth-step.test.ts b/packages/core/src/auth/auth-step.test.ts index ca4dd71..85eeb08 100644 --- a/packages/core/src/auth/auth-step.test.ts +++ b/packages/core/src/auth/auth-step.test.ts @@ -9,7 +9,9 @@ // Proxy-Authorization the hook adds to an unguarded NO_AUTH hop), AUTH-29 (the cross-origin marker skips the guard and // stamping, is cleared from the outbound headers, and suppresses the challenge reaction too -- so the // credential cannot re-enter via the 401), AUTH-25 (a 407 is answered from Proxy-Authenticate into -// Proxy-Authorization), AUTH-30 (401 + WWW-Authenticate invokes the hook; a replacement re-drives +// Proxy-Authorization), AUTH-12/AUTH-13 (EVERY value of the matching challenge header is parsed, each +// value on its own, so a repeated WWW-Authenticate/Proxy-Authenticate offers its later challenges too +// and a malformed earlier value cannot swallow them), AUTH-30 (401 + WWW-Authenticate invokes the hook; a replacement re-drives // exactly once through a fresh fork()), AUTH-31 (a non-replayable replacement body surfaces the // original challenge unchanged and unclosed), AUTH-32 (a throwing hook closes the challenge response // before propagating), AUTH-33 (no matching challenge header, or a hook yielding nothing -> unchanged), @@ -127,6 +129,27 @@ function challengeResponse( return {response, cancelCount: base.cancelCount}; } +/** + * A challenge response carrying the same challenge header SEVERAL times — the wire shape RFC 7616 + * §3.3 recommends for algorithm discovery, and the one `@dexpace/transport-undici` hands over as + * separate entries rather than one comma-joined value. + * + * `addInbound` in a loop, never `setInbound`: `set` REPLACES, so building the fixture with it would + * quietly assert against a single-valued header and the row would pass against the very bug it exists + * to catch. + */ +function repeatedChallengeResponse( + status: number, + headerName: string, + headerValues: readonly string[], +): {response: Response; cancelCount: () => number} { + const base = countingResponse(status); + const builder = base.response.headers.newBuilder(); + for (const value of headerValues) builder.addInbound(headerName, value); + const response = base.response.newBuilder().headers(builder.build()).build(); + return {response, cancelCount: base.cancelCount}; +} + /** A one-shot request body: `StreamBody.replayable` is `false` (AUTH-31's gate). */ function oneShotPost(url = 'https://example.com/a'): Request { const stream = new ReadableStream({ @@ -612,6 +635,84 @@ describe('authStep: answering a Digest challenge (AUTH-15..AUTH-22)', () => { }); }); +describe('authStep: repeated challenge headers (AUTH-12/AUTH-16/AUTH-25)', () => { + test('a later WWW-Authenticate entry is answered when the first is unsupported', async () => { + // RFC 7616 §3.3's algorithm-discovery shape: one header per algorithm, strongest first. Reading + // only `headers.get(...)` saw the SHA-512-256 line, found nothing satisfiable, and surfaced the + // 401 — while the identical pair comma-joined into ONE value authenticated (audit #67 / #74). + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="r", nonce="n", algorithm=SHA-512-256', + 'Digest realm="r", nonce="n", algorithm=SHA-256, qop="auth"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + const value = transport.calls[1]?.request.headers.get('Authorization'); + expect(value).toContain('algorithm=SHA-256'); + expect(value).toContain('qop=auth'); + }); + + test('a malformed FIRST entry cannot swallow a satisfiable later one (AUTH-13)', async () => { + // The row that fixes the parse strategy rather than only the read. Comma-joining the two values + // before parsing lets the unterminated quoted string in the first run on into the second — the + // scanner closes it at the `"` of `realm="r"`, and the whole satisfiable challenge disappears + // into a realm value. Parsing each value on its own bounds the damage at the value that carries + // it, which is what AUTH-13's "recovers to the next top-level comma" is reaching for. + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="unterminated', + 'Digest realm="r", nonce="n", qop="auth"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect(transport.calls[1]?.request.headers.get('Authorization')).toContain( + 'realm="r"', + ); + }); + + test('a repeated Proxy-Authenticate is read the same way (AUTH-25)', async () => { + const challenged = repeatedChallengeResponse(407, 'Proxy-Authenticate', [ + 'Negotiate abc123', + 'Basic realm="p"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {basic: new BasicCredential('u', 'p')}, + tiers: tiersFor('BASIC'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect( + transport.calls[1]?.request.headers + .get('Proxy-Authorization') + ?.startsWith('Basic '), + ).toBe(true); + }); +}); + describe('authStep: the replayability gate (AUTH-31)', () => { test('an unsatisfiable challenge leaves the response unchanged (AUTH-25/AUTH-33)', async () => { const challenged = challengeResponse( diff --git a/packages/core/src/auth/auth-step.ts b/packages/core/src/auth/auth-step.ts index 3097161..042eadf 100644 --- a/packages/core/src/auth/auth-step.ts +++ b/packages/core/src/auth/auth-step.ts @@ -396,7 +396,8 @@ async function planOutbound( } interface ChallengeSelection { - readonly value: string; + /** Every value the matching challenge header carried, in wire order. */ + readonly values: readonly string[]; readonly isProxy: boolean; } @@ -404,16 +405,37 @@ interface ChallengeSelection { * AUTH-25: a 401 is answered from `WWW-Authenticate`, a 407 from `Proxy-Authenticate`. Reading only * the header that matches the status keeps the pairing honest — a 401 carrying a stray * `Proxy-Authenticate` must not produce a `Proxy-Authorization`, and vice versa. + * + * `getAll`, not `get`. RFC 9110 §5.3 lets a server send a list-valued field either comma-joined into + * one line or repeated across several, and RFC 7616 §3.3 recommends the repeated form for Digest + * algorithm discovery — one challenge per algorithm, strongest first. `get()` returns the FIRST value + * only, so a 401 offering `algorithm=SHA-512-256` (unsupported here) on line one and + * `algorithm=SHA-256, qop="auth"` on line two ended with no `Authorization` at all, while the same two + * challenges comma-joined authenticated (audit #67 / #74). Which shape reaches this step is the + * transport's accident, not the server's intent: `@dexpace/transport-fetch` joins repeated values, + * `@dexpace/transport-undici` keeps them apart, and both are legal. */ function pickChallengeHeader( response: Response, ): ChallengeSelection | undefined { - if (response.status.code === 401) { - const www = response.headers.get('WWW-Authenticate'); - return www === undefined ? undefined : {value: www, isProxy: false}; - } - const proxy = response.headers.get('Proxy-Authenticate'); - return proxy === undefined ? undefined : {value: proxy, isProxy: true}; + const isProxy = response.status.code !== 401; + const values = response.headers.getAll( + isProxy ? 'Proxy-Authenticate' : 'WWW-Authenticate', + ); + return values.length === 0 ? undefined : {values, isProxy}; +} + +/** + * Every challenge the selected header offered, in wire order (AUTH-12). + * + * Each value is parsed on its OWN, and the lists are concatenated — never joined into one string + * first. `parseChallenges` is total (AUTH-13), but its recovery is bounded by the string it is handed: + * an unterminated quoted string terminates at end-of-input, so joining lets a malformed earlier value + * swallow a satisfiable later one whole. Parsing per value keeps the blast radius of a broken header + * line inside that line. + */ +function challengesOf(selection: ChallengeSelection): readonly Challenge[] { + return selection.values.flatMap(value => parseChallenges(value)); } interface DefaultHookContext { @@ -441,7 +463,7 @@ async function oauth2ChallengeHook( const rejected = request.headers.get(headerName); // AUTH-36: no Authorization on the rejected request -> surface the challenge unchanged. if (rejected === undefined) return undefined; - const challenges: readonly Challenge[] = parseChallenges(selection.value); + const challenges: readonly Challenge[] = challengesOf(selection); if (!challenges.some(challenge => challenge.scheme === 'bearer')) { return undefined; // AUTH-36: the response advertises no Bearer challenge } @@ -487,7 +509,7 @@ async function basicDigestChallengeHook( selection: ChallengeSelection, context: DefaultHookContext, ): Promise { - const challenges = parseChallenges(selection.value); + const challenges = challengesOf(selection); const url = request.url; // HTTP-5: a fresh URL per access, so read it once. const requestTarget = `${url.pathname}${url.search}`; const value = await context.composing.stamp(challenges, { From db430f4a236e85e90d4f01224eaec1348dda860f Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:14:23 +0300 Subject: [PATCH 028/100] fix(auth): emit cnonce for every -sess Digest algorithm, negotiated qop or not A -sess HA1 is H(H(user:realm:pass):nonce:cnonce) (RFC 7616 3.4.2). digestHandler hashed a fresh random cnonce into it and then left the value off the wire whenever the challenge negotiated no qop, so the server had nothing to recompute HA1 from: every MD5-sess/SHA-256-sess challenge without qop produced an unverifiable response, and AUTH-30 bounds the replay to one 401, so the request failed outright. cnonce is now emitted for any -sess algorithm. nc and qop stay conditional on a negotiated qop -- RFC 2069's response input carries no nonce count, so emitting one would advertise a count the response was not computed over. This departs from AUTH-22's letter ("emit cnonce/nc/qop only when qop is negotiated"), which is RFC 2617's RFC 2069-compatibility form and predates -sess. RFC 7616 3.4: "cnonce: This parameter MUST be used by all implementations". Recorded as a docs/deviations.md row. Audit #67 / #74. AUTH-17, AUTH-22. --- packages/core/src/auth/digest.test.ts | 66 ++++++++++++++++++++++++++- packages/core/src/auth/digest.ts | 17 ++++++- 2 files changed, 79 insertions(+), 4 deletions(-) diff --git a/packages/core/src/auth/digest.test.ts b/packages/core/src/auth/digest.test.ts index d887af9..4000448 100644 --- a/packages/core/src/auth/digest.test.ts +++ b/packages/core/src/auth/digest.test.ts @@ -9,7 +9,9 @@ // verified against independently-computed vectors), AUTH-18/AUTH-19 (nonce count: starts at 1, // increments only on nonce reuse, 8 lower-case hex digits, bounded and drained to the cap), // AUTH-20 (client nonce from crypto.getRandomValues, >=128 bits), AUTH-21 (UTF-8 vs ISO-8859-1 by -// charset), AUTH-22 (quoting, and cnonce/nc/qop emitted only when qop negotiated), AUTH-25 +// charset), AUTH-22 (quoting, and nc/qop emitted only when qop negotiated -- with cnonce emitted for +// every -sess algorithm whatever qop says, a recorded departure from AUTH-22's letter: RFC 7616 3.4.2 +// folds cnonce into a -sess HA1, so a server cannot verify a -sess response that omits it), AUTH-25 // (Authorization vs Proxy-Authorization is the CALLER's job -- stamp() returns only the value). import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; @@ -91,6 +93,20 @@ describe('computeDigestResponse (verified against RFC 2617/7616 vectors)', () => ).toBe('8e3825c57e897f5a0dec6c2d4e5059d0'); }); + test('MD5-sess, no qop -- cnonce still folds into HA1 (RFC 7616 3.4.2)', async () => { + // The vector the `-sess`-without-`qop` fix is pinned on. HA1 is + // H(H(user:realm:pass):nonce:cnonce) for every `-sess` algorithm, whatever `qop` the server + // offered, and the response is then RFC 2069's H(HA1:nonce:HA2) because no qop was negotiated. + // Recomputed independently from the same RFC 2617 3.5 inputs the vectors above use. + expect( + await computeDigestResponse({ + ...BASE, + algorithm: 'MD5-sess', + hasQopAuth: false, + }), + ).toBe('4726bc10c33fa6cb357eb27807b1cce8'); + }); + test('SHA-256, qop=auth', async () => { expect( await computeDigestResponse({ @@ -421,9 +437,55 @@ describe('digestHandler opaque and qop emission (AUTH-22)', () => { }); }); +describe('digestHandler -sess without qop (AUTH-17/AUTH-22)', () => { + test('stamp() emits cnonce -- and only cnonce -- for a -sess algorithm with no qop', async () => { + // A `-sess` HA1 is H(H(u:r:p):nonce:cnonce). Hashing a fresh random cnonce and then leaving it + // off the wire made every such response unverifiable by construction, and AUTH-30 bounds the + // replay to one 401, so the request simply failed (audit #67 / #74). RFC 7616 3.4: "cnonce: This + // parameter MUST be used by all implementations". `nc` and `qop` stay conditional on a negotiated + // qop -- a `deviations.md` row records the departure from AUTH-22's letter. + const handler = digestHandler('u', 'p'); + const value = await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD5-sess'}), + REQUEST_CONTEXT, + ); + expect(value).toContain('cnonce="'); + expect(value).not.toContain('qop='); + expect(value).not.toContain('nc='); + }); + + test('the emitted cnonce is the one the response was computed with (-sess, no qop)', async () => { + // The row that makes the header VERIFIABLE rather than merely populated: a cnonce emitted from a + // second draw would satisfy the assertion above and still be worthless to the server. Recomputing + // HA1 from the header's own cnonce has to reproduce the header's own response. + const handler = digestHandler('Mufasa', 'Circle Of Life'); + const value = await handler.stamp( + digestChallenge({realm: REALM, nonce: NONCE, algorithm: 'MD5-sess'}), + REQUEST_CONTEXT, + ); + const cnonce = /cnonce="(?[0-9a-f]+)"/u.exec(value)?.groups?.value; + const response = /response="(?[0-9a-f]+)"/u.exec(value)?.groups + ?.value; + expect(cnonce).toBeDefined(); + expect(response).toBe( + await computeDigestResponse({ + ...BASE, + isUtf8: false, // no charset on the challenge -- AUTH-21's ISO-8859-1 branch + algorithm: 'MD5-sess', + hasQopAuth: false, + cnonce: cnonce ?? '', + nc: '', + }), + ); + }); +}); + describe('digestHandler nonce-count sequencing (AUTH-18)', () => { - test('stamp() omits cnonce/nc/qop when the challenge negotiated no qop (AUTH-22)', async () => { + test('stamp() omits cnonce/nc/qop for a NON-sess algorithm with no qop (AUTH-22)', async () => { const handler = digestHandler('u', 'p'); + // No `algorithm` parameter, so AUTH-16's default applies: plain MD5, whose HA1 does not involve + // the client nonce at all. AUTH-22's "emit cnonce/nc/qop only when qop is negotiated" is exactly + // right here, and this is the branch it was written for. const challenge = digestChallenge({realm: REALM, nonce: NONCE}); const value = await handler.stamp(challenge, REQUEST_CONTEXT); expect(value).not.toContain('qop='); diff --git a/packages/core/src/auth/digest.ts b/packages/core/src/auth/digest.ts index ba4862d..d6a47d3 100644 --- a/packages/core/src/auth/digest.ts +++ b/packages/core/src/auth/digest.ts @@ -368,8 +368,19 @@ interface HeaderValueParams { /** * AUTH-22: quotes `username`/`realm`/`nonce`/`uri`/`response`/`cnonce`/`opaque`; leaves - * `qop`/`nc`/`algorithm` unquoted, with the full algorithm spelling; emits `cnonce`/`nc`/`qop` only - * when `qop` was actually negotiated. + * `qop`/`nc`/`algorithm` unquoted, with the full algorithm spelling; emits `nc`/`qop` only when `qop` + * was actually negotiated. + * + * **`cnonce` is emitted for every `-sess` algorithm, negotiated `qop` or not** — a deliberate + * departure from AUTH-22's letter, recorded in `docs/deviations.md`. A `-sess` HA1 is + * `H(H(user:realm:pass):nonce:cnonce)` (RFC 7616 §3.4.2), so a server handed a `-sess` response with + * no `cnonce` cannot recompute HA1 and cannot verify anything: the request is unanswerable by + * construction, and AUTH-30 bounds the replay to one 401, so it simply fails. RFC 7616 §3.4 states it + * outright — "cnonce: This parameter MUST be used by all implementations". AUTH-22's wording is RFC + * 2617's RFC 2069-compatibility form, which predates `-sess` entirely. + * + * `nc` stays conditional and stays out: RFC 2069's response input is `H(HA1:nonce:HA2)`, with no nonce + * count in it, so emitting one would advertise a count the response was not computed over. */ function buildHeaderValue(params: HeaderValueParams): string { const {username, info, uri, response, cnonce, nc} = params; @@ -385,6 +396,8 @@ function buildHeaderValue(params: HeaderValueParams): string { if (info.opaque !== undefined) parts.push(`opaque=${quote(info.opaque)}`); if (info.hasQopAuth) parts.push('qop=auth', `nc=${nc}`, `cnonce=${quote(cnonce)}`); + else if (info.algorithm.endsWith('-sess')) + parts.push(`cnonce=${quote(cnonce)}`); return `Digest ${parts.join(', ')}`; } From fb2a6317eb541c3eb414794429a1dd0ebcce45ed Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:16:11 +0300 Subject: [PATCH 029/100] test(rx): pin the SSE ownership transfer by counting the owned resource `sseEvents$`/`typedSse$` hand `() => stream.close()` to `fromAsyncIterable` as its `release`, which RxJS runs on every termination. That departs from ASYNC-21's "MUST NOT close the caller-owned source on any termination" clause and nothing pinned it, so a future reading could drop the callback and take both halves of the current behaviour with it. Seven cases across two `resource ownership` blocks. They count the release the OWNED RESOURCE sees, never `SseStream.close()` calls: `close()` memoizes its release promise (SSE-28), so a facade-level count reads "once" however many paths call it. `countingBody()` is a structural `ReadableStream` double because the platform's own `cancel` hook collapses the same way -- it fires at most once per stream by specification, and not at all once a producer has ended it. - end-of-source, source error (SSE-19 line cap, so the release itself stays clean and the typed error surfaces unwrapped), early unsubscribe, and a throwing mapper (SSE-36) each release the source half and the response half exactly once. - Unsubscribing while a pull is suspended settles the teardown, through the bare and the mapping path alike. This is the discriminating case: with the `release` argument removed, both fail as "the teardown did not settle within 500ms" -- `iterator.return()` on an async generator queues behind the in-flight `next()`, so only the release running ahead of it reaches a server that never sends again. `within()` exists so that regression surfaces as a named assertion rather than a runner timeout. Behaviour unchanged; `packages/rx/src/sse.ts` is untouched by this commit. Refs #75, #67. Decision D12 in docs/audit-67-decisions.md. --- packages/rx/src/sse.test.ts | 278 +++++++++++++++++++++++++++++++++++- 1 file changed, 277 insertions(+), 1 deletion(-) diff --git a/packages/rx/src/sse.test.ts b/packages/rx/src/sse.test.ts index 03e4a71..08425b4 100644 --- a/packages/rx/src/sse.test.ts +++ b/packages/rx/src/sse.test.ts @@ -2,13 +2,20 @@ // packages/rx/src/sse.test.ts // // Exercises: SSE-41 (reactive adapter), SSE-26 (single-pass: second subscription fails loudly), -// SSE-33-36 (typed adapter mapping over reactive stream), ASYNC-21, ASYNC-6. +// SSE-33-36 (typed adapter mapping over reactive stream), ASYNC-21, ASYNC-6, SSE-28 (idempotent release), +// SSE-30 (quiet automatic release). +// +// The `resource ownership` blocks pin the deliberate departure from ASYNC-21's "MUST NOT close the +// caller-owned source on any termination" clause -- see the ASYNC-21 row of `docs/deviations.md`. They count +// the release the OWNED RESOURCE sees, never `SseStream.close()` calls: `close()` memoizes its release +// promise (SSE-28), so a facade-level count reads "once" no matter how many paths call it. import {describe, expect, test} from 'bun:test'; import {firstValueFrom, toArray} from 'rxjs'; import { Protocol, Request, Response, + SseLineTooLongError, SseStream, SseStreamError, sseStreamFrom, @@ -76,6 +83,134 @@ function makeUnclosedSseStream(text: string, onCancel?: () => void): SseStream { return sseStreamFrom(response); } +/** + * The two halves of the ONE resource `sseStreamFrom` hands the facade -- `closingBoth(source, response)` in + * core's `sse/stream.ts`. `socket` is the byte stream's own teardown, which the platform invokes at most once + * per stream and not at all once a producer has already ended it, so it corroborates rather than carries the + * count. + */ +interface ReleaseCounts { + /** `BufferedSource.close()` -> `RetentionWindow.close()` -> `reader.cancel()`. */ + source: number; + /** `Response.close()` -> `body.cancel()`. */ + response: number; + /** The `ReadableStream`'s own `cancel` hook. */ + socket: number; +} + +interface CountingSseStream { + readonly stream: SseStream; + readonly releases: ReleaseCounts; + /** Settles the first time the byte stream itself is torn down. */ + readonly socketTornDown: Promise; +} + +/** + * A `ReadableStream` facade counting every `cancel()` the SDK routes through it, at both levels + * `sseStreamFrom` uses. + * + * A structural double rather than a subclass, because `ResponseBuilder.body()` stores what it is handed and + * runs no `instanceof` check. A platform `ReadableStream` cannot do this job on its own: its underlying + * `cancel` hook is invoked at most once by specification and never at all after the producer closed the + * controller, so a second release would collapse into the first and read as clean. + */ +function countingBody( + bytes: ReadableStream, + releases: ReleaseCounts, +): ReadableStream { + const reader = (): ReadableStreamDefaultReader => { + const real = bytes.getReader(); + return { + closed: real.closed, + read: () => real.read(), + releaseLock: () => { + real.releaseLock(); + }, + cancel: async (reason?: unknown) => { + releases.source += 1; + return real.cancel(reason); + }, + } as unknown as ReadableStreamDefaultReader; + }; + return { + get locked(): boolean { + return bytes.locked; + }, + getReader: reader, + cancel: async (reason?: unknown) => { + releases.response += 1; + return bytes.cancel(reason); + }, + } as unknown as ReadableStream; +} + +/** + * An `SseStream` over a body that reports every release it is asked for. + * + * `ended: false` leaves the producer's controller open, which is the idle state a live event stream sits in + * between events -- the reader stays suspended in a pull and only a cancel can settle it. + */ +function makeCountingSseStream( + text: string, + options: {readonly ended: boolean; readonly maxLineBytes?: number}, +): CountingSseStream { + const releases: ReleaseCounts = {source: 0, response: 0, socket: 0}; + let tornDown = (): void => undefined; + const socketTornDown = new Promise(resolve => { + tornDown = resolve; + }); + const bytes = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + if (options.ended) controller.close(); + }, + cancel() { + releases.socket += 1; + tornDown(); + }, + }); + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(countingBody(bytes, releases)) + .build(); + return { + stream: sseStreamFrom(response, {maxLineBytes: options.maxLineBytes}), + releases, + socketTornDown, + }; +} + +/** Lets the adapter's teardown, which is driven off the microtask queue, run to completion. */ +const settle = async (): Promise => { + await new Promise(resolve => setTimeout(resolve, 20)); +}; + +/** + * Awaits `promise`, failing with a line number rather than hanging when it never settles. + * + * The suspended-pull case regresses as a *hang*, and a bare `await` would surface that as a bare runner + * timeout naming no assertion. + */ +async function within(ms: number, promise: Promise): Promise { + let timer: ReturnType | undefined; + const deadline = new Promise((_resolve, reject) => { + timer = setTimeout(() => { + reject(new Error(`the teardown did not settle within ${String(ms)}ms`)); + }, ms); + }); + try { + await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + describe('sseEvents$', () => { test('emits every parsed SseEvent in order and completes at end-of-stream', async () => { const stream = makeSseStreamFixture('data: one\n\ndata: two\n\n'); @@ -203,3 +338,144 @@ describe('typedSse$', () => { expect(cancelCalled).toBe(true); }); }); + +describe('sseEvents$ resource ownership (ASYNC-21 departure, SSE-28)', () => { + test('end-of-source releases each half of the owned resource exactly once', async () => { + const {stream, releases} = makeCountingSseStream( + 'data: one\n\ndata: two\n\n', + {ended: true}, + ); + + const events = await firstValueFrom(sseEvents$(stream).pipe(toArray())); + + expect(events).toHaveLength(2); + await settle(); + // `socket: 0` is not a miss: the producer ended the byte stream, so there is nothing left for the + // platform to tear down. The two halves the facade owns are still released, once each. + expect(releases).toEqual({source: 1, response: 1, socket: 0}); + }); + + test('a source error releases each half exactly once and surfaces the error unwrapped', async () => { + const {stream, releases} = makeCountingSseStream( + `data: ${'x'.repeat(64)}\n\n`, + {ended: false, maxLineBytes: 16}, + ); + + const failure = await rejection(firstValueFrom(sseEvents$(stream))); + + // SSE-29 releases before the error propagates; the adapter's own release then finds it already done. + expect(failure).toBeInstanceOf(SseLineTooLongError); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('early unsubscribe releases each half exactly once', async () => { + const {stream, releases} = makeCountingSseStream( + 'data: one\n\ndata: two\n\ndata: three\n\n', + {ended: false}, + ); + + const subscription = sseEvents$(stream).subscribe({ + next() { + subscription.unsubscribe(); + }, + }); + + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('unsubscribing while a pull is suspended settles the teardown', async () => { + const {stream, releases, socketTornDown} = makeCountingSseStream( + 'data: one\n\n', + {ended: false}, + ); + const received: string[] = []; + const subscription = sseEvents$(stream).subscribe({ + next(event) { + const item = event.data[0]; + if (item !== undefined) received.push(item); + }, + }); + + await settle(); + expect(received).toEqual(['one']); + expect(releases.socket).toBe(0); + + // The server will never send another byte, so the reader is parked in a pull. Only the release running + // AHEAD of `iterator.return()` settles it -- a `return()` on an async generator queues behind the + // in-flight `next()`. Drop the release from `sseEvents$` and this never resolves. + subscription.unsubscribe(); + await within(500, socketTornDown); + + // `socketTornDown` fires inside the source half; the response half follows it (release order is reverse + // acquisition), so let the rest of the teardown run before counting. + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); +}); + +describe('typedSse$ resource ownership (ASYNC-21 departure, SSE-28)', () => { + test('end-of-source releases each half of the owned resource exactly once', async () => { + const {stream, releases} = makeCountingSseStream('data: 1\n\ndata: 2\n\n', { + ended: true, + }); + + const values = await firstValueFrom( + typedSse$(stream, (_event, data) => ({ + kind: 'value', + value: Number(data), + })).pipe(toArray()), + ); + + expect(values).toEqual([1, 2]); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 0}); + }); + + test('a throwing mapper releases each half exactly once (SSE-36)', async () => { + const {stream, releases} = makeCountingSseStream('data: one\n\n', { + ended: false, + }); + + // Three release paths converge here: `runMapper`'s explicit `close()`, the adapter's `release`, and the + // mapping generator's `return()` unwinding into the facade's own quiet release. + const failure = await rejection( + firstValueFrom( + typedSse$(stream, () => { + throw new TypeError('mapper blew up'); + }), + ), + ); + + expect(failure).toBeInstanceOf(TypeError); + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); + + test('unsubscribing while a pull is suspended settles the teardown through the mapping generator', async () => { + const {stream, releases, socketTornDown} = makeCountingSseStream( + 'data: 100\n\n', + {ended: false}, + ); + const subscription = typedSse$(stream, (_event, data) => ({ + kind: 'value', + value: Number(data), + })).subscribe({ + next() { + // ignore + }, + }); + + await settle(); + expect(releases.socket).toBe(0); + + subscription.unsubscribe(); + await within(500, socketTornDown); + + // `socketTornDown` fires inside the source half; the response half follows it (release order is reverse + // acquisition), so let the rest of the teardown run before counting. + await settle(); + expect(releases).toEqual({source: 1, response: 1, socket: 1}); + }); +}); From 624f53b5075e4d41b3fc0379b6c9c80c818e0d83 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:15:31 +0300 Subject: [PATCH 030/100] fix(auth): an empty Digest realm or nonce is unsatisfiable, not present parseDigestChallenge tested `=== undefined`. A truncated `WWW-Authenticate: Digest realm="r", nonce=` parses to `nonce: ''` -- which is AUTH-12's verbatim storage and AUTH-13's keep-what-parsed both behaving as specified -- and the client then answered with `nonce=""`, a response computed over a nonce no server can have issued. AUTH-16's "carries realm and nonce" is read as carrying a VALUE. Such a challenge is declined, so the next offered challenge is tried and a 401 offering nothing else surfaces unchanged (AUTH-25, AUTH-33). No deviation row: the reading is the requirement's, and the check sits at selection with AUTH-12's storage untouched. Audit #67 / #74. AUTH-16, AUTH-25. --- packages/core/src/auth/auth-step.test.ts | 50 ++++++++++++++++++------ packages/core/src/auth/digest.test.ts | 30 +++++++++++++- packages/core/src/auth/digest.ts | 16 ++++++-- 3 files changed, 80 insertions(+), 16 deletions(-) diff --git a/packages/core/src/auth/auth-step.test.ts b/packages/core/src/auth/auth-step.test.ts index 85eeb08..f8e266e 100644 --- a/packages/core/src/auth/auth-step.test.ts +++ b/packages/core/src/auth/auth-step.test.ts @@ -661,6 +661,32 @@ describe('authStep: repeated challenge headers (AUTH-12/AUTH-16/AUTH-25)', () => expect(value).toContain('qop=auth'); }); + test('a repeated Proxy-Authenticate is read the same way (AUTH-25)', async () => { + const challenged = repeatedChallengeResponse(407, 'Proxy-Authenticate', [ + 'Negotiate abc123', + 'Basic realm="p"', + ]); + const transport = new FakeTransport([ + challenged.response, + countingResponse(200).response, + ]); + const descriptor = authStep({ + credentials: {basic: new BasicCredential('u', 'p')}, + tiers: tiersFor('BASIC'), + }); + + await runThrough(descriptor, transport); + + expect(transport.sendCount).toBe(2); + expect( + transport.calls[1]?.request.headers + .get('Proxy-Authorization') + ?.startsWith('Basic '), + ).toBe(true); + }); +}); + +describe('authStep: one offered challenge declined, the next answered (AUTH-13/AUTH-16)', () => { test('a malformed FIRST entry cannot swallow a satisfiable later one (AUTH-13)', async () => { // The row that fixes the parse strategy rather than only the read. Comma-joining the two values // before parsing lets the unterminated quoted string in the first run on into the second — the @@ -688,28 +714,30 @@ describe('authStep: repeated challenge headers (AUTH-12/AUTH-16/AUTH-25)', () => ); }); - test('a repeated Proxy-Authenticate is read the same way (AUTH-25)', async () => { - const challenged = repeatedChallengeResponse(407, 'Proxy-Authenticate', [ - 'Negotiate abc123', - 'Basic realm="p"', + test('a challenge with an empty nonce is declined and the next one answered', async () => { + // Both halves of the same 401: a truncated first challenge that parses to `nonce: ''`, and a + // well-formed second. Declining is only useful if the step then keeps looking, which is what + // AUTH-25's "return no header when it cannot satisfy ANY offered challenge" requires -- the + // quantifier is over the whole offer, not over the first entry. + const challenged = repeatedChallengeResponse(401, 'WWW-Authenticate', [ + 'Digest realm="r", nonce=', + 'Digest realm="second", nonce="n", qop="auth"', ]); const transport = new FakeTransport([ challenged.response, countingResponse(200).response, ]); const descriptor = authStep({ - credentials: {basic: new BasicCredential('u', 'p')}, - tiers: tiersFor('BASIC'), + credentials: {digest: new DigestCredential('u', 'p')}, + tiers: tiersFor('DIGEST'), }); await runThrough(descriptor, transport); expect(transport.sendCount).toBe(2); - expect( - transport.calls[1]?.request.headers - .get('Proxy-Authorization') - ?.startsWith('Basic '), - ).toBe(true); + expect(transport.calls[1]?.request.headers.get('Authorization')).toContain( + 'realm="second"', + ); }); }); diff --git a/packages/core/src/auth/digest.test.ts b/packages/core/src/auth/digest.test.ts index 4000448..93b474e 100644 --- a/packages/core/src/auth/digest.test.ts +++ b/packages/core/src/auth/digest.test.ts @@ -5,7 +5,8 @@ // (the client nonce is drawn from a CSPRNG with >= 128 bits of entropy, never a non-cryptographic RNG), // AUTH-15 (exactly {MD5, MD5-sess, SHA-256, SHA-256-sess}, qop=auth or absent, declines // auth-int and unsupported algorithms), AUTH-16 (satisfiability: scheme/realm/nonce/qop/algorithm, -// and configured-preference order over wire order), AUTH-17 (HA1/HA2/response per RFC 7616/2069, +// and configured-preference order over wire order -- realm and nonce must carry a VALUE, so a +// truncated `nonce=` is declined rather than echoed back as `nonce=""`), AUTH-17 (HA1/HA2/response per RFC 7616/2069, // verified against independently-computed vectors), AUTH-18/AUTH-19 (nonce count: starts at 1, // increments only on nonce reuse, 8 lower-case hex digits, bounded and drained to the cap), // AUTH-20 (client nonce from crypto.getRandomValues, >=128 bits), AUTH-21 (UTF-8 vs ISO-8859-1 by @@ -258,6 +259,33 @@ describe('digestHandler: credential validation (AUTH-9/AUTH-22)', () => { }); }); +describe('digestHandler: a realm or nonce with no value (AUTH-16)', () => { + test('canHandle declines an EMPTY realm or nonce, not only an absent one (AUTH-16)', () => { + // A truncated `WWW-Authenticate: Digest realm="r", nonce=` parses to `nonce: ''` -- AUTH-12 stores + // values verbatim after unquoting and AUTH-13 keeps what it parsed before the malformed tail, both + // correctly. The `=== undefined` test downstream then read that as present, and the client sent + // `nonce=""` back: a response computed over an empty nonce, which no server can have issued + // (audit #67 / #74). AUTH-16's "carries realm and nonce" is read as carrying a VALUE. + const handler = digestHandler('u', 'p'); + expect(handler.canHandle(digestChallenge({realm: REALM, nonce: ''}))).toBe( + false, + ); + expect(handler.canHandle(digestChallenge({realm: '', nonce: NONCE}))).toBe( + false, + ); + }); + + test('rank is worst-possible for an empty realm or nonce, so a sibling wins', () => { + // The declining half is only useful if the composer can still get past it: `rank` and `canHandle` + // must agree, or a challenge that cannot be stamped would still sort first (AUTH-25's "return no + // header when it cannot satisfy any" applies per challenge, not per response). + const handler = digestHandler('u', 'p'); + expect(handler.rank?.(digestChallenge({realm: REALM, nonce: ''}))).toBe( + Number.MAX_SAFE_INTEGER, + ); + }); +}); + describe('digestHandler: challenge selection (AUTH-15/AUTH-16)', () => { test('canHandle accepts a well-formed Digest challenge', () => { const handler = digestHandler('u', 'p'); diff --git a/packages/core/src/auth/digest.ts b/packages/core/src/auth/digest.ts index d6a47d3..fdfaa2a 100644 --- a/packages/core/src/auth/digest.ts +++ b/packages/core/src/auth/digest.ts @@ -233,10 +233,17 @@ function isHeaderSafeEcho(info: { } /** - * AUTH-16: satisfiable if and only if the scheme is `digest`, `realm` and `nonce` are both present, - * `qop` is absent or contains `auth`, the algorithm (defaulting to `MD5`) is in the caller's - * configured preference list, and every field AUTH-22 echoes back is header-safe + * AUTH-16: satisfiable if and only if the scheme is `digest`, `realm` and `nonce` both carry a + * non-empty value, `qop` is absent or contains `auth`, the algorithm (defaulting to `MD5`) is in the + * caller's configured preference list, and every field AUTH-22 echoes back is header-safe * ({@link isHeaderSafeEcho}). + * + * "Carries realm and nonce" is read as carrying a VALUE, not merely a key. A truncated header — + * `Digest realm="r", nonce=` — parses to `nonce: ''`, which is AUTH-12 and AUTH-13 both behaving + * exactly as specified: values are stored verbatim after unquoting, and what was parsed before a + * malformed tail is kept. Testing `=== undefined` here accepted that and sent `nonce=""` back, a + * response computed over a nonce no server can have issued. The challenge is declined instead, so the + * next one is tried and a 401 offering nothing else surfaces unchanged (AUTH-25, AUTH-33). */ function parseDigestChallenge( challenge: Challenge, @@ -245,7 +252,8 @@ function parseDigestChallenge( if (challenge.scheme !== 'digest') return undefined; const realm = challenge.params.get('realm'); const nonce = challenge.params.get('nonce'); - if (realm === undefined || nonce === undefined) return undefined; + if (realm === undefined || realm === '') return undefined; + if (nonce === undefined || nonce === '') return undefined; const qopRaw = challenge.params.get('qop'); const hasQop = qopRaw !== undefined; From 9126a04714bba6612142776280f9072660dfd778 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:18:38 +0300 Subject: [PATCH 031/100] docs(rx): state the SSE ownership transfer outright MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The TSDoc said "unsubscribing closes the stream" and left the rest to be inferred. What a caller actually needs to know is who owns the `SseStream` after `subscribe()`, because the answer decides whether they must call `close()` themselves — and the honest answer is that they must not. Both `@public` blocks now open with the transfer: subscribing takes the stream's one iterator (SSE-26) and the adapter releases the stream on every termination, so do not close it yourself and do not iterate it afterwards. `sseEvents$` also carries why — the release runs ahead of `iterator.return()` because an async generator's `return()` queues behind a suspended `next()` and cannot settle an idle read on its own — and names the departure from ASYNC-21's non-closing clause rather than leaving the requirement citation on line 1 reading as satisfied. `packages/rx/README.md` gains a `Who owns the stream` section saying the same, next to the `for await` equivalent it matches: `SseStream` releases on iterator return either way, so the reactive form transfers ownership for the same reason the loop does. It also states, correctly, why `pageItems$`/`pages$` attach no release callback — the paginator's own walk owns each page, and its pulls are bounded exchanges. `bun run api:local` in `packages/rx` regenerates `etc/rx.api.md` byte-identically: no signature and no tag moved, and the run is what proves the `{@link sseEvents$}` references still resolve. Refs #75, #67. Decision D12 in docs/audit-67-decisions.md. --- packages/rx/README.md | 39 ++++++++++++++++++++++++++++++++++++++- packages/rx/src/sse.ts | 24 ++++++++++++++++++++---- 2 files changed, 58 insertions(+), 5 deletions(-) diff --git a/packages/rx/README.md b/packages/rx/README.md index 2ef018e..fd35cb4 100644 --- a/packages/rx/README.md +++ b/packages/rx/README.md @@ -45,7 +45,44 @@ own guard and surfaces its error through the `Observable`'s error channel. call (`PAGE-8`), so each subscription drives an independent fetch sequence. Both release their source on `unsubscribe()`, including while idle — an SSE stream waiting on the next event is -closed immediately rather than when the server next sends something. +closed immediately rather than when the server next sends something. Which of the two *owns* that source +differs, and that is the next section. + +## Who owns the stream + +`sseEvents$`/`typedSse$` **take ownership of the `SseStream` you hand them.** Subscribing takes the stream's one +iterator (`SSE-26`), and the adapter releases the stream on every termination — `unsubscribe()`, end-of-source +and a source error alike. Do not call `close()` on it yourself, and do not iterate it afterwards; `unsubscribe()` +is how you stop early. + +That covers stopping while the stream is **idle**, which is where a live event stream spends almost all of its +time. The adapter runs the release *ahead of* the iterator's `return()` on purpose: an async generator's +`return()` queues behind a suspended `next()`, so on its own it cannot settle a read that is waiting on a server +which will never send again. + +It is also what a plain `for await` over the same stream already does — `SseStream` releases its resource when +its iterator returns, so `break`ing out of the loop closes the response body too. The reactive form transfers +ownership for the same reason the loop does, not as an extra: + +```typescript +import {sseStreamFrom, type Response} from '@dexpace/core'; + +declare const response: Response; + +// No `close()` here either: leaving the loop releases the response body. +for await (const event of sseStreamFrom(response)) { + if (event.event === 'done') break; +} +``` + +This is a deliberate departure from `ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" +clause, and it is recorded as one — see the `ASYNC-21` row of [`docs/deviations.md`](../../docs/deviations.md). + +`pageItems$`/`pages$` transfer nothing, and attach no release callback. The `Paginator`'s own walk already owns +each page's response — `items()` closes a page before yielding any of its items (`PAGE-11`), and `pages()` +closes the held page from the generator's `finally` (`PAGE-12`), which the iterator's `return()` drives. That is +enough there because a paginator's pulls are bounded HTTP exchanges; an SSE pull is a wait on a server that may +never answer, which is the difference the callback exists for. ## Notes diff --git a/packages/rx/src/sse.ts b/packages/rx/src/sse.ts index e4a343f..c7686e0 100644 --- a/packages/rx/src/sse.ts +++ b/packages/rx/src/sse.ts @@ -12,13 +12,24 @@ import {fromAsyncIterable} from './from-async-iterable.js'; /** * Bridges an {@link @dexpace/core#SseStream} to an RxJS `Observable` (SSE-41, ASYNC-21). * + * **Subscribing transfers ownership of `stream` to the returned `Observable`.** The adapter releases it on + * every termination path -- unsubscription, end-of-source and a source error alike (ASYNC-6). Do not call + * `stream.close()` yourself, and do not iterate `stream` after passing it here: its iterator may be taken at + * most once (SSE-26) and this function takes it, so the subscription's lifetime is the stream's lifetime. + * Closing it out from under a live subscription corrupts nothing -- `close()` is idempotent (SSE-28) -- but it + * ends the stream from the wrong end; `unsubscribe()` is the supported way. + * + * That transfer is a deliberate departure from ASYNC-21's "MUST NOT close the caller-owned source on any + * termination" clause, recorded in this project's deviation register. It is what releases the response body + * when a subscriber unsubscribes from an *idle* stream -- the state a long-lived event stream sits in almost + * all of the time -- because an async generator's `return()` queues behind the suspended pull it would + * otherwise have to interrupt, and so cannot settle one on its own. + * * Single-subscription: `SseStream` wraps an already-open, single-use HTTP response body (BODY-14) and is itself * single-pass (SSE-26) -- obtaining an iterator succeeds at most once and a second attempt fails loudly. * Subscribing to the returned `Observable` a second time reaches `SseStream`'s own guard and surfaces an error * through the `Observable`'s error channel. * - * Unsubscribing closes the stream, releasing the response body even when no event is in flight (ASYNC-6). - * * Diagnostic context propagates on its own (ASYNC-8–ASYNC-11): every pull runs inside the continuation chain * that called `subscribe()`, which is exactly what Node's `AsyncLocalStorage` tracks. That holds only for the * `Observable` returned here -- a caller who pipes it through an RxJS scheduler operator (`observeOn`, @@ -38,8 +49,13 @@ export function sseEvents$(stream: SseStream): Observable { /** * Bridges an {@link @dexpace/core#SseStream} to a typed RxJS `Observable` via an {@link @dexpace/core#SseMapper} (SSE-41, ASYNC-21, SSE-33–SSE-36). * - * Shares every note on {@link sseEvents$}: single-subscription, release-on-unsubscribe, and automatic - * diagnostic-context propagation through the unscheduled path. + * **Subscribing transfers ownership of `stream` to the returned `Observable`,** exactly as in + * {@link sseEvents$} and for the same reasons: the adapter releases the stream on every termination path, so + * do not call `stream.close()` yourself and do not iterate `stream` after passing it here. The mapper sits + * between the stream and the subscriber; it does not change who owns the stream. + * + * Shares every other note on {@link sseEvents$} too: single-subscription, and automatic diagnostic-context + * propagation through the unscheduled path. * * A throwing mapper reaches the `Observable`'s error channel unwrapped (ASYNC-13), after `typedSseStream` has * released the stream (SSE-36). From f6d2510bdc9c02ebb3ab41ca6c0dcb42689573e6 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:20:01 +0300 Subject: [PATCH 032/100] test(node): pin the SSE ownership transfer's single release on Node The Bun cases added with this issue count the releases the owned resource sees. Whether three converging release paths collapse to one is not this package's decision on either runtime: it is Node's `ReadableStream` cancel semantics, and when Node resumes a generator parked in `return()`. That is the membership rule in `tests/node-conformance/README.md`, so the counts get a case here too. Three cases against the built artifact: end-of-source, unsubscribe while a pull is suspended, and a throwing mapper (SSE-36, where `runMapper`'s explicit `close()`, the adapter's `release` and the mapping generator's `return()` all run). `countingBody()` is a structural `ReadableStream` double for the same reason as the Bun side -- Node invokes a stream's own `cancel` hook at most once and never after the producer closed the controller, so a double release would collapse into the first and read as clean. Refs #75, #67. Decision D12 in docs/audit-67-decisions.md. --- tests/node-conformance/rx-bridge.test.mjs | 106 ++++++++++++++++++++++ 1 file changed, 106 insertions(+) diff --git a/tests/node-conformance/rx-bridge.test.mjs b/tests/node-conformance/rx-bridge.test.mjs index 2c1e90a..ae7a9dc 100644 --- a/tests/node-conformance/rx-bridge.test.mjs +++ b/tests/node-conformance/rx-bridge.test.mjs @@ -14,6 +14,10 @@ // suspended pull if the runtime's `ReadableStream` cancellation rejects/resolves the pending read. // 3. `pages$` unsubscribed mid-walk must close the in-hand page's response body (PAGE-11/PAGE-26) through // the same generator-return path. +// 4. The adapter's ownership transfer converges THREE release paths on one resource -- `sseEvents$`'s +// `release`, the iterator's `return()`, and `SseStream`'s own quiet release inside it. Whether they +// collapse to a single resource close is decided by Node's `ReadableStream` cancel semantics and by when +// Node resumes a generator parked in `return()`, not by this package. Bun agreeing proves nothing here. import assert from 'node:assert/strict'; import {describe, it} from 'node:test'; import {firstValueFrom, toArray} from 'rxjs'; @@ -258,3 +262,105 @@ describe('pageItems$/pages$ over Node (PAGE-8, ASYNC-6)', () => { assert.deepEqual(closed, [1], 'page 1 body must be released'); }); }); + +/** + * A `ReadableStream` facade counting every `cancel()` the SDK routes through it, at both the levels + * `sseStreamFrom` uses: the reader `BufferedSource` takes, and the stream `Response.close()` cancels. + * + * A structural double, because `ResponseBuilder.body()` stores what it is handed. The platform stream's own + * `cancel` hook cannot do this job: Node invokes it at most once per stream and never after the producer has + * closed the controller, so a second release would collapse into the first and read as clean. + */ +function countingBody(bytes, counts) { + return { + get locked() { + return bytes.locked; + }, + getReader() { + const real = bytes.getReader(); + return { + closed: real.closed, + read: () => real.read(), + releaseLock: () => real.releaseLock(), + cancel: reason => { + counts.source += 1; + return real.cancel(reason); + }, + }; + }, + cancel: reason => { + counts.response += 1; + return bytes.cancel(reason); + }, + }; +} + +/** As {@link openSseStream}/{@link closedSseStream}, but reporting every release the owned resource sees. */ +function countingSseStream(text, ended) { + const counts = {source: 0, response: 0, socket: 0}; + const bytes = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(text)); + if (ended) controller.close(); + }, + cancel() { + counts.socket += 1; + }, + }); + const request = Request.newBuilder() + .method('GET') + .url('https://example.com/events') + .build(); + const response = Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(countingBody(bytes, counts)) + .build(); + return {stream: sseStreamFrom(response), counts}; +} + +describe('SSE ownership transfer releases once on Node (ASYNC-21 departure, SSE-28)', () => { + it('end-of-source reaches each half of the owned resource exactly once', async () => { + const {stream, counts} = countingSseStream( + 'data: one\n\ndata: two\n\n', + true, + ); + + const events = await firstValueFrom(sseEvents$(stream).pipe(toArray())); + + assert.equal(events.length, 2); + await settle(); + // `socket: 0` is not a miss -- the producer ended the byte stream, so Node has nothing left to tear down. + assert.deepEqual(counts, {source: 1, response: 1, socket: 0}); + }); + + it('unsubscribing while a pull is suspended reaches each half exactly once', async () => { + const {stream, counts} = countingSseStream('data: one\n\n', false); + const subscription = sseEvents$(stream).subscribe({next: () => undefined}); + + await settle(); + assert.deepEqual(counts, {source: 0, response: 0, socket: 0}); + + subscription.unsubscribe(); + await settle(); + assert.deepEqual(counts, {source: 1, response: 1, socket: 1}); + }); + + it('a throwing mapper reaches each half exactly once, across three release paths (SSE-36)', async () => { + const {stream, counts} = countingSseStream('data: one\n\n', false); + + // `runMapper`'s explicit close, the adapter's `release`, and the mapping generator's `return()` unwinding + // into the facade's quiet release all run. Exactly one of them may reach the resource. + await assert.rejects(() => + firstValueFrom( + typedSse$(stream, () => { + throw new TypeError('mapper blew up'); + }), + ), + ); + + await settle(); + assert.deepEqual(counts, {source: 1, response: 1, socket: 1}); + }); +}); From 19b06022a8a6c81fb9cb27a09c9b2e75d467101c Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:20:54 +0300 Subject: [PATCH 033/100] test(transport): both transports must surface every repeated challenge header New fixture route /repeated-challenge answers 401 with two WWW-Authenticate lines -- the RFC 7616 3.3 algorithm-discovery shape, one challenge per algorithm. New conformance row asserts the CHALLENGE LIST recovered from headers.getAll() is identical through both adapters. The entry count deliberately is not asserted: WHATWG Headers comma-joins every name but Set-Cookie, so transport-fetch yields one entry and transport-undici yields two, and RFC 9110 5.3 makes the two wire shapes equivalent. Losing a value is what neither may do, and the row was confirmed red against a fixture sending only the first line. The core-side parse is asserted in packages/core/src/auth/auth-step.test.ts: @dexpace/core's parseChallenges is @internal and unreachable from this package, and #74 adds no core export. Also corrects undici-transport.ts's inbound-header comment, which claimed an array value meant Set-Cookie. undici arrays any repeated header. Audit #67 / #74. TRANSPORT-14, AUTH-12, AUTH-25. --- .../transport-conformance/src/fixtures.ts | 25 +++++++++ .../transport-conformance/src/run-suite.ts | 51 +++++++++++++++++-- .../transport-undici/src/undici-transport.ts | 7 ++- 3 files changed, 79 insertions(+), 4 deletions(-) diff --git a/packages/transport-conformance/src/fixtures.ts b/packages/transport-conformance/src/fixtures.ts index cf8fc78..593d32d 100644 --- a/packages/transport-conformance/src/fixtures.ts +++ b/packages/transport-conformance/src/fixtures.ts @@ -21,6 +21,18 @@ const SLOW_RESPONSE_MS = 5_000; const DRIP_INTERVAL_MS = 50; const DRIP_CHUNKS = 20; +/** + * `/repeated-challenge`'s two `WWW-Authenticate` lines, in wire order. Exported so the suite asserts + * against what the server actually sent rather than against a second copy that can drift from it. + * + * The first algorithm is deliberately one no SDK handler supports, so a transport that keeps only the + * first value produces a challenge nothing can answer — the shape audit #67 / #74 found. + */ +export const REPEATED_CHALLENGES: readonly string[] = [ + 'Digest realm="conformance", nonce="n1", algorithm=SHA-512-256', + 'Digest realm="conformance", nonce="n1", algorithm=SHA-256, qop="auth"', +]; + function route( pathname: string, req: IncomingMessage, @@ -101,6 +113,19 @@ function route( res.end('done'); }, SLOW_RESPONSE_MS).unref(); return; + case '/repeated-challenge': + // AUTH-12/AUTH-25: the same challenge header sent TWICE, which RFC 9110 5.3 permits for any + // list-valued field and RFC 7616 3.3 recommends for Digest algorithm discovery -- one challenge + // per algorithm, strongest first. The two transports legitimately surface it differently: + // WHATWG `Headers` comma-joins every name but `Set-Cookie`, so `@dexpace/transport-fetch` + // delivers one entry, while undici arrays any repeated header and `@dexpace/transport-undici` + // keeps two. Neither may LOSE one, which is what the row asserts. + // + // An array value in `writeHead`, not two `setHeader` calls: `setHeader` on the same name + // replaces, which would make the fixture single-valued and the row vacuous. + res.writeHead(401, {'www-authenticate': REPEATED_CHALLENGES}); + res.end(); + return; case '/redirect': res.writeHead(302, {location: '/echo-headers'}); res.end(); diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index b6eadfa..d9445b7 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -1,8 +1,9 @@ // SPDX-License-Identifier: MIT // packages/transport-conformance/src/run-suite.ts // The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot -// drift. Exercises: TRANSPORT-1..9, TRANSPORT-15..17, TRANSPORT-20..21, TRANSPORT-23..27, -// TRANSPORT-29, SEAM-12, SEAM-16, SEAM-30, NFR-15. TRANSPORT-10..14 are asserted at their source in +// drift. Exercises: TRANSPORT-1..9, TRANSPORT-14..17, TRANSPORT-20..21, TRANSPORT-23..27, +// TRANSPORT-29, SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is +// answerable for them (the repeated-challenge-header row). TRANSPORT-10..14 are asserted at their source in // @dexpace/transport-shared; TRANSPORT-18/28's collapses are Deviation Ledger rows; TRANSPORT-30's // full flow is transport-undici's challenge-handler.test.ts. TRANSPORT-22 is NOT driven from here -- // forcing an adaptation throw needs a per-transport hook into the native response, so each adapter @@ -20,7 +21,11 @@ import { type Logger, type Transport, } from '@dexpace/core'; -import {startFixtureServer, type TestServer} from './fixtures.js'; +import { + REPEATED_CHALLENGES, + startFixtureServer, + type TestServer, +} from './fixtures.js'; /** * The clauses `docs/product-spec/17-transport-adapter-conformance-contract.md` scopes to only one @@ -531,6 +536,45 @@ function registerHeaderRows(ctx: SuiteContext): void { }); } +/** + * The fixture's challenge list, recovered from however this transport chose to split it. + * + * Scoped to `/repeated-challenge` on purpose, and deliberately NOT a general RFC 7235 parser: the two + * fixture challenges are `Digest`-schemed and carry no comma inside a quoted value, so a split at each + * `, Digest ` boundary recovers exactly what the server wrote. `@dexpace/core`'s real parser is + * `@internal` and unreachable from here — the core-side rows for it live in + * `packages/core/src/auth/auth-step.test.ts`. + */ +function challengeList(values: readonly string[]): readonly string[] { + return values.join(', ').split(/,\s+(?=Digest )/u); +} + +function registerInboundHeaderRows(ctx: SuiteContext): void { + describe('TRANSPORT-14, AUTH-12/AUTH-25: a repeated inbound header keeps every value', () => { + test('two WWW-Authenticate lines reach the pipeline as the same challenge list', async () => { + // The two transports split this differently and both are right: WHATWG `Headers` comma-joins + // every name but `Set-Cookie`, so `@dexpace/transport-fetch` yields ONE `getAll` entry, while + // undici arrays any repeated header and `@dexpace/transport-undici` yields TWO. RFC 9110 5.3 + // makes the two wire shapes equivalent, so the entry count is not what either adapter is + // answerable for — the challenge list after the split is, and it must be identical. + // + // What this row guards is the loss: before audit #67 / #74 the auth step read + // `headers.get(...)`, saw only the first line, and left a 401 offering an answerable SHA-256 + // challenge unanswered through undici while the identical offer authenticated through fetch. + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/repeated-challenge')).build(), + ); + expect(response.status.code).toBe(401); + expect( + challengeList(response.headers.getAll('WWW-Authenticate')), + ).toEqual([...REPEATED_CHALLENGES]); + await response.close(); + }); + }); + }); +} + function registerDropSetRows(ctx: SuiteContext): void { describe('TRANSPORT-11/13: the transport-specific drop set', () => { test('the Connection header follows this transport’s documented drop set', async () => { @@ -641,6 +685,7 @@ export function runTransportConformanceSuite( registerCancellationRows(ctx); registerLifecycleRows(ctx); registerHeaderRows(ctx); + registerInboundHeaderRows(ctx); registerDropSetRows(ctx); registerScopedRows(ctx); }); diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index a901c97..0853169 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -331,7 +331,12 @@ function adaptResponse( const raw: [string, string][] = []; for (const [name, value] of Object.entries(result.headers)) { if (value === undefined) continue; - // An array means a genuinely repeated header (Set-Cookie); keep each value its own entry. + // An array means a genuinely repeated header -- undici arrays ANY name it saw more than once, + // not just `Set-Cookie`. Keep each value its own entry: `WWW-Authenticate` and + // `Proxy-Authenticate` arrive this way from a server following RFC 7616 3.3, and collapsing them + // to the first would hide every challenge after it (audit #67 / #74). `@dexpace/transport-fetch` + // comma-joins the same response, which RFC 9110 5.3 makes equivalent; the conformance row + // `a repeated inbound header keeps every value` asserts the two agree on the list. if (Array.isArray(value)) for (const each of value) raw.push([name, each]); else raw.push([name, value]); } From be1b82334d94a8040ad7ad1518053762e2ff8bb8 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:21:32 +0300 Subject: [PATCH 034/100] docs: ledger the -sess cnonce departure from AUTH-22's letter One row appended at the end of the "Deviations recorded outside a phase" table, per the run's D0. Audit #67 / #74. --- docs/deviations.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/deviations.md b/docs/deviations.md index 885fe34..1930686 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -480,6 +480,7 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **The two shipped transports disagree on `HTTP-6`'s optional reason phrase: `transport-fetch` sets it, `transport-undici` leaves it `undefined`.** `HTTP-6` (MUST) has a response carry "an optional reason phrase", and the domain model provides the slot. `fetch-transport` fills it from the WHATWG `Response.statusText`, normalizing the empty string to `undefined`. `undici-transport` does not call `.reasonPhrase()` at all, because undici's `ResponseData` carries `statusCode` and no phrase — HTTP/2 has no reason phrase and undici's parser does not surface HTTP/1.1's. **Platform-shaped, and it sits beside §10 item 13's `Protocol.HTTP_1_1` gap rather than under it.** Item 13 already ledgers the negotiated-version gap in both adapters for the same reason — the value is not readable — but it does not name the reason phrase, so this row does. The field is optional in the requirement, and no shipped step or conformance row reads it, so the disagreement is observable to a caller and to nothing else. Recorded rather than papered over: synthesizing a phrase from the status code in the undici adapter would report a value the server never sent | audit #67 / #69 | 2026-09-04 | `packages/transport-fetch/src/fetch-transport.ts:180` (`.reasonPhrase(...)` fed from the WHATWG `statusText`, with the empty string normalized to `undefined`) against `packages/transport-undici/src/undici-transport.ts:342-350` (the builder chain, with no `.reasonPhrase` call); `docs/product-spec/04-core-http-domain-model.md:13`; the neighbouring gap is item 13 of the audit above | not yet in §10 | | **`AUTH-8`'s redaction clause is read as covering EVERY credential type, not the three it enumerates.** The requirement's own list is "API key, name-key secret, bearer token", and it was implemented to the letter: `ApiKeyCredential`, `NameKeyCredential` and `BearerToken` shipped as classes with a `#private` secret, a redacted `toString()` and the `nodejs.util.inspect.custom` hook, while the Basic and Digest credentials shipped as structural interfaces with a public `readonly password: string`. That is a leak the same requirement's first four words forbid: `util.inspect` of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and `JSON.stringify` serialized both passwords. **Widened, deliberately.** Both are now classes on the same pattern, with the password reachable only through the in-package `credentialPassword()` friend hook. The cost is a public-shape change — `{username, password}` object literals no longer type-check — taken now because it is free before the first version bump. Validation is deliberately NOT duplicated onto the classes: AUTH-14's non-empty-whitespace-permitted rule and AUTH-16's acceptable-set rule stay single-sourced in `basicHandler()`/`digestHandler()`, which `authStep()` builds at construction, so a blank password still fails synchronously from that factory | audit #67 / #71 | 2026-09-04 | `docs/product-spec/11-authentication.md:12` is AUTH-8's three-type enumeration. `packages/core/src/auth/credential.ts:342` (`BasicCredential`) and `:393` (`DigestCredential`), with the friend hooks at `:299-300` and `credentialPassword()` at `:314`; the sole reader is `buildHandlers` in `packages/core/src/auth/auth-step.ts:131-152`. Pinned by "a whole AuthCredentialSet is diagnostic-safe (AUTH-8)" in `packages/core/src/auth/credential.test.ts`, which drives the real `util.inspect` rather than the hook it calls | not yet in §10 | | **`XCUT-16`'s replay guard is keyed on whether the hop was guarded, not on whether the replacement looks credentialed.** `XCUT-16` and `AUTH-28` say the guard applies "on any path where a credential will be attached", and carve out "a deliberately credential-free re-issue MAY proceed over any scheme". Deciding which of the two a challenge replacement is cannot be done by reading header names: the step's own `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, and a `challengeHook` may invent a carrier this step has never been told about. The port therefore reads "a credential will be attached" as a property of the HOP — if the outbound pass ran the HTTPS guard, so does the replay, whatever URL and headers the hook chose. **Strictly wider than the requirement's letter**, and knowingly so: it refuses a downgraded replacement that carries no credential at all, on a hop that is credentialed. The carve-out is preserved where it is observable — a `NO_AUTH` hop is never guarded outbound, and its replay is guarded only when the replacement carries `Authorization` or `Proxy-Authorization`, which is the previous rule kept as a second arm. *Rejected:* deriving the credential-carrying header names from configuration, which misses the hook-invented carrier and is the shape that let the reported leak through | audit #67 / #71 | 2026-09-04 | `docs/product-spec/19-cross-cutting-invariants-and-policies.md:44` is the requirement and its carve-out. `packages/core/src/auth/auth-step.ts:389` sets `OutboundPlan.guarded`; `:564-575` is `guardReplayScheme` and its two arms. Pinned by "a replacement carrying a NON-standard credential header over plaintext is refused" and "a header-free replacement over plaintext is refused too" in `packages/core/src/auth/auth-step.test.ts`, and by the "XCUT-16: a guarded hop stays guarded across a challenge replay" block in `tests/conformance/xcut/security-by-default.conformance.test.ts` | not yet in §10 | +| **`AUTH-22`'s "emit cnonce/nc/qop only when qop is negotiated" is not applied to `cnonce` for a `-sess` algorithm.** A `-sess` HA1 is `H(H(user:realm:pass):nonce:cnonce)` (RFC 7616 §3.4.2), so the client nonce is an *input to the hash* for `MD5-sess` and `SHA-256-sess` whatever `qop` the challenge offered. The port implemented AUTH-22 to the letter: it drew a fresh cnonce, folded it into HA1, and then omitted it from the header whenever `qop` was absent — a response no server can verify, because it has no way to reconstruct HA1. AUTH-30 bounds the re-challenge replay to one 401, so every such exchange simply failed. **`cnonce` is now emitted for any `-sess` algorithm; `nc` and `qop` stay conditional exactly as AUTH-22 says**, because RFC 2069's response input is `H(HA1:nonce:HA2)` and carries no nonce count, so emitting one would advertise a count the response was not computed over. RFC 7616 §3.4 states the wider rule outright — "cnonce: This parameter MUST be used by all implementations". AUTH-22's clause is RFC 2617's RFC 2069-compatibility form, written before `-sess` existed, and the requirement's own AUTH-15 mandates both `-sess` algorithms, so the two sentences cannot both be followed. *Rejected:* declining a `-sess`-without-`qop` challenge instead, which turns every such server into a guaranteed 401 for no security gain, when the value the server needs has already been computed | audit #67 / #74 | 2026-09-05 | `packages/core/src/auth/digest.ts:345-350` (the `-sess` HA1 that consumes the cnonce) against `:405-408` (`buildHeaderValue`, where the `else if` now emits it); `docs/product-spec/11-authentication.md:18` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:357` are AUTH-22's wording. Pinned by the `digestHandler -sess without qop (AUTH-17/AUTH-22)` block in `packages/core/src/auth/digest.test.ts` — one row asserting the header carries `cnonce` and neither `nc` nor `qop`, one recomputing the response from the header's OWN cnonce so a value drawn twice would fail — and by the `MD5-sess, no qop` vector in the same file | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) From 6152d27b24f0ad40aa86239963bfa609d18d5b2f Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:21:52 +0300 Subject: [PATCH 035/100] docs: record the challenge-list and -sess cnonce behaviour in auth.md Audit #67 / #74. --- docs/sdk-documentation/auth.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/sdk-documentation/auth.md b/docs/sdk-documentation/auth.md index bc997e1..8873c4e 100644 --- a/docs/sdk-documentation/auth.md +++ b/docs/sdk-documentation/auth.md @@ -161,6 +161,23 @@ when the deferral register was dissolved on 2026-09-04). **Basic and Digest never stamp preemptively.** They react to a challenge. That is an interpretation of `§11` rather than a stated requirement, and it is ledgered as one. +**Every challenge the response offered is considered, not just the first.** A server may send +`WWW-Authenticate` (or `Proxy-Authenticate`) once with several comma-separated challenges, or once per +challenge — RFC 9110 §5.3 permits both, and RFC 7616 §3.3 recommends the repeated form for Digest +algorithm discovery. Which of the two shapes reaches the step is partly the transport's accident: +`@dexpace/transport-fetch` comma-joins repeated values because WHATWG `Headers` does, +`@dexpace/transport-undici` keeps them apart. The step reads *every* value and parses each on its own, +so the offer is the same list either way, and `@dexpace/transport-conformance` carries a row asserting +that. A challenge the SDK cannot answer — an unsupported algorithm, an `auth-int`-only `qop`, a realm +this client cannot echo back, an empty `realm` or `nonce` — is declined and the next one is tried; a +`401` offering nothing answerable surfaces unchanged and unclosed. + +**Digest `-sess` sends `cnonce` whether or not `qop` was negotiated.** `MD5-sess` and `SHA-256-sess` +fold the client nonce into HA1 (RFC 7616 §3.4.2), so a server handed a `-sess` response without one +cannot verify it. `nc` and `qop` stay conditional on a negotiated `qop`. `AUTH-22`'s literal wording +says all three are conditional; the departure is a row in +[`deviations.md`](../deviations.md). + ## Redirects and credentials Credentials are attached at an origin and must not follow a request to a different one. The redirect From 74e4fd9633ede872b74897e2de01898d1541d57e Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:22:01 +0300 Subject: [PATCH 036/100] docs: ledger the RxJS SSE ownership transfer as an ASYNC-21 departure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit One row appended to the end of "Deviations recorded outside a phase", per D0. The adapter closes the caller-owned `SseStream` on every termination, which is the clause ASYNC-21 spends its last MUST forbidding, and nothing recorded it: not §10, not item 16, and not the Phase 8b checklist, whose ASYNC-21 gist dropped the clause before ticking the row. The row keeps the behaviour and says why, with the evidence for both grounds. It also records what the counterfactual measures — deleting the two `release` arguments turns four cases red and leaves every exactly-once release count green — because that is the difference between "we prefer this" and "the letter buys nothing here and costs a held socket". Refs #75, #67. Decision D12 in docs/audit-67-decisions.md. --- docs/deviations.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/deviations.md b/docs/deviations.md index 885fe34..07360b3 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -480,6 +480,7 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **The two shipped transports disagree on `HTTP-6`'s optional reason phrase: `transport-fetch` sets it, `transport-undici` leaves it `undefined`.** `HTTP-6` (MUST) has a response carry "an optional reason phrase", and the domain model provides the slot. `fetch-transport` fills it from the WHATWG `Response.statusText`, normalizing the empty string to `undefined`. `undici-transport` does not call `.reasonPhrase()` at all, because undici's `ResponseData` carries `statusCode` and no phrase — HTTP/2 has no reason phrase and undici's parser does not surface HTTP/1.1's. **Platform-shaped, and it sits beside §10 item 13's `Protocol.HTTP_1_1` gap rather than under it.** Item 13 already ledgers the negotiated-version gap in both adapters for the same reason — the value is not readable — but it does not name the reason phrase, so this row does. The field is optional in the requirement, and no shipped step or conformance row reads it, so the disagreement is observable to a caller and to nothing else. Recorded rather than papered over: synthesizing a phrase from the status code in the undici adapter would report a value the server never sent | audit #67 / #69 | 2026-09-04 | `packages/transport-fetch/src/fetch-transport.ts:180` (`.reasonPhrase(...)` fed from the WHATWG `statusText`, with the empty string normalized to `undefined`) against `packages/transport-undici/src/undici-transport.ts:342-350` (the builder chain, with no `.reasonPhrase` call); `docs/product-spec/04-core-http-domain-model.md:13`; the neighbouring gap is item 13 of the audit above | not yet in §10 | | **`AUTH-8`'s redaction clause is read as covering EVERY credential type, not the three it enumerates.** The requirement's own list is "API key, name-key secret, bearer token", and it was implemented to the letter: `ApiKeyCredential`, `NameKeyCredential` and `BearerToken` shipped as classes with a `#private` secret, a redacted `toString()` and the `nodejs.util.inspect.custom` hook, while the Basic and Digest credentials shipped as structural interfaces with a public `readonly password: string`. That is a leak the same requirement's first four words forbid: `util.inspect` of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and `JSON.stringify` serialized both passwords. **Widened, deliberately.** Both are now classes on the same pattern, with the password reachable only through the in-package `credentialPassword()` friend hook. The cost is a public-shape change — `{username, password}` object literals no longer type-check — taken now because it is free before the first version bump. Validation is deliberately NOT duplicated onto the classes: AUTH-14's non-empty-whitespace-permitted rule and AUTH-16's acceptable-set rule stay single-sourced in `basicHandler()`/`digestHandler()`, which `authStep()` builds at construction, so a blank password still fails synchronously from that factory | audit #67 / #71 | 2026-09-04 | `docs/product-spec/11-authentication.md:12` is AUTH-8's three-type enumeration. `packages/core/src/auth/credential.ts:342` (`BasicCredential`) and `:393` (`DigestCredential`), with the friend hooks at `:299-300` and `credentialPassword()` at `:314`; the sole reader is `buildHandlers` in `packages/core/src/auth/auth-step.ts:131-152`. Pinned by "a whole AuthCredentialSet is diagnostic-safe (AUTH-8)" in `packages/core/src/auth/credential.test.ts`, which drives the real `util.inspect` rather than the hook it calls | not yet in §10 | | **`XCUT-16`'s replay guard is keyed on whether the hop was guarded, not on whether the replacement looks credentialed.** `XCUT-16` and `AUTH-28` say the guard applies "on any path where a credential will be attached", and carve out "a deliberately credential-free re-issue MAY proceed over any scheme". Deciding which of the two a challenge replacement is cannot be done by reading header names: the step's own `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, and a `challengeHook` may invent a carrier this step has never been told about. The port therefore reads "a credential will be attached" as a property of the HOP — if the outbound pass ran the HTTPS guard, so does the replay, whatever URL and headers the hook chose. **Strictly wider than the requirement's letter**, and knowingly so: it refuses a downgraded replacement that carries no credential at all, on a hop that is credentialed. The carve-out is preserved where it is observable — a `NO_AUTH` hop is never guarded outbound, and its replay is guarded only when the replacement carries `Authorization` or `Proxy-Authorization`, which is the previous rule kept as a second arm. *Rejected:* deriving the credential-carrying header names from configuration, which misses the hook-invented carrier and is the shape that let the reported leak through | audit #67 / #71 | 2026-09-04 | `docs/product-spec/19-cross-cutting-invariants-and-policies.md:44` is the requirement and its carve-out. `packages/core/src/auth/auth-step.ts:389` sets `OutboundPlan.guarded`; `:564-575` is `guardReplayScheme` and its two arms. Pinned by "a replacement carrying a NON-standard credential header over plaintext is refused" and "a header-free replacement over plaintext is refused too" in `packages/core/src/auth/auth-step.test.ts`, and by the "XCUT-16: a guarded hop stays guarded across a challenge replay" block in `tests/conformance/xcut/security-by-default.conformance.test.ts` | not yet in §10 | +| **`ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" is not honoured: the RxJS SSE adapter takes ownership and closes.** `sseEvents$` and `typedSse$` pass `() => stream.close()` as `fromAsyncIterable`'s `release`, and RxJS runs a subscriber's finalizer on *every* termination — unsubscription, end-of-source and a source error alike, which is the complete list the clause names. **Kept, deliberately, on two grounds.** (1) **The clause has no subject on this platform.** It presumes a source whose iterator return leaves the source open; this port's `SseStream` is deliberately not that one. `#iterate`'s `finally` calls `#releaseQuietly()`, so the resource is released whenever the runtime drives `return()` — which `fromAsyncIterable` must do exactly once (`ASYNC-6`), and which a plain `for await` with `break` does too. Removing the callback would change which channel reports a release failure and when the release runs, not whether the caller-owned source ends up closed. (2) **The ordering is load-bearing.** The release runs *ahead of* `iterator.return()` because an async generator's `return()` queues behind a suspended `next()`, and an SSE stream idling between events is parked in exactly that pull — so without the callback an `unsubscribe()` stays pending until the server next sends a byte, holding the socket open indefinitely. Measured: deleting the two `release` arguments turns four cases red — the two suspended-pull ones, as "the teardown did not settle within 500ms", and the two pre-existing idle-unsubscribe assertions — while every exactly-once release count stays green, which is the shape of the claim. Pagination attaches no release for the complementary reason: a `Paginator`'s pulls are bounded HTTP exchanges, never a wait on a server that may never answer. *Rejected:* dropping the callback to match the letter (reintroduces the hang for no change in what closes). *Rejected:* a caller-facing `{ownership}` option (two behaviours to document for a case with one correct answer). The public TSDoc and `packages/rx/README.md` now state the transfer outright — subscribing hands the stream over, do not close it yourself and do not iterate it afterwards — rather than leaving the `ASYNC-21` citation on the doc comment's first line to read as satisfied | audit #67 / #75 | 2026-09-05 | `packages/rx/src/sse.ts:46` and `:73-75` are the two `release` arguments; `packages/rx/src/from-async-iterable.ts:103-108` is the teardown that runs one on every termination; `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:42` is the requirement. Ground 1: `packages/core/src/sse/stream.ts:136-139` (`#iterate`'s `finally` → `#releaseQuietly()`) with `:117-121` (`close()` memoized, `SSE-28`). Ground 2: `packages/rx/src/from-async-iterable.ts:44-48` states the ordering and why. Pinned by the two `resource ownership` blocks in `packages/rx/src/sse.test.ts`, which count the release the OWNED resource sees rather than `SseStream.close()` calls — the facade memoizes, so a facade-level count reads "once" however many paths call it — and by "SSE ownership transfer releases once on Node" in `tests/node-conformance/rx-bridge.test.mjs`. Phase 8b marked `ASYNC-21` ✅ with this clause dropped from its gist (`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67`); that is a dated record and is left as written. The other half is `SSE-41`'s own "documented source ownership" clause, which the same checklist marked ✅ (`:74`) on the strength of documentation that named unsubscription only — completed by the TSDoc and README rewrite this row accompanies | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) From 2cfbe5c3f9aef805b07b5d5be230bbc0e60f2500 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:24:55 +0300 Subject: [PATCH 037/100] docs(transport-conformance): the suite header no longer claims TRANSPORT-14 twice Adding the repeated-challenge-header row put TRANSPORT-14 in the Exercises list while the sentence below still assigned TRANSPORT-10..14 wholesale to @dexpace/transport-shared. What lives there is the SHARED outbound header pass; the rows in this file assert what each adapter decides for itself, which is why registerHeaderRows and registerDropSetRows are labelled TRANSPORT-10/11 and TRANSPORT-11/13 already. --- packages/transport-conformance/src/run-suite.ts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index d9445b7..e7a416f 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -3,8 +3,10 @@ // The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot // drift. Exercises: TRANSPORT-1..9, TRANSPORT-14..17, TRANSPORT-20..21, TRANSPORT-23..27, // TRANSPORT-29, SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is -// answerable for them (the repeated-challenge-header row). TRANSPORT-10..14 are asserted at their source in -// @dexpace/transport-shared; TRANSPORT-18/28's collapses are Deviation Ledger rows; TRANSPORT-30's +// answerable for them (the repeated-challenge-header row). TRANSPORT-10..13's SHARED half -- the one +// outbound header pass both adapters call -- is asserted at its source in +// @dexpace/transport-shared, and the rows here cover only what each adapter decides for itself. +// TRANSPORT-18/28's collapses are Deviation Ledger rows; TRANSPORT-30's // full flow is transport-undici's challenge-handler.test.ts. TRANSPORT-22 is NOT driven from here -- // forcing an adaptation throw needs a per-transport hook into the native response, so each adapter // asserts it against its own (transport-fetch's fetch-transport.test.ts:118, transport-undici's From 21fa0bab90bfe043e5f09191d574b955ec31d861 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:28:10 +0300 Subject: [PATCH 038/100] fix(transport-conformance): the fixture's header array must be mutable for node:http `node:http`'s OutgoingHttpHeader is `string | string[]`, so `writeHead` rejects the exported `readonly string[]` outright. Spread into a copy at the call site, which also stops node:http aliasing the exported constant. Caught by `bun run typecheck`, which `gts lint` does not stand in for: lint, the package's own `bun test` and both transports' conformance runs were all green against this line. --- packages/transport-conformance/src/fixtures.ts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/packages/transport-conformance/src/fixtures.ts b/packages/transport-conformance/src/fixtures.ts index 593d32d..fe20b7f 100644 --- a/packages/transport-conformance/src/fixtures.ts +++ b/packages/transport-conformance/src/fixtures.ts @@ -122,8 +122,10 @@ function route( // keeps two. Neither may LOSE one, which is what the row asserts. // // An array value in `writeHead`, not two `setHeader` calls: `setHeader` on the same name - // replaces, which would make the fixture single-valued and the row vacuous. - res.writeHead(401, {'www-authenticate': REPEATED_CHALLENGES}); + // replaces, which would make the fixture single-valued and the row vacuous. Spread into a + // MUTABLE copy -- `OutgoingHttpHeader` is `string | string[]`, so a `readonly string[]` does + // not satisfy it, and handing the exported constant itself to `node:http` would alias it. + res.writeHead(401, {'www-authenticate': [...REPEATED_CHALLENGES]}); res.end(); return; case '/redirect': From 36378b392baa69e652e26ba337bd243a2d01df8d Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:48:18 +0300 Subject: [PATCH 039/100] fix(retry): surface the final typed error, not a SuppressedError wrapper The retry engine wrapped its terminal failure in `suppress(error, folded, 'retry attempts exhausted')`, which made the surfaced CLASS a function of how many attempts ran: one attempt gave `TransportFailureError`, three gave a wrapper with it at `.error`. The row that catches it is XCUT-1's conformance clause -- "assert the surfaced error is the cancellation type" -- because a cancellation during backoff always has a non-empty trail, so `abortToSdkError` mapped the abort to `CancellationError` at engine.ts:385 and the wrapper undid the mapping at :386. RETRY-34 asks for the prior failures to be "attached to the surfaced exception as suppressed", which is the JVM's `addSuppressed`: the exception stays what it is and grows a list. It does not ask for the exception to be replaced by a container. So `withTrail` becomes `attachTrail`: the outcome's error is returned unchanged, and the priors go into a module-private WeakMap in the new `retry/attempt-trail.ts`, read back through a `@public` `retryAttempts(error)`. A side table rather than an own property, because the engine did not construct the throwable it is surfacing: it may be frozen (so `defineProperty` in the failure path would itself throw and replace the failure the caller cares about), it may be a primitive (which carries nothing at all, and passes through unannotated), and `.suppressed` already means "the one secondary" on `SuppressedErrorLike`. `suppress()` keeps its RECOV-12 job -- `withReleaseFailure` still pairs a release failure with the primary it must not mask. RETRY-34's skip-self guard is unchanged, and an empty trail now DELETES any entry a previous run left, so a transport reusing one error instance reports the run that just surfaced it rather than a stale one. Decision D10 of docs/audit-67-decisions.md. Refs #72, #67. --- packages/core/etc/core.api.md | 3 + packages/core/src/index.ts | 5 + packages/core/src/retry/attempt-trail.test.ts | 138 ++++++++++++++++ packages/core/src/retry/attempt-trail.ts | 91 +++++++++++ packages/core/src/retry/engine.test.ts | 147 ++++++++++++------ packages/core/src/retry/engine.ts | 49 +++--- packages/core/src/retry/errors.ts | 5 +- packages/core/src/retry/retry-dispatch.ts | 4 +- packages/core/src/retry/retry-step.ts | 8 + 9 files changed, 381 insertions(+), 69 deletions(-) create mode 100644 packages/core/src/retry/attempt-trail.test.ts create mode 100644 packages/core/src/retry/attempt-trail.ts diff --git a/packages/core/etc/core.api.md b/packages/core/etc/core.api.md index 3431dbb..ac916ca 100644 --- a/packages/core/etc/core.api.md +++ b/packages/core/etc/core.api.md @@ -1176,6 +1176,9 @@ export type ResponseStep = (response: Response_2) => Promise; // @public export const RETRYABLE_STATUSES: ReadonlySet; +// @public +export function retryAttempts(error: unknown): readonly unknown[]; + // @public export class RetryDiscardedResponseError extends DexpaceError { constructor(status: number, options?: ErrorOptions); diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index eff7f03..43cc879 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -113,6 +113,11 @@ export { ReservedStageError, } from './pipeline/errors.js'; export {retryStep} from './retry/retry-step.js'; +// RETRY-34's read side. The retry pillar surfaces the FINAL attempt's own error, so `instanceof` +// against it does not depend on how many attempts ran; this is how the earlier ones are reached. +// Exported alongside `retryStep` because the two are one contract: nothing else in the barrel can +// tell a caller that the error they caught is the third of three. +export {retryAttempts} from './retry/attempt-trail.js'; // The trail entry for a response the engine discarded whose status is outside 400-599 — reachable // only through a caller-widened `retryableStatuses`. export {RetryDiscardedResponseError} from './retry/errors.js'; diff --git a/packages/core/src/retry/attempt-trail.test.ts b/packages/core/src/retry/attempt-trail.test.ts new file mode 100644 index 0000000..b5835be --- /dev/null +++ b/packages/core/src/retry/attempt-trail.test.ts @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/attempt-trail.test.ts +// Exercises: RETRY-34 (the prior-attempt trail rides alongside the surfaced error rather than +// replacing it; the surfaced instance is skipped; a run with no priors leaves no trail behind), +// XCUT-1 (recording a trail never changes the surfaced value's class). +import {describe, expect, test} from 'bun:test'; +import {IoError} from '../io/errors.js'; +import {CancellationError} from '../seams/transport.js'; +import {recordAttempts, retryAttempts} from './attempt-trail.js'; + +describe('retryAttempts -- the read side', () => { + test('returns an empty list for an error that never went through the engine', () => { + expect(retryAttempts(new IoError('never retried'))).toEqual([]); + }); + + test('returns an empty list for a primitive, which cannot carry a trail at all', () => { + // RETRY-34's trail is keyed by identity, so a string, a number or a symbol throw passes + // through the engine unchanged and unannotated rather than being wrapped to make room. + expect(retryAttempts('a bare string throw')).toEqual([]); + expect(retryAttempts(42)).toEqual([]); + expect(retryAttempts(Symbol('thrown'))).toEqual([]); + expect(retryAttempts(null)).toEqual([]); + expect(retryAttempts(undefined)).toEqual([]); + }); + + test('returns the recorded attempts oldest first', () => { + const first = new IoError('first'); + const second = new IoError('second'); + const surfaced = new IoError('third'); + + recordAttempts(surfaced, [first, second]); + + expect(retryAttempts(surfaced)).toEqual([first, second]); + }); + + test('hands back a frozen list, so one caller cannot edit a trail another caller holds', () => { + const surfaced = new IoError('surfaced'); + recordAttempts(surfaced, [new IoError('prior')]); + + const attempts = retryAttempts(surfaced); + + expect(Object.isFrozen(attempts)).toBe(true); + }); + + test('does not read a trail off the cause chain -- only off the instance itself', () => { + const inner = new IoError('inner'); + recordAttempts(inner, [new IoError('prior')]); + const outer = new IoError('outer', {cause: inner}); + + expect(retryAttempts(outer)).toEqual([]); + }); +}); + +describe('recordAttempts -- the write side', () => { + test('leaves the class and identity of the error untouched (XCUT-1)', () => { + const surfaced = new CancellationError('operation cancelled'); + + recordAttempts(surfaced, [new IoError('prior')]); + + expect(surfaced).toBeInstanceOf(CancellationError); + expect(surfaced.name).toBe('CancellationError'); + }); + + test('adds no own property, so a JSON or structured-clone round trip is unchanged', () => { + const surfaced = new IoError('surfaced'); + const before = Object.getOwnPropertyNames(surfaced).sort(); + + recordAttempts(surfaced, [new IoError('prior')]); + + expect(Object.getOwnPropertyNames(surfaced).sort()).toEqual(before); + }); + + test('copies the trail, so a later push by the engine cannot mutate a published list', () => { + const surfaced = new IoError('surfaced'); + const trail: unknown[] = [new IoError('prior')]; + + recordAttempts(surfaced, trail); + trail.push(new IoError('added afterwards')); + + expect(retryAttempts(surfaced)).toHaveLength(1); + }); +}); + +describe('recordAttempts -- the values it can and cannot key on', () => { + test('is a no-op on a frozen error rather than throwing', () => { + // The reason the trail is a side table and not an own property: a foreign error may be frozen + // or non-extensible, and `defineProperty` inside the failure path of the engine would then replace + // the failure the caller cares about with a TypeError. + const surfaced = Object.freeze(new IoError('frozen by its author')); + const prior = new IoError('prior'); + + expect(() => { + recordAttempts(surfaced, [prior]); + }).not.toThrow(); + expect(retryAttempts(surfaced)).toEqual([prior]); + }); + + test('is a no-op for a primitive surfaced value', () => { + expect(() => { + recordAttempts('a bare string throw', [new IoError('prior')]); + }).not.toThrow(); + expect(retryAttempts('a bare string throw')).toEqual([]); + }); + + test('records against a thrown function, which is an object for these purposes', () => { + const surfaced = (): void => undefined; + const prior = new IoError('prior'); + + recordAttempts(surfaced, [prior]); + + expect(retryAttempts(surfaced)).toEqual([prior]); + }); +}); + +describe('recordAttempts -- a reused error instance', () => { + test('an empty trail clears any entry a previous run left on a reused instance', () => { + // Error singletons are ordinary in fakes and in transports that reuse one instance. The RETRY-34 + // clause "on eventual success the prior trail MUST be discarded" is worth nothing if the NEXT run + // to surface that same instance still reports the old one. + const reused = new IoError('reused across runs'); + recordAttempts(reused, [new IoError('from the first run')]); + + recordAttempts(reused, []); + + expect(retryAttempts(reused)).toEqual([]); + }); + + test('the latest recording wins for a reused instance', () => { + const reused = new IoError('reused across runs'); + const older = new IoError('from the first run'); + const newer = new IoError('from the second run'); + + recordAttempts(reused, [older]); + recordAttempts(reused, [newer]); + + expect(retryAttempts(reused)).toEqual([newer]); + }); +}); diff --git a/packages/core/src/retry/attempt-trail.ts b/packages/core/src/retry/attempt-trail.ts new file mode 100644 index 0000000..2885160 --- /dev/null +++ b/packages/core/src/retry/attempt-trail.ts @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: MIT +// packages/core/src/retry/attempt-trail.ts + +/** + * RETRY-34's prior-attempt trail, held in a side table keyed by the surfaced throwable rather than + * written onto it. + * + * A `WeakMap` because the key is the error a caller is about to catch: the entry has to disappear + * when that error does, and a `Map` here would pin every failed request's error graph -- including + * whatever the buffered `HttpStatusError` bodies hold -- for the life of the process. + */ +const attemptTrails = new WeakMap(); + +/** One shared frozen empty list, so the common "no trail" answer allocates nothing. */ +const NO_ATTEMPTS: readonly unknown[] = Object.freeze([]); + +/** + * The subset of throwables a `WeakMap` can key on. Objects and functions qualify; primitives do + * not, and a registered symbol throws when used as a weak key, so symbols are excluded outright + * rather than probed. + */ +function trailKey(value: unknown): object | undefined { + if (typeof value === 'function') return value; + return typeof value === 'object' && value !== null ? value : undefined; +} + +/** + * Records the errors of the earlier attempts against the error the retry engine is about to + * surface (`RETRY-34`). + * + * Written into a side table instead of onto the error for three reasons the engine cannot rule out + * about a throwable it did not construct: it may be frozen or otherwise non-extensible, so a + * `defineProperty` in the failure path would itself throw and replace the failure the caller cares + * about; it may be a primitive, which can carry nothing at all; and `suppressed` already means + * "the one secondary" on `SuppressedErrorLike`, so reusing the name would collide with + * `RECOV-12`'s pairing. + * + * `attempts` MUST already have the surfaced instance filtered out -- `RETRY-34`'s skip-self clause, + * applied by the engine's `attachTrail` (`engine.ts`), which is the one caller. An empty + * `attempts` DELETES any entry a previous run left, so a transport that reuses a single error + * instance across calls reports the trail of the run that just surfaced it rather than a stale one. + * + * The list is copied and frozen, so the engine's own mutable `trail` array cannot be observed + * growing after the fact. + * + * @param error - the throwable the engine is surfacing; a primitive is silently ignored. + * @param attempts - the earlier attempts' errors, oldest first, surfaced instance excluded. + * + * @internal + */ +export function recordAttempts( + error: unknown, + attempts: readonly unknown[], +): void { + const key = trailKey(error); + if (key === undefined) return; + if (attempts.length === 0) { + attemptTrails.delete(key); + return; + } + attemptTrails.set(key, Object.freeze([...attempts])); +} + +/** + * The errors of the attempts that came before the one you caught. + * + * The retry pillar surfaces the **final** attempt's own error, unwrapped: `instanceof` against it + * answers the same for one attempt as for ten, and a cancellation that ended a backoff wait arrives + * as `CancellationError` rather than as something carrying one. The earlier attempts are not + * discarded — they are recorded here, so `retryAttempts(caught).length + 1` is how many sends the + * pillar made. A worked example is in `docs/sdk-documentation/pipelines.md`. + * + * Oldest first, and the error you passed in is never a member of its own trail (`RETRY-34`'s + * skip-self clause, which matters because a transport may reuse one error instance across + * attempts). A run that succeeded, a failure that was never retried, and any error this SDK did not + * surface from a retry loop all answer with an empty list — this never throws and never returns + * `undefined`. + * + * The result is frozen, and it is read by identity: an error reached through another error's + * `cause` has its own trail or none, never its wrapper's. + * + * @param error - the throwable a retrying pipeline surfaced; any value, not necessarily an `Error`. + * @returns the earlier attempts' errors, oldest first, or an empty list when there are none. + * + * @public + */ +export function retryAttempts(error: unknown): readonly unknown[] { + const key = trailKey(error); + if (key === undefined) return NO_ATTEMPTS; + return attemptTrails.get(key) ?? NO_ATTEMPTS; +} diff --git a/packages/core/src/retry/engine.test.ts b/packages/core/src/retry/engine.test.ts index cbb1919..55d6350 100644 --- a/packages/core/src/retry/engine.test.ts +++ b/packages/core/src/retry/engine.test.ts @@ -3,7 +3,9 @@ // Exercises: RETRY-7/8 (both axes gate), RETRY-20 (a hint replaces the schedule, unjittered), RETRY-22 // (a pacing failure never masks the upstream failure), RETRY-26/31 (cancellable wait, zero delay // inline), RETRY-27/RECOV-20 (total-timeout budget with per-attempt shrinking), RETRY-32 (no attempts -// after cancellation), RETRY-34 (suppressed trail on failure, discarded on success, skip-self), +// after cancellation), RETRY-34 (the prior-attempt trail rides BESIDE the surfaced error, discarded +// on success, skip-self), XCUT-1 (the surfaced type does not depend on how many attempts ran -- the +// final attempt's own error is what the engine hands back, cancellation included), // RETRY-35/RECOV-16 (body released before the wait, bounded buffering), RETRY-36/RECOV-19 (503,503,200 // terminates on the 200; a surviving response is returned LIVE), RETRY-39/40 (delay precedence; a // throwing override is non-fatal), RETRY-42/RECOV-28 (per-call state). @@ -15,10 +17,12 @@ import {Protocol} from '../http/protocol.js'; import {Request} from '../http/request.js'; import {Response} from '../http/response.js'; import {Status} from '../http/status.js'; -import {IoError} from '../io/errors.js'; +import {IoError, TransportFailureError} from '../io/errors.js'; import {failure, success, type Outcome} from '../recovery/outcome.js'; +import {CancellationError} from '../seams/transport.js'; import type {SuppressedErrorLike} from '../suppress.js'; import {countingResponse} from '../testing/fake-transport.js'; +import {retryAttempts} from './attempt-trail.js'; import {runWithRetry, type RetryConfig, type RetryDispatch} from './engine.js'; import {retrySettings, type RetrySettings} from './settings.js'; @@ -29,9 +33,13 @@ const BARE_POST = Request.newBuilder() .build(); /** - * The suppressed pair is asserted on SHAPE, never `instanceof SuppressedError`: the native class is - * absent on this package's Node floor (>=20.3), where `suppress()` returns a structural stand-in and - * an `instanceof` assertion would silently assert nothing. + * The one remaining pairing this engine can build is RECOV-12's -- a release failure riding along + * with the primary it must not mask. The retry TRAIL is no longer a `SuppressedError` chain, so the + * only assertions below that use this predicate are the release ones. + * + * Asserted on SHAPE, never `instanceof SuppressedError`: the native class is absent on this + * package's Node floor (>=20.3), where `suppress()` returns a structural stand-in and an + * `instanceof` assertion would silently assert nothing. */ function isSuppressedShape(value: unknown): value is SuppressedErrorLike { return ( @@ -364,12 +372,11 @@ describe('cancellation (RETRY-26/32)', () => { }); }); -describe('suppressed trail (RETRY-34)', () => { - test('prior attempt failures ride along as suppressed on the surfaced error', async () => { - const dispatch = scriptedDispatch([ - failure(new IoError('first')), - failure(new IoError('second')), - ]); +describe('the prior-attempt trail (RETRY-34)', () => { + test('the FINAL attempt error is surfaced, with the priors reachable beside it', async () => { + const first = new IoError('first'); + const second = new IoError('second'); + const dispatch = scriptedDispatch([failure(first), failure(second)]); const outcome = await runWithRetry( GET, @@ -379,12 +386,45 @@ describe('suppressed trail (RETRY-34)', () => { expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); + // Not a wrapper: the surfaced value IS attempt 2's own error, so a caller's `instanceof` reads + // the same here as it does after a single attempt. + expect(outcome.error).toBe(second); + expect(retryAttempts(outcome.error)).toEqual([first]); + }); + + test('the surfaced TYPE does not depend on how many attempts ran (XCUT-1)', async () => { + const one = await runWithRetry( + GET, + scriptedDispatch([failure(new TransportFailureError('refused'))]), + configOf({maxAttempts: 1, fixedDelayMs: 0}), + ); + // Distinct instances, because `scriptedDispatch` repeats its last entry and RETRY-34's + // skip-self guard would otherwise empty the trail this row needs to be non-empty. + const three = await runWithRetry( + GET, + scriptedDispatch([ + failure(new TransportFailureError('refused')), + failure(new TransportFailureError('refused')), + failure(new TransportFailureError('refused')), + ]), + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); + + expect(one.kind).toBe('failure'); + expect(three.kind).toBe('failure'); + if (one.kind !== 'failure' || three.kind !== 'failure') return; + // Until 2026-09-05 the second of these was a `SuppressedError` and this row was false: the + // surfaced CLASS was a function of the attempt budget, which is what XCUT-1's conformance + // clause ("assert the surfaced error is the cancellation type") catches on the abort path. + expect(one.error).toBeInstanceOf(TransportFailureError); + expect(three.error).toBeInstanceOf(TransportFailureError); + expect(retryAttempts(three.error)).toHaveLength(2); }); test('the trail is discarded entirely on eventual success', async () => { + const first = new IoError('first'); const dispatch = scriptedDispatch([ - failure(new IoError('first')), + failure(first), success(countingResponse(200).response), ]); @@ -395,10 +435,12 @@ describe('suppressed trail (RETRY-34)', () => { ); expect(outcome.kind).toBe('success'); + // Nothing was recorded anywhere: the discarded failure carries no trail of its own either. + expect(retryAttempts(first)).toEqual([]); }); }); -describe('suppressed trail -- skip-self and single-attempt shapes (RETRY-34)', () => { +describe('the trail -- skip-self and single-attempt shapes (RETRY-34)', () => { test('a reused instance never suppresses itself (RETRY-34 skip-self)', async () => { const reused = new IoError('same instance every time'); const dispatch = scriptedDispatch([failure(reused)]); @@ -410,6 +452,8 @@ describe('suppressed trail -- skip-self and single-attempt shapes (RETRY-34)', ( ); expect(outcome).toEqual(failure(reused)); + // Skip-self leaves nothing at all for a transport that reuses one instance across attempts. + expect(retryAttempts(reused)).toEqual([]); }); test('a single failed attempt surfaces its error unwrapped', async () => { @@ -419,6 +463,7 @@ describe('suppressed trail -- skip-self and single-attempt shapes (RETRY-34)', ( expect(await runWithRetry(GET, dispatch, configOf())).toEqual( failure(only), ); + expect(retryAttempts(only)).toEqual([]); }); test('a discarded 503 becomes a buffered HttpStatusError in the trail (RECOV-16)', async () => { @@ -438,10 +483,8 @@ describe('suppressed trail -- skip-self and single-attempt shapes (RETRY-34)', ( expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.error).toBeInstanceOf(IoError); - expect(outcome.error.suppressed).toBeInstanceOf(HttpStatusError); + expect(outcome.error).toBeInstanceOf(IoError); + expect(retryAttempts(outcome.error)[0]).toBeInstanceOf(HttpStatusError); }); }); @@ -469,13 +512,10 @@ describe('a discarded response OUTSIDE the 400-599 band (V14/N2, XCUT-8)', () => expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.suppressed).toBeInstanceOf( - RetryDiscardedResponseError, - ); - expect(outcome.error.suppressed).not.toBeInstanceOf(HttpStatusError); - expect((outcome.error.suppressed as {status: number}).status).toBe(200); + const [discarded] = retryAttempts(outcome.error); + expect(discarded).toBeInstanceOf(RetryDiscardedResponseError); + expect(discarded).not.toBeInstanceOf(HttpStatusError); + expect((discarded as {status: number}).status).toBe(200); }); test('a discarded 404 still becomes an HttpStatusError -- the band is unchanged', async () => { @@ -496,9 +536,7 @@ describe('a discarded response OUTSIDE the 400-599 band (V14/N2, XCUT-8)', () => expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.suppressed).toBeInstanceOf(HttpStatusError); + expect(retryAttempts(outcome.error)[0]).toBeInstanceOf(HttpStatusError); }); }); @@ -599,9 +637,7 @@ describe('the inter-attempt wait -- degenerate and hostile delays', () => { // instead of silently becoming an extra attempt. expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.error).toBeInstanceOf(RangeError); + expect(outcome.error).toBeInstanceOf(RangeError); expect(dispatch.calls).toHaveLength(1); }); }); @@ -663,6 +699,30 @@ describe('cancellation while an attempt is in flight (RETRY-32)', () => { }); }); +describe('a cancelled backoff surfaces the cancellation TYPE (XCUT-1)', () => { + test('the surfaced error is CancellationError, with the prior attempt beside it', async () => { + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch: RetryDispatch = () => { + controller.abort(); + return Promise.resolve(failure(first)); + }; + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + // `abortToSdkError` maps the abort to this type one line before the trail is attached; until + // 2026-09-05 the trail wrapper undid the mapping immediately, and a cancelled backoff ALWAYS + // has a non-empty trail -- so `instanceof CancellationError` was false for every one of them. + expect(outcome.error).toBeInstanceOf(CancellationError); + expect(retryAttempts(outcome.error)).toEqual([first]); + }); +}); + describe('a throwing attempt still carries the trail (RETRY-33/34)', () => { test('a throw from inside the attempt is folded into a failure outcome, trail intact', async () => { const calls: number[] = []; @@ -681,16 +741,14 @@ describe('a throwing attempt still carries the trail (RETRY-33/34)', () => { expect(calls).toEqual([1, 2]); expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect(outcome.error.error).toBeInstanceOf(RangeError); + expect(outcome.error).toBeInstanceOf(RangeError); // RETRY-34: attempt 1's failure would have been lost had the throw escaped as a rejection. - expect((outcome.error.suppressed as Error).message).toBe('first'); + expect((retryAttempts(outcome.error)[0] as Error).message).toBe('first'); }); }); -describe('the suppressed trail with more than two entries (RETRY-34)', () => { - test('three distinct attempt failures fold into a nested chain', async () => { +describe('the trail with more than two entries (RETRY-34)', () => { + test('three distinct attempt failures list flat, oldest first', async () => { const dispatch = scriptedDispatch([ failure(new IoError('first')), failure(new IoError('second')), @@ -705,15 +763,12 @@ describe('the suppressed trail with more than two entries (RETRY-34)', () => { expect(outcome.kind).toBe('failure'); if (outcome.kind !== 'failure') return; - expect(isSuppressedShape(outcome.error)).toBe(true); - if (!isSuppressedShape(outcome.error)) return; - expect((outcome.error.error as Error).message).toBe('third'); - // The two priors folded into a nested pair, oldest innermost. - const folded = outcome.error.suppressed; - expect(isSuppressedShape(folded)).toBe(true); - if (!isSuppressedShape(folded)) return; - expect((folded.error as Error).message).toBe('second'); - expect((folded.suppressed as Error).message).toBe('first'); + expect((outcome.error as Error).message).toBe('third'); + // A flat list in wire order. The `SuppressedError` pair is binary, so N priors used to fold + // into a nested chain a caller had to walk; nothing about RETRY-34 asked for that shape. + expect( + retryAttempts(outcome.error).map(entry => (entry as Error).message), + ).toEqual(['first', 'second']); }); }); diff --git a/packages/core/src/retry/engine.ts b/packages/core/src/retry/engine.ts index 0403d76..a394c17 100644 --- a/packages/core/src/retry/engine.ts +++ b/packages/core/src/retry/engine.ts @@ -8,7 +8,7 @@ import type {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; import {failure, type Outcome} from '../recovery/outcome.js'; import {releaseQuietly, withReleaseFailure} from '../recovery/release.js'; -import {suppress} from '../suppress.js'; +import {recordAttempts} from './attempt-trail.js'; import {stampAttempt} from './attempt-stamp.js'; import {computeDelay} from './backoff.js'; import {RetryDiscardedResponseError} from './errors.js'; @@ -247,27 +247,35 @@ async function decideRetry( } /** - * RETRY-34: prior failures ride along as `suppressed` on the surfaced error; the surfaced instance - * itself is skipped, so a reused throwable cannot suppress itself. On success the trail is discarded - * whole. + * RETRY-34: prior failures ride ALONGSIDE the surfaced error, recorded in `attempt-trail.ts`'s side + * table and read back through the public `retryAttempts()`. The surfaced instance itself is skipped, + * so a reused throwable never appears in its own trail. On success the trail is discarded whole -- + * nothing is written, and the outcome is returned untouched. * - * The suppressed pair is a binary shape, so N entries fold into a nested chain. Built through Phase - * 4b's `suppress()` helper rather than `new SuppressedError(...)`: the native class reached Node only - * in 24.0.0 and this package's floor is `>=20.3`, so the direct form neither type-checks nor runs - * there. Argument order is controlled explicitly -- native `using` disposal builds the pair the other - * way round, making the *later* error primary. + * **The outcome's error is returned unchanged, class and identity intact.** Until 2026-09-05 this + * function wrapped it in a `SuppressedError` pair instead, which made the surfaced TYPE a function of + * how many attempts ran: one attempt surfaced `TransportFailureError`, three surfaced a wrapper with + * the `TransportFailureError` at `.error`. XCUT-1's conformance clause -- "assert the surfaced error + * is the cancellation type" -- is the row that catches it, because a cancellation during backoff + * ALWAYS has a non-empty trail: `abortToSdkError` maps the abort to `CancellationError` below, and + * the wrapper undid that mapping on the very next line. RETRY-34 asks for the prior failures to be + * "attached to the surfaced exception", which is the JVM's `addSuppressed` -- the exception stays + * what it is and grows a list -- not for the exception to be replaced by a container. + * + * `suppress()` keeps its RECOV-12 job elsewhere in this file: `withReleaseFailure` pairs a release + * failure with the primary it must not mask. That is a genuine two-value pairing; an N-entry attempt + * history folded into a binary shape was never one. */ -function withTrail( +function attachTrail( outcome: Outcome, trail: readonly unknown[], ): Outcome { if (outcome.kind === 'success') return outcome; - const prior = trail.filter(entry => entry !== outcome.error); - if (prior.length === 0) return outcome; - const folded = prior.reduce((accumulated, entry) => - suppress(entry, accumulated, 'earlier retry attempt failed'), + recordAttempts( + outcome.error, + trail.filter(entry => entry !== outcome.error), ); - return failure(suppress(outcome.error, folded, 'retry attempts exhausted')); + return outcome; } /** @@ -346,12 +354,13 @@ function maybeEmitExhausted( * iterative, so N retries build no continuation chain and no stack growth. RETRY-33's "every * terminal path returns an Outcome" is honored literally -- an attempt that throws is folded into a * failure outcome carrying the trail, rather than left to surface as a bare rejected promise that - * would drop RETRY-34's suppressed attempts on the floor. + * would drop RETRY-34's prior attempts on the floor. * * @param request - the captured template every attempt re-sends. * @param dispatch - performs one attempt and reports its outcome without throwing. * @param config - settings, clock, randomness, signal, and the optional delay override. - * @returns the terminal outcome, with RETRY-34's suppressed trail attached on failure. + * @returns the terminal outcome. On failure the error is the FINAL attempt's own, unwrapped, with + * RETRY-34's prior-attempt trail recorded beside it for `retryAttempts()`. * * @internal */ @@ -383,7 +392,7 @@ export async function runWithRetry( // and silently missed a cancelled backoff. The raw reason is kept as `.cause`. if (config.signal?.aborted === true) { const cancellation = abortToSdkError(config.signal, config.signal.reason); - return withTrail(failure(cancellation), trail); + return attachTrail(failure(cancellation), trail); } try { @@ -396,7 +405,7 @@ export async function runWithRetry( const decision = await runAttempt(dispatch, state); if (decision.kind === 'stop') { maybeEmitExhausted(decision.outcome, trail.length, state); - return withTrail(decision.outcome, trail); + return attachTrail(decision.outcome, trail); } trail.push(decision.error); @@ -418,7 +427,7 @@ export async function runWithRetry( // `stampAttempt`'s header build, `toHttpError`'s body drain, and a misbehaving injected // clock's `sleep` -- and letting any of them escape would discard the whole suppressed trail // RETRY-34 requires the surfaced failure to carry. - return withTrail(failure(error), trail); + return attachTrail(failure(error), trail); } } } diff --git a/packages/core/src/retry/errors.ts b/packages/core/src/retry/errors.ts index ed0ba07..bfaeccc 100644 --- a/packages/core/src/retry/errors.ts +++ b/packages/core/src/retry/errors.ts @@ -7,8 +7,9 @@ import {DexpaceError} from '../http/errors.js'; * * Reachable only when a caller widens `RetrySettings.retryableStatuses` to include a non-error code: * the engine then retries a 2xx or 3xx, and every response it discards still owes `RETRY-34` an - * entry in the suppressed trail. `toHttpError` correctly returns `null` for such a status - * (`BODY-31` hands a non-error response back intact), so there is nothing for it to build. + * entry in the prior-attempt trail `retryAttempts()` reads. `toHttpError` correctly returns `null` + * for such a status (`BODY-31` hands a non-error response back intact), so there is nothing for it + * to build. * * Until 2026-09-02 the engine fabricated `new HttpStatusError(200, …)` here — precisely the * "successful exception" `XCUT-8` forbids, constructed by core itself, and contradicting diff --git a/packages/core/src/retry/retry-dispatch.ts b/packages/core/src/retry/retry-dispatch.ts index b87b29e..9672fca 100644 --- a/packages/core/src/retry/retry-dispatch.ts +++ b/packages/core/src/retry/retry-dispatch.ts @@ -42,7 +42,9 @@ function attemptVia(config: RetryDispatchConfig): RetryDispatch { * @param request - the request to prepare, send, and possibly re-send. * @param config - the recovery chains, transport, and retry policy. * @returns the response of the terminal successful attempt. - * @throws Whatever the terminal Failure carries, with RETRY-34's suppressed trail attached. + * @throws Whatever the FINAL attempt failed with, unwrapped -- the same class a single-attempt run + * would have thrown. RETRY-34's earlier attempts are recorded beside it and read back through + * `retryAttempts()`. * * @internal */ diff --git a/packages/core/src/retry/retry-step.ts b/packages/core/src/retry/retry-step.ts index da146c0..3199311 100644 --- a/packages/core/src/retry/retry-step.ts +++ b/packages/core/src/retry/retry-step.ts @@ -113,6 +113,14 @@ function configFrom( * `ctx.fork` is asserted rather than checked -- RETRY is in `PILLAR_STAGES`, so its absence means the * descriptor was installed somewhere it cannot be, which is a programmer error. * + * **What it throws when it gives up is the FINAL attempt's own error, unwrapped.** The class you + * catch does not depend on how many attempts ran: a transport failure surfaces as + * `TransportFailureError` whether `maxAttempts` was 1 or 3, and an abort that ended a backoff wait + * surfaces as `CancellationError` (`XCUT-1`). The earlier attempts' errors are not lost -- read them + * with `retryAttempts(caught)`, oldest first (`RETRY-34`). A response the loop discards is always + * closed first; the response that ENDS the loop is returned live and unread, and closing it is + * yours. + * * @param options - settings overrides and the injected clock, randomness, and delay override. * @returns the descriptor to install in a pipeline's RETRY slot. * From 32c9f2b9141022dba7ae29934c39ac4a7ee96730 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:48:18 +0300 Subject: [PATCH 040/100] test(xcut): conformance rows for the surfaced retry error and its trail Four rows across the two suites the issue names, all four red against the parent commit's engine and green against this one: retry-safety.conformance.test.ts - one GET at a closed port surfaces `TransportFailureError` for maxAttempts 1 AND for 3; the dispatch count is what proves the budgets actually differed (XCUT-1) - the two priors are reachable through `retryAttempts()`, oldest first, with the surfaced instance excluded (RETRY-34) cancellation-and-timeout.conformance.test.ts - `expect(surfaced).toBeInstanceOf(CancellationError)` at the TOP level, which is XCUT-1's conformance clause unqualified. The existing row asserted only on a chain walk, and its own comment said why: the top level was the `SuppressedError`. A caller writing `catch` has no walk. - the 500 that provoked the retry is still reachable in the trail as a buffered `HttpStatusError` (RETRY-34/RECOV-16) `chainOf`'s doc comment is rewritten rather than deleted: the walk still earns its place proving the raw abort survived as `cause` and that no `TimeoutError` hides one hop down (XCUT-3), but it is no longer the only assertion. Refs #72, #67. --- ...ncellation-and-timeout.conformance.test.ts | 65 ++++++++++++++---- .../xcut/retry-safety.conformance.test.ts | 67 ++++++++++++++++++- 2 files changed, 117 insertions(+), 15 deletions(-) diff --git a/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts b/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts index c1a1952..a97008a 100644 --- a/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts +++ b/tests/conformance/xcut/cancellation-and-timeout.conformance.test.ts @@ -9,7 +9,12 @@ // These run the invariants through the fully composed retry+redirect+auth+logging pipeline over a // real socket, which is what this suite adds over 5a's and Phase 2's own unit-level coverage. import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; -import {CancellationError, Request} from '@dexpace/core'; +import { + CancellationError, + HttpStatusError, + Request, + retryAttempts, +} from '@dexpace/core'; import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; import {rejectionOf} from './fixtures/settle.js'; @@ -20,10 +25,14 @@ let server: XcutFixtureServer; * Walks everything a surfaced failure can nest a prior error under, visiting by identity so a cyclic * chain terminates -- the same discipline `XCUT-9` puts on the classifier. * - * The walk is necessary because 5a's engine folds the retry trail into a `SuppressedError` - * (`RETRY-34`), so the cancellation that ended a backoff wait arrives as `.error` beneath a wrapper - * rather than as the top-level throwable. Asserting on the top level alone would test the wrapping, - * not the invariant. + * Kept after #72 made the top-level assertion possible, because the two answer different questions. + * `CancellationError` carries the raw `AbortSignal.reason` as its `cause`, so the walk is what + * proves the ambient abort was not swallowed on the way out, and it is also what catches a + * `TimeoutError` hiding one hop down where `XCUT-3` forbids one. What it must no longer be is the + * ONLY assertion: 5a's engine folded the retry trail into a `SuppressedError` (`RETRY-34`), the + * cancellation arrived as `.error` beneath that wrapper, and a chain walk was the only way to find + * it -- which is precisely the defect, since a caller writing `catch (e) { e instanceof + * CancellationError }` has no walk. Every row below asserts the top level too. */ function* chainOf(error: unknown): Generator { const seen = new Set(); @@ -176,13 +185,17 @@ describe('XCUT-3: an inter-attempt wait is promptly cancellable', () => { }, 50).unref(); const surfaced = await rejectionOf(pending); - // Asserted on the chain rather than the top-level type, because the top level is a - // `SuppressedError` pairing the cancellation with the prior attempt's failure. What the chain - // must carry is the SDK's OWN type: until 2026-09-02 the retry engine surfaced `signal.reason` - // verbatim, so a cancelled backoff arrived as a bare DOMException `AbortError` while the - // transport path mapped the identical abort to `CancellationError` -- one requirement, two - // types, depending on which layer noticed. Both layers now map through the same shape, so this - // asserts the type as well as XCUT-3's letter. + // XCUT-1's conformance clause, at the top level and unqualified: "assert the surfaced error is + // the cancellation type". Two separate defects had to be fixed for this line to hold. Until + // 2026-09-02 the engine surfaced `signal.reason` verbatim, so a cancelled backoff arrived as a + // bare DOMException `AbortError` while the transport mapped the identical abort to + // `CancellationError` -- one requirement, two types, depending on which layer noticed. Until + // 2026-09-05 the mapping was then undone one line later by the retry trail's `SuppressedError` + // wrapper, and a cancelled backoff ALWAYS has a non-empty trail, so this row was false for + // every reachable case. + expect(surfaced).toBeInstanceOf(CancellationError); + // The chain still has to carry the raw abort (the ambient flag survived the mapping) and must + // not carry a timeout, which is XCUT-3's own letter. expect(carriesCancellation(surfaced)).toBe(true); expect(carriesSdkCancellationError(surfaced)).toBe(true); expect(carriesTimeout(surfaced)).toBe(false); @@ -211,3 +224,31 @@ describe('XCUT-3: an inter-attempt wait is promptly cancellable', () => { await pipeline.close(); }); }); + +describe('RETRY-34: the trail survives the unwrapped cancellation', () => { + test('the attempt the cancelled wait was scheduled for stays reachable', async () => { + const pipeline = buildComposedPipeline({ + retry: {settings: {maxAttempts: 5, initialDelayMs: 60_000}}, + }); + const controller = new AbortController(); + const pending = pipeline.runtime.send( + Request.newBuilder().url(`${server.url}/fail-500`).build(), + undefined, + controller.signal, + ); + setTimeout(() => { + controller.abort(); + }, 50).unref(); + const surfaced = await rejectionOf(pending); + + // Surfacing the cancellation unwrapped is not allowed to LOSE the 500 that provoked the retry + // in the first place -- that was the one thing the `SuppressedError` wrapper did buy. It rides + // in the trail instead, and the retired response is a buffered `HttpStatusError` (RECOV-16). + const priors = retryAttempts(surfaced); + expect(priors).toHaveLength(1); + expect(priors[0]).toBeInstanceOf(HttpStatusError); + expect((priors[0] as HttpStatusError).status).toBe(500); + + await pipeline.close(); + }); +}); diff --git a/tests/conformance/xcut/retry-safety.conformance.test.ts b/tests/conformance/xcut/retry-safety.conformance.test.ts index 67aa4d7..0be9dd0 100644 --- a/tests/conformance/xcut/retry-safety.conformance.test.ts +++ b/tests/conformance/xcut/retry-safety.conformance.test.ts @@ -2,15 +2,25 @@ // tests/conformance/xcut/retry-safety.conformance.test.ts // Exercises: XCUT-10 (retry-SAFETY is decided at the retry step independently of retryability, and // applies uniformly to protocol AND transport failures -- the gate must not special-case a transport -// error that never reached the server). +// error that never reached the server), XCUT-1 (the class of the surfaced error does not depend on +// how many attempts the pillar spent), RETRY-34 (the earlier attempts stay reachable beside it). // -// The five rows are the ones XCUT-10's own conformance clause names, run for the first time against +// The five XCUT-10 rows are the ones its own conformance clause names, run for the first time against // the composed pipeline rather than 5a's unit-level harness. Each asserts on dispatches that actually // reached the terminal transport, which is the only vantage point where "was it re-sent?" is visible. +// The two XCUT-1/RETRY-34 rows below need the same vantage point for the opposite reason: the budget +// they vary is only observable as a dispatch count. import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; -import {Request, streamBody, stringBody} from '@dexpace/core'; +import { + Request, + retryAttempts, + streamBody, + stringBody, + TransportFailureError, +} from '@dexpace/core'; import {buildComposedPipeline} from './fixtures/composed-pipeline.js'; import {startFixtureServer, type XcutFixtureServer} from './fixtures/server.js'; +import {rejectionOf} from './fixtures/settle.js'; let server: XcutFixtureServer; @@ -27,6 +37,16 @@ function retrying(): {settings: {maxAttempts: number; initialDelayMs: number}} { return {settings: {maxAttempts: 3, initialDelayMs: 1}}; } +/** + * A GET at a closed port. Retry-SAFE (idempotent, body-less) and retryABLE (the transports map a + * refused connection to `TransportFailureError`, an `IoError`), so the pillar spends its whole + * budget and every attempt fails the same way -- which is what makes the surfaced CLASS the only + * variable between the two budgets below. + */ +function unreachable(): Request { + return Request.newBuilder().url('http://127.0.0.1:1/').method('GET').build(); +} + describe('XCUT-10: retry-safety on a body-less request follows method idempotence', () => { test('retries a body-less GET against a retryable protocol failure', async () => { const pipeline = buildComposedPipeline({retry: retrying()}); @@ -104,3 +124,44 @@ describe('XCUT-10: retry-safety on a body-bearing request follows body replayabi await pipeline.close(); }); }); + +describe('XCUT-1/RETRY-34: what a retrying pipeline surfaces when it gives up', () => { + test('the same failure surfaces as TransportFailureError for maxAttempts 1 and for 3', async () => { + const once = buildComposedPipeline({ + retry: {settings: {maxAttempts: 1}}, + }); + const thrice = buildComposedPipeline({retry: retrying()}); + + const afterOne = await rejectionOf(once.runtime.send(unreachable())); + const afterThree = await rejectionOf(thrice.runtime.send(unreachable())); + + expect(once.dispatches()).toBe(1); + expect(thrice.dispatches()).toBe(3); + // The row #72 exists for. Until 2026-09-05 the three-attempt case surfaced a `SuppressedError` + // holding the transport failure at `.error`, so one condition had two surfaced classes and the + // discriminator was the attempt budget -- something no caller writing `catch` can see. + expect(afterOne).toBeInstanceOf(TransportFailureError); + expect(afterThree).toBeInstanceOf(TransportFailureError); + + await once.close(); + await thrice.close(); + }); + + test('the earlier attempts are reachable beside it, oldest first, self excluded', async () => { + const pipeline = buildComposedPipeline({retry: retrying()}); + + const surfaced = await rejectionOf(pipeline.runtime.send(unreachable())); + + // RETRY-34 through the composed pipeline: three sends, so two priors, and the surfaced instance + // is not a member of its own trail. + const priors = retryAttempts(surfaced); + expect(pipeline.dispatches()).toBe(3); + expect(priors).toHaveLength(2); + expect(priors.every(prior => prior instanceof TransportFailureError)).toBe( + true, + ); + expect(priors).not.toContain(surfaced); + + await pipeline.close(); + }); +}); From 0778a882407aaa62c5072c02d62091a36fc1cb50 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:48:18 +0300 Subject: [PATCH 041/100] docs: the retry pillar throws the last attempt's error, and retryAttempts reads the rest - pipelines.md gains the note as the FIRST of the retry bullets, with a worked example that `check-fences.mjs` typechecks against the built package. The lead sentence said "two notes each"; it is three now. - errors.md: the `CancellationError`/`TransportFailureError` narrowing section states outright that installing a retry pillar does not change either check, which is the whole point of #72. The `RetryDiscardedResponseError` row at :188 stops calling the trail "suppressed". - write-a-response-handler.md: RECOV-12's release pairing is now documented as the ONLY reason this SDK builds a `SuppressedError`. Verified against every `suppress()` and `withReleaseFailure()` call site in packages/core/src, all of which are a close() that threw with an error already in flight. Both new cross-links resolve against pipelines.md's real headings (checked by script, not by eye). Refs #72, #67. --- docs/sdk-documentation/errors.md | 7 +++- docs/sdk-documentation/pipelines.md | 38 ++++++++++++++++++- .../write-a-response-handler.md | 12 ++++++ 3 files changed, 54 insertions(+), 3 deletions(-) diff --git a/docs/sdk-documentation/errors.md b/docs/sdk-documentation/errors.md index b9985fa..fd62950 100644 --- a/docs/sdk-documentation/errors.md +++ b/docs/sdk-documentation/errors.md @@ -112,6 +112,11 @@ export async function call(): Promise { the two apart at the point of abort — which is exactly how a transport decides which of the two errors to raise. +**Neither narrowing changes when a retry pillar is installed.** What `retryStep` throws once it gives +up is the final attempt's own error, unwrapped, so the two `instanceof` checks above read the same at +`maxAttempts: 1` and at `maxAttempts: 3`. The earlier attempts ride beside it and are read with +`retryAttempts(error)` — see [`pipelines.md`](./pipelines.md#the-four-shipped-pillars). + ## Narrowing helpers Three predicates exist for the cases where `instanceof` on a union is tedious: @@ -185,7 +190,7 @@ Two more joined them on the same date, both from `XCUT-8`'s "never fabricate a s | Error | Raised by | |---|---| | `HttpStatusValidationError` | `new HttpStatusError(status, …)` when `status` is not an integer in 400–599. The constructor validated nothing before, so a consumer could build an `HttpStatusError` claiming a `200`. `toHttpError` is the total form — it returns `null` instead of throwing | -| `RetryDiscardedResponseError` | the retry engine's suppressed trail, for a response it discarded whose status is outside 400–599. Reachable only if you widen `RetrySettings.retryableStatuses` to include a non-error code; the trail previously said `HttpStatusError` for it, which claimed an HTTP failure that had not happened | +| `RetryDiscardedResponseError` | the retry engine's prior-attempt trail (`retryAttempts()`), for a response it discarded whose status is outside 400–599. Reachable only if you widen `RetrySettings.retryableStatuses` to include a non-error code; the trail previously said `HttpStatusError` for it, which claimed an HTTP failure that had not happened | All of them descend from `DexpaceError`, so the broad catch works too. diff --git a/docs/sdk-documentation/pipelines.md b/docs/sdk-documentation/pipelines.md index 1336c5e..f6fe142 100644 --- a/docs/sdk-documentation/pipelines.md +++ b/docs/sdk-documentation/pipelines.md @@ -153,8 +153,42 @@ All three carry the same `InstrumentationBundle`, so trace and span identity sur | Auth | `authStep(settings)` | `credentials`, `tiers`, `challengeHook`, `bearerMarginMs` — see [`auth.md`](./auth.md) | | Logging | `loggingStep(settings?)` | `granularity`, `severity`, `previewSizeBytes`, `droppedHeaderPolicy`, `logger`, `meter`, `tracerFactory` | -Retry and redirect are worth two notes each, because both surprise people: - +Retry and redirect are worth a few notes each, because both surprise people: + +- **What retry throws is the last attempt's own error.** The class you catch does not depend on how + many attempts ran: a refused connection is a `TransportFailureError` whether `maxAttempts` was 1 or + 3, and an abort that lands during a backoff wait is a `CancellationError` (`XCUT-1`). The earlier + attempts are not thrown away — `retryAttempts(caught)` returns them, oldest first, with the error + you passed in excluded from its own trail (`RETRY-34`): + + ```typescript + import { + retryAttempts, + TransportFailureError, + type Request, + type Runtime, + } from '@dexpace/core'; + + declare const runtime: Runtime; + declare const request: Request; + + export async function send(): Promise { + try { + await runtime.send(request); + } catch (error) { + if (error instanceof TransportFailureError) { + const spent = retryAttempts(error).length + 1; + console.error(`gave up after ${String(spent)} sends`, error.message); + } + throw error; + } + } + ``` + + The trail is a side table keyed by the error, not a property on it, so nothing is added to an + object you may not own; an error that never went through a retry loop answers with an empty list. + Until 2026-09-05 the pillar wrapped its terminal failure in a `SuppressedError` instead, which made + the surfaced class a function of the attempt budget. - **Retry pacing honours the server.** `Retry-After`, `X-RateLimit-Reset` and friends are parsed in a fixed precedence and win over computed backoff. Every computed delta is clamped to a 365-day ceiling that `RETRY-18` mandates — so a server that sends `X-RateLimit-Reset` in milliseconds diff --git a/docs/sdk-documentation/write-a-response-handler.md b/docs/sdk-documentation/write-a-response-handler.md index da5d364..7611147 100644 --- a/docs/sdk-documentation/write-a-response-handler.md +++ b/docs/sdk-documentation/write-a-response-handler.md @@ -82,6 +82,18 @@ release failure. **`instanceof SuppressedError` is not a valid test**: the class project's declared Node floor and a structurally identical stand-in is built there instead. Test the shape, or read `.error` unconditionally. +**A failed release is the only thing in this SDK that builds one.** Every site that constructs the +pairing — the response chain, the redirect and auth and retry pillars, the serde handlers, the SSE +stream, the paginator — is doing this one job: a `close()` that threw while an error was already in +flight. It is a genuine two-value shape, which is why it fits. + +The retry pillar used to build one for a *second* reason, folding N prior attempt errors into a +nested chain of pairs so that what you caught after three attempts was a wrapper rather than the +failure. It no longer does: it surfaces the last attempt's error unwrapped and records the earlier +ones beside it, read back with `retryAttempts()` (see [`pipelines.md`](./pipelines.md#the-four-shipped-pillars)). +So if you catch a `'SuppressedError'` from this SDK, `.suppressed` is a teardown failure, never an +earlier attempt. + **3. Only payload failures are re-typed.** `SERDE-12`: a malformed body or a shape mismatch becomes a `DeserializationError` with the original chained; a genuine stream failure propagates untouched, because re-wrapping it would tell a caller their payload was malformed when their socket dropped. From c377fea3cee76c992a9eac1684b81cc0437b2101 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:48:18 +0300 Subject: [PATCH 042/100] test(node): the retry trail case moves from "same wrapper" to "no wrapper" `tests/node-conformance/retry.test.mjs` names four runtime-divergent points in its header, and point 2 was RETRY-34's trail going through `suppress()` -- whose native-vs-fallback branch is decided by the runtime, and whose two legs this suite's matrix actually runs (`lts/*` has the global, the pinned `20.3.0` floor does not). #72 takes that branch off the retry path, so the case that asserted "the wrapper has the same shape on either runtime" would now be asserting a shape nothing builds. It becomes the stronger claim: neither leg produces a wrapper, `outcome.error` IS the last attempt's error, and `retryAttempts()` reads the trail through the `@dexpace/core` specifier and the built `dist/`, as a consumer does. The real-timer abort case gains one line: `outcome.error instanceof CancellationError`, over a real `AbortSignal` and a real `setTimeout`, which is the half the unit suite's injected clock cannot reach. OUT OF THE TASK FILE'S STATED PARTITION, deliberately and reported: the partition lists the two `tests/conformance/xcut/` files but not the Node tree, and this case asserts the exact behaviour D10 changes, so `test:node` (and therefore the preflight) is red without it. No wave-3 sibling touches this file -- D2 gives #74 auth plus transport-conformance and #75 rx. Refs #72, #67. --- tests/node-conformance/retry.test.mjs | 44 +++++++++++++++++++-------- 1 file changed, 31 insertions(+), 13 deletions(-) diff --git a/tests/node-conformance/retry.test.mjs b/tests/node-conformance/retry.test.mjs index 8fb48d7..c686241 100644 --- a/tests/node-conformance/retry.test.mjs +++ b/tests/node-conformance/retry.test.mjs @@ -9,9 +9,13 @@ // `AbortSignal.timeout()`, whose class and `name` are the runtime's, not this package's -- if // Node named it anything but `TimeoutError`, every timed-out request would silently stop being // retried and `bun test` would still be green. -// 2. RETRY-34's suppressed trail goes through `suppress()`, which picks the native `SuppressedError` -// or the shape-compatible fallback depending on the runtime. Bun has the global; the declared -// floor (`engines.node >=20.3`) does not. The trail's SHAPE has to be identical either way. +// 2. RETRY-34's trail used to go through `suppress()`, which picks the native `SuppressedError` or +// the shape-compatible fallback depending on the runtime -- Bun has the global, the declared +// floor (`engines.node >=20.3`) does not, and this suite's matrix runs both legs. #72 took that +// branch off the retry path entirely: the final attempt's own error is surfaced and the trail +// rides in a side table. So the assertion moved with it, from "the wrapper has the same shape on +// either runtime" to "neither runtime produces a wrapper", which is the stronger claim and the +// one a reintroduced `suppress()` would break differently on Node 20 than on Node 24. // 3. RETRY-35/RECOV-16's "release the discarded response" rides on Web Streams: a retired response is // drained to EOF by `toHttpError()`, an abandoned one is cancelled by `Response.close()`. Node's // `cancel()`/`pull()` timing is an independent implementation of Bun's. @@ -24,7 +28,14 @@ // `dist/` file path, per this suite's import rule. import assert from 'node:assert/strict'; import {describe, it} from 'node:test'; -import {Protocol, Request, Response, Status} from '@dexpace/core'; +import { + CancellationError, + Protocol, + Request, + Response, + retryAttempts, + Status, +} from '@dexpace/core'; import { isRetryableFailure, RETRYABLE_STATUSES, @@ -123,13 +134,12 @@ describe('retry classification on the declared Node floor', () => { }); describe('the retry engine on the declared Node floor', () => { - it('folds the suppressed trail into the same shape whether or not the runtime has SuppressedError', async () => { + it('surfaces the final attempt error itself, on a runtime that may lack SuppressedError', async () => { // Timeout aborts, because they are the retryable throwable this suite can build without reaching - // into another `dist/` module -- two of them exhaust the budget and produce a two-entry trail. - const dispatch = scriptedDispatch([ - failure(new DOMException('timed out', 'TimeoutError')), - failure(new DOMException('timed out again', 'TimeoutError')), - ]); + // into another `dist/` module -- two of them exhaust the budget and produce a one-entry trail. + const first = new DOMException('timed out', 'TimeoutError'); + const last = new DOMException('timed out again', 'TimeoutError'); + const dispatch = scriptedDispatch([failure(first), failure(last)]); const outcome = await runWithRetry( GET, @@ -139,9 +149,13 @@ describe('the retry engine on the declared Node floor', () => { assert.equal(dispatch.calls.length, 2); assert.equal(outcome.kind, 'failure'); - assert.equal(outcome.error.name, 'SuppressedError'); - assert.ok('error' in outcome.error); - assert.ok('suppressed' in outcome.error); + assert.equal(outcome.error, last); + assert.notEqual(outcome.error.name, 'SuppressedError'); + // RETRY-34's trail, read through the accessor as a CONSUMER reaches it -- the `@dexpace/core` + // specifier and the built `dist/`, not the engine's own module path. + const priors = retryAttempts(outcome.error); + assert.equal(priors.length, 1); + assert.equal(priors[0], first); }); it('releases a discarded response through the drain route, over Node Web Streams (RETRY-35)', async () => { @@ -217,5 +231,9 @@ describe('the retry engine on the declared Node floor', () => { assert.equal(outcome.kind, 'failure'); // The point of the case: it returned instead of sleeping out the full 60s backoff. assert.ok(defaultClock.monotonic() - startedAt < 5_000); + // XCUT-1 over a REAL AbortSignal and a REAL timer, which is the half the unit suite's injected + // clock cannot reach. The trail is non-empty here by construction, so before #72 this was a + // `SuppressedError` and the assertion below was false. + assert.ok(outcome.error instanceof CancellationError); }); }); From 4578841168ba499a21776dcc26ff19996433e556 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:48:18 +0300 Subject: [PATCH 043/100] =?UTF-8?q?docs(retry):=20the=20trail=20is=20not?= =?UTF-8?q?=20an=20attempt=20count=20=E2=80=94=20drop=20the=20false=20arit?= =?UTF-8?q?hmetic?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `retryAttempts(caught).length + 1` was documented as "how many sends the pillar made". It is not, and it is wrong on exactly the path #72 exists for: a cancellation observed at the RETRY-32 gate is synthesized by the engine (`engine.ts:394`), never raised by a send, so the trail already covers every attempt and the sum overstates by one. Two more reachable paths make the same sum wrong, both in the engine's own catch: `stampAttempt` throws before `dispatch` is called (`runAttempt` stamps first), and a `Clock.sleep` rejecting for something other than an abort fails after the attempt it followed is already in the trail. And narrowing the catch does not rescue it. `abortToSdkError` branches on `isTimeoutSignal` (`cancellation.ts:37-39`), so the RETRY-32 gate can synthesize a `TransportFailureError` too — which means `pipelines.md`'s worked example was wrong under the very class it narrowed to. That example no longer counts anything; it iterates the priors, which is what the accessor is for. Three prose sites reworded to state what the trail actually holds — one entry per attempt that failed BEFORE the surfaced error — plus the caveat: the surfaced error is an attempt's own only when it came from one. Two new engine cases pin it mechanically rather than by assertion in prose: a caller abort and a timeout abort each after ONE send, both asserting `dispatch.sends === 1` beside `retryAttempts(...).length === 1`, so any future `+ 1` claim has a red test under it. `core.api.md` unchanged: TSDoc prose only, api-extractor reports no signature change. Round 2 of #72. Refs #72, #67. --- docs/sdk-documentation/pipelines.md | 12 ++++- packages/core/src/retry/attempt-trail.ts | 14 ++++- packages/core/src/retry/engine.test.ts | 68 ++++++++++++++++++++++-- packages/core/src/retry/retry-step.ts | 7 +-- 4 files changed, 90 insertions(+), 11 deletions(-) diff --git a/docs/sdk-documentation/pipelines.md b/docs/sdk-documentation/pipelines.md index f6fe142..0ff6532 100644 --- a/docs/sdk-documentation/pipelines.md +++ b/docs/sdk-documentation/pipelines.md @@ -177,14 +177,22 @@ Retry and redirect are worth a few notes each, because both surprise people: await runtime.send(request); } catch (error) { if (error instanceof TransportFailureError) { - const spent = retryAttempts(error).length + 1; - console.error(`gave up after ${String(spent)} sends`, error.message); + for (const prior of retryAttempts(error)) { + console.error('an earlier attempt failed:', prior); + } } throw error; } } ``` + One entry per attempt that failed *before* the one you caught — which is not an attempt count, so + resist writing `length + 1`. The surfaced error is an attempt's own only when it came from one, and + sometimes it did not: a cancellation or timeout the engine observes between attempts is synthesized + at that gate, and so is a failure from stamping the attempt header, which runs before the request + goes out. On those paths the trail already covers every send. Narrowing the catch does not help — + a timeout signal is mapped to `TransportFailureError`, the same class a real send failure raises. + The trail is a side table keyed by the error, not a property on it, so nothing is added to an object you may not own; an error that never went through a retry loop answers with an empty list. Until 2026-09-05 the pillar wrapped its terminal failure in a `SuppressedError` instead, which made diff --git a/packages/core/src/retry/attempt-trail.ts b/packages/core/src/retry/attempt-trail.ts index 2885160..b526913 100644 --- a/packages/core/src/retry/attempt-trail.ts +++ b/packages/core/src/retry/attempt-trail.ts @@ -67,8 +67,18 @@ export function recordAttempts( * The retry pillar surfaces the **final** attempt's own error, unwrapped: `instanceof` against it * answers the same for one attempt as for ten, and a cancellation that ended a backoff wait arrives * as `CancellationError` rather than as something carrying one. The earlier attempts are not - * discarded — they are recorded here, so `retryAttempts(caught).length + 1` is how many sends the - * pillar made. A worked example is in `docs/sdk-documentation/pipelines.md`. + * discarded — they are recorded here, one entry per attempt that failed BEFORE the error you caught. + * A worked example is in `docs/sdk-documentation/pipelines.md`. + * + * **That is not an attempt count, and `length + 1` is not one either.** The arithmetic holds only + * when the surfaced error is itself an attempt's, and on three reachable paths it is not: a + * cancellation or timeout the engine observes at its `RETRY-32` gate is synthesized there rather + * than raised by a send; a failure from stamping the attempt header is raised before the request + * goes out; and a `Clock.sleep` that rejects for something other than an abort fails after the + * attempt it followed is already in the trail. On each of those the trail already accounts for every + * send, so adding one overstates it. Narrowing the catch does not rescue the sum — `abortToSdkError` + * yields `TransportFailureError` for a timeout signal, so even that class can reach you without a + * send behind it. * * Oldest first, and the error you passed in is never a member of its own trail (`RETRY-34`'s * skip-self clause, which matters because a transport may reuse one error instance across diff --git a/packages/core/src/retry/engine.test.ts b/packages/core/src/retry/engine.test.ts index 55d6350..3405c96 100644 --- a/packages/core/src/retry/engine.test.ts +++ b/packages/core/src/retry/engine.test.ts @@ -700,13 +700,24 @@ describe('cancellation while an attempt is in flight (RETRY-32)', () => { }); describe('a cancelled backoff surfaces the cancellation TYPE (XCUT-1)', () => { - test('the surfaced error is CancellationError, with the prior attempt beside it', async () => { - const controller = new AbortController(); - const first = new IoError('reset'); - const dispatch: RetryDispatch = () => { + /** Aborts from inside attempt 1, so the loop reaches its RETRY-32 check with a one-entry trail. */ + function abortAfterOneAttempt( + controller: AbortController, + first: unknown, + ): RetryDispatch & {sends: number} { + const dispatch = (): Promise> => { + dispatch.sends += 1; controller.abort(); return Promise.resolve(failure(first)); }; + dispatch.sends = 0; + return dispatch; + } + + test('the surfaced error is CancellationError, with the prior attempt beside it', async () => { + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch = abortAfterOneAttempt(controller, first); const outcome = await runWithRetry(GET, dispatch, { ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), @@ -721,6 +732,55 @@ describe('a cancelled backoff surfaces the cancellation TYPE (XCUT-1)', () => { expect(outcome.error).toBeInstanceOf(CancellationError); expect(retryAttempts(outcome.error)).toEqual([first]); }); + + test('the trail already covers every send, so length is NOT one less than the count', async () => { + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch = abortAfterOneAttempt(controller, first); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + // The surfaced error is SYNTHESIZED at the RETRY-32 gate, not raised by a send, so it is not an + // attempt's error and the trail already accounts for all of them. `length + 1` would say two + // sends where one happened -- which is why no TSDoc here offers that arithmetic. + expect(dispatch.sends).toBe(1); + expect(retryAttempts(outcome.error)).toHaveLength(1); + }); +}); + +describe('the RETRY-32 gate can synthesize a TransportFailureError too', () => { + test('a TIMEOUT signal takes the same synthesized path, as TransportFailureError', async () => { + // `abortToSdkError` branches on `isTimeoutSignal`, so the engine's own RETRY-32 gate can + // synthesize a `TransportFailureError` too. Narrowing a catch to that class therefore does NOT + // guarantee the caught error came from a send. + const controller = new AbortController(); + const first = new IoError('reset'); + const dispatch: RetryDispatch & {sends: number} = Object.assign( + (): Promise> => { + dispatch.sends += 1; + controller.abort(new DOMException('timed out', 'TimeoutError')); + return Promise.resolve(failure(first)); + }, + {sends: 0}, + ); + + const outcome = await runWithRetry(GET, dispatch, { + ...configOf({fixedDelayMs: 60_000, maxAttempts: 5}), + signal: controller.signal, + }); + + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(TransportFailureError); + expect(outcome.error).not.toBeInstanceOf(CancellationError); + expect(dispatch.sends).toBe(1); + expect(retryAttempts(outcome.error)).toHaveLength(1); + }); }); describe('a throwing attempt still carries the trail (RETRY-33/34)', () => { diff --git a/packages/core/src/retry/retry-step.ts b/packages/core/src/retry/retry-step.ts index 3199311..9a58392 100644 --- a/packages/core/src/retry/retry-step.ts +++ b/packages/core/src/retry/retry-step.ts @@ -117,9 +117,10 @@ function configFrom( * catch does not depend on how many attempts ran: a transport failure surfaces as * `TransportFailureError` whether `maxAttempts` was 1 or 3, and an abort that ended a backoff wait * surfaces as `CancellationError` (`XCUT-1`). The earlier attempts' errors are not lost -- read them - * with `retryAttempts(caught)`, oldest first (`RETRY-34`). A response the loop discards is always - * closed first; the response that ENDS the loop is returned live and unread, and closing it is - * yours. + * with `retryAttempts(caught)`, oldest first: one entry per attempt that failed BEFORE the error you + * caught, which is not the same as an attempt count (`RETRY-34`, and see `retryAttempts` for why the + * difference bites). A response the loop discards is always closed first; the response that ENDS the + * loop is returned live and unread, and closing it is yours. * * @param options - settings overrides and the injected clock, randomness, and delay override. * @returns the descriptor to install in a pipeline's RETRY slot. From c644473eb43ed6bdb44a8031fe2a47c2e56aff03 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:56:30 +0300 Subject: [PATCH 044/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20wave=203?= =?UTF-8?q?=20landed;=20re-anchor=20deviations=20item=203=20after=20#72?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 51 ++++++++++++++++++++++++++++++++++++++ docs/deviations.md | 6 ++--- 2 files changed, 54 insertions(+), 3 deletions(-) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index 157f3b6..d00d387 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -149,6 +149,19 @@ than its letter. `scripts/verify-consumer-types.mjs`'s fixture changed because n **Trap for later subtasks (api-extractor).** `{@link SomeError.message}` does not resolve (`message` is inherited from `Error`; `ae-unresolved-link`). Write it as backticked prose. +## Wave 3 — landed 2026-09-05 +PR #87 (#75), PR #89 (#74) and PR #88 (#72) merged into the umbrella at `2ea8b3e`, in that order. One +conflict (the `deviations.md` table tail: #75's `ASYNC-21` row and #74's `AUTH-22` row), unioned. Merged tree +preflighted in a throwaway worktree before the merge (byte-identical result): all 20 steps passed. The three +stale `retry/*` citations in `deviations.md` item 3 that #72 shifted were re-anchored on the umbrella after the +merge (`engine.ts:367`, `retry-step.ts:151`, `retry-dispatch.ts:55`). + +**Trap for every later subtask (git identity).** The #72 agent committed with +`git -c user.email="oaljarrah@dexpace.org"`, lifted from the harness's "user's email address" context line — +that address is the Claude login, and GitHub attributes it to a different account. Rewritten with +`--reset-author` and force-pushed before the merge; nothing on the umbrella carries it. Contract item 10 now +forbids any author override; the supervisor checks `git log --format=%ae` on a branch before merging it. + ## Decisions taken for wave 3 (M3) — pre-taken 2026-09-05, before dispatch ### D10 — #72: the final typed error is surfaced as-is; the trail rides in a side table, read through `retryAttempts()` @@ -226,6 +239,41 @@ afterwards"), and `packages/rx/README.md` says the same beside its `for await` g *Rejected:* dropping `release` (above). *Rejected:* a caller-facing `{ownership}` option (two behaviours to document for a case with one correct answer). No changeset (D1). + +**D10 outcome (2026-09-05, #72, PR #88).** `withTrail` became `attachTrail`; the trail lives in a new leaf +module `packages/core/src/retry/attempt-trail.ts` (`recordAttempts` internal, `retryAttempts` public, one +shared frozen empty list). An empty trail *deletes* a prior entry for a reused error instance; symbols are +excluded as weak keys rather than probed. Round 2 removed a public claim that `retryAttempts(e).length + 1` is +the send count — false on three reachable paths (the RETRY-32 gate, `stampAttempt` throwing, a non-abort +`Clock.sleep` rejection), and the `pipelines.md` example was unsound even narrowed to +`TransportFailureError`, because `abortToSdkError` synthesizes one for a timeout signal. Two engine cases pin +sends against trail length. `tests/node-conformance/retry.test.mjs` was outside the partition and edited anyway: +its case asserted the wrapper by name and was red the moment D10 landed. *Correction to D10's text:* +`classify.ts` never walked `.error`, and it ran per attempt before the terminal wrap, so the "hop" the +constraint on #78 described was not live; the end state it names is right. + +**D11 outcome (2026-09-05, #74, PR #89).** `pickChallengeHeader` returns every value (`getAll`), parsed per value +by a new `challengesOf`; a test with an unterminated quoted string in the first value discriminates parse-each +from comma-join. `cnonce` emitted for any `-sess` algorithm (row in `deviations.md`). Empty `realm`/`nonce` +decline (`!== ''`, not trimmed). **Deviation from D11's letter, accepted:** the conformance row cannot assert the +*parsed* list — `parseChallenges` is `@internal` and D11 forbade a new core export, and a real `authStep` drive is +blocked by AUTH-28 on the plain-`http` fixture — so `run-suite.ts`'s `challengeList()` splits the joined +`getAll` at `, Digest ` boundaries, scoped to the fixture's own two challenges and documented as not a parser. +The transport is answerable only for surfacing both values, which the row proves (red against a one-line +fixture). Found and fixed on the way: `run-suite.ts`'s header claimed TRANSPORT-10..14 were asserted elsewhere +while labelling rows with them; `node:http`'s `writeHead` rejects a `readonly string[]` header value, and +`gts lint` did not catch it — only `typecheck` did. *Constrains #82:* the new fixture route +`/repeated-challenge` and `registerInboundHeaderRows` exist; the `TRANSPORT-14` label is the closest MUST. + +**D12 outcome (2026-09-05, #75, PR #87).** No behaviour change. Ten `sse.test.ts` cases count the release the +*owned resource* sees (a structural `ReadableStream` double counting `reader.cancel()` and `body.cancel()` +separately) plus one in `tests/node-conformance/rx-bridge.test.mjs`. Measured counterfactual: deleting both +`release` arguments turns the two suspended-pull cases and two pre-existing idle-unsubscribe assertions red while +every exactly-once count stays green — so only the suspended-pull case discriminates the design; the counts pin +against a future double release, and the row says which is which. The row also closes `SSE-41`'s "documented +source ownership" clause, which Phase 8b marked done on documentation that named unsubscription only. New +`packages/rx/README.md` section "Who owns the stream"; `rx.api.md` regenerated byte-identical (prose only). + ### Wave 3 partition | Task | Owns | Shared, append-only | |---|---|---| @@ -243,3 +291,6 @@ TSDoc beside #72's new export. Supervisor resolves both, as in waves 1 and 2. | #70 / PR #85 | patch changeset for `@dexpace/core`: redirect error messages now carry redacted URLs (and `[malformed url]` for unparseable input); `http.redirect.rejected` gained `url.full` | | #71 / PR #86 | minor changeset for `@dexpace/core`: `BasicCredential`/`DigestCredential` become classes (breaking for object-literal callers); patch note for `authStep`'s `@throws PlaintextCredentialError` prose; `docs/first-release.md` untouched though this is its "free before the first bump" class | | #69 / PR #84 | patch changeset for `@dexpace/core`: `noopInstrumentationBundle.activeSpan` changed from `undefined` to `NOOP_SPAN` (documented default of a `@public` interface) | +| #72 / PR #88 | minor changeset for `@dexpace/core`: retry surfaces the final typed error (was a `SuppressedError` wrapper); new `retryAttempts()` export | +| #74 / PR #89 | patch changeset for `@dexpace/core`: every `WWW-Authenticate`/`Proxy-Authenticate` value is read; `cnonce` emitted for `-sess`; empty `realm`/`nonce` declined | +| #75 / PR #87 | changeset for `@dexpace/rx` (issue says minor; only `.d.ts` prose moved, so patch is arguable): SSE ownership transfer documented | diff --git a/docs/deviations.md b/docs/deviations.md index 4bd08e4..6bf260b 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -141,9 +141,9 @@ non-bridge clause survives and is enforced as an ordinary obligation on `send()` ## 3. Two retry stacks collapse into one, with the total-timeout budget explicitly opt-in -**Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:358`) — with exactly two thin -callers: the pillar step (`packages/core/src/retry/retry-step.ts:142`) and the dispatch adapter -(`packages/core/src/retry/retry-dispatch.ts:53`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | +**Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:367`) — with exactly two thin +callers: the pillar step (`packages/core/src/retry/retry-step.ts:151`) and the dispatch adapter +(`packages/core/src/retry/retry-dispatch.ts:55`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | undefined` and undefined by default (`packages/core/src/retry/settings.ts:27`), pinned by a test named for `RETRY-28` (`packages/core/src/retry/settings.test.ts:20`). From 25b987a515edfbaadb45c2d38d8c55344d747075 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 10:59:32 +0300 Subject: [PATCH 045/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20pre-take?= =?UTF-8?q?=20wave=204=20decisions=20D13=E2=80=93D15=20and=20the=20file=20?= =?UTF-8?q?partition?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 79 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index d00d387..a8a810d 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -283,6 +283,85 @@ source ownership" clause, which Phase 8b marked done on documentation that named Known merge seams: the `deviations.md` table tail (#74 + #75), and `core.api.md` if #74 changes a `@public` TSDoc beside #72's new export. Supervisor resolves both, as in waves 1 and 2. +## Decisions taken for wave 4 (#73 M3, #76 + #77 M4) — pre-taken 2026-09-05, before dispatch + +### D13 — #73: the request recovery chain runs once, above the retry loop +`dispatchWithRetry` applies `config.requestChain` **once** and hands the prepared request to `runWithRetry`; +each attempt then re-executes transport + response chain over `stampAttempt`'s fresh copy of that prepared +request. That is the layering `idempotencyKeyStep`'s `@public` TSDoc has claimed since `1f48926` ("runs ONCE per +call, upstream of retry"), and it is what RECOV-32's "one stable key across every retry" and RETRY-38's +"preserving any idempotency key" both presuppose — a key the engine re-sends from a pre-chain template is not +preserved, it is regenerated. **Reading of RETRY-44:** its "downstream chain" is whatever sits below the retry +loop; in the recovery stack that is transport + response chain, and its "upstream steps MUST NOT mutate the +shared in-flight request" clause is satisfied by construction, because upstream steps no longer run between +attempts at all. One `deviations.md` row records the reading (D0), because the port's own test named RETRY-44 +as the reason the request chain re-ran (`retry-dispatch.test.ts:67`, to be renamed). A request-chain failure +happens before the loop and is not retried: it never reached the wire, and it still passes through the response +chain once so RECOV-10/11's outcome handling is unchanged. *Rejected:* memoizing the key on the template +(`WeakMap` keyed by the `Request` instance) — a caller who deliberately sends one immutable `Request` value twice +would replay the key and have the server drop a real second call. *Rejected:* re-running the chain over the +*prepared* request each attempt — the chain reads its own output, which is the mutation RETRY-44 forbids in +different clothes, and every step would have to be proven idempotent. Enumerate the shipped `RequestStep`s in +`recovery/` and state in the ledger outcome that none needs per-attempt re-execution; the attempt ordinal is +the engine's (RETRY-38). Tests: N attempts, one `generate()` call, the same header value on every wire send; +rename the RETRY-44 test to what it now proves. TSDoc on `dispatchWithRetry` and the orchestrator; no +public shape change (`dispatchWithRetry` is `@internal`). `api:local` if `idempotencyKeyStep`'s prose moves. +*Constrains #78:* `engine.ts` is edited here; wave 5 lands on top. + +### D14 — #76: reject at the setter, in the `DexpaceError` tree, and say so in `@throws` +- **Path params:** `Object.hasOwn(pathParams, name)`; a missing own property is `OperationAssemblyError` + (SEAM-27's "every placeholder MUST have a supplied value"). `{constructor}` with `{}` is the pin. +- **Dates:** `ifModifiedSince` / `ifUnmodifiedSince` throw `RequestConditionsValidationError` on + `Number.isNaN(date.getTime())`. +- **`timeoutMs`:** the setter rejects a non-integer and anything above `2**32 - 1` with + `RequestOptionsValidationError` — HTTP-35 puts the range check at the setter, and `AbortSignal.timeout()`'s + range is the only one a transport can honour. Rewrite the TSDoc paragraph that argues a fractional + millisecond is meaningful and flip the test that pins it. *Rejected:* rounding/clamping in `composeSignal` + (hides the caller's error where HTTP-35 says to surface it). Add `@throws` to `composeSignal` for whatever + it can still raise. +- **Lone surrogates:** `String.prototype.isWellFormed()` at the call site that supplied the value — + `QueryParams` builder `add`/`set` (name and value) and `substitutePathParams` — throwing the error class + that call site already throws for invalid input (`OperationAssemblyError` for path params; for query params + the class the builder uses today, or `UrlConstructionError` if it has none — no new error class). Then + prove by test that no `URIError` can escape `encode()`, `equals()` or `buildRequest`; if one still can, + report the path rather than adding a second mechanism. +- **`getAll`:** one shared frozen empty array in `Headers` and `QueryParams`; a test that the present-name + array is frozen too. +- **`Headers.equals`:** direct cases (name order, value order, subset, case). +- **`TeeSink`:** `Number.isInteger(tapLimit) || tapLimit === Number.POSITIVE_INFINITY`, else the error the + constructor already throws for a negative limit. +- `api:local` on core after the `@throws` edits. No changeset (D1). Do not touch `http/media-type.ts` (#77's). + +### D15 — #77: contract violation on an empty chunk, quote the boundary, gate the tap on `closed` +- **Empty chunks:** `#writeExactly` throws `SourceContractViolationError` on `value.length === 0` for a + positive request, same wording as `io/retention-window.ts:177-183`; a declared length of 0 stays a + legitimate empty write (BODY-10). Tests: an empty-only source, and an empty chunk between real chunks. +- **Boundary:** keep RFC 2046 `bchars` as the accepted grammar (HTTP-51 says reject what *violates* it, not + narrow it) and **quote** the `boundary=` parameter whenever it is not a pure `tchar` token, using + `http/media-type.ts`'s existing token/quoted-string rendering (export an internal helper from that file if + the class API does not reach it; #77 owns `media-type.ts` this wave). Round-trip test through + `Response.formData()` on Bun, and the same case in `tests/node-conformance/body-lifecycle.test.mjs`. + *Rejected:* narrowing `validateBoundary` to `tchar` (rejects boundaries RFC 2046 allows for a problem the + renderer owns). +- **Logging tap after `close()`:** `startDrain`, `snapshot` and `read` check `state.closed` first — + `snapshot()` returns the captured prefix without starting a drain, `read()` rejects with + `ClosedResourceError`, `error()` reports only a genuine drain failure. Tests: close-then-snapshot, + close-then-read, close-then-error. +- **Node conformance:** cases for `toReadableStream`, `toWritableStream`, `TeeSink`'s bridge, + `withRequestLogging` and `withResponseLogging` go into the **existing** topic files + (`io-byte-stream.test.mjs`, `body-lifecycle.test.mjs`), not one new file per bridge — the tree is + topic-named and flat, and its README lists members. Pull, cancel and lock behaviour is what to assert. +- `api:local` on core if a `@throws` changes. No changeset (D1). + +### Wave 4 partition +| Task | Owns | +|---|---| +| #73 | `packages/core/src/recovery/**`, `packages/core/src/retry/{retry-dispatch,engine}.ts` + tests, `tests/node-conformance/recovery-chain.test.mjs` and `retry.test.mjs` if a case belongs there, `docs/deviations.md` (one row at table end), `core.api.md` if `idempotencyKeyStep`'s TSDoc moves | +| #76 | `packages/core/src/http/{headers,query-params,request-options,request-conditions,rfc3986}.ts`, `packages/core/src/seams/{operation,transport}.ts`, `packages/core/src/io/tee-sink.ts` + their tests, `core.api.md` (`@throws` on those) | +| #77 | `packages/core/src/body/{stream-body,multipart-body,response-body-logging}.ts` + tests, `packages/core/src/http/media-type.ts` (helper export only), `tests/node-conformance/{io-byte-stream,body-lifecycle}.test.mjs` + that tree's README, `core.api.md` if a `@throws` changes | +Known merge seams: `core.api.md` (up to three), which the supervisor regenerates on the merged tree rather than +hand-merging. No two tasks share a source file. + ## Deferred — release machinery (recoverable list) | Issue / PR | Deferred item | |---|---| From dba77433757bd6be2adbfa205dcbf8cbace651da Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:05:09 +0300 Subject: [PATCH 046/100] fix(core): a path placeholder is satisfied only by an own property (SEAM-27) `substitutePathParams` read `pathParams?.[name]`, which walks the prototype chain. A `{constructor}` placeholder against `{}` therefore resolved to `Object.prototype.constructor`, stringified, and shipped `/users/function%20Object%28%29%20%7B%20%5Bnative%20code%5D%20%7D` as a path segment instead of failing assembly; `{toString}`, `{hasOwnProperty}`, `{valueOf}` and `{__proto__}` behaved the same way. SEAM-27 requires every placeholder to have a *supplied* value, and a name the caller never supplied is missing whatever `Object.prototype` happens to carry. `Object.hasOwn(pathParams, name)` now gates the read, so an inherited member is absent and an own property named like one is still honored. A null-prototype `pathParams` keeps working. Found by: audit #67 / #76. --- packages/core/src/seams/operation.test.ts | 57 +++++++++++++++++++++++ packages/core/src/seams/operation.ts | 30 ++++++++---- 2 files changed, 78 insertions(+), 9 deletions(-) diff --git a/packages/core/src/seams/operation.test.ts b/packages/core/src/seams/operation.test.ts index 3927e4e..65342ee 100644 --- a/packages/core/src/seams/operation.test.ts +++ b/packages/core/src/seams/operation.test.ts @@ -158,6 +158,63 @@ describe('SEAM-27: dot-segment path-param values are rejected, not silently norm }); }); +describe('SEAM-27: a placeholder is satisfied only by an OWN property of pathParams', () => { + // `pathParams?.[name]` reached the whole prototype chain, so `{constructor}` against `{}` resolved to + // `Object`'s own constructor, stringified, and shipped + // `/users/function%20Object%28%29%20%7B%20%5Bnative%20code%5D%20%7D` instead of failing assembly. Every + // placeholder MUST have a *supplied* value (SEAM-27); a name the caller never supplied is a missing value + // whatever `Object.prototype` happens to carry. Measured on the pre-fix tree, audit #67 / #76. + test.each([ + 'constructor', + 'toString', + 'hasOwnProperty', + 'valueOf', + '__proto__', + ])( + 'a {%s} placeholder against empty pathParams throws OperationAssemblyError', + name => { + expect(() => + buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: `/users/{${name}}`, + pathParams: {}, + }), + ).toThrow(OperationAssemblyError); + }, + ); + + test('the error names the placeholder, not the inherited member it resolved to', () => { + expect(() => + buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{constructor}', + pathParams: {}, + }), + ).toThrow(/missing value for path parameter "constructor"/); + }); + + test('an own property named like a prototype member is still honored', () => { + const request = buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{constructor}', + pathParams: {constructor: 'me'}, + }); + expect(request.url.pathname).toBe('/users/me'); + }); + + test('a null-prototype pathParams object still resolves its own keys', () => { + const pathParams = Object.assign(Object.create(null) as object, { + id: 'x', + }) as Record; + const request = buildRequest('https://api.example.com', { + method: 'GET', + pathTemplate: '/users/{id}', + pathParams, + }); + expect(request.url.pathname).toBe('/users/x'); + }); +}); + describe('a path-param value containing / is encoded, not split (property)', () => { test('holds for arbitrary generated path-param values', () => { fc.assert( diff --git a/packages/core/src/seams/operation.ts b/packages/core/src/seams/operation.ts index b6213f2..5974a45 100644 --- a/packages/core/src/seams/operation.ts +++ b/packages/core/src/seams/operation.ts @@ -10,9 +10,10 @@ import {encodeRfc3986Component} from '../http/rfc3986.js'; /** * Thrown when `buildRequest()` cannot assemble a request from its descriptor: a `{name}` - * placeholder in `pathTemplate` has no value in `pathParams`, or a supplied value is a dot segment - * (`.`/`..`) that the WHATWG URL parser would normalize into a path rewrite instead of keeping as - * one literal segment. + * placeholder in `pathTemplate` has no OWN value in `pathParams` — an inherited member such as + * `constructor` does not satisfy one — or a supplied value is a dot segment (`.`/`..`) that the + * WHATWG URL parser would normalize into a path rewrite instead of keeping as one literal + * segment. * * @public */ @@ -50,9 +51,10 @@ export interface OperationDescriptor { readonly pathTemplate: string; /** - * Values for `pathTemplate`'s `{name}` placeholders. Every placeholder must have a value here; - * each value is percent-encoded as a single path segment, so a value containing `/` cannot inject - * an extra segment (SEAM-27). Defaults to empty. + * Values for `pathTemplate`'s `{name}` placeholders. Every placeholder must have an OWN property + * here — a name reachable only through the prototype chain, such as `constructor`, is treated as + * absent — and each value is percent-encoded as a single path segment, so a value containing `/` + * cannot inject an extra segment (SEAM-27). Defaults to empty. */ readonly pathParams?: Readonly> | undefined; @@ -103,7 +105,16 @@ function substitutePathParams( pathParams: Readonly> | undefined, ): string { return template.replace(PATH_PARAM_RE, (_match, name: string) => { - const value = pathParams?.[name]; + // `Object.hasOwn`, not `pathParams?.[name]`: the indexed read walks the prototype chain, so + // `{constructor}` against `{}` resolved to `Object.prototype.constructor`, stringified through + // `encodeRfc3986Component`, and shipped a native-code source text as a path segment instead of + // failing assembly. SEAM-27 requires every placeholder to have a *supplied* value, and a name + // the caller never supplied is missing whatever `Object.prototype` happens to carry + // (audit #67 / #76). + const value = + pathParams !== undefined && Object.hasOwn(pathParams, name) + ? pathParams[name] + : undefined; if (value === undefined) { throw new OperationAssemblyError( `missing value for path parameter "${name}"`, @@ -158,8 +169,9 @@ function composeQuery( * @param baseUrl - the absolute base URL to project the operation onto. * @param operation - the operation to assemble into a request. * @returns the assembled request. - * @throws {@link OperationAssemblyError} when a `{name}` placeholder has no value in `pathParams`, - * or a supplied value is a dot segment (`.`/`..`) — fix the descriptor; no request was assembled. + * @throws {@link OperationAssemblyError} when a `{name}` placeholder has no own value in + * `pathParams` (a name inherited from `Object.prototype` does not count as supplied), or a supplied + * value is a dot segment (`.`/`..`) — fix the descriptor; no request was assembled. * @throws {@link UrlConstructionError} when `baseUrl` is malformed, non-absolute, or carries a * fragment — supply a clean absolute base URL. * @throws {@link RequestBodyNotAllowedError} when the descriptor pairs a body with GET, HEAD, TRACE From 3629ec57878705f67d9a551d78139b3c75ce4337 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:05:54 +0300 Subject: [PATCH 047/100] fix(core): reject an invalid Date in the conditional-request setters (HTTP-50) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `toRfc1123` is `date.toUTCString()`, which is total: a NaN time value renders the literal string `Invalid Date` instead of throwing. That string is HTAB-free printable ASCII, so HTTP-18's outbound header grammar accepts it and `If-Modified-Since: Invalid Date` reached the wire — a header no server can evaluate, produced by a caller mistake made several frames earlier. `ifModifiedSince` and `ifUnmodifiedSince` now reject `Number.isNaN(date.getTime())` with `RequestConditionsValidationError`, naming the header in the message. HTTP-50's "emit RFC 1123 dates" is not satisfiable from a NaN instant, so the setter that was handed the bad `Date` is where it fails. Found by: audit #67 / #76. --- .../core/src/http/request-conditions.test.ts | 53 ++++++++++++++++++- packages/core/src/http/request-conditions.ts | 32 +++++++++-- 2 files changed, 81 insertions(+), 4 deletions(-) diff --git a/packages/core/src/http/request-conditions.test.ts b/packages/core/src/http/request-conditions.test.ts index 4382f64..b89c3bb 100644 --- a/packages/core/src/http/request-conditions.test.ts +++ b/packages/core/src/http/request-conditions.test.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/request-conditions.test.ts -// Exercises: HTTP-50 (comma-joined If-Match/If-None-Match, RFC 1123 dates, idempotent apply, any-tag exclusivity) +// Exercises: HTTP-50 (comma-joined If-Match/If-None-Match, RFC 1123 dates, idempotent apply, any-tag +// exclusivity, and an invalid Date rejected at the setter rather than emitted as `Invalid Date`) import {describe, expect, test} from 'bun:test'; import { RequestConditions, @@ -134,3 +135,53 @@ describe('ifUnmodifiedSince (HTTP-50)', () => { ]); }); }); + +describe('an invalid Date is rejected at the setter (HTTP-50)', () => { + // `toRfc1123` is `date.toUTCString()`, which renders the literal string `Invalid Date` for a NaN + // time value. `Invalid Date` is HTAB-free printable ASCII, so HTTP-18's outbound header grammar + // waves it through and it reaches the wire as `If-Modified-Since: Invalid Date` — a header no + // server can evaluate, produced from a caller mistake made several frames earlier. HTTP-50's + // "emit RFC 1123 dates" is not satisfiable from a NaN instant, so the setter is where it fails. + // Measured on the pre-fix tree, audit #67 / #76. + test.each([ + ['new Date("nope")', new Date('nope')], + ['new Date(NaN)', new Date(Number.NaN)], + ])( + 'ifModifiedSince rejects %s with RequestConditionsValidationError', + (_label, date) => { + expect(() => + RequestConditions.newBuilder().ifModifiedSince(date), + ).toThrow(RequestConditionsValidationError); + }, + ); + + test.each([ + ['new Date("nope")', new Date('nope')], + ['new Date(NaN)', new Date(Number.NaN)], + ])( + 'ifUnmodifiedSince rejects %s with RequestConditionsValidationError', + (_label, date) => { + expect(() => + RequestConditions.newBuilder().ifUnmodifiedSince(date), + ).toThrow(RequestConditionsValidationError); + }, + ); + + test('the message names the setter, so the caller knows which field to fix', () => { + expect(() => + RequestConditions.newBuilder().ifModifiedSince(new Date('nope')), + ).toThrow(/If-Modified-Since/); + expect(() => + RequestConditions.newBuilder().ifUnmodifiedSince(new Date('nope')), + ).toThrow(/If-Unmodified-Since/); + }); + + test('no invalid instant can reach applyTo, so no header renders "Invalid Date"', () => { + const builder = RequestConditions.newBuilder(); + expect(() => builder.ifModifiedSince(new Date('nope'))).toThrow( + RequestConditionsValidationError, + ); + const headers = builder.build().applyTo(Headers.newBuilder().build()); + expect(headers.has('If-Modified-Since')).toBe(false); + }); +}); diff --git a/packages/core/src/http/request-conditions.ts b/packages/core/src/http/request-conditions.ts index a973e88..db1d564 100644 --- a/packages/core/src/http/request-conditions.ts +++ b/packages/core/src/http/request-conditions.ts @@ -30,6 +30,25 @@ function toRfc1123(date: Date): string { return date.toUTCString(); } +/** + * Copies `date` after rejecting a NaN time value. + * + * `toUTCString()` is total — it renders the literal string `Invalid Date` rather than throwing — + * and `Invalid Date` is HTAB-free printable ASCII, so HTTP-18's outbound header grammar accepts it + * and it reaches the wire as `If-Modified-Since: Invalid Date`. HTTP-50 requires an RFC 1123 date, + * which a NaN instant cannot produce, so the failure belongs at the setter that was handed the bad + * `Date` rather than several frames downstream (audit #67 / #76). + */ +function copyValidInstant(date: Date, headerName: string): Date { + const time = date.getTime(); + if (Number.isNaN(time)) { + throw new RequestConditionsValidationError( + `${headerName}: date must be a valid instant, got an invalid Date`, + ); + } + return new Date(time); +} + // eslint-disable-next-line max-params -- private, builder-internal plumbing; field count fixed by HTTP-50 let createRequestConditions: ( ifMatch: readonly ETag[], @@ -45,7 +64,9 @@ let createRequestConditions: ( * Multiple entity-tags emit as one comma-separated header; dates emit in RFC 1123 form. * {@link RequestConditions.applyTo} uses `set`, never `add`, so applying the same conditions twice * cannot duplicate a header. The any-tag (`*`) is mutually exclusive with concrete entity-tags, and - * repeated `*` collapses to one — enforced when the tag is added, not at emission. + * repeated `*` collapses to one — enforced when the tag is added, not at emission. An invalid + * `Date` is likewise rejected by the setter that was handed it, so no instance can emit the literal + * header value `Invalid Date`. * * @example * ```ts @@ -204,9 +225,12 @@ export class RequestConditionsBuilder implements Builder { * @param date - the instant; copied, not aliased, so a caller mutating its own `Date` after * `build()` cannot change what {@link RequestConditions.applyTo} emits. * @returns this builder, for chaining. + * @throws {@link RequestConditionsValidationError} when `date` carries a NaN time value. An + * invalid `Date` renders as the literal `Invalid Date`, which the outbound header grammar accepts + * and no server can evaluate, so it is rejected here rather than emitted (HTTP-50). */ ifModifiedSince(date: Date): this { - this.#ifModifiedSince = new Date(date.getTime()); + this.#ifModifiedSince = copyValidInstant(date, 'If-Modified-Since'); return this; } @@ -216,9 +240,11 @@ export class RequestConditionsBuilder implements Builder { * @param date - the instant; copied, not aliased, exactly as in * {@link RequestConditionsBuilder.ifModifiedSince}. * @returns this builder, for chaining. + * @throws {@link RequestConditionsValidationError} when `date` carries a NaN time value, for the + * reason given on {@link RequestConditionsBuilder.ifModifiedSince} (HTTP-50). */ ifUnmodifiedSince(date: Date): this { - this.#ifUnmodifiedSince = new Date(date.getTime()); + this.#ifUnmodifiedSince = copyValidInstant(date, 'If-Unmodified-Since'); return this; } From 8036fbc2e20ea960cfe9887e076461b0909717f9 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:07:06 +0300 Subject: [PATCH 048/100] fix(core): raise on a zero-length chunk during an exact-length body copy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `StreamBody.#writeExactly` read, checked the over-run, then wrote — with no guard on `value.length === 0`. Two consequences, neither diagnosable anywhere else: - A source that only ever yields empty chunks never signals `done`, so the delivered-of-declared check at the end of the loop is never reached. The audit's probe forwarded 200,000 empty chunks before the source closed. - Every empty chunk reached the transport sink. To an HTTP/1.1 chunked-encoding transport a zero-length chunk is the TERMINATING chunk — `io/buffered-sink.ts:66-71` says so and refuses to emit one — so tolerating it ends the request body early while the copy still believes it is mid-body. `assertNonEmptyChunk` now raises `SourceContractViolationError` with `io/retention-window.ts:177-183`'s wording and error type, which is also what `body/response-body-logging.ts:82-88` does for BODY-25 on the response side. A request body reaches both this copy and `BufferedSource`; a divergence would make the same upstream fail or succeed depending only on which wrapper it passed through. A declared length of 0 stays a legitimate empty write (BODY-10): that is a source signalling `{done: true}` immediately, which never reaches the check. The `pipeTo` path (unknown declared length) is unchanged — the runtime owns that pump, and HTTP-39/BODY-10's rule is scoped to the exact-length copy. `bun run api:local` on core: no change, the report records signatures only. Refs #77, #67. HTTP-39, BODY-10. --- packages/core/src/body/stream-body.test.ts | 95 +++++++++++++++++++++- packages/core/src/body/stream-body.ts | 34 +++++++- 2 files changed, 126 insertions(+), 3 deletions(-) diff --git a/packages/core/src/body/stream-body.test.ts b/packages/core/src/body/stream-body.test.ts index 3530875..b966275 100644 --- a/packages/core/src/body/stream-body.test.ts +++ b/packages/core/src/body/stream-body.test.ts @@ -6,11 +6,13 @@ // delivered-of-declared, and an overrunning stream is stopped BEFORE the extra bytes reach the sink), // IO-3 (a contentLength below the -1 sentinel is rejected), HTTP-26/HTTP-51 (a media type is // header-safe), RECOV-12 (a close failure never masks the primary write failure), HTTP-1 (frozen at -// construction so the declared length cannot be desynced from the written bytes) +// construction so the declared length cannot be desynced from the written bytes), HTTP-39/BODY-10 again +// (a zero-length delivery during an exact-length copy is a source-contract violation, never a no-op and +// never spun on, and no empty chunk reaches the sink) import {describe, expect, test} from 'bun:test'; import {MediaTypeParseError} from '../http/errors.js'; import {InvariantViolation} from '../invariant.js'; -import {EndOfStreamError} from '../io/errors.js'; +import {EndOfStreamError, SourceContractViolationError} from '../io/errors.js'; import {ConsumedBodyError} from './errors.js'; import {streamBody} from './stream-body.js'; @@ -49,6 +51,16 @@ function collectingSink(): { }; } +/** Awaits a rejection and returns its reason, failing loudly when the promise resolves instead. */ +async function rejection(promise: Promise): Promise { + try { + await promise; + } catch (error: unknown) { + return error; + } + throw new Error('expected a rejection, but the promise resolved'); +} + describe('caller stream ownership (BODY-8)', () => { test('a sink failure does not cancel the caller stream on the unknown-length path', async () => { // `pipeTo`'s default (`preventCancel: false`) cancels the SOURCE when the destination errors, @@ -230,6 +242,85 @@ describe('a mis-framed body never reaches the wire (HTTP-39/BODY-10)', () => { }); }); +describe('a zero-length delivery is a source-contract violation (HTTP-39/BODY-10)', () => { + /** + * The auditor's probe, bounded so an unguarded run terminates instead of hanging the suite: a source + * that delivers nothing but empty chunks, then finally ends. Unbounded is the real-world shape, and + * the pull count below is what proves the copy did not spin on it. + */ + function emptyOnly(limit: number): { + stream: ReadableStream; + pulls: () => number; + } { + let pulls = 0; + const stream = new ReadableStream({ + pull(controller) { + pulls += 1; + if (pulls > limit) { + controller.close(); + return; + } + controller.enqueue(new Uint8Array(0)); + }, + }); + return {stream, pulls: () => pulls}; + } + + test('an empty-only source raises on the first empty chunk rather than spinning on it', async () => { + const {stream, pulls} = emptyOnly(1000); + const {state, sink} = probeSink(); + const error = await rejection( + streamBody(stream, undefined, 3).writeTo(sink), + ); + + // Not EndOfStreamError after a thousand futile reads: an unbounded source of empty chunks never + // ends, so nothing downstream can ever diagnose it, and every one of those chunks reaches the sink. + expect(error).toBeInstanceOf(SourceContractViolationError); + // Two, not one: the default queuing strategy reads one chunk ahead, so the source is pulled again + // the moment our read drains its queue. What matters is that it is not `limit`. + expect(pulls()).toBeLessThanOrEqual(2); + expect(state.written).toEqual([]); + expect(state.aborted).toBe(true); // a mis-framed body is never signalled as a clean close + }); + + test('an empty chunk between real chunks is raised, and never reaches the sink', async () => { + const emptyBetween = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + controller.enqueue(new Uint8Array(0)); + controller.enqueue(Uint8Array.from([3])); + controller.close(); + }, + }); + const chunkLengths: number[] = []; + const sink = new WritableStream({ + write: chunk => void chunkLengths.push(chunk.length), + }); + + expect( + await rejection(streamBody(emptyBetween, undefined, 3).writeTo(sink)), + ).toBeInstanceOf(SourceContractViolationError); + // `io/buffered-sink.ts` writeString: a zero-length chunk is HTTP/1.1 chunked encoding's TERMINATING + // chunk, so forwarding one ends the request body early on the wire. + expect(chunkLengths).toEqual([2]); + }); + + test('a declared length of 0 over a source that just closes stays a legitimate empty write (BODY-10)', async () => { + const {state, sink} = probeSink(); + await streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ).writeTo(sink); + expect(state.written).toEqual([]); + expect(state.closed).toBe(true); + }); +}); + describe('StreamBody media type and failure propagation', () => { test('rejects a media type carrying CR/LF (HTTP-26/HTTP-51)', () => { expect(() => diff --git a/packages/core/src/body/stream-body.ts b/packages/core/src/body/stream-body.ts index c0ba250..093e0a7 100644 --- a/packages/core/src/body/stream-body.ts +++ b/packages/core/src/body/stream-body.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/body/stream-body.ts -import {EndOfStreamError} from '../io/errors.js'; +import {EndOfStreamError, SourceContractViolationError} from '../io/errors.js'; import {invariant} from '../invariant.js'; import type {Body} from './body.js'; import {ConsumedBodyError} from './errors.js'; @@ -8,6 +8,33 @@ import {freezeBody} from './freeze-body.js'; import {assertHeaderSafeMediaType} from './media-type-safety.js'; import {withBodyWriter} from './write-body.js'; +/** + * HTTP-39/BODY-10: a zero-length delivery during an exact-length copy is a source-contract violation, + * never end-of-stream and never something to spin on -- `{done: true}` is the only end signal. + * + * Two failure modes, and neither one is diagnosable anywhere else. A source that only ever yields empty + * chunks never ends, so the delivered-of-declared check below is never reached; and every empty chunk + * that gets past here reaches the transport sink, where to an HTTP/1.1 chunked-encoding transport a + * zero-length chunk is the TERMINATING chunk (`io/buffered-sink.ts`'s `writeString`) -- so tolerating + * one ends the request body early on the wire while the copy still believes it is mid-body. + * + * The wording and the error type are `io/retention-window.ts`'s deliberately: `read()` carries no + * requested count, so the requirement's "for a positive requested count" has no literal analog, and a + * request body reaches both this copy and `BufferedSource` -- a divergence would make the same upstream + * fail or succeed depending only on which wrapper it passed through. `body/response-body-logging.ts` + * makes the same call for BODY-25 on the response side. + * + * A declared length of 0 is still a legitimate empty write (BODY-10): that is a source that signals + * `{done: true}` immediately, which never reaches this check. + */ +function assertNonEmptyChunk(value: Uint8Array): void { + if (value.length === 0) { + throw new SourceContractViolationError( + 'source delivered 0 bytes without signalling end of stream', + ); + } +} + /** * A single-use body backed by a caller-supplied stream. * @@ -50,6 +77,8 @@ export class StreamBody implements Body { * @throws {@link ConsumedBodyError} on a second call -- this body is single-use (BODY-3). * @throws EndOfStreamError when a declared `contentLength` disagrees with the bytes the stream * actually yields, in either direction (HTTP-39/BODY-10). + * @throws SourceContractViolationError when the stream delivers a zero-length chunk without + * signalling end of stream during a declared-length write (HTTP-39/BODY-10). */ async writeTo(sink: WritableStream): Promise { if (this.#consumed) throw new ConsumedBodyError('stream'); @@ -80,6 +109,7 @@ export class StreamBody implements Body { // Serial by necessity: each read depends on the previous one advancing the cursor. const {done, value} = await reader.read(); if (done) break; + assertNonEmptyChunk(value); // HTTP-39/BODY-10 // Checked BEFORE the write, not after the loop: once a transport has stamped the declared // Content-Length, an overrun byte sits on the socket where the peer reads it as the start of // the next message, and a thrown error cannot recall bytes already written (HTTP-39/BODY-10). @@ -109,6 +139,8 @@ export class StreamBody implements Body { * @throws ConsumedBodyError from `writeTo` when the body has already been written once (BODY-3). * @throws EndOfStreamError from `writeTo` when the stream yields a byte count other than the declared * `contentLength` (HTTP-39/BODY-10). + * @throws SourceContractViolationError from `writeTo` when the stream delivers a zero-length chunk + * without signalling end of stream during a declared-length write (HTTP-39/BODY-10). * @public */ export function streamBody( From ef7df30990d17c57e672e1351c802c3d8d4ff7e0 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:09:36 +0300 Subject: [PATCH 049/100] fix(core): quote the multipart boundary parameter when it is not a token MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `MultipartBody` built its own `Content-Type` by interpolation — `multipart/form-data; boundary=${boundary}` — while `BOUNDARY_PATTERN` (`multipart-body.ts:29-31`) admits the full RFC 2046 `bchars` set. `bchars` and RFC 9110 `tchar` are different: ' ', ',', ':', '=', '?', '/', '(' and ')' are legal in a boundary and illegal bare in a header parameter value. So `multipartBody(parts, 'a,b')` emitted `boundary=a,b`, which an RFC 9110 parameter parser reads as `boundary=a` plus a junk parameter and then never finds a delimiter for. Node's own FormData parser rejects the whole body with `TypeError: Failed to parse body as FormData`. The header is now rendered through `MediaType.of(...).render()`, the module that already owns HTTP-25's token-or-quoted-string decision and guarantees `parse(render(x)) === x`. A boundary that IS a bare token still renders bare, so the generated default and every existing expectation are byte-identical. Narrowing `validateBoundary` to `tchar` was rejected (D15): HTTP-51 asks that a boundary violating the RFC 2046 grammar be refused, not that a conforming one be. The defect was in the rendering. The round-trip case is a parser disagreement, not a one-runtime bug: Bun's `Response.formData()` tolerates the unquoted form, Node's does not. The Bun rows are the regression guard; the reproducer is the matching case in `tests/node-conformance/body-lifecycle.test.mjs` (next commit). Refs #77, #67. HTTP-51. --- packages/core/src/body/multipart-body.test.ts | 58 +++++++++++++++++-- packages/core/src/body/multipart-body.ts | 33 ++++++++++- 2 files changed, 84 insertions(+), 7 deletions(-) diff --git a/packages/core/src/body/multipart-body.test.ts b/packages/core/src/body/multipart-body.test.ts index 5b03d39..b767fc4 100644 --- a/packages/core/src/body/multipart-body.test.ts +++ b/packages/core/src/body/multipart-body.test.ts @@ -1,8 +1,9 @@ // SPDX-License-Identifier: MIT // packages/core/src/body/multipart-body.test.ts // Exercises: BODY-2 (composite replayability, unknown-length collapse), HTTP-51 (shared framing routine, -// boundary generation/validation, header quoting, and a part media type that cannot break the framing), -// HTTP-26 (a media type is header-safe), RECOV-12 (a close failure never masks the primary failure) +// boundary generation/validation, header quoting, a boundary parameter rendered so an RFC 9110 parser +// can read it, and a part media type that cannot break the framing), HTTP-26 (a media type is +// header-safe), RECOV-12 (a close failure never masks the primary failure) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {MediaTypeParseError} from '../http/errors.js'; @@ -66,9 +67,9 @@ function collectingSink(): { }; } -async function drain(body: { +async function drainBytes(body: { writeTo: (sink: WritableStream) => Promise; -}): Promise { +}): Promise { const chunks: Uint8Array[] = []; await body.writeTo(new WritableStream({write: c => void chunks.push(c)})); const total = chunks.reduce((s, c) => s + c.length, 0); @@ -78,7 +79,13 @@ async function drain(body: { out.set(c, offset); offset += c.length; } - return new TextDecoder().decode(out); + return out; +} + +async function drain(body: { + writeTo: (sink: WritableStream) => Promise; +}): Promise { + return new TextDecoder().decode(await drainBytes(body)); } describe('MultipartBody replayability and length (BODY-2)', () => { @@ -186,6 +193,47 @@ describe('MultipartBody boundary generation and validation (HTTP-51)', () => { }); }); +describe('the rendered boundary parameter is a parseable one (HTTP-51)', () => { + // RFC 2046 `bchars` and RFC 9110 `tchar` are different sets: ' ', ',', ':', '=', '?', '/', '(' and + // ')' are legal in a boundary and illegal bare in a header parameter value. `validateBoundary` admits + // the first grammar and the renderer owes the second, so a boundary the constructor accepts must come + // back out quoted rather than bare. + test('a boundary that is not a bare token is quoted', () => { + expect( + multipartBody([{name: 'a', body: stringBody('x')}], 'a,b').mediaType, + ).toBe('multipart/form-data; boundary="a,b"'); + }); + + test('a boundary that IS a bare token is left unquoted', () => { + expect( + multipartBody([{name: 'a', body: stringBody('x')}], 'plain-1').mediaType, + ).toBe('multipart/form-data; boundary=plain-1'); + // The generated default stays byte-identical: it is drawn from ALPHA/DIGIT only. + expect( + multipartBody([{name: 'a', body: stringBody('x')}]).mediaType, + ).toMatch(/^multipart\/form-data; boundary=dexpace-[A-Za-z0-9]{32}$/); + }); + + test.each([['a,b'], ['bound ary'], ['a:b'], ['a=b'], ['a?b'], ['(a)/b']])( + 'the header a peer receives round-trips through a real parameter parser: %p', + async boundary => { + // The runtime's own multipart parser, standing in for the peer. Bun's happens to tolerate the + // unquoted form, so this is the regression guard and NOT the reproducer: Node's (undici's) + // rejects the whole body with `TypeError: Failed to parse body as FormData`, which is why the + // same case is also in `tests/node-conformance/body-lifecycle.test.mjs`. Two independent + // parsers disagreeing about our own Content-Type is exactly what that tree is for. + const body = multipartBody( + [{name: 'field', body: stringBody('value')}], + boundary, + ); + const response = new globalThis.Response(await drainBytes(body), { + headers: {'content-type': body.mediaType}, + }); + expect((await response.formData()).get('field')).toBe('value'); + }, + ); +}); + describe('MultipartBodyBuilder (HTTP-2, HTTP-3)', () => { test('static newBuilder and instance newBuilder pre-populates parts and boundary', async () => { const original = MultipartBody.newBuilder() diff --git a/packages/core/src/body/multipart-body.ts b/packages/core/src/body/multipart-body.ts index 6df3020..e1affd3 100644 --- a/packages/core/src/body/multipart-body.ts +++ b/packages/core/src/body/multipart-body.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/core/src/body/multipart-body.ts import type {Builder} from '../http/builder.js'; +import {MediaType} from '../http/media-type.js'; import {EndOfStreamError} from '../io/errors.js'; import {invariant} from '../invariant.js'; import type {Body} from './body.js'; @@ -77,6 +78,31 @@ function renderPartHeader(part: MultipartPart, boundary: string): Uint8Array { return new TextEncoder().encode(header); } +/** + * HTTP-51: the `Content-Type` a peer actually parses. + * + * RFC 2046 `bchars` and RFC 9110 `tchar` are different sets. `BOUNDARY_PATTERN` above admits ' ', ',', + * ':', '=', '?', '/', '(' and ')', none of which is a `tchar`, so interpolating the boundary bare + * produces a parameter value that stops at the first offending byte -- `boundary=a,b` reads as + * `boundary=a` plus a junk parameter, and the peer then never finds a delimiter. Node's own FormData + * parser rejects such a body outright with `TypeError: Failed to parse body as FormData`. + * + * Rendered through {@link MediaType} rather than a second quoting routine here: it is the module that + * owns HTTP-25's token-or-quoted-string decision, and `parse(render(x)) === x` is its guarantee. A + * boundary that IS a bare token still renders bare, so the generated default is byte-identical to what + * this class emitted before. + * + * Narrowing `validateBoundary` to `tchar` instead was rejected: HTTP-51 asks that a boundary VIOLATING + * the RFC 2046 grammar be refused, not that a conforming one be. The defect is in the rendering. + */ +function renderMediaType(boundary: string): string { + return MediaType.of( + 'multipart', + 'form-data', + new Map([['boundary', boundary]]), + ).render(); +} + function trailerBytes(boundary: string): Uint8Array { return new TextEncoder().encode(`--${boundary}--\r\n`); } @@ -144,7 +170,10 @@ function nonClosingSink( export class MultipartBody implements Body { /** Discriminates this variant within the {@link Body} union. */ readonly kind = 'multipart' as const; - /** `multipart/form-data` carrying the boundary this instance frames its parts with. */ + /** + * `multipart/form-data` carrying the boundary this instance frames its parts with, with the + * `boundary` parameter quoted whenever it is not a bare RFC 9110 token (HTTP-51). + */ readonly mediaType: string; /** The total framed byte count, or -1 when any part's own length is unknown (BODY-2). */ readonly contentLength: number; @@ -157,7 +186,7 @@ export class MultipartBody implements Body { if (boundary !== undefined) validateBoundary(boundary); this.#boundary = boundary ?? generateBoundary(); this.#parts = [...parts]; - this.mediaType = `multipart/form-data; boundary=${this.#boundary}`; + this.mediaType = renderMediaType(this.#boundary); this.replayable = this.#parts.every(part => part.body.replayable); this.contentLength = computeContentLength(this.#parts, this.#boundary); invariant( From 5a49bb1053d57672e7cba21e98cfffdb8c748619 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:10:20 +0300 Subject: [PATCH 050/100] fix(core): the timeout setter enforces AbortSignal.timeout()'s own range (HTTP-35) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `RequestOptionsBuilder.timeoutMs` accepted a fractional value on the argument that "a timeout is a duration and a fractional millisecond is meaningful", and accepted anything finite above `2**32 - 1`. Nothing downstream can express either. The only consumer of the field is `composeSignal`, which hands it to `AbortSignal.timeout()` — so `timeoutMs(1.5)` failed one seam away from the call site that supplied it, inside the transport, as an unwrapped platform error. HTTP-35 puts the range check at the setter; the range that setter must check is the one a transport can honor. The setter now requires an integer in `1 .. 2**32 - 1`, and the TSDoc paragraph arguing the other way is rewritten. The test that pinned `timeoutMs(1.5)` as accepted is flipped, and a property states the invariant: every admitted value is inside the range, everything else raises `RequestOptionsValidationError`. `composeSignal` gains the `@throws` it never had, and it is deliberately not a clamp — a value reaching it out of range now comes only from a transport's own unvalidated `defaultTimeoutMs` construction option. Runtime divergence, measured 2026-09-05: `AbortSignal.timeout(1.5)` and `AbortSignal.timeout(2**32)` raise `RangeError` on Node and are ACCEPTED on Bun, and a negative delay is `RangeError` on Node against `TypeError` on Bun. `bun test` therefore cannot assert either half, so the Node case goes to `tests/node-conformance/seams.test.mjs` per CLAUDE.md's membership rule — the file already owns `composeSignal`, and it is outside the letter of #76's partition for that reason. Found by: audit #67 / #76. --- .../core/src/http/request-options.test.ts | 70 ++++++++++++++++++- packages/core/src/http/request-options.ts | 36 ++++++++-- packages/core/src/seams/transport.test.ts | 30 ++++++++ packages/core/src/seams/transport.ts | 14 +++- tests/node-conformance/seams.test.mjs | 36 +++++++++- 5 files changed, 174 insertions(+), 12 deletions(-) diff --git a/packages/core/src/http/request-options.test.ts b/packages/core/src/http/request-options.test.ts index 7bec354..bcfc28f 100644 --- a/packages/core/src/http/request-options.test.ts +++ b/packages/core/src/http/request-options.test.ts @@ -3,6 +3,7 @@ // Exercises: HTTP-34 (EMPTY sentinel, defensive tag copy), HTTP-35 (timeout/maxRetries validation), // AUTH-4 (the per-call auth descriptor tier, added in Phase 5c) import {describe, expect, test} from 'bun:test'; +import fc from 'fast-check'; import {createAuthDescriptor} from '../auth/descriptor.js'; import {createAuthRequirement} from '../auth/requirement.js'; import {RequestOptions} from './request-options.js'; @@ -74,6 +75,31 @@ describe('operation auth descriptor (AUTH-4, docs/work/mvp/2026-09-04-open-items }); }); +const timeoutCandidate = fc.oneof( + fc.double({noNaN: false}), + fc.integer({min: -10, max: 10}), + fc.constantFrom(2 ** 32 - 1, 2 ** 32, Number.MAX_SAFE_INTEGER), +); + +/** + * Either the builder refuses `candidate` with the typed error, or it admits a value inside + * `AbortSignal.timeout()`'s range. There is no third outcome — that is the whole HTTP-35 claim. + */ +function expectAdmittedTimeoutInRange(candidate: number): void { + let accepted: number | undefined; + try { + accepted = RequestOptions.newBuilder() + .timeoutMs(candidate) + .build().timeoutMs; + } catch (e: unknown) { + expect(e).toBeInstanceOf(RequestOptionsValidationError); + return; + } + expect(Number.isInteger(accepted)).toBe(true); + expect(accepted).toBeGreaterThanOrEqual(1); + expect(accepted).toBeLessThanOrEqual(2 ** 32 - 1); +} + describe('timeout validation (HTTP-35)', () => { test('rejects zero or negative timeout', () => { expect(() => RequestOptions.newBuilder().timeoutMs(0)).toThrow( @@ -93,9 +119,47 @@ describe('timeout validation (HTTP-35)', () => { ); }); - test('accepts a fractional millisecond timeout, which a deadline can honor', () => { - expect(RequestOptions.newBuilder().timeoutMs(1.5).build().timeoutMs).toBe( - 1.5, + // Flipped by audit #67 / #76. The old case pinned `timeoutMs(1.5)` as accepted "because a deadline + // can honor a fractional millisecond". Nothing downstream can: the only consumer is + // `composeSignal`, which hands the value to `AbortSignal.timeout()`, and that throws + // `RangeError: The value of "delay" is out of range. It must be an integer.` — inside the + // transport, one seam away from the setter that accepted it. HTTP-35 puts the range check at the + // setter, so the range checked is `AbortSignal.timeout()`'s, the only one a transport can honor. + test('rejects a fractional timeout, which no transport deadline can honor', () => { + expect(() => RequestOptions.newBuilder().timeoutMs(1.5)).toThrow( + RequestOptionsValidationError, + ); + }); + + test("rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1", () => { + expect(() => RequestOptions.newBuilder().timeoutMs(2 ** 32)).toThrow( + RequestOptionsValidationError, + ); + expect(() => + RequestOptions.newBuilder().timeoutMs(Number.MAX_SAFE_INTEGER), + ).toThrow(RequestOptionsValidationError); + }); + + test('accepts the ceiling itself, so the boundary is inclusive', () => { + expect( + RequestOptions.newBuilder() + .timeoutMs(2 ** 32 - 1) + .build().timeoutMs, + ).toBe(2 ** 32 - 1); + }); + + // The invariant, stated runtime-independently: a value this setter accepts is inside + // `AbortSignal.timeout()`'s documented range, and anything else fails here. It is asserted as a + // property rather than against `AbortSignal.timeout()` itself because the two runtimes disagree — + // Bun accepts `1.5` and `2**32` where Node raises `RangeError` — which is precisely why the range + // is checked in the model instead of left to whichever runtime the caller happens to be on. + // `tests/node-conformance/seams.test.mjs` closes the Node half. + test('every accepted timeout is an integer in 1..2**32 - 1 (property)', () => { + fc.assert( + fc.property(timeoutCandidate, candidate => { + expectAdmittedTimeoutInRange(candidate); + }), + {numRuns: 500}, ); }); diff --git a/packages/core/src/http/request-options.ts b/packages/core/src/http/request-options.ts index 77e7e7c..b0258dd 100644 --- a/packages/core/src/http/request-options.ts +++ b/packages/core/src/http/request-options.ts @@ -4,6 +4,13 @@ import type {AuthDescriptor} from '../auth/descriptor.js'; import type {Builder} from './builder.js'; import {RequestOptionsValidationError} from './errors.js'; +/** + * The largest timeout `AbortSignal.timeout()` accepts, and therefore the largest one this model + * will hold: the platform rejects anything above it with `RangeError: The value of "delay" is out + * of range. It must be >= 0 && <= 4294967295.` (HTTP-35, audit #67 / #76). + */ +const MAX_TIMEOUT_MS = 2 ** 32 - 1; + // eslint-disable-next-line max-params -- private, builder-internal plumbing; one parameter per HTTP-34 field let createRequestOptions: ( timeoutMs: number | undefined, @@ -92,7 +99,11 @@ export class RequestOptions { .operationAuth(this.#operationAuth); } - /** The per-call timeout in milliseconds, or `undefined` to use the configured default. */ + /** + * The per-call timeout in milliseconds, or `undefined` to use the configured default. Always an + * integer in `1 .. 2**32 - 1` when defined — {@link RequestOptionsBuilder.timeoutMs} admits + * nothing else, so a transport can pass it to `AbortSignal.timeout()` unchecked. + */ get timeoutMs(): number | undefined { return this.#timeoutMs; } @@ -172,20 +183,31 @@ export class RequestOptionsBuilder implements Builder { * The range check is the FULL range, not merely its lower bound. `Infinity` and `NaN` are as out * of range as `-1`: a non-finite deadline is one no clock can compare against, so it degrades to * "no deadline" silently rather than failing at the call site that supplied it, which is exactly - * what HTTP-35 exists to prevent. Not required to be integral, unlike `maxRetries` -- a timeout is - * a duration and a fractional millisecond is meaningful. + * what HTTP-35 exists to prevent. + * + * The range is `AbortSignal.timeout()`'s — an integer in `1 .. 2**32 - 1` — because that is the + * only range a transport can honor. `composeSignal` is the one consumer of this value and it + * hands it straight to `AbortSignal.timeout()`, which throws a `RangeError` on a fractional + * millisecond and on anything above `4294967295`. A timeout this setter accepted and a transport + * then refused is the failure HTTP-35 exists to move to the call site, so integrality is checked + * here and not treated as an implementation detail of one transport (audit #67 / #76 flipped the + * earlier reading, which accepted `1.5` on the argument that a duration may be fractional; no + * consumer of this field can express one). * * @param value - the timeout in milliseconds, or `undefined` for no override. Zero is rejected * rather than reinterpreted: it means "no timeout" in one transport and is an error in another * (HTTP-35). * @returns this builder, for chaining. - * @throws {@link RequestOptionsValidationError} when a defined value is zero, negative, or not - * finite. + * @throws {@link RequestOptionsValidationError} when a defined value is zero, negative, not + * finite, not an integer, or greater than `2**32 - 1`. */ timeoutMs(value: number | undefined): this { - if (value !== undefined && !(Number.isFinite(value) && value > 0)) { + if ( + value !== undefined && + !(Number.isInteger(value) && value > 0 && value <= MAX_TIMEOUT_MS) + ) { throw new RequestOptionsValidationError( - `timeout must be a finite positive duration, got ${String(value)}`, + `timeout must be an integer number of milliseconds in 1..${String(MAX_TIMEOUT_MS)}, got ${String(value)}`, ); } this.#timeoutMs = value; diff --git a/packages/core/src/seams/transport.test.ts b/packages/core/src/seams/transport.test.ts index 81a78ea..d43d497 100644 --- a/packages/core/src/seams/transport.test.ts +++ b/packages/core/src/seams/transport.test.ts @@ -3,12 +3,16 @@ // Exercises: SEAM-18's residual (composeSignal is the per-call-options-threading helper's cancellation half), // XCUT-2 (timeout vs. caller-cancellation told apart by signal.reason.name, not a message string). // No stub Transport is constructed — neither composeSignal nor isTimeoutSignal takes or returns one. +// Also HTTP-35 (composeSignal's documented RangeError is AbortSignal.timeout()'s own, and no value +// RequestOptionsBuilder accepts can produce it). import {describe, expect, test} from 'bun:test'; import { composeSignal, isTimeoutSignal, CancellationError, } from './transport.js'; +import {RequestOptions} from '../http/request-options.js'; +import {RequestOptionsValidationError} from '../http/errors.js'; describe('composeSignal', () => { test('returns undefined when neither input is supplied', () => { @@ -55,3 +59,29 @@ describe('isTimeoutSignal', () => { expect(isTimeoutSignal(new AbortController().signal)).toBe(false); }); }); + +describe('composeSignal timeout range (HTTP-35)', () => { + // `composeSignal` hands `timeoutMs` straight to `AbortSignal.timeout()`, and what that does with + // an out-of-range delay is a RUNTIME decision, measured 2026-09-05: Node raises + // `RangeError: The value of "delay" is out of range` for `1.5`, for `2**32` and for `-1`; Bun + // accepts `1.5` and `2**32` and raises a `TypeError` for `-1`. So the only claim assertable on + // both is the one below — that no value `RequestOptionsBuilder` accepts can reach that fork at + // all. `tests/node-conformance/seams.test.mjs` asserts the Node half, where the throw is real. + // This is why audit #67 / #76 put the range check in the model rather than clamping here. + test('every timeout RequestOptionsBuilder accepts composes without throwing', () => { + for (const value of [1, 1000, 2 ** 32 - 1]) { + const accepted = RequestOptions.newBuilder() + .timeoutMs(value) + .build().timeoutMs; + expect(() => composeSignal(undefined, accepted)).not.toThrow(); + } + }); + + test('a timeout the builder rejects never reaches composeSignal', () => { + for (const value of [1.5, 2 ** 32, -1, 0]) { + expect(() => RequestOptions.newBuilder().timeoutMs(value)).toThrow( + RequestOptionsValidationError, + ); + } + }); +}); diff --git a/packages/core/src/seams/transport.ts b/packages/core/src/seams/transport.ts index 8cd62fc..8388603 100644 --- a/packages/core/src/seams/transport.ts +++ b/packages/core/src/seams/transport.ts @@ -76,8 +76,20 @@ export interface Transport { * logic. * * @param userSignal - an optional caller-supplied abort signal. - * @param timeoutMs - an optional timeout, in milliseconds. + * @param timeoutMs - an optional timeout, in milliseconds. Must be an integer in + * `1 .. 2**32 - 1` — the range Node's `AbortSignal.timeout()` accepts. A value taken from + * {@link RequestOptions.timeoutMs} always is, because {@link RequestOptionsBuilder.timeoutMs} + * rejects everything else at the call site (HTTP-35). A transport's own `defaultTimeoutMs` + * construction option is NOT validated by this package and is the one remaining way an + * out-of-range value reaches here. * @returns the composed signal, the sole supplied signal, or `undefined` when neither is supplied. + * @throws Whatever the host runtime's `AbortSignal.timeout()` raises for an out-of-range delay, + * unwrapped. The runtimes disagree, measured 2026-09-05: Node raises `RangeError` for a fractional + * value, for anything above `4294967295`, and for a negative one; Bun accepts the first two and + * raises `TypeError` for the third. Not wrapped in a `DexpaceError` and not clamped here — it is a + * programming error in whatever supplied the value, and the divergence is exactly why the range + * lives on {@link RequestOptionsBuilder.timeoutMs}, which rejects every such value identically on + * both runtimes (HTTP-35, audit #67 / #76). * * @public */ diff --git a/tests/node-conformance/seams.test.mjs b/tests/node-conformance/seams.test.mjs index ba8f420..1ddc013 100644 --- a/tests/node-conformance/seams.test.mjs +++ b/tests/node-conformance/seams.test.mjs @@ -12,7 +12,7 @@ // 20.3.0. 20.3.0 is the first release carrying both. import assert from 'node:assert/strict'; import {describe, it} from 'node:test'; -import {composeSignal, isTimeoutSignal} from '@dexpace/core'; +import {composeSignal, isTimeoutSignal, RequestOptions} from '@dexpace/core'; describe('composeSignal on the declared Node floor', () => { it('returns a distinct AbortSignal.any() result when both a signal and a timeout are supplied', () => { @@ -101,3 +101,37 @@ describe('Web Crypto on the declared Node floor', () => { ); }); }); + +describe('composeSignal timeout range on Node (HTTP-35)', () => { + // Runtime-divergent by measurement, 2026-09-05: `AbortSignal.timeout(1.5)` and + // `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are accepted on Bun, and a + // negative delay raises `RangeError` on Node against `TypeError` on Bun. `bun test` therefore + // cannot assert either half of this, which is what puts the case here rather than only in + // `packages/core/src/seams/transport.test.ts`. Added by audit #67 / #76, which moved the range + // check onto `RequestOptionsBuilder.timeoutMs` for this reason. + it('accepts every timeout RequestOptionsBuilder accepts, at both ends of the range', () => { + for (const value of [1, 1000, 2 ** 32 - 1]) { + const accepted = RequestOptions.newBuilder() + .timeoutMs(value) + .build().timeoutMs; + assert.equal(accepted, value); + assert.ok( + composeSignal(undefined, accepted) instanceof AbortSignal, + `composeSignal must accept the timeout ${value}, which the model admits`, + ); + } + }); + + it('would raise RangeError on the values the model now rejects', () => { + for (const value of [1.5, 2 ** 32]) { + assert.throws( + () => composeSignal(undefined, value), + RangeError, + `Node's AbortSignal.timeout() must still reject ${value}; the model is what keeps it unreachable`, + ); + assert.throws(() => RequestOptions.newBuilder().timeoutMs(value), { + name: 'RequestOptionsValidationError', + }); + } + }); +}); From 58a4d3ecae68315a16c6f7d599dfbc8fcb991fe6 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:11:54 +0300 Subject: [PATCH 051/100] fix(core): stop the response logging tap from poisoning itself after close() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `closeDelegate` releases the reader (`response-body-logging.ts:62`), but `snapshot()` called `startDrain` unconditionally. So a `snapshot()` after `close()` read from a detached reader, `drainOnce`'s catch cached the raw `TypeError: Invalid state: The reader is not attached to a stream` as this wrapper's `failure`, and `error()` then reported a fabricated upstream failure forever — over a capture that never failed. `read()` after `close()` rejected with the same raw TypeError instead of an error from the SDK's own tree. `startDrain` now returns immediately when the wrapper is closed and no drain was ever started; a drain already in flight is left alone, because on the fits-cap path the drain closes the delegate itself and its promise is what `read()` awaits. `read()` gains a `ClosedResourceError('LoggedResponseBody')` after the fits-cap and tail-consumed checks, in that order: - fits-cap still serves a fresh non-consuming view after close — BODY-23 and BODY-28 both require it, so "closed" must not mean "unreadable"; - a consumed tail still reports `ConsumedBodyError`, which says more than "closed" (BODY-24); - everything else with the delegate gone is IO-42's state error, because the captured prefix is not the whole body and serving it would hand the consumer a silently truncated response. `error()` is unchanged and now cannot be poisoned. `snapshot()` is the post-mortem accessor BODY-28 asks for. `bun run api:local` on core: no change — `LoggedResponseBody` is `@internal`. Refs #77, #67. BODY-26, BODY-27, BODY-28, IO-42. --- .../src/body/response-body-logging.test.ts | 90 ++++++++++++++++++- .../core/src/body/response-body-logging.ts | 33 ++++++- 2 files changed, 118 insertions(+), 5 deletions(-) diff --git a/packages/core/src/body/response-body-logging.test.ts b/packages/core/src/body/response-body-logging.test.ts index d118273..9fe6b83 100644 --- a/packages/core/src/body/response-body-logging.test.ts +++ b/packages/core/src/body/response-body-logging.test.ts @@ -4,11 +4,16 @@ // reads), BODY-24 (exceeds-cap: prefix+tail once, second read fails), BODY-26 (drain failure cached, // partial bytes retained, error() does not drain), BODY-27 (close-once shared guard), BODY-28 (captured // buffer survives close), BODY-29 (reported length), BODY-32 (negative cap rejected), BODY-25 (a -// zero-length delegate chunk is a stream-contract violation, never end-of-stream) +// zero-length delegate chunk is a stream-contract violation, never end-of-stream), BODY-27/BODY-28 again +// (close() ends the drain rather than poisoning the wrapper: snapshot still serves the captured prefix, +// read() reports IO-42's state error, error() reports only a genuine drain failure) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {InvariantViolation} from '../invariant.js'; -import {SourceContractViolationError} from '../io/errors.js'; +import { + ClosedResourceError, + SourceContractViolationError, +} from '../io/errors.js'; import {withResponseLogging} from './response-body-logging.js'; function readableOf(...chunks: number[][]): ReadableStream { @@ -282,6 +287,87 @@ describe('the tail path enforces the same chunk contract (BODY-25)', () => { }); }); +describe('the tap is inert after close(), not poisoned by it (BODY-27, BODY-28)', () => { + // closeDelegate releases the reader. Every entry point that used to start a drain unconditionally then + // read from a detached reader, so `snapshot()` cached a raw `TypeError: Invalid state: The reader is + // not attached to a stream` as the wrapper's failure -- and `error()` reported that forever, over a + // capture that never failed. BODY-28 says the captured bytes survive close; they cannot survive it + // behind a fabricated error. + + test('close-then-snapshot returns the captured prefix and starts no drain', async () => { + const logged = withResponseLogging(readableOf([1, 2], [3, 4]), 2); + await logged.read(); // exceeds-cap: the prefix is captured, the delegate stays live + await logged.close(); + + expect([...logged.snapshot()]).toEqual([1, 2]); + await new Promise(resolve => setTimeout(resolve, 0)); // a drain started here would settle by now + expect(logged.error()).toBeNull(); + }); + + test('close-before-any-read leaves snapshot empty and error() null', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + await logged.close(); + + expect([...logged.snapshot()]).toEqual([]); + await new Promise(resolve => setTimeout(resolve, 0)); + expect(logged.error()).toBeNull(); + }); + + test('close-then-read rejects with ClosedResourceError, not a detached-reader TypeError', async () => { + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + await logged.close(); + + const error = await rejection(logged.read()); + expect(error).toBeInstanceOf(ClosedResourceError); + expect(error.message).toBe('LoggedResponseBody is closed'); + }); + + test('close-then-read in the exceeds-cap regime rejects too -- there is no live tail left', async () => { + // The drain stopped at the cap and nobody took the tail, so the captured prefix is NOT the whole + // body. Serving it would hand the consumer a silently truncated response; the delegate that held + // the rest is gone. + const logged = withResponseLogging(readableOf([1, 2], [3, 4]), 2); + logged.snapshot(); // starts the drain without taking the tail + await new Promise(resolve => setTimeout(resolve, 0)); + expect([...logged.snapshot()]).toEqual([1, 2]); + await logged.close(); + + expect(await rejection(logged.read())).toBeInstanceOf(ClosedResourceError); + }); + + test('close-then-error reports only a genuine drain failure (BODY-26)', async () => { + const boom = new Error('upstream reset'); + const failing = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + }, + pull(controller) { + controller.error(boom); + }, + }); + const logged = withResponseLogging(failing, 100); + expect(await rejection(logged.read())).toBe(boom); + // `cancel()` on an errored stream rejects with that stream's own stored error, which the + // 'a non-TypeError from cancel() propagates' case above already pins. Not what this test is about. + await logged.close().catch(() => undefined); + + // The real failure is not displaced by a close-induced one, and snapshot still shows the partial + // capture BODY-26 asked to be retained. + expect(logged.error()).toBe(boom); + expect([...logged.snapshot()]).toEqual([1, 2]); + }); + + test('the fits-cap regime still serves repeatable reads after its own close (BODY-23, BODY-28)', async () => { + // The drain itself closes the delegate on this path, so "closed" must NOT mean "unreadable". + const logged = withResponseLogging(readableOf([1, 2, 3]), 100); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + await logged.close(); + expect([...(await readAll(await logged.read()))]).toEqual([1, 2, 3]); + expect([...logged.snapshot()]).toEqual([1, 2, 3]); + expect(logged.error()).toBeNull(); + }); +}); + describe('snapshot is a drain trigger (BODY-22)', () => { test('calling snapshot starts the drain, without a read()', async () => { const logged = withResponseLogging(readableOf([1, 2, 3]), 100); diff --git a/packages/core/src/body/response-body-logging.ts b/packages/core/src/body/response-body-logging.ts index a5ebc0a..0e23e01 100644 --- a/packages/core/src/body/response-body-logging.ts +++ b/packages/core/src/body/response-body-logging.ts @@ -2,7 +2,10 @@ // packages/core/src/body/response-body-logging.ts import {invariant} from '../invariant.js'; import {ByteQueue} from '../io/byte-queue.js'; -import {SourceContractViolationError} from '../io/errors.js'; +import { + ClosedResourceError, + SourceContractViolationError, +} from '../io/errors.js'; import {MAX_BYTE_ARRAY_LENGTH} from '../io/limits.js'; import {ConsumedBodyError} from './errors.js'; @@ -20,11 +23,19 @@ export interface LoggedResponseBody { * (BODY-22). Fits-cap regime: every call, including calls after the first, returns a fresh * non-consuming view over the captured bytes (BODY-23). Exceeds-cap regime: exactly one call is * allowed; a second throws (BODY-24). If the drain failed, every call re-throws the cached error. + * After `close()` in any regime but fits-cap, throws `ClosedResourceError`: the delegate is gone and + * the captured prefix is not the whole body (BODY-27, BODY-28). */ read(): Promise>; - /** Non-consuming; reflects whatever has been captured so far, even after a failed drain (BODY-26). */ + /** + * Non-consuming; reflects whatever has been captured so far, even after a failed drain (BODY-26) and + * after `close()` (BODY-28), which it never restarts a drain past. + */ snapshot(): Uint8Array; - /** The cached drain failure, or null. MUST NOT trigger a drain (BODY-26). */ + /** + * The cached drain failure, or null. MUST NOT trigger a drain (BODY-26), and reports only a genuine + * upstream failure -- never one manufactured by reading past `close()`. + */ error(): Error | null; /** Captured size iff fully captured within the cap, else the delegate's declared length (BODY-29). */ readonly contentLength: number; @@ -129,8 +140,16 @@ async function drainOnce(state: DrainState): Promise { * The detached `.catch` matters: a snapshot-triggered drain has no awaiter, so without it a drain failure * becomes an unhandled rejection. Attaching a handler to a *copy* leaves the stored promise rejected, so * `read()` still re-throws the cached failure on every call (BODY-26). + * + * BODY-28: after `close()` there is nothing left to drain -- `closeDelegate` released the reader, so + * starting one here reads from a detached reader, raises a raw `TypeError: Invalid state`, and + * `drainOnce`'s catch caches it as this wrapper's `failure`. `error()` would then report a fabricated + * upstream failure forever, over a capture that never failed, and the captured bytes BODY-28 promises + * survive close would be reachable only past that lie. A drain already in flight is left alone: on the + * fits-cap path the drain closes the delegate itself, and its own promise is what `read()` awaits. */ function startDrain(state: DrainState): Promise { + if (state.closed && state.started === undefined) return Promise.resolve(); state.started ??= drainOnce(state); void state.started.catch(() => undefined); return state.started; @@ -226,10 +245,17 @@ export function withResponseLogging( return { async read(): Promise> { await startDrain(state); // a cached failure re-throws here on every call (BODY-26) + // Ordered deliberately. `fits` first: on that path the drain closed the delegate itself, and + // BODY-23 still requires every later read to be a fresh non-consuming view -- "closed" there does + // not mean "unreadable" (BODY-28). if (state.regime === 'fits') return capturedStream(state); if (state.tailConsumed) { throw new ConsumedBodyError('logged-response'); } + // Anything else with the delegate gone: there is no live tail to continue from, and the captured + // prefix is not the whole body, so serving it would hand the consumer a silently truncated + // response. IO-42's state error, not the raw `TypeError` a detached reader throws. + if (state.closed) throw new ClosedResourceError('LoggedResponseBody'); state.tailConsumed = true; return tailStream(state); }, @@ -238,6 +264,7 @@ export function withResponseLogging( // so it starts the drain and returns what has been captured so far rather than awaiting it; a // later read() awaits the very same in-flight promise, so the delegate is still read exactly once. // (BODY-26's "snapshot returns the partial bytes without throwing" is why it cannot await here.) + // After close() `startDrain` is a no-op, so this is the post-mortem accessor BODY-28 asks for. void startDrain(state); return state.captured.snapshot(); }, From fbaf269b18c085bc74eb89f03ab7739fa1f2012b Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:13:01 +0300 Subject: [PATCH 052/100] fix(core): one idempotency key per logical request across retry attempts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `dispatchWithRetry` re-ran the ENTIRE recovery chain per attempt, request chain included, so `idempotencyKeyStep`'s `generate()` fired once per attempt and three attempts of one logical request reached the server under three different keys — defeating exactly what the header is bought for (RECOV-32) and contradicting the `@public` TSDoc that has claimed "runs ONCE per call, upstream of retry" since 1f48926. Split `dispatchWithRecovery` into its two halves and let the retry adapter compose them differently: - `prepareRequest(request, chain)` — the request chain, once per LOGICAL request, with RECOV-2's throw-to-Failure conversion already applied. - `dispatchPrepared(prepared, config)` — the transport hop, the response chain and RECOV-10's unwrap, once per WIRE SEND. It takes an `Outcome` so a request-chain failure still gets its one trip through the response and recovery hooks with the transport untouched. `dispatchWithRecovery` is now the composition of the two and is behaviourally unchanged. `dispatchWithRetry` applies the chain once above `runWithRetry` and each attempt repeats only `dispatchPrepared` over `stampAttempt`'s fresh copy, so RETRY-38's ordinal varies while the key does not. RETRY-44 is satisfied, not traded away: its "downstream chain" is what sits below the retry point — transport plus response chain — and its "upstream steps MUST NOT mutate the shared in-flight request between attempts" clause now holds by construction, because upstream steps no longer run between attempts at all. The test that asserted the old layering by name (`re-runs the request recovery chain on every attempt (RETRY-44)`) is renamed to what the code now proves. Five new/changed cases in `retry-dispatch.test.ts`, all red before this change: chain applied once; one `generate()` across three attempts with the same key on all three sends; the same on the exhausting run; the ordinal varying while the key holds; and a request-chain throw that never reaches the transport, is not retried, and passes the recovery phase exactly once. Refs #73, #67. --- packages/core/src/recovery/idempotency-key.ts | 14 +- packages/core/src/recovery/orchestrator.ts | 141 +++++++++++--- packages/core/src/recovery/request-chain.ts | 9 +- packages/core/src/retry/engine.ts | 5 +- .../core/src/retry/retry-dispatch.test.ts | 172 +++++++++++++++++- packages/core/src/retry/retry-dispatch.ts | 48 ++++- 6 files changed, 340 insertions(+), 49 deletions(-) diff --git a/packages/core/src/recovery/idempotency-key.ts b/packages/core/src/recovery/idempotency-key.ts index 4f81df0..e2b967f 100644 --- a/packages/core/src/recovery/idempotency-key.ts +++ b/packages/core/src/recovery/idempotency-key.ts @@ -30,9 +30,17 @@ export interface IdempotencyKeyOptions { /** * A `RequestStep` that stamps an idempotency key on write requests (RECOV-32). * - * Runs ONCE per call, upstream of retry -- not per attempt. `retry/attempt-stamp.ts` is its sibling: - * that one writes the attempt ordinal on each per-attempt copy and preserves whatever this wrote - * (RETRY-38), so the server sees one stable key across every retry of the same logical request. + * Runs ONCE per logical request, upstream of retry -- not per attempt. `retry/attempt-stamp.ts` is + * its sibling: that one writes the attempt ordinal on each per-attempt copy and preserves whatever + * this wrote (RETRY-38), so the server sees one stable key across every retry of the same logical + * request. + * + * **That is a property of the composition, and the SDK's own retry adapter is what supplies it**: + * `retry/retry-dispatch.ts` applies the `RequestRecoveryChain` once, above the retry loop, and each + * attempt re-sends a copy of the request it produced. On its own a step can only promise RECOV-32's + * letter -- `generate()` is invoked at most once per *application* to an applicable request -- so a + * caller who re-applies their own chain per attempt will get a fresh key per attempt. Install the + * chain once and let the retry layer sit below it. * * @param options - the key strategy plus the header name, method set, and existing-key policy. * @returns the request step to install in a `RequestRecoveryChain`. diff --git a/packages/core/src/recovery/orchestrator.ts b/packages/core/src/recovery/orchestrator.ts index 3f3e9c8..16e9645 100644 --- a/packages/core/src/recovery/orchestrator.ts +++ b/packages/core/src/recovery/orchestrator.ts @@ -29,47 +29,98 @@ export interface DispatchConfig { } /** - * The unified recovery-chain orchestrator (RECOV-2, RECOV-10, RECOV-11). + * {@link DispatchConfig} without the request chain: everything the phases BELOW that chain need. * - * One `try`/`catch` wraps both the request chain's `apply()` and the transport invocation, so every - * throwable from either is caught and converted into a Failure before the response chain runs — a - * before-request throw cannot skip after-error handling. That conversion goes through - * {@link wrapCancellation} (RECOV-11), this orchestrator's catch being its only call site; RECOV-2's - * guarantee rests on that helper never throwing, since this catch clause is the last place a - * throwable could escape without meeting the recovery hooks. + * A retry loop drives {@link dispatchPrepared} through this, having already run the request chain + * once above itself. Naming it as a subset rather than duplicating the fields keeps the two shapes + * from drifting when `DispatchConfig` grows. * - * The final unwrap returns the response on a Success, or rethrows the Failure's throwable - * **unchanged** — no wrapping, no substitution (RECOV-10). Surfacing a typed exception is a - * recovery step's own responsibility, never this function's. + * @internal + */ +export type PreparedDispatchConfig = Omit; + +/** + * The request chain, run ONCE per logical request, with RECOV-2's conversion already applied: a + * throwing step becomes a Failure here rather than propagating, so no caller has to catch it. That + * conversion goes through {@link wrapCancellation} (RECOV-11); RECOV-2's guarantee rests on that + * helper never throwing, since this `catch` is the last place a request-chain throwable could + * escape without meeting the recovery hooks. * - * @param request - the request to prepare and send. - * @param config - transport, chains, and the per-call options and signal. - * @returns the response the terminal outcome carries. - * @throws Whatever the terminal Failure carries, by identity — any value, not necessarily an - * `Error`. + * Split out of {@link dispatchWithRecovery} on 2026-09-05 so `retry/retry-dispatch.ts` can run this + * half once for a whole logical request and repeat only the half below it per attempt. Before that + * split every retry attempt re-ran the request chain, and `recovery/idempotency-key.ts` generated a + * fresh key on each one — three attempts of one logical request reached the server as three + * distinct idempotency keys, which is exactly what RECOV-32's key is bought to prevent. * - * @public + * @param request - the request to prepare. + * @param requestChain - the ordered request steps (RECOV-3). + * @returns a Success carrying the prepared request, or a Failure carrying whatever a step threw. + * Never throws, for any input. + * + * @internal */ -export async function dispatchWithRecovery( +export async function prepareRequest( request: Request, - config: DispatchConfig, -): Promise { - let outcome: Outcome; + requestChain: RequestRecoveryChain, +): Promise> { try { - const preparedRequest = await config.requestChain.apply(request); - outcome = success( + return success(await requestChain.apply(request)); + } catch (error) { + return wrapCancellation(error); + } +} + +/** + * The transport hop, with RECOV-2's conversion applied to whatever it throws. + * + * A `prepared` that is already a Failure short-circuits it: the transport is not called, and the + * failure is handed on for the response chain to see. Both branches widen `Outcome` and + * `Outcome` to `Outcome` without a cast, because the failure variant does not + * mention the type parameter — and neither branch throws, which is what keeps RECOV-2 absolute. + */ +async function sendPrepared( + prepared: Outcome, + config: PreparedDispatchConfig, +): Promise> { + if (prepared.kind === 'failure') return prepared; + try { + return success( await config.transport.send( - preparedRequest, + prepared.value, config.options, config.signal, ), ); } catch (error) { - // RECOV-11: `Outcome` widens to `Outcome` without a cast, and never throws, - // which is what keeps RECOV-2 absolute. - outcome = wrapCancellation(error); + return wrapCancellation(error); } - const finalOutcome = await config.responseChain.apply(outcome); +} + +/** + * Everything below the request chain — the transport hop, the response chain, and RECOV-10's + * terminal unwrap. This is the part a retry loop repeats, once per wire send (RETRY-44's + * "downstream chain"). + * + * It takes {@link prepareRequest}'s outcome rather than a bare `Request` because a request-chain + * failure still owes RECOV-2 a trip through the response and recovery chains before it surfaces. + * On that input the transport is not called at all, which is the whole difference between the two + * variants. + * + * @param prepared - {@link prepareRequest}'s result for this logical request. + * @param config - transport, response chain, and the per-call options and signal. + * @returns the response the terminal outcome carries. + * @throws Whatever the terminal Failure carries, by identity — any value, not necessarily an + * `Error`. + * + * @internal + */ +export async function dispatchPrepared( + prepared: Outcome, + config: PreparedDispatchConfig, +): Promise { + const finalOutcome = await config.responseChain.apply( + await sendPrepared(prepared, config), + ); return fold( finalOutcome, response => response, @@ -78,3 +129,39 @@ export async function dispatchWithRecovery( }, ); } + +/** + * The unified recovery-chain orchestrator (RECOV-2, RECOV-10, RECOV-11). + * + * The two halves it composes are named: `prepareRequest` runs the request chain, and + * `dispatchPrepared` runs the transport hop and the response chain. Neither is exported from the + * package, so both are backticked rather than `{@link}`ed — api-extractor cannot resolve a + * reference out of the published surface into one, and the unresolved link is an error, not a + * warning to live with. Every throwable from either half is caught and converted into a Failure + * before the response chain runs — a before-request throw cannot skip after-error handling. + * + * The final unwrap returns the response on a Success, or rethrows the Failure's throwable + * **unchanged** — no wrapping, no substitution (RECOV-10). Surfacing a typed exception is a + * recovery step's own responsibility, never this function's. + * + * **One dispatch is one wire send.** Nothing here retries; a caller that wants retries composes the + * two halves itself so that the request chain runs once and only the second half repeats + * (`retry/retry-dispatch.ts`). + * + * @param request - the request to prepare and send. + * @param config - transport, chains, and the per-call options and signal. + * @returns the response the terminal outcome carries. + * @throws Whatever the terminal Failure carries, by identity — any value, not necessarily an + * `Error`. + * + * @public + */ +export async function dispatchWithRecovery( + request: Request, + config: DispatchConfig, +): Promise { + return dispatchPrepared( + await prepareRequest(request, config.requestChain), + config, + ); +} diff --git a/packages/core/src/recovery/request-chain.ts b/packages/core/src/recovery/request-chain.ts index 7876e87..3426126 100644 --- a/packages/core/src/recovery/request-chain.ts +++ b/packages/core/src/recovery/request-chain.ts @@ -13,8 +13,13 @@ export type RequestStep = (request: Request) => Promise; /** * A sequential left-to-right fold over request steps (RECOV-3): the output of step N is the input * of step N+1, an empty chain returns its input unchanged, and a throwing step aborts the remainder - * and propagates — `dispatchWithRecovery` (`orchestrator.ts`) converts that propagation into a - * `Failure` per RECOV-2, which is the only reason propagating here is safe. + * and propagates — `prepareRequest` (`orchestrator.ts`) converts that propagation into a `Failure` + * per RECOV-2, which is the only reason propagating here is safe. Every entry point runs the chain + * through that one helper, the retry adapter included. + * + * **Apply the chain once per logical request, not once per wire send.** Steps here are the ones + * whose output must stay stable across a retry — `idempotencyKeyStep` above all — and the SDK's own + * retry adapter sits below this chain for that reason (`retry/retry-dispatch.ts`). * * Safe under concurrent `apply()` calls (RECOV-14): after construction the instance holds nothing * but its step array, and every piece of per-call state lives in `apply()`'s locals. A later phase diff --git a/packages/core/src/retry/engine.ts b/packages/core/src/retry/engine.ts index a394c17..edce8be 100644 --- a/packages/core/src/retry/engine.ts +++ b/packages/core/src/retry/engine.ts @@ -356,7 +356,10 @@ function maybeEmitExhausted( * failure outcome carrying the trail, rather than left to surface as a bare rejected promise that * would drop RETRY-34's prior attempts on the floor. * - * @param request - the captured template every attempt re-sends. + * @param request - the captured template every attempt re-sends. Whatever the caller captured is + * already final: the pillar adapter passes the request arriving at the RETRY stage, and the + * recovery adapter passes the output of a request chain it applied ONCE, above this loop + * (RECOV-32 -- one idempotency key per logical request). This loop only ever copies it. * @param dispatch - performs one attempt and reports its outcome without throwing. * @param config - settings, clock, randomness, signal, and the optional delay override. * @returns the terminal outcome. On failure the error is the FINAL attempt's own, unwrapped, with diff --git a/packages/core/src/retry/retry-dispatch.test.ts b/packages/core/src/retry/retry-dispatch.test.ts index 4d82b0b..de83837 100644 --- a/packages/core/src/retry/retry-dispatch.test.ts +++ b/packages/core/src/retry/retry-dispatch.test.ts @@ -1,42 +1,101 @@ // SPDX-License-Identifier: MIT // packages/core/src/retry/retry-dispatch.test.ts -// Exercises: RECOV-17..20 (the recovery stack's retry lands here), RETRY-44 (each attempt re-runs the -// WHOLE recovery chain -- request chain, transport, response chain), RETRY-13/14/RECOV-30 (both entry -// points share one engine, so the schedule cannot drift). +// Exercises: RECOV-17..20 (the recovery stack's retry lands here), RECOV-32 (one idempotency key per +// LOGICAL request -- the strategy runs once and every wire send carries its result), RETRY-38 (the +// per-attempt ordinal is stamped on a fresh copy and preserves that key), RETRY-44 (fresh per-attempt +// state below the retry point; upstream steps do not run between attempts), RECOV-2 (a request-chain +// throw still meets the response and recovery hooks), RETRY-13/14/RECOV-30 (both entry points share +// one engine, so the schedule cannot drift). import {describe, expect, test} from 'bun:test'; +import {stringBody} from '../body/simple-bodies.js'; import type {Clock} from '../config/clock.js'; import {Request} from '../http/request.js'; +import type {Response} from '../http/response.js'; import {IoError} from '../io/errors.js'; +import {idempotencyKeyStep} from '../recovery/idempotency-key.js'; +import {failure, success, type Outcome} from '../recovery/outcome.js'; import {RequestRecoveryChain} from '../recovery/request-chain.js'; -import {ResponseRecoveryChain} from '../recovery/response-chain.js'; +import { + ResponseRecoveryChain, + type RecoveryStep, +} from '../recovery/response-chain.js'; import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; import {dispatchWithRetry, type RetryDispatchConfig} from './retry-dispatch.js'; -import {retrySettings} from './settings.js'; +import {retrySettings, type RetrySettings} from './settings.js'; const GET = Request.newBuilder().url('https://example.com').build(); +/** A POST the re-send gate lets through: RETRY-5 wants a body, and a replayable one. */ +function replayablePost(): Request { + return Request.newBuilder() + .method('POST') + .url('https://example.com') + .body(stringBody('payload')) + .build(); +} + const zeroClock: Clock = { now: () => 0, monotonic: () => 0, sleep: () => Promise.resolve(), }; +/** The third parameter is an object because `max-params` errors at four. */ +interface ConfigExtras { + readonly settings?: Partial; + readonly recoverySteps?: readonly RecoveryStep[]; +} + function configOf( transport: FakeTransport, requestSteps = new RequestRecoveryChain([]), + extras: ConfigExtras = {}, ): RetryDispatchConfig { return { transport, requestChain: requestSteps, - responseChain: new ResponseRecoveryChain([], []), + responseChain: new ResponseRecoveryChain([], extras.recoverySteps ?? []), retry: { - settings: retrySettings({maxAttempts: 3, fixedDelayMs: 0}), + settings: retrySettings({ + maxAttempts: 3, + fixedDelayMs: 0, + ...extras.settings, + }), clock: zeroClock, random: () => 0.5, }, }; } +/** + * A one-step request chain whose key strategy is COUNTED and whose keys are DISTINCT. Both matter: + * a strategy returning one constant value would pass every assertion below even if it were called + * once per attempt, which is the bug these cases exist to catch. + */ +function countingKeyChain(): { + chain: RequestRecoveryChain; + generated: () => number; +} { + let generated = 0; + const chain = new RequestRecoveryChain([ + idempotencyKeyStep({ + generate: () => { + generated += 1; + return `key-${String(generated)}`; + }, + }), + ]); + return {chain, generated: () => generated}; +} + +/** The named header seen by each wire send, in order. */ +function headerSent( + transport: FakeTransport, + name = 'Idempotency-Key', +): (string | undefined)[] { + return transport.calls.map(call => call.request.headers.get(name)); +} + /** * Captures a rejection reason. `expect(...).rejects` is typed as returning `void` under this * runner's type definitions, so awaiting it trips `@typescript-eslint/await-thenable`; this helper @@ -64,7 +123,7 @@ describe('dispatchWithRetry', () => { expect(transport.sendCount).toBe(2); }); - test('re-runs the request recovery chain on every attempt (RETRY-44)', async () => { + test('applies the request recovery chain ONCE per logical request, not per attempt (RETRY-44)', async () => { let applications = 0; const chain = new RequestRecoveryChain([ request => { @@ -79,7 +138,8 @@ describe('dispatchWithRetry', () => { await dispatchWithRetry(GET, configOf(transport, chain)); - expect(applications).toBe(2); + expect(transport.sendCount).toBe(2); + expect(applications).toBe(1); }); test('rethrows the terminal failure unchanged in shape', async () => { @@ -104,3 +164,97 @@ describe('dispatchWithRetry', () => { expect(transport.sendCount).toBe(1); }); }); + +describe('dispatchWithRetry and the idempotency key (RECOV-32)', () => { + test('generates ONE key for three attempts and sends it on all three', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([ + new IoError('reset'), + new IoError('reset'), + countingResponse(200).response, + ]); + + const response = await dispatchWithRetry( + replayablePost(), + configOf(transport, chain), + ); + + expect(response.status.code).toBe(200); + expect(transport.sendCount).toBe(3); + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + }); + + test('the run that exhausts the budget sends the same key on every attempt too', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([new IoError('reset')]); + + await rejectionOf( + dispatchWithRetry(replayablePost(), configOf(transport, chain)), + ); + + expect(transport.sendCount).toBe(3); + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + }); + + test('the attempt ordinal varies per send while the key does not (RETRY-38)', async () => { + const {chain, generated} = countingKeyChain(); + const transport = new FakeTransport([ + new IoError('reset'), + new IoError('reset'), + countingResponse(200).response, + ]); + + await dispatchWithRetry( + replayablePost(), + configOf(transport, chain, {settings: {attemptHeaderName: 'X-Attempt'}}), + ); + + // The ordinal is the ENGINE's, written per attempt on `stampAttempt`'s fresh copy; the key is + // the request chain's, written once above the loop. Both survive on every send. + expect(generated()).toBe(1); + expect(headerSent(transport)).toEqual(['key-1', 'key-1', 'key-1']); + expect(headerSent(transport, 'X-Attempt')).toEqual(['1', '2', '3']); + }); +}); + +describe('dispatchWithRetry and a failing request chain (RECOV-2)', () => { + const boom = new IoError('request step failed'); + const throwingChain = (): RequestRecoveryChain => + new RequestRecoveryChain([() => Promise.reject(boom)]); + + test('does not retry it, never reaches the transport, and runs the recovery phase once', async () => { + const seen: Outcome[] = []; + const recovery: RecoveryStep = outcome => { + seen.push(outcome); + return Promise.resolve(outcome); + }; + const transport = new FakeTransport([countingResponse(200).response]); + + const thrown = await rejectionOf( + dispatchWithRetry( + GET, + configOf(transport, throwingChain(), {recoverySteps: [recovery]}), + ), + ); + + expect(thrown).toBe(boom); + expect(transport.sendCount).toBe(0); + expect(seen).toEqual([failure(boom)]); + }); + + test('a recovery step may still convert that failure into a success', async () => { + const substitute = countingResponse(204).response; + const recovery: RecoveryStep = () => Promise.resolve(success(substitute)); + const transport = new FakeTransport([countingResponse(200).response]); + + const response = await dispatchWithRetry( + GET, + configOf(transport, throwingChain(), {recoverySteps: [recovery]}), + ); + + expect(response).toBe(substitute); + expect(transport.sendCount).toBe(0); + }); +}); diff --git a/packages/core/src/retry/retry-dispatch.ts b/packages/core/src/retry/retry-dispatch.ts index 9672fca..f641b44 100644 --- a/packages/core/src/retry/retry-dispatch.ts +++ b/packages/core/src/retry/retry-dispatch.ts @@ -3,7 +3,8 @@ import type {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; import { - dispatchWithRecovery, + dispatchPrepared, + prepareRequest, type DispatchConfig, } from '../recovery/orchestrator.js'; import {failure, fold, success} from '../recovery/outcome.js'; @@ -18,10 +19,15 @@ export interface RetryDispatchConfig extends DispatchConfig { readonly retry: RetryConfig; } +/** + * One attempt: the transport hop and the response chain over the ALREADY-prepared request the + * engine handed back from `stampAttempt`. The request chain is deliberately not in here — see + * {@link dispatchWithRetry}. + */ function attemptVia(config: RetryDispatchConfig): RetryDispatch { return async request => { try { - return success(await dispatchWithRecovery(request, config)); + return success(await dispatchPrepared(success(request), config)); } catch (error) { return failure(error); } @@ -32,9 +38,31 @@ function attemptVia(config: RetryDispatchConfig): RetryDispatch { * The recovery-chain entry point for retry (RECOV-17..RECOV-20). * * NOT a `RecoveryStep` -- a recovery step receives an outcome and has no way to re-dispatch. This - * wraps 4b's orchestrator instead, mirroring its `(request, config)` shape, so each attempt re-runs - * the ENTIRE recovery chain: request chain, transport, response chain. That is the recovery-side - * mirror of what `ctx.fork()` does for the pillar step (RETRY-44). + * composes 4b's orchestrator halves instead, mirroring its `(request, config)` shape. + * + * **The request chain runs ONCE, above the loop; each attempt repeats only what is below it** -- + * the transport hop and the response chain, over `stampAttempt`'s fresh copy of the one prepared + * request. That is the layering `recovery/idempotency-key.ts` documents and RECOV-32 needs: its + * `generate()` is invoked once per logical request, so all N attempts reach the server under one + * key, and RETRY-38's per-attempt ordinal is written on the copy without disturbing it. Until + * 2026-09-05 this function re-ran the whole recovery chain per attempt and three attempts produced + * three different keys, defeating the header's entire purpose (audit #67, issue #73). + * + * RETRY-44 is satisfied, not traded away. Its "downstream chain" is whatever sits below the retry + * point, which here is transport plus response chain, and that is re-executed with fresh + * per-attempt state every time. Its second clause -- upstream steps MUST NOT mutate the shared + * in-flight request between attempts -- holds by construction now, because upstream steps no longer + * run between attempts at all. + * + * A request-chain failure is NOT retried: it never reached the wire, so RETRY-5's re-send gate has + * nothing to judge and re-running the step that just threw would only throw again. It still passes + * through the response and recovery chains exactly once, so RECOV-2's "no throwable bypasses the + * recovery hooks" and RECOV-10's unwrap are unchanged. + * + * One consequence worth naming: the engine's re-send gate (RETRY-5/RECOV-18, `isResendable`) now + * reads the PREPARED request rather than the caller's, so a request step that swaps in a + * non-replayable body makes the call non-retryable -- which is the honest answer, since the + * prepared request is what a retry would have to re-send. * * Shares `runWithRetry` with the pillar adapter, which is what makes RETRY-13/RETRY-14/RECOV-30's * "the two stacks must not drift" structural rather than a discipline. @@ -44,7 +72,7 @@ function attemptVia(config: RetryDispatchConfig): RetryDispatch { * @returns the response of the terminal successful attempt. * @throws Whatever the FINAL attempt failed with, unwrapped -- the same class a single-attempt run * would have thrown. RETRY-34's earlier attempts are recorded beside it and read back through - * `retryAttempts()`. + * `retryAttempts()`. A request-chain throwable surfaces the same way, with no trail. * * @internal */ @@ -52,7 +80,13 @@ export async function dispatchWithRetry( request: Request, config: RetryDispatchConfig, ): Promise { - const outcome = await runWithRetry(request, attemptVia(config), config.retry); + const prepared = await prepareRequest(request, config.requestChain); + if (prepared.kind === 'failure') return dispatchPrepared(prepared, config); + const outcome = await runWithRetry( + prepared.value, + attemptVia(config), + config.retry, + ); return fold( outcome, response => response, From dad81b43fa6b40327bc7787212e7787d3eda5f86 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:13:59 +0300 Subject: [PATCH 053/100] docs: record the RETRY-44 reading behind the single-application request chain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit D0 puts a remediation-run deviation in `docs/deviations.md` under "Deviations recorded outside a phase", not in a `docs/work/` phase ledger. One row: what `RETRY-44`'s "downstream chain" means in the recovery stack, why both of its clauses hold under the new layering, the two rejected alternatives, and the re-send-gate consequence — with the four pinning tests named by line. Also re-anchors item 3's `retry-dispatch.ts:55` citation, which the previous commit shifted to `:85`. Same file, same reason waves 1-3 re-anchored theirs. Refs #73, #67. --- docs/deviations.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/deviations.md b/docs/deviations.md index 6bf260b..ded2cde 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -143,7 +143,7 @@ non-bridge clause survives and is enforced as an ordinary obligation on `send()` **Verified.** One engine — `runWithRetry` (`packages/core/src/retry/engine.ts:367`) — with exactly two thin callers: the pillar step (`packages/core/src/retry/retry-step.ts:151`) and the dispatch adapter -(`packages/core/src/retry/retry-dispatch.ts:55`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | +(`packages/core/src/retry/retry-dispatch.ts:85`). `totalTimeoutMs` is `readonly totalTimeoutMs?: number | undefined` and undefined by default (`packages/core/src/retry/settings.ts:27`), pinned by a test named for `RETRY-28` (`packages/core/src/retry/settings.test.ts:20`). @@ -482,6 +482,7 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **`XCUT-16`'s replay guard is keyed on whether the hop was guarded, not on whether the replacement looks credentialed.** `XCUT-16` and `AUTH-28` say the guard applies "on any path where a credential will be attached", and carve out "a deliberately credential-free re-issue MAY proceed over any scheme". Deciding which of the two a challenge replacement is cannot be done by reading header names: the step's own `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, and a `challengeHook` may invent a carrier this step has never been told about. The port therefore reads "a credential will be attached" as a property of the HOP — if the outbound pass ran the HTTPS guard, so does the replay, whatever URL and headers the hook chose. **Strictly wider than the requirement's letter**, and knowingly so: it refuses a downgraded replacement that carries no credential at all, on a hop that is credentialed. The carve-out is preserved where it is observable — a `NO_AUTH` hop is never guarded outbound, and its replay is guarded only when the replacement carries `Authorization` or `Proxy-Authorization`, which is the previous rule kept as a second arm. *Rejected:* deriving the credential-carrying header names from configuration, which misses the hook-invented carrier and is the shape that let the reported leak through | audit #67 / #71 | 2026-09-04 | `docs/product-spec/19-cross-cutting-invariants-and-policies.md:44` is the requirement and its carve-out. `packages/core/src/auth/auth-step.ts:389` sets `OutboundPlan.guarded`; `:564-575` is `guardReplayScheme` and its two arms. Pinned by "a replacement carrying a NON-standard credential header over plaintext is refused" and "a header-free replacement over plaintext is refused too" in `packages/core/src/auth/auth-step.test.ts`, and by the "XCUT-16: a guarded hop stays guarded across a challenge replay" block in `tests/conformance/xcut/security-by-default.conformance.test.ts` | not yet in §10 | | **`ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" is not honoured: the RxJS SSE adapter takes ownership and closes.** `sseEvents$` and `typedSse$` pass `() => stream.close()` as `fromAsyncIterable`'s `release`, and RxJS runs a subscriber's finalizer on *every* termination — unsubscription, end-of-source and a source error alike, which is the complete list the clause names. **Kept, deliberately, on two grounds.** (1) **The clause has no subject on this platform.** It presumes a source whose iterator return leaves the source open; this port's `SseStream` is deliberately not that one. `#iterate`'s `finally` calls `#releaseQuietly()`, so the resource is released whenever the runtime drives `return()` — which `fromAsyncIterable` must do exactly once (`ASYNC-6`), and which a plain `for await` with `break` does too. Removing the callback would change which channel reports a release failure and when the release runs, not whether the caller-owned source ends up closed. (2) **The ordering is load-bearing.** The release runs *ahead of* `iterator.return()` because an async generator's `return()` queues behind a suspended `next()`, and an SSE stream idling between events is parked in exactly that pull — so without the callback an `unsubscribe()` stays pending until the server next sends a byte, holding the socket open indefinitely. Measured: deleting the two `release` arguments turns four cases red — the two suspended-pull ones, as "the teardown did not settle within 500ms", and the two pre-existing idle-unsubscribe assertions — while every exactly-once release count stays green, which is the shape of the claim. Pagination attaches no release for the complementary reason: a `Paginator`'s pulls are bounded HTTP exchanges, never a wait on a server that may never answer. *Rejected:* dropping the callback to match the letter (reintroduces the hang for no change in what closes). *Rejected:* a caller-facing `{ownership}` option (two behaviours to document for a case with one correct answer). The public TSDoc and `packages/rx/README.md` now state the transfer outright — subscribing hands the stream over, do not close it yourself and do not iterate it afterwards — rather than leaving the `ASYNC-21` citation on the doc comment's first line to read as satisfied | audit #67 / #75 | 2026-09-05 | `packages/rx/src/sse.ts:46` and `:73-75` are the two `release` arguments; `packages/rx/src/from-async-iterable.ts:103-108` is the teardown that runs one on every termination; `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:42` is the requirement. Ground 1: `packages/core/src/sse/stream.ts:136-139` (`#iterate`'s `finally` → `#releaseQuietly()`) with `:117-121` (`close()` memoized, `SSE-28`). Ground 2: `packages/rx/src/from-async-iterable.ts:44-48` states the ordering and why. Pinned by the two `resource ownership` blocks in `packages/rx/src/sse.test.ts`, which count the release the OWNED resource sees rather than `SseStream.close()` calls — the facade memoizes, so a facade-level count reads "once" however many paths call it — and by "SSE ownership transfer releases once on Node" in `tests/node-conformance/rx-bridge.test.mjs`. Phase 8b marked `ASYNC-21` ✅ with this clause dropped from its gist (`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67`); that is a dated record and is left as written. The other half is `SSE-41`'s own "documented source ownership" clause, which the same checklist marked ✅ (`:74`) on the strength of documentation that named unsubscription only — completed by the TSDoc and README rewrite this row accompanies | not yet in §10 | | **`AUTH-22`'s "emit cnonce/nc/qop only when qop is negotiated" is not applied to `cnonce` for a `-sess` algorithm.** A `-sess` HA1 is `H(H(user:realm:pass):nonce:cnonce)` (RFC 7616 §3.4.2), so the client nonce is an *input to the hash* for `MD5-sess` and `SHA-256-sess` whatever `qop` the challenge offered. The port implemented AUTH-22 to the letter: it drew a fresh cnonce, folded it into HA1, and then omitted it from the header whenever `qop` was absent — a response no server can verify, because it has no way to reconstruct HA1. AUTH-30 bounds the re-challenge replay to one 401, so every such exchange simply failed. **`cnonce` is now emitted for any `-sess` algorithm; `nc` and `qop` stay conditional exactly as AUTH-22 says**, because RFC 2069's response input is `H(HA1:nonce:HA2)` and carries no nonce count, so emitting one would advertise a count the response was not computed over. RFC 7616 §3.4 states the wider rule outright — "cnonce: This parameter MUST be used by all implementations". AUTH-22's clause is RFC 2617's RFC 2069-compatibility form, written before `-sess` existed, and the requirement's own AUTH-15 mandates both `-sess` algorithms, so the two sentences cannot both be followed. *Rejected:* declining a `-sess`-without-`qop` challenge instead, which turns every such server into a guaranteed 401 for no security gain, when the value the server needs has already been computed | audit #67 / #74 | 2026-09-05 | `packages/core/src/auth/digest.ts:345-350` (the `-sess` HA1 that consumes the cnonce) against `:405-408` (`buildHeaderValue`, where the `else if` now emits it); `docs/product-spec/11-authentication.md:18` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:357` are AUTH-22's wording. Pinned by the `digestHandler -sess without qop (AUTH-17/AUTH-22)` block in `packages/core/src/auth/digest.test.ts` — one row asserting the header carries `cnonce` and neither `nc` nor `qop`, one recomputing the response from the header's OWN cnonce so a value drawn twice would fail — and by the `MD5-sess, no qop` vector in the same file | not yet in §10 | +| **`RETRY-44`'s "downstream chain" is read as everything BELOW the retry point, which in the recovery stack excludes the request chain.** The requirement has two clauses: each attempt re-executes the downstream chain with fresh per-attempt state, and "upstream steps MUST NOT mutate the shared in-flight request between attempts". The port originally read the first clause as covering the *whole* recovery chain and re-ran `RequestRecoveryChain.apply()` per attempt, with a test that said so by name. That makes `packages/core/src/recovery/idempotency-key.ts` generate a fresh key on every attempt, so three attempts of one logical request reach the server as three unrelated writes — the precise failure `RECOV-32` exists to prevent, and the opposite of what that step's own `@public` TSDoc promises. **The chain is now applied once, above the loop; each attempt re-executes transport plus response chain over `stampAttempt`'s fresh copy of the prepared request.** Under this reading both clauses hold and the second holds *by construction*: upstream steps cannot mutate the in-flight request between attempts because they no longer run between attempts. The pillar stack is untouched — there "downstream" is the forked continuation (`ctx.fork()`), and `retryStep` still re-drives it per attempt. *Rejected:* memoizing the key on the template (a `WeakMap` keyed by the `Request` instance) — a caller who deliberately sends one immutable `Request` value twice would replay the key and have the server drop a genuine second call. *Rejected:* re-running the chain over the *prepared* request each attempt — the chain would read its own output, which is clause two's mutation in different clothes, and every shipped and caller-written step would have to be proven idempotent. One consequence recorded rather than assumed: the re-send gate (`RETRY-5`/`RECOV-18`) now judges the prepared request rather than the caller's, which is what a retry would actually re-send | audit #67 / #73 | 2026-09-05 | `packages/core/src/retry/retry-dispatch.ts:83-88` (the chain applied once, then `runWithRetry`) against `:27-33` (the per-attempt half); `packages/core/src/recovery/orchestrator.ts:62` (`prepareRequest`) and `:117` (`dispatchPrepared`); `packages/core/src/retry/engine.ts:243` is the gate that now reads the prepared request. `RETRY-44`'s wording is `docs/product-spec/09-retry-and-resilience.md:35` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:306`. Pinned by `packages/core/src/retry/retry-dispatch.test.ts:126` (chain applied once), `:169` (one `generate()`, one key on three sends), `:201` (the `RETRY-38` ordinal varies while the key does not) and `:227` (a request-chain throw is not retried and meets the recovery phase exactly once) | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) From 01dfdfbe5de56cbb87f0705a1ef37675439d7b35 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:15:48 +0300 Subject: [PATCH 054/100] test(node-conformance): cover the five Web Streams bridges, and pin the three #77 fixes on Node MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `toReadableStream`, `toWritableStream`, `TeeSink`'s bridge, `withRequestLogging` and `withResponseLogging` had zero cases under `tests/node-conformance/`. Every one is a hand-written underlying source or sink object, so what they exercise is the runtime's own pull scheduling, cancel dispatch and reader-lock bookkeeping — CLAUDE.md's runtime-divergent surface, and the tree's membership rule. `io-byte-stream.test.mjs` (9 cases): one-chunk-at-a-time pull rather than an eager drain; natural EOF closing the bridge but not the owning source, so an outstanding peek is still readable (IO-19); cancel closing the source AND releasing the caller stream's lock; a mid-stream read failure closing the source itself, because the spec does not dispatch `cancel` on an errored stream; `pipeTo` chaining close through the bridge to the destination; abort carrying its reason instead of collapsing into a graceful close; a zero-length chunk dropped rather than forwarded as a chunked-encoding terminator; the tee bridge routing through the tee so the tap still sees the bytes; abort reaching the primary while the tap survives. `body-lifecycle.test.mjs` (13 cases): the multipart boundary round-trip; the zero-length-chunk refusal and the declared-length-0 allowance; the request tap's mirror, its clear-between-writes and its abort of a sink the delegate never touched; the response tap's prefix-then-tail, its close-once cancel, its inertness after `close()` and its repeatable fits-cap reads. The multipart rows are the reason this is not optional. Bun's `Response.formData()` accepts `boundary=a,b`; Node's rejects the whole body with `TypeError: Failed to parse body as FormData`. The Bun suite was green over a Content-Type no Node peer could parse, and only a case here reproduces it. Both taps are `@internal`, so they come in by direct `dist/` path like `io/`. Refs #77, #67. IO-16, IO-19, IO-26, HTTP-39, BODY-10, HTTP-51, BODY-17..28. --- tests/node-conformance/README.md | 14 +- .../node-conformance/body-lifecycle.test.mjs | 184 ++++++++++++++++++ .../node-conformance/io-byte-stream.test.mjs | 160 +++++++++++++++ 3 files changed, 354 insertions(+), 4 deletions(-) diff --git a/tests/node-conformance/README.md b/tests/node-conformance/README.md index b9c62df..ef802e0 100644 --- a/tests/node-conformance/README.md +++ b/tests/node-conformance/README.md @@ -51,10 +51,16 @@ Node). **A phase that touches a runtime-divergent surface adds a case here, not only to `bun test`** (§5.9:378). Since Phase 4 that has meant most phases — pipelines, retry, redirect, auth, serde, SSE, pagination, -configuration, observability, the two concrete transports, and the RxJS bridge all have cases here. Two are -worth naming as the shape to aim for: 8a's `fetch`/`undici` transports, where this stops being precautionary -and becomes the point, and 8b's RxJS bridge, whose reason for being hand-written is a cancellation path the -runtime decides. +configuration, observability, the two concrete transports, and the RxJS bridge all have cases here — as, since +audit #67's #77, do the five Web Streams bridges that had none: `BufferedSource.toReadableStream`, +`BufferedSink.toWritableStream`, `TeeSink.toWritableStream` in `io-byte-stream.test.mjs`, and the +`withRequestLogging` / `withResponseLogging` body taps in `body-lifecycle.test.mjs`. Every one of them is a +hand-written underlying source or sink object, so what they exercise is the runtime's own pull scheduling, +cancel dispatch and reader-lock bookkeeping. Three are worth naming as the shape to aim for: 8a's +`fetch`/`undici` transports, where this stops being precautionary and becomes the point; 8b's RxJS bridge, +whose reason for being hand-written is a cancellation path the runtime decides; and #77's multipart +`Content-Type`, where Bun's `Response.formData()` accepted a header Node's `Response.formData()` rejected — +the Bun rows were green over a body no Node peer could parse, and only the case here reproduced it. ## Which cases exist diff --git a/tests/node-conformance/body-lifecycle.test.mjs b/tests/node-conformance/body-lifecycle.test.mjs index ec37f82..931f5ec 100644 --- a/tests/node-conformance/body-lifecycle.test.mjs +++ b/tests/node-conformance/body-lifecycle.test.mjs @@ -24,6 +24,14 @@ import { stringBody, toHttpError, } from '@dexpace/core'; +// The two logging taps are `@internal` -- `body/index.ts` holds them and the public barrel deliberately +// does not, so they are reached by direct `dist/` file path, exactly as `io-byte-stream.test.mjs` reaches +// `io/`. Still the BUILT artifact, never `src/`. +import {withRequestLogging} from '../../packages/core/dist/body/request-body-logging.js'; +import {withResponseLogging} from '../../packages/core/dist/body/response-body-logging.js'; + +/** `Response` above is the SDK's model class, which shadows the platform global this file also needs. */ +const PlatformResponse = globalThis.Response; function streamOf(bytes) { return new ReadableStream({ @@ -227,3 +235,179 @@ describe('toHttpError buffering on Node', () => { assert.deepEqual([...(await response.bytes())], [1, 2, 3]); }); }); + +describe("the multipart Content-Type parses in Node's own FormData reader (HTTP-51)", () => { + // The reproducer for the boundary-quoting fix, and the reason it belongs here rather than only in + // `bun test`: Bun's `Response.formData()` tolerates an unquoted `boundary=a,b`, Node's (undici's) + // rejects the whole body with `TypeError: Failed to parse body as FormData`. Two independent parsers + // disagreeing about a header this SDK generates is precisely what this tree exists to catch. + for (const boundary of ['a,b', 'bound ary', 'a:b', 'a=b', 'a?b', '(a)/b']) { + it(`round-trips a body framed with ${JSON.stringify(boundary)}`, async () => { + const body = multipartBody( + [{name: 'field', body: stringBody('value')}], + boundary, + ); + const parsed = await new PlatformResponse(await collect(body), { + headers: {'content-type': body.mediaType}, + }).formData(); + assert.equal(parsed.get('field'), 'value'); + }); + } + + it('leaves a boundary that is already a bare token unquoted', () => { + const body = multipartBody([{name: 'a', body: stringBody('x')}], 'plain-1'); + assert.equal(body.mediaType, 'multipart/form-data; boundary=plain-1'); + }); +}); + +describe('an exact-length copy refuses a zero-length delivery on Node (HTTP-39/BODY-10)', () => { + it('raises rather than forwarding a chunked-encoding terminator to the sink', async () => { + const source = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2])); + controller.enqueue(new Uint8Array(0)); + controller.enqueue(Uint8Array.from([3])); + controller.close(); + }, + }); + const chunkLengths = []; + + await assert.rejects( + streamBody(source, undefined, 3).writeTo( + new WritableStream({write: c => void chunkLengths.push(c.length)}), + ), + error => error.name === 'SourceContractViolationError', + ); + assert.deepEqual(chunkLengths, [2]); + }); + + it('still allows a declared length of 0 over a source that just closes', async () => { + let closed = false; + await streamBody( + new ReadableStream({ + start: c => { + c.close(); + }, + }), + undefined, + 0, + ).writeTo( + new WritableStream({ + write: c => void c, + close: () => void (closed = true), + }), + ); + assert.equal(closed, true); + }); +}); + +describe('withRequestLogging over Node Web Streams (BODY-17..21)', () => { + it('mirrors into the tap while the full untruncated payload reaches the primary', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([1, 2, 3, 4, 5])), + 2, + ); + assert.deepEqual([...(await collect(logged))], [1, 2, 3, 4, 5]); + assert.deepEqual([...logged.snapshot()], [1, 2]); + }); + + it('clears the tap between writes so a retry does not accumulate stale bytes (BODY-18)', async () => { + const logged = withRequestLogging( + byteArrayBody(Uint8Array.from([9, 9])), + 8, + ); + await collect(logged); + await collect(logged); + assert.deepEqual([...logged.snapshot()], [9, 9]); + }); + + it('aborts the real sink when the delegate refuses before ever touching the adapter', async () => { + // A `ConsumedBodyError` on a second write reaches neither handler on the adapter stream, so without + // the wrapper's own catch the primary writer stays open and locked forever -- a held connection. + // Whether an abort dispatched on a writer reaches the underlying sink's algorithm, and does so + // instead of the close algorithm, is runtime plumbing rather than logic. + const logged = withRequestLogging( + streamBody(streamOf([1, 2, 3]), undefined, 3), + 8, + ); + await collect(logged); // consumes the single-use delegate + + let abortReason = 'NOT-ABORTED'; + let closed = false; + const destination = new WritableStream({ + write: chunk => void chunk, + close: () => void (closed = true), + abort: reason => void (abortReason = reason), + }); + await assert.rejects( + logged.writeTo(destination), + error => error.name === 'ConsumedBodyError', + ); + assert.equal(abortReason.name, 'ConsumedBodyError'); + // Never closed: a broken message must not be committed downstream as a well-formed short one. + assert.equal(closed, false); + }); +}); + +describe('withResponseLogging over Node Web Streams (BODY-22..28)', () => { + function chunked(...chunks) { + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) controller.enqueue(Uint8Array.from(chunk)); + controller.close(); + }, + }); + } + + async function readAll(stream) { + const out = []; + for await (const chunk of stream) out.push(...chunk); + return out; + } + + it('serves the prefix then the still-live tail, one pull at a time (BODY-24)', async () => { + const logged = withResponseLogging(chunked([1, 2], [3, 4, 5]), 3); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3, 4, 5]); + assert.deepEqual([...logged.snapshot()], [1, 2, 3]); + }); + + it('cancelling the tail stream cancels the delegate exactly once (BODY-27)', async () => { + let cancels = 0; + const delegate = chunked([1, 2], [3, 4]); + const inner = delegate.cancel.bind(delegate); + delegate.cancel = async reason => { + cancels += 1; + return inner(reason); + }; + const logged = withResponseLogging(delegate, 1); + + await (await logged.read()).cancel(); + await logged.close(); + assert.equal(cancels, 1); + }); + + it('close() leaves the tap inert instead of poisoning it with a detached-reader TypeError', async () => { + // Node reports a released reader as `TypeError [ERR_INVALID_STATE]: Invalid state: The reader is not + // attached to a stream`. That message used to be cached as this wrapper's drain failure and reported + // by error() forever, over a capture that never failed. + const logged = withResponseLogging(chunked([1, 2, 3]), 100); + await logged.close(); + + assert.deepEqual([...logged.snapshot()], []); + await new Promise(resolve => setTimeout(resolve, 0)); + assert.equal(logged.error(), null); + await assert.rejects( + logged.read(), + error => error.name === 'ClosedResourceError', + ); + }); + + it('a fits-cap capture stays repeatably readable after close (BODY-23, BODY-28)', async () => { + const logged = withResponseLogging(chunked([1, 2, 3]), 100); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3]); + await logged.close(); + assert.deepEqual(await readAll(await logged.read()), [1, 2, 3]); + assert.deepEqual([...logged.snapshot()], [1, 2, 3]); + assert.equal(logged.error(), null); + }); +}); diff --git a/tests/node-conformance/io-byte-stream.test.mjs b/tests/node-conformance/io-byte-stream.test.mjs index c9165ce..c86f68c 100644 --- a/tests/node-conformance/io-byte-stream.test.mjs +++ b/tests/node-conformance/io-byte-stream.test.mjs @@ -224,3 +224,163 @@ describe('BufferedSink and TeeSink over a Node WritableStream', () => { ); }); }); + +// The five Web Streams bridges had no case here at all until #77. Every one of them is a hand-written +// `ReadableStream`/`WritableStream` underlying-source or -sink object, so what they exercise is the +// runtime's own pull scheduling, cancel dispatch and reader-lock bookkeeping — the three things §5.9 +// names and the three that two independent Streams implementations are most likely to differ on. + +describe('BufferedSource.toReadableStream on Node (IO-16)', () => { + it('pulls one chunk at a time instead of draining the source eagerly', async () => { + let pulls = 0; + const stream = new ReadableStream({ + pull(controller) { + pulls += 1; + if (pulls > 4) { + controller.close(); + return; + } + controller.enqueue(Uint8Array.from([pulls])); + }, + }); + const bridge = BufferedSource.overStream(stream).toReadableStream(); + const reader = bridge.getReader(); + + const first = await reader.read(); + assert.deepEqual([...first.value], [1]); + // Node's default queuing strategy reads one chunk ahead, so at most one pull beyond the one just + // served. The assertion that matters is that the whole 4-chunk source has not been materialized. + assert.ok(pulls <= 2, `expected at most 2 pulls, saw ${pulls}`); + await reader.cancel(); + }); + + it('closes the bridge at natural EOF without tearing down the owning source', async () => { + // IO-19: closing the source here would invalidate every outstanding peek/slice view, defeating the + // bridge's most natural usage — take a preview, hand the bridge to the transport, read the preview + // afterwards. Only an explicit cancel closes the source (next case). + const source = BufferedSource.overStream(streamOfChunks([[1, 2], [3]])); + const preview = source.peek(); + const collected = []; + for await (const chunk of source.toReadableStream()) + collected.push(...chunk); + + assert.deepEqual(collected, [1, 2, 3]); + assert.deepEqual([...(await preview.readBytes())], [1, 2, 3]); + assert.equal(source.closed, false); + await source.close(); + }); + + it('cancelling the bridge closes the source AND releases the caller stream lock', async () => { + const stream = streamOfChunks([[1, 2, 3]]); + const source = BufferedSource.overStream(stream); + assert.equal(stream.locked, true); + + await source.toReadableStream().cancel(); + assert.equal(source.closed, true); + // cancel() cancels the stream but never releases the reader's lock; only releaseLock() does, and a + // leaked lock on a connection-backed source is a held socket. + assert.equal(stream.locked, false); + }); + + it('a mid-stream read failure closes the source rather than stranding the lock', async () => { + // The Streams spec does NOT invoke `cancel` on an errored stream, so the bridge has to close the + // source itself on this path. A runtime that dispatched cancel here would hide the bug. + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(Uint8Array.from([1, 2, 3])); + }, + pull() { + throw new Error('mid-stream read failure'); + }, + }); + const source = BufferedSource.overStream(stream); + const reader = source.toReadableStream().getReader(); + + assert.deepEqual([...(await reader.read()).value], [1, 2, 3]); + await assert.rejects(reader.read(), /mid-stream read failure/); + assert.equal(source.closed, true); + assert.equal(stream.locked, false); + }); +}); + +describe('BufferedSink.toWritableStream on Node (IO-16)', () => { + it('carries a pipeTo through to the destination and closes it', async () => { + // `pipeTo` closes its destination on natural EOF, and IO-16 says closing the bridge closes the + // sink, which closes the caller's stream. Three closes chained through two runtimes' plumbing. + const written = []; + let closed = false; + const destination = new WritableStream({ + write: chunk => void written.push(...chunk), + close: () => void (closed = true), + }); + const sink = BufferedSink.overStream(destination); + + await streamOfChunks([[1, 2], [3]]).pipeTo(sink.toWritableStream()); + assert.deepEqual(written, [1, 2, 3]); + assert.equal(sink.closed, true); + assert.equal(closed, true); + }); + + it('aborting the bridge aborts the sink and carries the reason, rather than closing it', async () => { + // Collapsing an abort into a graceful close commits a cancelled request body downstream as a + // well-formed complete one, so the peer cannot tell an aborted upload from a successful short one. + let closed = false; + let abortReason = 'NOT-ABORTED'; + const destination = new WritableStream({ + write: chunk => void chunk, + close: () => void (closed = true), + abort: reason => void (abortReason = reason), + }); + const sink = BufferedSink.overStream(destination); + const writer = sink.toWritableStream().getWriter(); + await writer.write(Uint8Array.from([1, 2, 3])); + + const reason = new Error('user cancelled'); + await writer.abort(reason); + assert.equal(abortReason, reason); + assert.equal(closed, false); + assert.equal(sink.closed, true); + }); + + it('drops a zero-length chunk rather than forwarding a chunked-encoding terminator', async () => { + const {stream, written} = collectingStream(); + const sink = BufferedSink.overStream(stream); + const writer = sink.toWritableStream().getWriter(); + await writer.write(new Uint8Array(0)); + await writer.write(Uint8Array.from([7])); + await writer.close(); + assert.deepEqual([...written()], [7]); + }); +}); + +describe('TeeSink.toWritableStream on Node (IO-16, IO-26)', () => { + it('routes through the tee, so bytes written to the bridge still reach the tap', async () => { + // Handing callers the PRIMARY's bridge instead would let every byte written through it bypass the + // tap, silently producing an empty capture. + const {stream, written} = collectingStream(); + const tee = new TeeSink(BufferedSink.overStream(stream), 2); + + await streamOfChunks([ + [1, 2], + [3, 4, 5], + ]).pipeTo(tee.toWritableStream()); + assert.deepEqual([...written()], [1, 2, 3, 4, 5]); + assert.deepEqual([...tee.snapshot()], [1, 2]); + }); + + it('forwards an abort to the primary while the tap survives to record what was attempted', async () => { + let abortReason = 'NOT-ABORTED'; + const destination = new WritableStream({ + write: chunk => void chunk, + abort: reason => void (abortReason = reason), + }); + const tee = new TeeSink(BufferedSink.overStream(destination), 4); + const writer = tee.toWritableStream().getWriter(); + await writer.write(Uint8Array.from([1, 2, 3])); + + const reason = new Error('deadline exceeded'); + await writer.abort(reason); + assert.equal(abortReason, reason); + assert.deepEqual([...tee.snapshot()], [1, 2, 3]); + }); +}); From 6c1343d1ccfb90797183182824707630bb935584 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:16:00 +0300 Subject: [PATCH 055/100] fix(core): reject an unpaired surrogate where it is supplied (HTTP-29, HTTP-31, SEAM-27) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` on a string with no UTF-8 form. That escaped the `DexpaceError` tree entirely, and it escaped from the wrong place: `QueryParams.encode()` and `QueryParams.equals()` document no throw at all, and `buildRequest`'s `@throws` list did not name it either. The `add()` that accepted the value never complained. `rfc3986.ts` now single-sources the rule as `hasLoneSurrogate()` (strict) and `toWellFormed()` (lenient), and each call site uses the one its own contract allows: - `QueryParamsBuilder.add` rejects a non-well-formed name or value with `UrlConstructionError` — the class D14 named, since the builder threw none of its own. - `substitutePathParams` rejects one with `OperationAssemblyError`, naming the parameter. - `QueryParams.parse` substitutes U+FFFD instead. HTTP-31 is MUST-level that parsing never throws, so the strict path cannot be the one `parse` uses on text it did not choose; this is the outbound/inbound split `Headers` already draws for HTTP-18 against HTTP-19. Replacement is what the platform's own serializer does — `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD`. A well-formed surrogate pair is ordinary text and stays accepted; the check is "no LONE surrogate", not "no astral character". Deviation from D14's letter: `String.prototype.isWellFormed()` is ES2024 and does not type-check on this repo's declared `lib` (`tsconfig.base.json` pins `ES2023`), though the `engines.node >= 20.3` runtime has it. `/\p{Surrogate}/u` is exact — in `u` mode a well-formed pair is one non-surrogate code point — and verified equivalent to `isWellFormed()`/`toWellFormed()` on the cases tested. Two properties pin the closure: no `URIError` escapes `encode()`, `equals()` or `buildRequest` for any surrogate-bearing input, and `parse()` never throws for any. Found by: audit #67 / #76. --- packages/core/src/http/query-params.test.ts | 93 ++++++++++++++++++++- packages/core/src/http/query-params.ts | 46 ++++++++-- packages/core/src/http/rfc3986.ts | 55 +++++++++++- packages/core/src/seams/operation.test.ts | 79 ++++++++++++++++- packages/core/src/seams/operation.ts | 20 ++++- 5 files changed, 279 insertions(+), 14 deletions(-) diff --git a/packages/core/src/http/query-params.test.ts b/packages/core/src/http/query-params.test.ts index 2865794..ed0021a 100644 --- a/packages/core/src/http/query-params.test.ts +++ b/packages/core/src/http/query-params.test.ts @@ -1,7 +1,8 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/query-params.test.ts // Exercises: HTTP-28 (case-sensitive, multi-value, value-less param), HTTP-29/32 (RFC 3986 encoding), -// HTTP-30 (order-sensitive equality, empty-list dropped), HTTP-31 (lenient parse) +// HTTP-30 (order-sensitive equality, empty-list dropped), HTTP-31 (lenient parse), +// HTTP-5 (getAll returns a frozen list on every path, present name or absent) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import { @@ -9,6 +10,22 @@ import { decodeQueryComponent, encodeQueryComponent, } from './query-params.js'; +import {UrlConstructionError} from './errors.js'; + +/** + * Strings that mix ordinary text with UNPAIRED surrogate code units. `fc.string()` alone never + * produces one — its default unit is printable ASCII — so the URIError path it is here to cover + * would go ungenerated. + */ +const surrogateBearingString = fc.string({ + unit: fc.oneof( + fc.constantFrom('a', 'b', ' ', '=', '&', '%', '+', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + maxLength: 8, +}); describe('case-sensitive names and multi-value (HTTP-28)', () => { test('page and Page are distinct names', () => { @@ -149,3 +166,77 @@ test('the component decoder treats a literal + as data, not a space (HTTP-29, PA test('the decoder falls back to raw text on malformed percent-encoding (HTTP-31)', () => { expect(decodeQueryComponent('a%zzb')).toBe('a%zzb'); }); + +describe('lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)', () => { + // `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + // on a string carrying an unpaired surrogate. Before audit #67 / #76 that escaped the + // `DexpaceError` tree entirely, and it escaped from `encode()` and `equals()` — accessors whose + // TSDoc documents no throw at all — rather than from the `add()` that accepted the value. + const LONE_HIGH = '\uD800'; + const LONE_LOW = '\uDFFF'; + + test.each([ + ['a lone high surrogate value', 'c', LONE_HIGH], + ['a lone low surrogate value', 'c', LONE_LOW], + ['a lone surrogate name', LONE_HIGH, 'v'], + ['a lone surrogate inside a longer value', 'c', `ok${LONE_HIGH}ok`], + ])('add() rejects %s with UrlConstructionError', (_label, name, value) => { + expect(() => QueryParams.newBuilder().add(name, value)).toThrow( + UrlConstructionError, + ); + }); + + test('a well-formed surrogate PAIR is accepted and encoded as UTF-8', () => { + // U+1F600, one code point spelled with two code units. Rejecting this too would make the check + // "no astral characters", which HTTP-29 does not say. + expect( + QueryParams.newBuilder().add('emoji', '\u{1F600}').build().encode(), + ).toBe('emoji=%F0%9F%98%80'); + }); + + test('parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing', () => { + // HTTP-31 is MUST-level about `parse` never throwing, so the strict `add()` path cannot be the + // one `parse` uses for a value it did not choose. Replacement matches what the platform's own + // query serializer does with the same input: `new URL('https://x/?a=\uD800').search` is + // `?a=%EF%BF%BD` (measured 2026-09-05). This is `Headers`' outbound/inbound split, applied to + // the query model. + const parsed = QueryParams.parse(`a=${LONE_HIGH}`); + expect(parsed.get('a')).toBe('�'); + expect(parsed.encode()).toBe('a=%EF%BF%BD'); + }); + + test('parse() sanitizes the NAME as well as the value', () => { + const parsed = QueryParams.parse(`${LONE_HIGH}=v`); + expect(parsed.has('�')).toBe(true); + expect(parsed.encode()).toBe('%EF%BF%BD=v'); + }); + + test('no URIError escapes encode() or equals(), whatever the builder admitted (property)', () => { + fc.assert( + fc.property(surrogateBearingString, surrogateBearingString, (n, v) => { + let params: QueryParams; + try { + params = QueryParams.newBuilder().add(n, v).build(); + } catch (e: unknown) { + expect(e).toBeInstanceOf(UrlConstructionError); + return; + } + expect(() => params.encode()).not.toThrow(); + expect(() => + params.equals(QueryParams.newBuilder().build()), + ).not.toThrow(); + }), + {numRuns: 500}, + ); + }); + + test('no anything escapes parse(), whatever it is handed (property, HTTP-31)', () => { + fc.assert( + fc.property(surrogateBearingString, raw => { + const parsed = QueryParams.parse(raw); + expect(() => parsed.encode()).not.toThrow(); + }), + {numRuns: 500}, + ); + }); +}); diff --git a/packages/core/src/http/query-params.ts b/packages/core/src/http/query-params.ts index a990680..e720b3c 100644 --- a/packages/core/src/http/query-params.ts +++ b/packages/core/src/http/query-params.ts @@ -1,7 +1,12 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/query-params.ts import type {Builder} from './builder.js'; -import {encodeRfc3986Component} from './rfc3986.js'; +import {UrlConstructionError} from './errors.js'; +import { + encodeRfc3986Component, + hasLoneSurrogate, + toWellFormed, +} from './rfc3986.js'; /** * @internal @@ -28,6 +33,22 @@ export function decodeQueryComponent(value: string): string { } } +/** + * The strict half of the surrogate rule, applied by `add()`. + * + * `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + * on an unpaired surrogate — outside the `DexpaceError` tree, and out of `encode()` or `equals()` + * rather than out of the call that supplied the value. Rejecting here puts the failure at the call + * site, which is the same place HTTP-35 and HTTP-17/18 put theirs (audit #67 / #76). + */ +function requireWellFormed(kind: 'name' | 'value', text: string): void { + if (hasLoneSurrogate(text)) { + throw new UrlConstructionError( + `query parameter ${kind} contains an unpaired surrogate and cannot be percent-encoded`, + ); + } +} + let createQueryParams: ( valuesByName: ReadonlyMap, insertionOrder: readonly string[], @@ -43,7 +64,9 @@ let createQueryParams: ( * Encoding and parsing are deliberately asymmetric and kept as separate operations. * {@link QueryParams.encode} is strict RFC 3986 percent-encoding — not * `application/x-www-form-urlencoded`, so a space is `%20` and never `+` (HTTP-29/32). - * {@link QueryParams.parse} is lenient and never throws (HTTP-31). + * {@link QueryParams.parse} is lenient and never throws (HTTP-31). The asymmetry extends to + * unpaired surrogates: {@link QueryParamsBuilder.add} rejects one, while + * {@link QueryParams.parse} substitutes U+FFFD for it. * * @example * ```ts @@ -99,8 +122,10 @@ export class QueryParams { * (HTTP-31). * * A `null`, `undefined`, or blank input yields empty parameters; a leading `?` is tolerated; a - * segment with no `=` or a trailing `=` yields an empty-string value; a stray `&` is skipped; and - * malformed percent-encoding falls back to the raw text rather than failing. + * segment with no `=` or a trailing `=` yields an empty-string value; a stray `&` is skipped; + * malformed percent-encoding falls back to the raw text rather than failing; and an unpaired + * surrogate is replaced with U+FFFD rather than rejected the way + * {@link QueryParamsBuilder.add} rejects one, so the result is always encodable. * * @param raw - the query string, with or without its leading `?`. * @returns the parsed, frozen parameters. @@ -116,9 +141,12 @@ export class QueryParams { const eqIndex = segment.indexOf('='); const rawName = eqIndex === -1 ? segment : segment.slice(0, eqIndex); const rawValue = eqIndex === -1 ? '' : segment.slice(eqIndex + 1); + // HTTP-31 is MUST-level that parsing never throws, so the strict `add()` path above cannot be + // the one `parse` uses on text it did not choose — exactly the split `Headers` draws between + // its outbound (`add`) and inbound (`addInbound`) methods for HTTP-18 against HTTP-19. builder.add( - decodeQueryComponent(rawName), - decodeQueryComponent(rawValue), + toWellFormed(decodeQueryComponent(rawName)), + toWellFormed(decodeQueryComponent(rawValue)), ); } return builder.build(); @@ -201,9 +229,15 @@ export class QueryParamsBuilder implements Builder { * @param value - the value; `null` records a value-less parameter as a single empty string * (HTTP-28). * @returns this builder, for chaining. + * @throws {@link UrlConstructionError} when the name or the value carries an unpaired surrogate. + * Such a string has no UTF-8 form, so RFC 3986 percent-encoding is undefined for it and + * {@link QueryParams.encode} could only fail — it is rejected here, at the call that supplied it. + * A well-formed surrogate pair is ordinary text and is accepted (HTTP-29). */ add(name: string, value: string | null): this { const actualValue = value ?? ''; + requireWellFormed('name', name); + requireWellFormed('value', actualValue); if (!this.#valuesByName.has(name)) { this.#insertionOrder.push(name); this.#valuesByName.set(name, []); diff --git a/packages/core/src/http/rfc3986.ts b/packages/core/src/http/rfc3986.ts index a2d6db5..c96955a 100644 --- a/packages/core/src/http/rfc3986.ts +++ b/packages/core/src/http/rfc3986.ts @@ -1,13 +1,66 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/rfc3986.ts +/** + * Matches an UNPAIRED surrogate code unit, and only an unpaired one. + * + * In a `u`-mode pattern the engine works in code points, so a well-formed surrogate pair is one + * non-surrogate code point and does not match, while a lone high or low unit stays a surrogate code + * point and does. Equivalent to `!String.prototype.isWellFormed()`, which is ES2024 and so outside + * this repo's declared `lib` (`tsconfig.base.json` pins `ES2023`) even though the + * `engines.node >= 20.3` runtime has it — raising `lib` for one predicate is a wider change than + * the predicate is worth. + * + * Two patterns, one rule: `.test()` must not carry `lastIndex` between calls, and `.replace()` must + * be global. Neither is exported; the two functions below are, so no caller can pick the wrong one. + */ +const LONE_SURROGATE = /\p{Surrogate}/u; +const LONE_SURROGATE_GLOBAL = /\p{Surrogate}/gu; + +/** Unicode's replacement character, what a lenient repair substitutes for an unpaired surrogate. */ +const REPLACEMENT_CHARACTER = '\uFFFD'; + +/** + * Whether `value` carries an unpaired surrogate, and so has no UTF-8 form and cannot be + * percent-encoded. The strict half of the rule: a call site that was HANDED such a string rejects + * it (audit #67 / #76). + * + * @param value - the string to inspect. + * @returns `true` when at least one surrogate code unit is unpaired. + */ +export function hasLoneSurrogate(value: string): boolean { + return LONE_SURROGATE.test(value); +} + +/** + * `value` with every unpaired surrogate replaced by U+FFFD. The lenient half, for a call site that + * MUST NOT throw — `QueryParams.parse` under HTTP-31. Matches what the platform's own query + * serializer does with the same input: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` + * (measured 2026-09-05). + * + * @param value - the string to repair. + * @returns `value` with unpaired surrogates replaced; the same string when there are none. + */ +export function toWellFormed(value: string): string { + return value.replace(LONE_SURROGATE_GLOBAL, REPLACEMENT_CHARACTER); +} + /** * Percent-encodes a single URL component per RFC 3986, patching `encodeURIComponent`'s divergence: * `encodeURIComponent` leaves `! * ' ( )` unescaped, but none of them are in RFC 3986's unreserved * set (HTTP-29). * - * @param value - the raw component value. + * Deliberately NOT total, and deliberately not guarded here. `encodeURIComponent` throws + * `URIError: URI malformed` on a string carrying an unpaired surrogate, because such a string has + * no UTF-8 form. Every caller in this package rejects or repairs that input BEFORE reaching here — + * `QueryParamsBuilder.add`, `QueryParams.parse` and `substitutePathParams` each do, and each throws + * the error class its own call site already throws — so a second, silent guard inside the encoder + * would only move the failure back off the call site (audit #67 / #76). + * + * @param value - the raw component value; must not carry an unpaired surrogate. * @returns the percent-encoded component. + * @throws A platform `URIError` when `value` carries an unpaired surrogate. Callers validate first; + * see the note above. */ export function encodeRfc3986Component(value: string): string { return encodeURIComponent(value).replace( diff --git a/packages/core/src/seams/operation.test.ts b/packages/core/src/seams/operation.test.ts index 65342ee..5e588a9 100644 --- a/packages/core/src/seams/operation.test.ts +++ b/packages/core/src/seams/operation.test.ts @@ -1,8 +1,10 @@ // SPDX-License-Identifier: MIT // packages/core/src/seams/operation.test.ts // Exercises: SEAM-26 (the four projections default to empty), SEAM-27 (buildRequest's encoding and base-URL -// composition rules), HTTP-7 (a body projected onto a body-forbidding method fails assembly), reusing -// HTTP-29's encodeRfc3986Component for path-segment encoding. +// composition rules, and that a placeholder is satisfied only by an OWN property of pathParams), +// HTTP-7 (a body projected onto a body-forbidding method fails assembly), reusing +// HTTP-29's encodeRfc3986Component for path-segment encoding — including the unpaired-surrogate input it +// cannot encode, which is rejected here rather than allowed to escape as a bare URIError. import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import { @@ -215,6 +217,79 @@ describe('SEAM-27: a placeholder is satisfied only by an OWN property of pathPar }); }); +describe('SEAM-27: an unpaired surrogate in a path-param value is rejected', () => { + // `encodeRfc3986Component` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` + // on a string with no UTF-8 form. Before audit #67 / #76 that escaped `buildRequest` outside the + // `DexpaceError` tree and outside its `@throws` list. It is now `OperationAssemblyError`, the + // class this call site already throws for a path-param value it cannot use. + test.each([ + ['a lone high surrogate', '\uD800'], + ['a lone low surrogate', '\uDFFF'], + ['a lone surrogate inside a longer value', 'ok\uD800ok'], + ])('%s throws OperationAssemblyError', (_label, value) => { + expect(() => + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: value}, + }), + ).toThrow(OperationAssemblyError); + }); + + test('a well-formed surrogate pair is ordinary text and is encoded', () => { + const request = buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: '\u{1F600}'}, + }); + expect(request.url.pathname).toBe('/things/%F0%9F%98%80'); + }); + + test('the query projection cannot carry one either, from a builder or from parse()', () => { + // `composeQuery` calls `operationQuery.encode()`, the second `encodeRfc3986Component` path into + // `buildRequest`. Both ways of obtaining a `QueryParams` are now closed: the builder rejects an + // unpaired surrogate, `parse` replaces it. + expect(() => QueryParams.newBuilder().add('c', '\uD800')).toThrow( + /unpaired surrogate/, + ); + const request = buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things', + query: QueryParams.parse('c=\uD800'), + }); + expect(request.url.search).toBe('?c=%EF%BF%BD'); + }); + + test('no URIError escapes buildRequest, whatever the descriptor carries (property)', () => { + fc.assert( + fc.property( + fc.string({ + unit: fc.oneof( + fc.constantFrom('a', '/', '.', '%', ' ', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + minLength: 1, + maxLength: 6, + }), + value => { + try { + buildRequest('https://host', { + method: 'GET', + pathTemplate: '/things/{id}', + pathParams: {id: value}, + }); + } catch (e: unknown) { + expect(e).toBeInstanceOf(OperationAssemblyError); + } + }, + ), + {numRuns: 500}, + ); + }); +}); + describe('a path-param value containing / is encoded, not split (property)', () => { test('holds for arbitrary generated path-param values', () => { fc.assert( diff --git a/packages/core/src/seams/operation.ts b/packages/core/src/seams/operation.ts index 5974a45..0683691 100644 --- a/packages/core/src/seams/operation.ts +++ b/packages/core/src/seams/operation.ts @@ -6,14 +6,14 @@ import type {Headers} from '../http/headers.js'; import type {QueryParams} from '../http/query-params.js'; import type {Method} from '../http/method.js'; import {UrlConstructionError, DexpaceError} from '../http/errors.js'; -import {encodeRfc3986Component} from '../http/rfc3986.js'; +import {encodeRfc3986Component, hasLoneSurrogate} from '../http/rfc3986.js'; /** * Thrown when `buildRequest()` cannot assemble a request from its descriptor: a `{name}` * placeholder in `pathTemplate` has no OWN value in `pathParams` — an inherited member such as * `constructor` does not satisfy one — or a supplied value is a dot segment (`.`/`..`) that the - * WHATWG URL parser would normalize into a path rewrite instead of keeping as one literal - * segment. + * WHATWG URL parser would normalize into a path rewrite instead of keeping as one literal segment, + * or a supplied value carries an unpaired surrogate and so has no percent-encoded form. * * @public */ @@ -131,6 +131,17 @@ function substitutePathParams( name, ); } + // A string carrying an unpaired surrogate has no UTF-8 form, so `encodeRfc3986Component` — i.e. + // `encodeURIComponent` — throws a bare `URIError: URI malformed`. That escaped `buildRequest` + // outside the `DexpaceError` tree and outside its documented `@throws`. Rejected here with the + // class this call site already throws, rather than guarded inside the encoder, so the failure + // names the parameter (audit #67 / #76). + if (hasLoneSurrogate(value)) { + throw new OperationAssemblyError( + `path parameter "${name}" contains an unpaired surrogate and cannot be percent-encoded`, + name, + ); + } return encodeRfc3986Component(value); }); } @@ -171,7 +182,8 @@ function composeQuery( * @returns the assembled request. * @throws {@link OperationAssemblyError} when a `{name}` placeholder has no own value in * `pathParams` (a name inherited from `Object.prototype` does not count as supplied), or a supplied - * value is a dot segment (`.`/`..`) — fix the descriptor; no request was assembled. + * value is a dot segment (`.`/`..`), or a supplied value carries an unpaired surrogate — fix the + * descriptor; no request was assembled. * @throws {@link UrlConstructionError} when `baseUrl` is malformed, non-absolute, or carries a * fragment — supply a clean absolute base URL. * @throws {@link RequestBodyNotAllowedError} when the descriptor pairs a body with GET, HEAD, TRACE From 2d3785c87864fd679a11603e92d45ef5c56845f3 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:19:48 +0300 Subject: [PATCH 056/100] fix(test): type the multipart drain helper as a view over a plain ArrayBuffer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `BodyInit` excludes a view over a `SharedArrayBuffer`, so `Promise` — whose default parameter is `ArrayBufferLike` — is not assignable to the platform `Response` constructor the boundary round-trip feeds. Bun's runner does not typecheck, so this only surfaced under `bun run typecheck`. Refs #77, #67. --- packages/core/src/body/multipart-body.test.ts | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/packages/core/src/body/multipart-body.test.ts b/packages/core/src/body/multipart-body.test.ts index b767fc4..d616008 100644 --- a/packages/core/src/body/multipart-body.test.ts +++ b/packages/core/src/body/multipart-body.test.ts @@ -67,9 +67,11 @@ function collectingSink(): { }; } +// `Uint8Array`, not the bare alias: `BodyInit` excludes a view over a `SharedArrayBuffer`, +// so the default `ArrayBufferLike` parameter is not assignable to the platform `Response` below. async function drainBytes(body: { writeTo: (sink: WritableStream) => Promise; -}): Promise { +}): Promise> { const chunks: Uint8Array[] = []; await body.writeTo(new WritableStream({write: c => void chunks.push(c)})); const total = chunks.reduce((s, c) => s + c.length, 0); From e414e266940a485b8b1814811492552e6753a73a Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:18:29 +0300 Subject: [PATCH 057/100] fix(core): getAll returns a frozen list on the absent-name path too (HTTP-5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Headers.getAll` and `QueryParams.getAll` both promise "a read-only, frozen list of values — empty when the name is absent" and both returned a fresh, unfrozen `[]` on a miss. A caller who pushed into it saw the push succeed, which is the mutable-collection surface HTTP-5 exists to close, and it was a fresh allocation on every miss besides. One frozen `EMPTY_VALUE_LIST` per model — sharing an instance is safe precisely because it is frozen. Declared in each of the two files rather than once in `http/builder.ts`: that would be the single home, but it is outside #76's file partition, and a frozen empty array cannot drift. Consolidating it is a one-line follow-up for whoever owns that file next. Tests assert the frozen-ness of both the absent-name list and, for the first time, the present-name list: `build()` freezes each value list and `getAll` hands back that same reference, so nothing else in the suite would notice if that freeze were dropped. Also closes the other `headers.ts` gap the same audit found, since it is the same file and the same kind of missing assertion: `Headers.equals` had no direct test and was exercised only through `Request.equals`. Nine cases now cover name folding, value case-sensitivity, distinct-name order (irrelevant), value order under one name (relevant), strict subsets in both directions, disjoint names at equal name count — the case that proves the per-name lookup runs rather than the length pre-check — differing value counts, and two empty instances. No behaviour change from these: the implementation was already correct on all nine. Found by: audit #67 / #76. --- packages/core/src/http/headers.test.ts | 120 +++++++++++++++++++- packages/core/src/http/headers.ts | 22 +++- packages/core/src/http/query-params.test.ts | 19 ++++ packages/core/src/http/query-params.ts | 19 +++- 4 files changed, 175 insertions(+), 5 deletions(-) diff --git a/packages/core/src/http/headers.test.ts b/packages/core/src/http/headers.test.ts index 762158a..618e6bb 100644 --- a/packages/core/src/http/headers.test.ts +++ b/packages/core/src/http/headers.test.ts @@ -9,7 +9,9 @@ // XCUT-15's ingested-collection clause (a builder defensively copies what it is handed, so mutating that // collection after build() cannot alter the built model, and a derived builder never aliases its source), // HTTP-17 (outbound name validation + trim), HTTP-18 (outbound value validation), HTTP-19 (inbound leniency), -// HTTP-20 (no value echo, escaped name), HTTP-21 (typed HeaderName interop) +// HTTP-20 (no value echo, escaped name), HTTP-21 (typed HeaderName interop), +// HTTP-5 again (getAll returns a FROZEN list on both the present-name and the absent-name path), +// HTTP-13 once more (Headers.equals asserted directly, not only through Request.equals) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {Headers, HeaderName} from './headers.js'; @@ -251,3 +253,119 @@ describe('HeaderName.lowerCased (HTTP-21)', () => { expect(name.raw).toBe('Content-Type'); }); }); + +describe('getAll returns a frozen list on every path (HTTP-5)', () => { + const headers = Headers.newBuilder() + .add('X-Tag', 'a') + .add('X-Tag', 'b') + .build(); + + test('the present-name list is frozen', () => { + // Asserted directly for the first time by audit #67 / #76. `build()` freezes each value list, + // and `getAll` returns that same reference rather than a copy, so the freeze is the whole of + // HTTP-5's "cannot mutate the model through the returned value" on this accessor — if the + // freeze were ever dropped, nothing else would notice. + const values = headers.getAll('X-Tag'); + expect(Object.isFrozen(values)).toBe(true); + expect(() => (values as string[]).push('c')).toThrow(TypeError); + expect(headers.getAll('X-Tag')).toEqual(['a', 'b']); + }); + + test('the absent-name list is frozen too, and is the same shared instance', () => { + // It was a fresh `[]` — unfrozen, against the TSDoc's promise of a frozen list, and a fresh + // allocation on every miss. + const first = headers.getAll('nope'); + const second = headers.getAll('also-nope'); + expect(Object.isFrozen(first)).toBe(true); + expect(first).toBe(second); + expect(() => (first as string[]).push('x')).toThrow(TypeError); + }); +}); + +// Reached only through `Request.equals` until audit #67 / #76. Each case below is a way the +// comparison could be wrong that a Request-level test would not isolate. +function buildHeaders(pairs: readonly (readonly [string, string])[]): Headers { + const builder = Headers.newBuilder(); + for (const [name, value] of pairs) builder.add(name, value); + return builder.build(); +} + +describe('Headers.equals directly (HTTP-13): names and casing', () => { + test('is reflexive and true for an identical construction', () => { + const a = buildHeaders([['X-A', '1']]); + expect(a.equals(a)).toBe(true); + expect(a.equals(buildHeaders([['X-A', '1']]))).toBe(true); + }); + + test('name casing does not participate — HTTP-13 folds names', () => { + expect( + buildHeaders([['X-A', '1']]).equals(buildHeaders([['x-a', '1']])), + ).toBe(true); + }); + + test('value casing DOES participate — only names are folded', () => { + expect( + buildHeaders([['X-A', 'v']]).equals(buildHeaders([['X-A', 'V']])), + ).toBe(false); + }); + + test('the order of distinct NAMES does not matter', () => { + const ab = buildHeaders([ + ['X-A', '1'], + ['X-B', '2'], + ]); + const ba = buildHeaders([ + ['X-B', '2'], + ['X-A', '1'], + ]); + expect(ab.equals(ba)).toBe(true); + expect(ba.equals(ab)).toBe(true); + }); +}); + +describe('Headers.equals directly (HTTP-13): values, order and subsets', () => { + test('the order of VALUES under one name does matter (HTTP-14)', () => { + const ab = buildHeaders([ + ['X-T', 'a'], + ['X-T', 'b'], + ]); + const ba = buildHeaders([ + ['X-T', 'b'], + ['X-T', 'a'], + ]); + expect(ab.equals(ba)).toBe(false); + }); + + test('a strict subset is not equal, in either direction', () => { + const one = buildHeaders([['X-A', '1']]); + const two = buildHeaders([ + ['X-A', '1'], + ['X-B', '2'], + ]); + expect(one.equals(two)).toBe(false); + expect(two.equals(one)).toBe(false); + }); + + test('same name count, disjoint names, is not equal', () => { + // The length pre-check passes here, so this is the case that proves the per-name lookup runs. + expect( + buildHeaders([['X-A', '1']]).equals(buildHeaders([['X-B', '1']])), + ).toBe(false); + }); + + test('same names with different value COUNTS is not equal', () => { + const one = buildHeaders([['X-T', 'a']]); + const two = buildHeaders([ + ['X-T', 'a'], + ['X-T', 'a'], + ]); + expect(one.equals(two)).toBe(false); + expect(two.equals(one)).toBe(false); + }); + + test('two empty instances are equal', () => { + expect( + Headers.newBuilder().build().equals(Headers.newBuilder().build()), + ).toBe(true); + }); +}); diff --git a/packages/core/src/http/headers.ts b/packages/core/src/http/headers.ts index 7cf0350..8d91fac 100644 --- a/packages/core/src/http/headers.ts +++ b/packages/core/src/http/headers.ts @@ -8,6 +8,20 @@ import { hasForbiddenInboundValueByte, } from './ascii-validation.js'; +/** + * The one frozen empty list this model's multi-value accessor returns for an absent name. + * + * Shared, not allocated per miss, and frozen for the same reason the present-name lists are: + * HTTP-5's accessors "MUST NOT let a caller mutate the model through the returned value", and + * `getAll`'s TSDoc promises a frozen list on every path. It returned a fresh `[]` on a miss, which + * was neither (audit #67 / #76). Sharing is safe precisely because it is frozen — there is no + * state in it to alias. + * + * Declared per model rather than shared with {@link QueryParams} through `builder.ts`: the two + * models deliberately import nothing from each other, and a frozen empty array cannot drift. + */ +const EMPTY_VALUE_LIST: readonly string[] = Object.freeze([]); + function validateName(name: string): string { const trimmed = name.trim(); if (trimmed === '' || hasForbiddenNameByte(trimmed)) { @@ -130,10 +144,14 @@ export class Headers { * Returns every value stored under `name`, in insertion order. * * @param name - the header name, as a string or a {@link HeaderName}. - * @returns a read-only, frozen list of values — empty when the name is absent. + * @returns a read-only, frozen list of values — the shared frozen empty list when the name is + * absent. Frozen on both paths, so mutating it cannot reach the model (HTTP-5). */ getAll(name: string | HeaderName): readonly string[] { - return this.#valuesByLowerName.get(toRawName(name).toLowerCase()) ?? []; + return ( + this.#valuesByLowerName.get(toRawName(name).toLowerCase()) ?? + EMPTY_VALUE_LIST + ); } /** diff --git a/packages/core/src/http/query-params.test.ts b/packages/core/src/http/query-params.test.ts index ed0021a..fb51ebf 100644 --- a/packages/core/src/http/query-params.test.ts +++ b/packages/core/src/http/query-params.test.ts @@ -240,3 +240,22 @@ describe('lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31 ); }); }); + +describe('getAll returns a frozen list on every path (HTTP-5)', () => { + const params = QueryParams.newBuilder().add('x', '1').add('x', '2').build(); + + test('the present-name list is frozen', () => { + const values = params.getAll('x'); + expect(Object.isFrozen(values)).toBe(true); + expect(() => (values as string[]).push('3')).toThrow(TypeError); + expect(params.getAll('x')).toEqual(['1', '2']); + }); + + test('the absent-name list is frozen too, and is the same shared instance', () => { + const first = params.getAll('nope'); + const second = params.getAll('also-nope'); + expect(Object.isFrozen(first)).toBe(true); + expect(first).toBe(second); + expect(() => (first as string[]).push('x')).toThrow(TypeError); + }); +}); diff --git a/packages/core/src/http/query-params.ts b/packages/core/src/http/query-params.ts index e720b3c..88d6907 100644 --- a/packages/core/src/http/query-params.ts +++ b/packages/core/src/http/query-params.ts @@ -8,6 +8,20 @@ import { toWellFormed, } from './rfc3986.js'; +/** + * The one frozen empty list this model's multi-value accessor returns for an absent name. + * + * Shared, not allocated per miss, and frozen for the same reason the present-name lists are: + * HTTP-5's accessors "MUST NOT let a caller mutate the model through the returned value", and + * `getAll`'s TSDoc promises a frozen list on every path. It returned a fresh `[]` on a miss, which + * was neither (audit #67 / #76). Sharing is safe precisely because it is frozen — there is no + * state in it to alias. + * + * Declared per model rather than shared with {@link Headers} through `builder.ts`: the two + * models deliberately import nothing from each other, and a frozen empty array cannot drift. + */ +const EMPTY_VALUE_LIST: readonly string[] = Object.freeze([]); + /** * @internal * RFC 3986 component encoding (HTTP-29): space → `%20` (never `+`), literal `+` → `%2B`, everything outside @@ -166,10 +180,11 @@ export class QueryParams { * Returns every value stored under `name`, in insertion order. * * @param name - the parameter name. - * @returns a read-only, frozen list of values — empty when the name is absent. + * @returns a read-only, frozen list of values — the shared frozen empty list when the name is + * absent. Frozen on both paths, so mutating it cannot reach the model (HTTP-5). */ getAll(name: string): readonly string[] { - return this.#valuesByName.get(name) ?? []; + return this.#valuesByName.get(name) ?? EMPTY_VALUE_LIST; } /** From 54b5b9057a0c8799945b694f54b1109459274d39 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:21:47 +0300 Subject: [PATCH 058/100] fix(core): TeeSink rejects a non-integral tap limit at the constructor (IO-3, IO-26) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The constructor checked `tapLimit >= 0` only, so `new TeeSink(primary, 2.5)` was accepted and the fault was deferred to the first write: `#mirror` computes `room = tapLimit - tap.size`, hands it to `ByteQueue.copyTo`, and `assertCount` rejects it as `count must be a non-negative integer, got 2.5`. That names the wrong parameter, fires at the wrong call, and does so on a path that has already taken bytes from the caller. A tap cap is a byte count, so IO-3's integrality rule for `count` is the same rule. `Infinity` is admitted explicitly — it is not an integer and it is the documented unbounded default (IO-26). The error stays `InvariantViolation`, the class this constructor already threw for a negative limit; only its message changed, to name `tapLimit`. Found by: audit #67 / #76. --- packages/core/src/io/tee-sink.test.ts | 45 ++++++++++++++++++++++++++- packages/core/src/io/tee-sink.ts | 13 ++++++-- 2 files changed, 55 insertions(+), 3 deletions(-) diff --git a/packages/core/src/io/tee-sink.test.ts b/packages/core/src/io/tee-sink.test.ts index 32560d8..0ac8c76 100644 --- a/packages/core/src/io/tee-sink.test.ts +++ b/packages/core/src/io/tee-sink.test.ts @@ -7,7 +7,8 @@ // IO-42 (write after close rejects with the source intact), // IO-13 (the tap mirrors the primary's exact encoded bytes, and refuses a label identically), // IO-16 (the tee's own writable bridge still feeds the tap), -// IO-3 (a negative count is an argument error, rejected before any transfer) +// IO-3 (a negative count is an argument error, rejected before any transfer; and a non-integral +// tapLimit is one too, rejected at the constructor rather than at the first write) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {BufferedSink} from './buffered-sink.js'; @@ -15,6 +16,7 @@ import {BufferedSource} from './buffered-source.js'; import {ByteQueue} from './byte-queue.js'; import {ClosedResourceError} from './errors.js'; import {TeeSink} from './tee-sink.js'; +import {InvariantViolation} from '../invariant.js'; import {writeAll} from './pump.js'; import { collectingWritableStream, @@ -300,4 +302,45 @@ describe('argument validation (IO-3)', () => { 'InvariantViolation', ); }); + + // The constructor checked `tapLimit >= 0` only, so a fractional cap was accepted and the fault + // surfaced at the FIRST write instead: `#mirror` computes `room = tapLimit - tap.size`, hands it + // to `ByteQueue.copyTo`, and `assertCount` rejects it as `count must be a non-negative integer, + // got 2.5` — a message about the wrong parameter, at the wrong call, on a path where the primary + // write has not run yet. A byte count is integral for the same reason `count` is (IO-3, IO-26). + // Measured on the pre-fix tree, audit #67 / #76. + test.each([2.5, 0.5, -0.5, Number.NaN, Number.NEGATIVE_INFINITY])( + 'a tapLimit of %p is rejected at the constructor', + tapLimit => { + const {stream} = collectingWritableStream(); + expect( + () => new TeeSink(BufferedSink.overStream(stream), tapLimit), + ).toThrow(InvariantViolation); + }, + ); + + test('the message names tapLimit, not count', () => { + const {stream} = collectingWritableStream(); + expect(() => new TeeSink(BufferedSink.overStream(stream), 2.5)).toThrow( + /tapLimit/, + ); + }); + + test.each([0, 1, 4096, Number.POSITIVE_INFINITY])( + 'a tapLimit of %p is still accepted', + tapLimit => { + const {stream} = collectingWritableStream(); + expect( + () => new TeeSink(BufferedSink.overStream(stream), tapLimit), + ).not.toThrow(); + }, + ); + + test('the unbounded default stays Infinity, which is not an integer', () => { + // `Number.isInteger(Infinity)` is false, so the check has to admit it explicitly — it is the + // documented default and the cap `#mirror` reads as "no cap" (IO-26). + const {stream} = collectingWritableStream(); + const tee = new TeeSink(BufferedSink.overStream(stream)); + expect(tee.snapshot().length).toBe(0); + }); }); diff --git a/packages/core/src/io/tee-sink.ts b/packages/core/src/io/tee-sink.ts index dc6e560..83b1d15 100644 --- a/packages/core/src/io/tee-sink.ts +++ b/packages/core/src/io/tee-sink.ts @@ -31,9 +31,18 @@ export class TeeSink implements Sink { readonly #tapLimit: number; constructor(primary: Sink, tapLimit: number = Number.POSITIVE_INFINITY) { + // Integrality, not merely `>= 0`. A tap cap is a byte count, so IO-3's rule for `count` applies + // to it — and a fractional cap was not merely odd, it was deferred: `#mirror` computes + // `room = tapLimit - tap.size` and hands it to `ByteQueue.copyTo`, where `assertCount` rejects + // it as "count must be a non-negative integer, got 2.5". That names the wrong parameter, fires + // at the first write rather than at the construction that supplied it, and does so before the + // primary write on a path that has already taken bytes from the caller. `Infinity` is admitted + // explicitly: it is not an integer, and it is the documented unbounded default (IO-26). + // Audit #67 / #76. invariant( - tapLimit >= 0, - `tapLimit must be non-negative, got ${String(tapLimit)}`, + (Number.isInteger(tapLimit) && tapLimit >= 0) || + tapLimit === Number.POSITIVE_INFINITY, + `tapLimit must be a non-negative integer or Infinity, got ${String(tapLimit)}`, ); this.#primary = primary; this.#tapLimit = tapLimit; From d357f61fb1c57c05615f3dd02bdfa3e4496ae855 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:27:56 +0300 Subject: [PATCH 059/100] refactor(core): consolidate EMPTY_VALUE_LIST in http/builder.ts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 2 of #76 widens the partition to cover this file. The frozen empty list was declared twice — once in `headers.ts`, once in `query-params.ts` — because `builder.ts` was outside the original partition. It is one constant now, in the module both models already import their shared construction helpers from. No behaviour change: `verify:import-cycles` stays clean, and the same `getAll` tests cover it. Found by: audit #67 / #76. --- packages/core/src/http/builder.ts | 16 ++++++++++++++++ packages/core/src/http/headers.ts | 16 +--------------- packages/core/src/http/query-params.ts | 16 +--------------- 3 files changed, 18 insertions(+), 30 deletions(-) diff --git a/packages/core/src/http/builder.ts b/packages/core/src/http/builder.ts index 5411659..c4bc1e4 100644 --- a/packages/core/src/http/builder.ts +++ b/packages/core/src/http/builder.ts @@ -20,6 +20,22 @@ export interface Builder { build(): T; } +/** + * The one frozen empty list every multi-value accessor returns for an absent name. + * + * Shared, not allocated per miss, and frozen for the same reason the present-name lists are: + * HTTP-5's accessors "MUST NOT let a caller mutate the model through the returned value", and the + * TSDoc on `Headers.getAll` and `QueryParams.getAll` promises a frozen list on every path. Both + * returned a fresh `[]` on a miss, which was neither (audit #67 / #76). Sharing one instance is + * safe precisely because it is frozen — there is no state in it to alias, and no caller can add + * any. + * + * Lives here rather than in either model because both need it and the two models deliberately + * import nothing from each other; this module is already the shared construction helper they both + * import from. + */ +export const EMPTY_VALUE_LIST: readonly string[] = Object.freeze([]); + /** * Returns `value` when present, throwing a field-named error when it is `null` or `undefined`. * diff --git a/packages/core/src/http/headers.ts b/packages/core/src/http/headers.ts index 8d91fac..f452d7e 100644 --- a/packages/core/src/http/headers.ts +++ b/packages/core/src/http/headers.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/headers.ts -import type {Builder} from './builder.js'; +import {EMPTY_VALUE_LIST, type Builder} from './builder.js'; import {HeaderValidationError} from './errors.js'; import { hasForbiddenNameByte, @@ -8,20 +8,6 @@ import { hasForbiddenInboundValueByte, } from './ascii-validation.js'; -/** - * The one frozen empty list this model's multi-value accessor returns for an absent name. - * - * Shared, not allocated per miss, and frozen for the same reason the present-name lists are: - * HTTP-5's accessors "MUST NOT let a caller mutate the model through the returned value", and - * `getAll`'s TSDoc promises a frozen list on every path. It returned a fresh `[]` on a miss, which - * was neither (audit #67 / #76). Sharing is safe precisely because it is frozen — there is no - * state in it to alias. - * - * Declared per model rather than shared with {@link QueryParams} through `builder.ts`: the two - * models deliberately import nothing from each other, and a frozen empty array cannot drift. - */ -const EMPTY_VALUE_LIST: readonly string[] = Object.freeze([]); - function validateName(name: string): string { const trimmed = name.trim(); if (trimmed === '' || hasForbiddenNameByte(trimmed)) { diff --git a/packages/core/src/http/query-params.ts b/packages/core/src/http/query-params.ts index 88d6907..0166b9b 100644 --- a/packages/core/src/http/query-params.ts +++ b/packages/core/src/http/query-params.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/http/query-params.ts -import type {Builder} from './builder.js'; +import {EMPTY_VALUE_LIST, type Builder} from './builder.js'; import {UrlConstructionError} from './errors.js'; import { encodeRfc3986Component, @@ -8,20 +8,6 @@ import { toWellFormed, } from './rfc3986.js'; -/** - * The one frozen empty list this model's multi-value accessor returns for an absent name. - * - * Shared, not allocated per miss, and frozen for the same reason the present-name lists are: - * HTTP-5's accessors "MUST NOT let a caller mutate the model through the returned value", and - * `getAll`'s TSDoc promises a frozen list on every path. It returned a fresh `[]` on a miss, which - * was neither (audit #67 / #76). Sharing is safe precisely because it is frozen — there is no - * state in it to alias. - * - * Declared per model rather than shared with {@link Headers} through `builder.ts`: the two - * models deliberately import nothing from each other, and a frozen empty array cannot drift. - */ -const EMPTY_VALUE_LIST: readonly string[] = Object.freeze([]); - /** * @internal * RFC 3986 component encoding (HTTP-29): space → `%20` (never `+`), literal `+` → `%2B`, everything outside From 63e2aec85e68c13a1bdec39ece79f3f52af7a72c Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:29:16 +0300 Subject: [PATCH 060/100] docs(core): errors.ts TSDoc matches what the two classes now cover MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both blocks shipped in the `.d.ts` and both were false, one of them before this run started. `RequestOptionsValidationError` read "a non-null timeout that is zero or negative, or a negative max-retries" — HTTP-35's own wording, and already incomplete: `Infinity`, `NaN` and a fractional `maxRetries` were rejected too, and #76 added a non-integer and an out-of-ceiling timeout. It now states both full ranges and why they are the full ranges. `UrlConstructionError` read "when a request URL is malformed or not absolute" and named one of the three throw sites. It is also raised for a malformed, non-absolute or fragment-bearing base URL in `buildRequest`, and — per D14, which chose reuse over a new class — for a query-parameter name or value carrying an unpaired surrogate. That third case is the one worth documenting: it sets no `cause` (nothing was caught), and `QueryParams.parse` deliberately does not raise it. Backticked prose throughout; no `{@link}` into an inherited member. `bun run api` clean over all nine reports; `core.api.md` unchanged, since api-extractor records signatures rather than prose. Found by: audit #67 / #76. --- packages/core/src/http/errors.ts | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/packages/core/src/http/errors.ts b/packages/core/src/http/errors.ts index 9a3d055..66c0f83 100644 --- a/packages/core/src/http/errors.ts +++ b/packages/core/src/http/errors.ts @@ -127,16 +127,39 @@ export class MediaTypeParseError extends DexpaceError {} export class ProtocolParseError extends DexpaceError {} /** - * Thrown when a request URL is malformed or not absolute; the message carries the offending input - * and the underlying parse failure is chained as `cause` (HTTP-47). + * Thrown when a URL cannot be constructed from what a caller supplied. + * + * Three cases, all of them "this input has no URL form": + * + * - A request URL that is malformed or not absolute (HTTP-47). The message carries the offending + * input and the underlying parse failure is chained as `cause`. + * - A base URL handed to `buildRequest()` that is malformed, not absolute, or carries a fragment + * (SEAM-27). + * - A query-parameter name or value carrying an unpaired surrogate. Such a string has no UTF-8 + * form, so RFC 3986 percent-encoding is undefined for it and `QueryParams.encode()` could only + * fail; `QueryParamsBuilder.add` rejects it at the call that supplied it instead. `cause` is not + * set on this path — nothing was caught, the input was inspected (HTTP-29, audit #67 / #76). + * `QueryParams.parse` does NOT throw it: HTTP-31 makes parsing lenient, so it substitutes U+FFFD. * * @public */ export class UrlConstructionError extends DexpaceError {} /** - * Thrown when a per-call operational override is out of range — a non-null timeout that is zero or - * negative, or a negative max-retries (HTTP-35). + * Thrown when a per-call operational override is out of range (HTTP-35). + * + * The ranges checked are the FULL ranges, not the lower bounds the requirement's own wording names. + * HTTP-35's point is that an out-of-range override is a loud error at the call site that supplied + * it, never a value reinterpreted downstream, and a value that only *some* consumer refuses is the + * same failure moved one seam away: + * + * - `timeoutMs` must be an integer in `1 .. 2**32 - 1` — the range `AbortSignal.timeout()` accepts, + * which is the only one a transport can honour. Zero, negatives, `Infinity`, `NaN`, a fractional + * millisecond and anything above the ceiling are all rejected. Zero is rejected rather than + * reinterpreted: it means "no timeout" in one transport and is an error in another. + * - `maxRetries` must be a non-negative integer. `0` is accepted and means "disable retries for + * this call", distinct from `undefined`; `Infinity` and `NaN` are rejected because they make a + * retry driver's ceiling test permanently false and its loop unbounded. * * @public */ From 18ba4207c8351604ac63f5e69a0b0a0c68069991 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:29:39 +0300 Subject: [PATCH 061/100] docs: http.md states the real timeoutMs range MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "A fractional `timeoutMs` is accepted — a timeout is a duration, not a count" was true when it was written and is false as of this branch: the setter now requires an integer in `1 .. 2**32 - 1`, `AbortSignal.timeout()`'s own range and the only one a transport can honor. Replaced with the range as it stands and the reason for it. `bun run build && node .claude/skills/housekeeping/check-fences.mjs`: 47 fences from 21 files, PASS. Found by: audit #67 / #76. --- docs/sdk-documentation/http.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/sdk-documentation/http.md b/docs/sdk-documentation/http.md index b570319..4ef7dc6 100644 --- a/docs/sdk-documentation/http.md +++ b/docs/sdk-documentation/http.md @@ -110,8 +110,9 @@ const options = RequestOptions.newBuilder() `RequestOptions.EMPTY` is the shared no-op instance. A step reads it as `ctx.options`, and a transport receives it as `send()`'s second argument. Both range checks are the **full** range, not only the lower bound: `maxRetries` rejects anything that is not a non-negative integer, and -`timeoutMs` rejects zero, negatives, `Infinity` and `NaN` alike (`HTTP-35`). A fractional -`timeoutMs` is accepted — a timeout is a duration, not a count. +`timeoutMs` rejects zero, negatives, `Infinity`, `NaN`, a fractional value and anything above +`2**32 - 1` (`HTTP-35`) — the range is `AbortSignal.timeout()`'s, the only one a transport can +honor. `auth` on the builder is the **per-call** auth tier, the highest-precedence one; see [`auth.md`](./auth.md). From 179696ea0f96ad01a561f6459d2324a5daa6db22 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:31:44 +0300 Subject: [PATCH 062/100] =?UTF-8?q?docs:=20deviations=20=E2=80=94=20two=20?= =?UTF-8?q?rows=20for=20#76,=20per=20D0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both are readings this branch takes that depart from a requirement's literal text, appended at the end of "Deviations recorded outside a phase" as D0 prescribes. Round 2 widens #76's partition to cover this file; #73 appends to the same tail and the supervisor merges that seam. 1. **HTTP-35's timeout check is read as the FULL range `AbortSignal.timeout()` accepts.** The requirement names zero and negative; the port also rejects non-finite (shipped unledgered before this audit), non-integer and `> 2**32 - 1`. Stricter than the letter, and deliberately: the field has exactly one consumer, so a value the setter admits and that consumer refuses is HTTP-35's own failure mode with the seam moved. The row records the measured two-runtime divergence that makes deferring the check to the platform unsound. 2. **HTTP-31's "falls back to raw text" is satisfied for an unpaired surrogate by substituting U+FFFD.** The prescribed fallback is unavailable for that input — the raw text has no UTF-8 form, so keeping it only defers the `URIError` from `parse` into `encode()`. The row records the repair, the `Headers` HTTP-18/HTTP-19 precedent it follows, and the two rejected alternatives. Both cells carry `file:line` evidence, verified mechanically: every path resolves, every line number is in range and holds what the row says it holds, and every row in the table has five cells. Found by: audit #67 / #76. --- docs/deviations.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/deviations.md b/docs/deviations.md index 6bf260b..86426bb 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -482,6 +482,8 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **`XCUT-16`'s replay guard is keyed on whether the hop was guarded, not on whether the replacement looks credentialed.** `XCUT-16` and `AUTH-28` say the guard applies "on any path where a credential will be attached", and carve out "a deliberately credential-free re-issue MAY proceed over any scheme". Deciding which of the two a challenge replacement is cannot be done by reading header names: the step's own `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, and a `challengeHook` may invent a carrier this step has never been told about. The port therefore reads "a credential will be attached" as a property of the HOP — if the outbound pass ran the HTTPS guard, so does the replay, whatever URL and headers the hook chose. **Strictly wider than the requirement's letter**, and knowingly so: it refuses a downgraded replacement that carries no credential at all, on a hop that is credentialed. The carve-out is preserved where it is observable — a `NO_AUTH` hop is never guarded outbound, and its replay is guarded only when the replacement carries `Authorization` or `Proxy-Authorization`, which is the previous rule kept as a second arm. *Rejected:* deriving the credential-carrying header names from configuration, which misses the hook-invented carrier and is the shape that let the reported leak through | audit #67 / #71 | 2026-09-04 | `docs/product-spec/19-cross-cutting-invariants-and-policies.md:44` is the requirement and its carve-out. `packages/core/src/auth/auth-step.ts:389` sets `OutboundPlan.guarded`; `:564-575` is `guardReplayScheme` and its two arms. Pinned by "a replacement carrying a NON-standard credential header over plaintext is refused" and "a header-free replacement over plaintext is refused too" in `packages/core/src/auth/auth-step.test.ts`, and by the "XCUT-16: a guarded hop stays guarded across a challenge replay" block in `tests/conformance/xcut/security-by-default.conformance.test.ts` | not yet in §10 | | **`ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" is not honoured: the RxJS SSE adapter takes ownership and closes.** `sseEvents$` and `typedSse$` pass `() => stream.close()` as `fromAsyncIterable`'s `release`, and RxJS runs a subscriber's finalizer on *every* termination — unsubscription, end-of-source and a source error alike, which is the complete list the clause names. **Kept, deliberately, on two grounds.** (1) **The clause has no subject on this platform.** It presumes a source whose iterator return leaves the source open; this port's `SseStream` is deliberately not that one. `#iterate`'s `finally` calls `#releaseQuietly()`, so the resource is released whenever the runtime drives `return()` — which `fromAsyncIterable` must do exactly once (`ASYNC-6`), and which a plain `for await` with `break` does too. Removing the callback would change which channel reports a release failure and when the release runs, not whether the caller-owned source ends up closed. (2) **The ordering is load-bearing.** The release runs *ahead of* `iterator.return()` because an async generator's `return()` queues behind a suspended `next()`, and an SSE stream idling between events is parked in exactly that pull — so without the callback an `unsubscribe()` stays pending until the server next sends a byte, holding the socket open indefinitely. Measured: deleting the two `release` arguments turns four cases red — the two suspended-pull ones, as "the teardown did not settle within 500ms", and the two pre-existing idle-unsubscribe assertions — while every exactly-once release count stays green, which is the shape of the claim. Pagination attaches no release for the complementary reason: a `Paginator`'s pulls are bounded HTTP exchanges, never a wait on a server that may never answer. *Rejected:* dropping the callback to match the letter (reintroduces the hang for no change in what closes). *Rejected:* a caller-facing `{ownership}` option (two behaviours to document for a case with one correct answer). The public TSDoc and `packages/rx/README.md` now state the transfer outright — subscribing hands the stream over, do not close it yourself and do not iterate it afterwards — rather than leaving the `ASYNC-21` citation on the doc comment's first line to read as satisfied | audit #67 / #75 | 2026-09-05 | `packages/rx/src/sse.ts:46` and `:73-75` are the two `release` arguments; `packages/rx/src/from-async-iterable.ts:103-108` is the teardown that runs one on every termination; `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:42` is the requirement. Ground 1: `packages/core/src/sse/stream.ts:136-139` (`#iterate`'s `finally` → `#releaseQuietly()`) with `:117-121` (`close()` memoized, `SSE-28`). Ground 2: `packages/rx/src/from-async-iterable.ts:44-48` states the ordering and why. Pinned by the two `resource ownership` blocks in `packages/rx/src/sse.test.ts`, which count the release the OWNED resource sees rather than `SseStream.close()` calls — the facade memoizes, so a facade-level count reads "once" however many paths call it — and by "SSE ownership transfer releases once on Node" in `tests/node-conformance/rx-bridge.test.mjs`. Phase 8b marked `ASYNC-21` ✅ with this clause dropped from its gist (`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67`); that is a dated record and is left as written. The other half is `SSE-41`'s own "documented source ownership" clause, which the same checklist marked ✅ (`:74`) on the strength of documentation that named unsubscription only — completed by the TSDoc and README rewrite this row accompanies | not yet in §10 | | **`AUTH-22`'s "emit cnonce/nc/qop only when qop is negotiated" is not applied to `cnonce` for a `-sess` algorithm.** A `-sess` HA1 is `H(H(user:realm:pass):nonce:cnonce)` (RFC 7616 §3.4.2), so the client nonce is an *input to the hash* for `MD5-sess` and `SHA-256-sess` whatever `qop` the challenge offered. The port implemented AUTH-22 to the letter: it drew a fresh cnonce, folded it into HA1, and then omitted it from the header whenever `qop` was absent — a response no server can verify, because it has no way to reconstruct HA1. AUTH-30 bounds the re-challenge replay to one 401, so every such exchange simply failed. **`cnonce` is now emitted for any `-sess` algorithm; `nc` and `qop` stay conditional exactly as AUTH-22 says**, because RFC 2069's response input is `H(HA1:nonce:HA2)` and carries no nonce count, so emitting one would advertise a count the response was not computed over. RFC 7616 §3.4 states the wider rule outright — "cnonce: This parameter MUST be used by all implementations". AUTH-22's clause is RFC 2617's RFC 2069-compatibility form, written before `-sess` existed, and the requirement's own AUTH-15 mandates both `-sess` algorithms, so the two sentences cannot both be followed. *Rejected:* declining a `-sess`-without-`qop` challenge instead, which turns every such server into a guaranteed 401 for no security gain, when the value the server needs has already been computed | audit #67 / #74 | 2026-09-05 | `packages/core/src/auth/digest.ts:345-350` (the `-sess` HA1 that consumes the cnonce) against `:405-408` (`buildHeaderValue`, where the `else if` now emits it); `docs/product-spec/11-authentication.md:18` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:357` are AUTH-22's wording. Pinned by the `digestHandler -sess without qop (AUTH-17/AUTH-22)` block in `packages/core/src/auth/digest.test.ts` — one row asserting the header carries `cnonce` and neither `nc` nor `qop`, one recomputing the response from the header's OWN cnonce so a value drawn twice would fail — and by the `MD5-sess, no qop` vector in the same file | not yet in §10 | +| **`HTTP-35`'s timeout check is read as the FULL range `AbortSignal.timeout()` accepts, not the lower bound the requirement enumerates.** `HTTP-35` says the options builder "MUST reject a non-null timeout that is zero or negative". `RequestOptionsBuilder.timeoutMs` rejects three more classes: non-finite (shipped unledgered before this audit), non-integer, and anything above `2**32 - 1`. **Strictly stricter than the letter, and deliberately so.** The field has exactly one consumer — `composeSignal` hands it to `AbortSignal.timeout()` — so a value this setter admits and that function refuses is `HTTP-35`'s own failure mode with the seam moved: the error surfaces inside a transport, as an unwrapped platform `RangeError`, one frame away from the call that supplied it. The earlier reading accepted `1.5` and argued in TSDoc that "a timeout is a duration and a fractional millisecond is meaningful"; no consumer of the field can express one. **The range checked is Node's, and that is the point:** `AbortSignal.timeout(1.5)` and `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are ACCEPTED on Bun, and a negative delay is `RangeError` on Node against `TypeError` on Bun (measured 2026-09-05), so leaving the check to the runtime would make an SDK-level contract depend on which runtime the caller happens to be on. *Rejected:* rounding with `Math.ceil` and clamping inside `composeSignal`, which hides the caller's mistake in the one place `HTTP-35` exists to surface it. `composeSignal` is documented as still able to raise, because a transport's own `defaultTimeoutMs` construction option bypasses this setter and is not validated by core — recorded for #81/#82, not fixed here | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:48` is `HTTP-35`'s wording. `packages/core/src/http/request-options.ts:12` (`MAX_TIMEOUT_MS`) and `:204-214` (the check and the rewritten TSDoc paragraph); `packages/core/src/seams/transport.ts:86-92` is `composeSignal`'s new `@throws`, which states the two-runtime divergence rather than naming one error class. Pinned by "rejects a fractional timeout, which no transport deadline can honor" (`packages/core/src/http/request-options.test.ts:128`, the FLIPPED case — it pinned acceptance until this audit), "rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1" (`:134`), "accepts the ceiling itself" (`:143`) and the `every accepted timeout is an integer in 1..2**32 - 1` property (`:157`); the Node half is `composeSignal timeout range on Node (HTTP-35)` in `tests/node-conformance/seams.test.mjs:105`, which cannot live in `bun test` because Bun accepts both rejected values | not yet in §10 | +| **`HTTP-31`'s "falls back to raw text rather than throwing" is satisfied for an unpaired surrogate by SUBSTITUTING U+FFFD, not by keeping the raw text.** `HTTP-31` (MUST) makes `QueryParams.parse` lenient and enumerates the lenient cases, ending with "malformed percent-encoding falling back to raw text rather than throwing". An unpaired surrogate is a fourth kind of malformed input the enumeration does not name, and the fallback it prescribes is not available for it: the raw text has no UTF-8 form, so keeping it produces a `QueryParams` whose `encode()` throws `URIError` — the throw merely deferred out of `parse` and into an accessor that documents no throw at all. **The port repairs instead.** `parse` runs `toWellFormed()` over each decoded name and value, so every instance it returns is encodable, which is what "parsing MUST invert encode" needs to mean. The strict half of the rule is unaffected and is where `#76` puts the rejection: `QueryParamsBuilder.add` throws `UrlConstructionError` for the same input, and `substitutePathParams` throws `OperationAssemblyError`. That asymmetry is not new to the query model — it is exactly the outbound/inbound split `Headers` already draws for `HTTP-18` against `HTTP-19`, applied to the one requirement pair that needs it here. Substitution matches the platform rather than inventing a policy: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` (measured 2026-09-05). *Rejected:* letting `parse` throw the builder's error, which breaks a MUST. *Rejected:* dropping the offending parameter, which loses a name the caller may be matching on | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:42` carries `HTTP-31`'s wording (shared with `HTTP-30`). `packages/core/src/http/rfc3986.ts:17-18` are the two patterns, `:31` `hasLoneSurrogate` (strict) and `:44` `toWellFormed` (lenient) — one rule, two entry points, so no caller can pick the wrong one; `packages/core/src/http/query-params.ts:144-150` is `parse`'s repair with the `HTTP-18`/`HTTP-19` comparison stated inline, against `:44-50` and `:240-241` for the strict `add` path; `packages/core/src/seams/operation.ts:139-144` is the path-param half. `/\p{Surrogate}/u` rather than `String.prototype.isWellFormed()` because the latter is ES2024 and `tsconfig.base.json:5-11` pins `lib: ES2023`, though the `engines.node >= 20.3` runtime has it. Pinned by the `lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)` block in `packages/core/src/http/query-params.test.ts:170` — "parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing" (`:197`) and the `no anything escapes parse()` property (`:233`) | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) From 8e32ee28930eff768662c90754e2ee1c3aa08f8f Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:38:58 +0300 Subject: [PATCH 063/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20wave=204?= =?UTF-8?q?=20landed;=20pre-take=20wave=205=20decisions=20D16=E2=80=93D18?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 143 +++++++++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index a8a810d..56db4f4 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -283,6 +283,24 @@ source ownership" clause, which Phase 8b marked done on documentation that named Known merge seams: the `deviations.md` table tail (#74 + #75), and `core.api.md` if #74 changes a `@public` TSDoc beside #72's new export. Supervisor resolves both, as in waves 1 and 2. +## Wave 4 — landed 2026-09-05 +PR #90 (#73), PR #91 (#77) and PR #92 (#76) merged into the umbrella at `36c3d04`, in that order. One +conflict (the `deviations.md` table tail: #73's `RETRY-44` row against #76's `HTTP-35` and `HTTP-31` rows), +unioned. Merged tree preflighted in a throwaway worktree before the merge (byte-identical result): all 20 +steps passed. Every commit on all three branches checked as `wahbehmo20@gmail.com` before merging. + +**Traps for every later subtask, added this wave.** +- `stripInternal` is wider than D3 recorded: **any** leading comment of a declaration that contains the + `@internal` substring — inside backticks, in ordinary prose — strips that declaration from the `.d.ts`. + #73 lost `dispatchWithRecovery` from `orchestrator.d.ts` by writing "Both are `@internal`" in its TSDoc. +- `bun run api:local` exits 0 on `ae-unresolved-link` *warnings*; `api:ci` (what `bun run api` and CI run) + exits 1 on them. A `{@link}` to an `@internal` name from a `@public` block therefore looks clean locally + and is red in CI. Backticked prose, as for the `.message` trap. +- `gts lint` and a green `bun test` do not type-check what `tsc` does: #74's `readonly string[]` header value + and #77's `Promise`-as-`BodyInit` both surfaced only on the preflight's `typecheck` step. +- `String.prototype.isWellFormed()` exists on the Node floor but not on `lib: ES2023`; use + `/\p{Surrogate}/u` (now in `http/rfc3986.ts`). + ## Decisions taken for wave 4 (#73 M3, #76 + #77 M4) — pre-taken 2026-09-05, before dispatch ### D13 — #73: the request recovery chain runs once, above the retry loop @@ -353,6 +371,38 @@ public shape change (`dispatchWithRetry` is `@internal`). `api:local` if `idempo topic-named and flat, and its README lists members. Pull, cancel and lock behaviour is what to assert. - `api:local` on core if a `@throws` changes. No changeset (D1). + +**D13 outcome (2026-09-05, #73, PR #90).** `orchestrator.ts` split into `prepareRequest()` (request chain, +once, RECOV-2's throw-to-`Failure` applied) and `dispatchPrepared()` (transport + response chain + unwrap), +both `@internal`; `dispatchWithRecovery` is their composition, behaviour unchanged. `dispatchWithRetry` +applies the chain once and retries `dispatchPrepared` over `stampAttempt`'s copy; a request-chain failure +gets one trip through the response chain and no retry. `engine.ts` needed no functional edit (D13's +constraint on #78 was over-cautious). The shipped `RequestStep` list is exactly one — `idempotencyKeyStep`; +the issue's "client identity, auth stamps" are pipeline-stack `StepDescriptor`s, already re-driven per +attempt by `retryStep`'s `ctx.fork()`, which is RETRY-44 correctly applied to *that* stack. *Correction to +D13's text:* "one stable key across every retry" is `idempotency-key.ts`'s own TSDoc, not RECOV-32's, whose +letter is "invoked at most once per applicable request"; the row records it as a reading. + +**D14 outcome (2026-09-05, #76, PR #92).** All seven bullets landed test-first. Departures, accepted: +`/\p{Surrogate}/u` instead of `isWellFormed()` (not on `lib: ES2023`); `QueryParams.parse` substitutes +U+FFFD rather than throwing, because HTTP-31 is a MUST that `parse` never throws (row); `QueryParamsBuilder` +has no `set`, so `add` only; `tests/node-conformance/seams.test.mjs` edited outside the partition because +`AbortSignal.timeout()` is runtime-divergent (Bun accepts `1.5` and `2**32`; Node rejects both). Round 2 +fixed `http.md`'s now-false fractional-timeout sentence, the two error-class TSDocs, consolidated +`EMPTY_VALUE_LIST` into `http/builder.ts`, and appended the HTTP-35 and HTTP-31 rows. `core.api.md` +byte-identical (signatures unchanged). *Handed to #79:* the last `URIError` escape, `pagination/query-splice.ts` +with a server-supplied cursor. *Handed to #81/#82:* `defaultTimeoutMs` unvalidated on both transports. + +**D15 outcome (2026-09-05, #77, PR #91).** Empty chunk → `SourceContractViolationError` via +`assertNonEmptyChunk`, applied for `declared === 0` too (the only reading that also honours "never an +infinite spin"; recorded in a source comment, as IO-17 and BODY-25 did for the same qualifier — no row). +Boundary rendered through `MediaType.of(...).render()`, so nothing new is exported from `media-type.ts`. +Tap gated on `closed` with the checks ordered fits-cap → tail-consumed → closed, because a literal +closed-first gate breaks BODY-23's repeatable read. 22 Node-conformance cases across the two existing topic +files; **Bun's `Response.formData()` accepts `boundary=a,b`, Node's rejects it**, so the Bun round-trip rows +are regression guards and only the Node tree reproduces the bug — the clearest case yet for that tree +existing. The unknown-length `pipeTo` path still forwards empty chunks; out of HTTP-39's scope, noted. + ### Wave 4 partition | Task | Owns | |---|---| @@ -362,6 +412,96 @@ public shape change (`dispatchWithRetry` is `@internal`). `api:local` if `idempo Known merge seams: `core.api.md` (up to three), which the supervisor regenerates on the merged tree rather than hand-merging. No two tasks share a source file. +## Decisions taken for wave 5 (M4: #78, #79, #80) — pre-taken 2026-09-05, before dispatch + +### D16 — #78: `instanceof IoError` stays; it means "transport-layer failure", and item 17 says so +The classifier keeps `current instanceof IoError` (option b). The four flat leaves are SDK-internal contract +and lifecycle failures, deterministic on re-send: `SourceContractViolationError` and `ClosedResourceError` +are caller programming errors, `AllocationLimitError` is a cap the same request will hit again, and +`EndOfStreamError` is the exact-length-copy contract inside this package — a *wire* truncation is the +transport's to surface, as `TransportFailureError`, which is why `TRANSPORT-20` makes that class an `IoError` +and the leaves not. RETRY-2's "an I/O error" is read as that boundary. Five `classify.test.ts` cases pin one +answer per class; `docs/deviations.md` item 17's rationale paragraph and the anchor-correction block are +rewritten to state the rule (the row itself stays; #68 left the decision to #78), and `io/index.ts`'s +comment says what the cause-walk actually matches. *Rejected:* switching to `isIoError` and deciding per leaf +(only `EndOfStreamError` was ever a candidate, and it is the wrong layer to decide wire truncation). +- `delayOverride` returning a non-finite number (`NaN`, `±Infinity`) is treated exactly like one that throws + (RETRY-40): the computed schedule is used and the loop continues; log through the same path a throwing + override uses. A finite negative override keeps today's behaviour (inline, no wait — pinned already). +- `computeDelay`: `initialDelayMs === 0` short-circuits to `0` before the power is taken, so `0 * Infinity` + never happens; keep the `Math.min` saturation for the positive case. A test at the overflow attempt. +- No changeset (D1); `api:local` only if a `@public` TSDoc changes. *Constrains #80:* item 17 is a section + edit mid-file in `deviations.md`; #80 edits the OBS-29 row and appends — different regions. + +### D17 — #79: race the read against the signal; close on every paginator exit; a present `null` is a `DeserializationError` +- **Abort:** `deserializeFrom` and `serializeTo` race each pending `reader.read()` / `writer.write()` against + the signal (abort listener added once, removed in `finally`), then release the lock as today, so the + documented "an aborted call never leaves the caller's source locked" becomes true rather than narrowed. A + module-private helper inside `packages/codec-json` — core exports no abort-race utility and codec-json has + no dependencies. Mid-drain tests in `json-serde.test.ts` and in `tests/node-conformance/serde.test.mjs` + (Web Streams + `AbortSignal`: runtime-divergent, so the Node case is required, not optional). + *Rejected:* narrowing `write-a-serde.md` and `seams/serde.ts`'s promise to "checked between chunks". +- **Paginator:** the two invariants use the same close-then-rethrow discipline as `parseOrClose`; test with + `== null` so `items: null` fails the invariant, not the spread. The error class is the one `invariant()` + already throws; the message names the invariant. Tests: `parse → undefined`, `parse → {items: null}`, + resource `close` called exactly once (PAGE-27). +- **`tristate()`:** runtime `=== null` check after `inner.parse`, throwing `DeserializationError` (SERDE-14's + fourth state is a decode failure, not a programmer error). +- **SSE:** `#releaseWithInFlightError` no longer calls `onReleaseFailure`; the release failure rides as + `suppressed` only (SSE-30 scopes the hook to the clean-terminal path). Adjust the test that expected both. +- **Handed over from #76:** the one `URIError` still escaping core is `pagination/query-splice.ts` + (`spliceQueryParam` / `readQueryParam`, reached from `strategies.ts:44,79,85` with a server-supplied cursor; + repro `spliceQueryParam(new URL('https://h/?a=1'), 'cursor', '\uD800')`). Handle it here with the same + `hasLoneSurrogate` helper #76 added to `http/rfc3986.ts`, throwing the error class the strategies already use + for a malformed cursor; add the test. +- Two packages: `api:local` in `core` and `codec-json` if a `@public` TSDoc changes. No changesets (D1). + +### D18 — #80: `run`, not `enterWith`, in the runtime; a public `instrumentation` option; document the config wiring +- **Context leak:** `Runtime.send()` wraps its awaited body in `AsyncLocalStorage.run()` for both stores + (diagnostic fields, active span), so what the caller observes after `await send()` is what it had before. + The handle-based `pushDiagnosticFields` / `AsyncScopedStore.enter` stay for callers, with TSDoc stating + that the returned restore works only inside the continuation that called it; the runtime stops using them. + Tests: log *after* `await send()` and assert no `trace.id`; the second `send()` gets its own + `http.client.operation` span (OBS-29 — this is the leak that suppressed it). +- **Public path:** `PipelineBuilder`'s constructor gains an options bag `{instrumentation?, operationName?}` + (second parameter, optional, so no caller breaks), threaded to `createRuntime`'s `contextInit`; + `StandardResilienceOptions` gains the same two fields and forwards them. `operationName` reaches + `promoteToRequest` (CTX-16). Rewrite `tracerFactory`'s TSDoc and the `deviations.md` OBS-29 row to "met" + with the new `file:line`. `api:local` on core. The example in + `.changeset/2026-09-04-per-operation-span.md` is a dated release note: leave it, list it as deferred (D1). +- **Store hygiene:** `contextStore.install` moves inside the `try`, or the `try` opens before + `startOperationSpan`; `span.end()` runs once, behind an `ended` flag. Test: `store.size` equal before and + after a send whose `tracerFactory` throws. +- **Body-drain diagnostics:** both catches emit `http.instrumentation.bodyCaptureFailed` through the + existing `safeEmit` with `cause`, then return the empty capture as today (OBS-20). +- **`DEXPACE_LOG_LEVEL`:** the global configuration's default stays empty — defaulting it to + `defaultConfiguration()` would read the process environment at import time, which is a behaviour change + with no phase behind it. Instead: `docs/sdk-documentation/pipelines.md` (logging section) documents + `setGlobalConfiguration(defaultConfiguration())` as the wiring, and `LoggingStepSettings` gains an optional + `configKey` whose default is the current constant. Check whether an OBS-35 row already exists in + `deviations.md`; if not, append one recording that the default key is baked in (OBS-35's letter) and why + a required key was not chosen (every caller would have to name one to get any logging at all). +- **Smaller items:** `noopInstrumentationBundle.activeSpan` is DONE (D3, #69) — skip. `logging-step.ts`'s + per-request span is ended in a `finally`. `redactUrl(string)`'s WHATWG normalisation (host case, default + port, path) is documented in its TSDoc as inherent to parsing and left as is — re-rendering the original + authority by hand is a second URL renderer for no security gain. +- No changeset (D1). `tests/node-conformance/observability.test.mjs` gets the post-`send()` context case + (`AsyncLocalStorage` behaviour is Node's, and the Bun suite is the one that passed over the leak). + +**Carried to wave 6 (#81/#82):** `defaultTimeoutMs` is unvalidated on both transports (`fetch-transport.ts:73,205,215`, +`undici-transport.ts:95,401,419`) and is now the only path by which an out-of-range delay reaches +`AbortSignal.timeout()`; Node rejects `1.5`/`2**32`/`-1` with `RangeError`, Bun accepts the first two. Validate at the +transport factory the way `RequestOptionsBuilder.timeoutMs` now does (D14), with a conformance row. + +### Wave 5 partition +| Task | Owns | +|---|---| +| #78 | `packages/core/src/retry/{classify,backoff,engine}.ts` + tests, `packages/core/src/io/index.ts` (comment), `docs/deviations.md` **item 17 section only**, `core.api.md` if a TSDoc changes | +| #79 | `packages/codec-json/**`, `packages/core/src/pagination/{paginator,page}.ts` + tests, the `tristate` schema module + test, `packages/core/src/sse/stream.ts` + tests, `docs/sdk-documentation/write-a-serde.md`, `tests/node-conformance/serde.test.mjs`, `codec-json.api.md` | +| #80 | `packages/core/src/observability/{diagnostic-context,logging-step,redaction}.ts`, `packages/core/src/pipeline/{runtime,builder}.ts`, `packages/core/src/auth/preset.ts`, `packages/core/src/context/instrumentation.ts` (TSDoc), their tests, `docs/sdk-documentation/pipelines.md` and the observability guide, `docs/deviations.md` **OBS-29 row edit + one appended row**, `tests/node-conformance/observability.test.mjs`, `core.api.md` | +Known merge seams: `deviations.md` (item 17 section vs OBS-29 row — disjoint regions); `core.api.md` +(regenerated on the merged tree). + ## Deferred — release machinery (recoverable list) | Issue / PR | Deferred item | |---|---| @@ -373,3 +513,6 @@ hand-merging. No two tasks share a source file. | #72 / PR #88 | minor changeset for `@dexpace/core`: retry surfaces the final typed error (was a `SuppressedError` wrapper); new `retryAttempts()` export | | #74 / PR #89 | patch changeset for `@dexpace/core`: every `WWW-Authenticate`/`Proxy-Authenticate` value is read; `cnonce` emitted for `-sess`; empty `realm`/`nonce` declined | | #75 / PR #87 | changeset for `@dexpace/rx` (issue says minor; only `.d.ts` prose moved, so patch is arguable): SSE ownership transfer documented | +| #73 / PR #90 | patch changeset for `@dexpace/core`: one idempotency key per logical request across retry attempts; `.changeset/2026-08-26-recovery-chain-primitives.md:14`'s "single `try`/`catch`" description of `dispatchWithRecovery` is now two functions (guarantee unchanged) | +| #76 / PR #92 | patch changeset for `@dexpace/core`: `timeoutMs` rejects non-integers and values above `2**32-1` (was accepted, threw at send); lone surrogates rejected at `QueryParamsBuilder.add` and path substitution; `getAll` frozen on every path | +| #77 / PR #91 | patch changeset for `@dexpace/core`: empty chunk in `streamBody` is a contract violation; multipart boundary quoted when not a token; logging tap safe after `close()`; `.d.ts` `@throws` prose changed | From 71c99979747a59aa051b794c72e025cfb0015d15 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:46:21 +0300 Subject: [PATCH 064/100] test(retry): pin one retryability answer per I/O error class `classify.ts`'s cause-walk tests `current instanceof IoError`. After the 2026-09-04 tier flattening only two of `io/errors.ts`'s six classes satisfy that: `IoError` and `TransportFailureError`. `EndOfStreamError`, `SourceContractViolationError`, `ClosedResourceError` and `AllocationLimitError` extend `DexpaceError` directly and are grouped only by `isIoError`, so the walk does not match them -- undecided behaviour, asserted nowhere. Audit #67 / #78 (decision D16) keeps `instanceof IoError` and reads it as the boundary it already draws: the branch means "the wire failed". `TRANSPORT-20` makes `TransportFailureError` an `IoError` for exactly that reason; the four flat leaves are this package's own contract and lifecycle failures and are deterministic on re-send. Six cases, one per class, each asserting both what `isIoError` says and what the classifier says, so the disagreement reads as a decision rather than an oversight. Counterfactual measured: switching the branch to `isIoError(current)` turns exactly the four leaf cases red. Refs #78, #67 --- packages/core/src/retry/classify.test.ts | 89 ++++++++++++++++++++++-- packages/core/src/retry/classify.ts | 17 +++++ 2 files changed, 101 insertions(+), 5 deletions(-) diff --git a/packages/core/src/retry/classify.test.ts b/packages/core/src/retry/classify.test.ts index 521fd5a..03ee970 100644 --- a/packages/core/src/retry/classify.test.ts +++ b/packages/core/src/retry/classify.test.ts @@ -1,10 +1,12 @@ // SPDX-License-Identifier: MIT // packages/core/src/retry/classify.test.ts // Exercises: RETRY-1 (single-sourced status set, 501/505 excluded), RETRY-2 (iterative -// identity-tracking cause walk, cycle-safe), RETRY-3 (retryability derived from status, not a stored -// flag), RETRY-4 (transport failures always retryable), RETRY-5/6/7 (re-sendability), RETRY-8 (both -// axes required), RETRY-23/24 (cancellation vs timeout), RETRY-25 (allow-list makes the fatal -// exclusion vacuous), RETRY-37 (configured set is authoritative -- widens AND narrows), +// identity-tracking cause walk, cycle-safe; and the I/O boundary the walk tests -- one case per +// error class in `io/errors.ts`, see the block below), RETRY-3 (retryability derived from status, +// not a stored flag), RETRY-4 (transport failures always retryable), RETRY-5/6/7 (re-sendability), +// RETRY-8 (both axes required), RETRY-23/24 (cancellation vs timeout), RETRY-25 (allow-list makes +// the fatal exclusion vacuous), RETRY-37 (configured set is authoritative -- widens AND narrows), +// TRANSPORT-20 (a no-response send surfaces as a retryable I/O subtype), // XCUT-5 (the baked retryability flag comes from ONE shared status classifier covering 408/429/all // 5xx except 501 and 505 -- asserted below. This port has no separately-cached boolean field: the // classifier is a pure function of HttpStatusError.status, which never changes post-construction @@ -16,7 +18,15 @@ import {stringBody} from '../body/simple-bodies.js'; import {streamBody} from '../body/stream-body.js'; import type {Body} from '../body/body.js'; import {Request} from '../http/request.js'; -import {IoError} from '../io/errors.js'; +import { + AllocationLimitError, + ClosedResourceError, + EndOfStreamError, + IoError, + isIoError, + SourceContractViolationError, + TransportFailureError, +} from '../io/errors.js'; import {CancellationError} from '../seams/transport.js'; import { RETRYABLE_STATUSES, @@ -119,6 +129,75 @@ describe('isRetryableFailure', () => { }); }); +/** + * One case per class in `io/errors.ts`, pinning the boundary RETRY-2's "an I/O error" is read as. + * + * `isIoError` accepts all six classes the file declares; `classify.ts`'s walk tests + * `instanceof IoError`, which two of them satisfy. That gap was undecided until audit #67 / #78 + * decided it (`docs/deviations.md` item 17): the branch means "the wire failed", so `IoError` and + * `TransportFailureError` retry and the four flat leaves do not. Each leaf case asserts BOTH halves + * -- that `isIoError` accepts the value, and what the classifier answers for it -- because the two + * disagreeing is the decision, and a test that only asserted the classifier would read as an + * oversight rather than a choice. + * + * These are the guard on re-parenting: moving any leaf back under `IoError`, or switching the branch + * to `isIoError`, turns four of them red instead of quietly making a deterministic failure retryable. + * Measured 2026-09-05 by making that one-line change: exactly these four fail. + */ +describe('the I/O boundary the cause-walk tests (RETRY-2/RETRY-4, TRANSPORT-20)', () => { + test('IoError itself is retryable', () => { + const error = new IoError('connection refused'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(true); + }); + + test('TransportFailureError is retryable (TRANSPORT-20)', () => { + // The one class TRANSPORT-20 requires to BE an IoError. A send that produced no response is the + // canonical retryable condition (RETRY-4), and the `extends` is what carries it here. + const error = new TransportFailureError('ECONNREFUSED'); + expect(error).toBeInstanceOf(IoError); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(true); + }); + + test('EndOfStreamError is NOT retryable, buried in a cause chain either', () => { + // The exact-length-copy contract inside io/, not a wire truncation: a short copy repeats on the + // next attempt. A truncated response is the transport's to report, as TransportFailureError. + const error = new EndOfStreamError(3, 8); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + // Asserted through a wrapper too: the walk is what would rescue it if the branch widened, so the + // shallow case alone would not catch a change made one hop up. + expect( + isRetryableFailure( + new Error('read failed', {cause: error}), + RETRYABLE_STATUSES, + ), + ).toBe(false); + }); + + test('SourceContractViolationError is NOT retryable', () => { + // A foreign source that returned zero bytes for a positive read (IO-17) is a programming error + // in the source, deterministic on re-send. + const error = new SourceContractViolationError('source returned 0 bytes'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); + + test('ClosedResourceError is NOT retryable', () => { + // Using a closed resource (IO-42) is a caller lifecycle error; the resource stays closed. + const error = new ClosedResourceError('response body'); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); + + test('AllocationLimitError is NOT retryable', () => { + // A cap the same request hits again (IO-9); retrying spends the budget to fail identically. + const error = new AllocationLimitError(2 ** 32, 2 ** 31 - 1); + expect(isIoError(error)).toBe(true); + expect(isRetryableFailure(error, RETRYABLE_STATUSES)).toBe(false); + }); +}); + describe('isRetryableFailure -- cancellation, timeouts, and the allow-list', () => { test('a user abort is never retryable (RETRY-23)', () => { const controller = new AbortController(); diff --git a/packages/core/src/retry/classify.ts b/packages/core/src/retry/classify.ts index 7073395..f911e70 100644 --- a/packages/core/src/retry/classify.ts +++ b/packages/core/src/retry/classify.ts @@ -53,6 +53,22 @@ function causeOf(value: unknown): unknown { * surfaces as an `IoError` subclass and is therefore retryable unconditionally at this level * (RETRY-4). * + * **The I/O branch tests `instanceof IoError`, not `isIoError`, and the difference is the rule.** + * `io/errors.ts` groups six classes under `isIoError`, but only two of them descend from `IoError`: + * `IoError` itself, and `TransportFailureError` -- the class TRANSPORT-20 requires a send that + * produced no response to surface, and the reason that `extends` is a requirement rather than a + * modelling choice (`docs/deviations.md` item 17). Those two mean "the wire failed", and RETRY-2's + * "an I/O error" is read as exactly that boundary. The other four -- `EndOfStreamError`, + * `SourceContractViolationError`, `ClosedResourceError`, `AllocationLimitError` -- extend + * `DexpaceError` directly and are deliberately outside it: they are this package's own contract and + * lifecycle failures and are deterministic on re-send. A closed resource or a violated source + * contract is a caller programming error, an allocation cap is a limit the same request hits again, + * and `EndOfStreamError` is the exact-length-copy contract inside `io/` -- a *wire* truncation is the + * transport's to report, as a `TransportFailureError`. Widening this branch to `isIoError` would + * retry all four. Decided by audit #67 / #78; one case per class in `classify.test.ts` pins the + * answer, so a later re-parenting of any leaf under `IoError` changes a test rather than passing + * silently. + * * @param error - whatever was thrown; any value, not necessarily an `Error`. * @param statuses - the CONFIGURED set, authoritative on its own -- it both widens and narrows * relative to `RETRYABLE_STATUSES`, and the built-in classifier is not AND-ed in (RETRY-37). @@ -70,6 +86,7 @@ export function isRetryableFailure( seen.add(current); // RETRY-3: derived from the carried status at classification time, never a stored per-subclass flag. if (current instanceof HttpStatusError) return statuses.has(current.status); + // Deliberately `instanceof IoError`, never `isIoError` -- see the boundary paragraph above. if (current instanceof IoError) return true; if (isTimeoutAbort(current)) return true; current = causeOf(current); From b9371752728d2c9619e43d7b2f4cb104d473701b Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:47:31 +0300 Subject: [PATCH 065/100] fix(retry): a zero initial delay never yields a NaN backoff `computeDelay` took `initialDelayMs * multiplier ** (attempt - 1)` and relied on `Math.min` to absorb an overflowing power into the cap. That holds for a positive base and not for a zero one: `0 * Infinity` is `NaN`, and `Math.min` propagates `NaN` instead of clamping it. `retrySettings()` accepts `initialDelayMs: 0` with any finite `multiplier >= 1`, so `{initialDelayMs: 0, multiplier: 1e200}` produced `NaN` at attempt 3 and `{initialDelayMs: 0, multiplier: 2}` at attempt 1100 -- against RETRY-11's "overflow-safe, saturating rather than throwing". `NaN` is worse downstream than a large delay, because it fails every comparison: the budget check, the overshoot check and the engine's `delayMs <= 0` short-circuit all read false and it reaches `Clock.sleep`, which rejects with a `RangeError` that replaces the failure being retried. Short-circuit the zero base before the power is taken. Exact, not a repair: the schedule is `0` at every attempt there, and jitter around `0` is `0` for any sample. Three tests -- the two overflow attempts, the jittered case at both sample extremes, and a property over the ranges `retrySettings()` admits (its `initialDelayMs` arm draws an explicit `0`, because the failing region needs a zero base and an overflowing power together and 100 unbiased runs never paired them). Refs #78, #67 --- packages/core/src/retry/backoff.test.ts | 53 +++++++++++++++++++++++++ packages/core/src/retry/backoff.ts | 13 ++++++ 2 files changed, 66 insertions(+) diff --git a/packages/core/src/retry/backoff.test.ts b/packages/core/src/retry/backoff.test.ts index 836ca61..0f309ae 100644 --- a/packages/core/src/retry/backoff.test.ts +++ b/packages/core/src/retry/backoff.test.ts @@ -39,6 +39,59 @@ describe('exponential schedule', () => { expect(() => computeDelay(0, SETTINGS, never)).toThrow(); expect(() => computeDelay(-1, SETTINGS, never)).toThrow(); }); + + test('a zero initial delay stays zero where the power overflows (RETRY-11)', () => { + // `0 * Infinity` is NaN, and NaN is the one value the saturation above cannot absorb: `Math.min` + // propagates it, `overshootsBudget` reads false for it, the engine's `delayMs <= 0` guard reads + // false for it, and it lands in `Clock.sleep` as a RangeError that replaces the real failure. + // `retrySettings` accepts both of these (`initialDelayMs >= 0`, finite `multiplier >= 1`), so + // "overflow-safe, saturating rather than throwing" has to hold for them too. + const hugeMultiplier: BackoffSettings = { + ...SETTINGS, + initialDelayMs: 0, + multiplier: 1e200, + }; + expect(computeDelay(3, hugeMultiplier, never)).toBe(0); + + const manyAttempts: BackoffSettings = {...SETTINGS, initialDelayMs: 0}; + // 2 ** 1099 is Infinity: the first attempt at which the doubling schedule overflows a double. + expect(computeDelay(1100, manyAttempts, never)).toBe(0); + }); + + test('a zero initial delay stays zero under jitter too (RETRY-10/11)', () => { + const jitteredZero: BackoffSettings = { + ...SETTINGS, + initialDelayMs: 0, + multiplier: 1e200, + jitter: 1, + }; + expect(computeDelay(4, jitteredZero, () => 0)).toBe(0); + expect(computeDelay(4, jitteredZero, () => 1)).toBe(0); + }); + + test('property: every accepted schedule is finite and non-negative (RETRY-11)', () => { + // The ranges are exactly what `retrySettings()` admits, so a passing property means no + // configuration a caller can build reaches the engine as a non-finite delay. `initialDelayMs` + // is drawn through an explicit `constant(0)` arm: the failing region needs a zero base AND an + // overflowing power together, and 100 runs of an unbiased double never produced the pair. + fc.assert( + fc.property( + fc.integer({min: 1, max: 5000}), + fc.oneof(fc.constant(0), fc.double({min: 0, max: 1e9, noNaN: true})), + fc.double({min: 1, max: 1e300, noNaN: true}), + fc.double({min: 0, max: 1, noNaN: true}), + (attempt, initialDelayMs, multiplier, jitter) => { + const delay = computeDelay( + attempt, + {initialDelayMs, multiplier, maxDelayMs: 8000, jitter}, + never, + ); + expect(Number.isFinite(delay)).toBe(true); + expect(delay).toBeGreaterThanOrEqual(0); + }, + ), + ); + }); }); describe('symmetric jitter', () => { diff --git a/packages/core/src/retry/backoff.ts b/packages/core/src/retry/backoff.ts index 63f8700..b505258 100644 --- a/packages/core/src/retry/backoff.ts +++ b/packages/core/src/retry/backoff.ts @@ -49,6 +49,17 @@ function applyJitter( * Overflow-safe by construction (RETRY-11): a large attempt makes `**` return `Infinity`, which * `Math.min` absorbs into the cap. It saturates; it never throws. * + * Except at a zero base, where the saturation does not hold and the guard below is what supplies it. + * `0 * Infinity` is `NaN`, and `Math.min` propagates `NaN` rather than clamping it -- so + * `initialDelayMs: 0` with any multiplier above 1 produced a `NaN` delay at the attempt where the + * power overflows (`multiplier: 2` reaches it at attempt 1100; `multiplier: 1e200` at attempt 3). + * `retrySettings()` accepts both configurations. Downstream, `NaN` is worse than a large number: it + * fails every comparison, so the engine's budget check, its overshoot check and its `delayMs <= 0` + * short-circuit all read false and it arrives at `Clock.sleep`, which rejects with a `RangeError` + * that replaces the failure being retried. Short-circuiting the zero base before the power is taken + * is exact rather than a repair: the schedule's value there is `0` at every attempt, and jitter + * around `0` is `0` for any sample (audit #67 / #78). + * * `random` is injected so jitter is assertable rather than statistical -- the same determinism seam * CFG-15 wants for the clock. * @@ -71,6 +82,8 @@ export function computeDelay( `retry attempt must be 1-indexed and >= 1, got ${String(attempt)}`, ); if (settings.fixedDelayMs !== undefined) return settings.fixedDelayMs; + // Before the power, not after: `0 * Infinity` is the one product `Math.min` cannot absorb. + if (settings.initialDelayMs === 0) return 0; const growth = settings.initialDelayMs * settings.multiplier ** (attempt - 1); return applyJitter( Math.min(growth, settings.maxDelayMs), From 426838b9e4539965810c0baadc6eed09d35d9e67 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:49:50 +0300 Subject: [PATCH 066/100] fix(retry): a non-finite delayOverride falls back like a throwing one RETRY-40 makes a bad caller delay-override non-fatal, and `callerOverride` implemented that for a throw only. A non-finite RETURN was the more damaging case: `NaN` and the infinities pass every guard downstream -- `overshootsBudget` and `budgetExhausted` compare false, `waitFor`'s `delayMs <= 0` short-circuit compares false -- and reach `Clock.sleep`, which rejects a non-finite duration with a `RangeError`. RETRY-33's catch-all folds that rejection into the terminal failure, so `delayOverride: () => NaN` with `maxAttempts: 3` gave ONE send and surfaced `RangeError: Clock.sleep: durationMs must be a non-negative finite number, got NaN`, with the real `TransportFailureError` demoted to the attempt trail. Screen the result for finiteness at the source and treat a non-finite one exactly as a throw: report `http.retry.delayOverrideFailed` at warning level through the same emit path, return `undefined`, let RETRY-39's precedence fall through to the computed schedule. The check sits outside the `try` so a logger that throws while reporting it is not re-reported as an override that threw. Finiteness alone is the screen -- a finite negative keeps continuing inline (RETRY-31), and a fractional or very large delay is one RETRY-39 gives the caller precedence for; both pinned. Four engine cases: the schedule the clock is actually asked to sleep for under all three non-finite values, the reported symptom against a clock that guards its input the way `defaultClock` does, the finite passthrough, and the log event with the cause naming the rejected value. Refs #78, #67 --- packages/core/src/retry/engine.test.ts | 149 ++++++++++++++++++++++++- packages/core/src/retry/engine.ts | 64 ++++++++--- 2 files changed, 199 insertions(+), 14 deletions(-) diff --git a/packages/core/src/retry/engine.test.ts b/packages/core/src/retry/engine.test.ts index 3405c96..542f1b4 100644 --- a/packages/core/src/retry/engine.test.ts +++ b/packages/core/src/retry/engine.test.ts @@ -8,7 +8,9 @@ // final attempt's own error is what the engine hands back, cancellation included), // RETRY-35/RECOV-16 (body released before the wait, bounded buffering), RETRY-36/RECOV-19 (503,503,200 // terminates on the 200; a surviving response is returned LIVE), RETRY-39/40 (delay precedence; a -// throwing override is non-fatal), RETRY-42/RECOV-28 (per-call state). +// throwing override is non-fatal -- and a non-finite RETURN from one is the same case, falling back +// to the schedule and logging through the same event; audit #67 / #78), RETRY-42/RECOV-28 (per-call +// state). import {describe, expect, test} from 'bun:test'; import {HttpStatusError} from '../body/http-status-error.js'; import type {Clock} from '../config/clock.js'; @@ -228,6 +230,115 @@ describe('delay resolution (RETRY-39/40)', () => { expect(outcome.kind).toBe('success'); expect(dispatch.calls).toHaveLength(2); }); + + test('a non-finite override falls back to the schedule, like a throwing one (RETRY-40)', async () => { + // RETRY-40 makes a bad override non-fatal. A throw was handled; a non-finite RETURN was not, and + // it is the worse of the two, because `NaN` fails every comparison downstream instead of failing + // loudly here. Audit #67 / #78 reads the two as one case: drop the value, use the computed + // schedule, keep going. Asserted on the delays the clock was ASKED for -- three sends alone would + // also pass on a clock that quietly slept for `NaN`. + for (const bad of [ + Number.NaN, + Number.POSITIVE_INFINITY, + Number.NEGATIVE_INFINITY, + ]) { + const slept: number[] = []; + const config: RetryConfig = { + settings: retrySettings({ + maxAttempts: 3, + initialDelayMs: 200, + multiplier: 2, + jitter: 0, + }), + clock: { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => { + slept.push(durationMs); + return Promise.resolve(); + }, + }, + random: () => 0.5, + delayOverride: () => bad, + }; + const dispatch = scriptedDispatch([ + failure(new TransportFailureError('connection refused')), + ]); + + const outcome = await runWithRetry(GET, dispatch, config); + + expect(dispatch.calls).toHaveLength(3); + expect(slept).toEqual([200, 400]); + expect(outcome.kind).toBe('failure'); + } + }); + + test('a non-finite override never reaches Clock.sleep as a duration (RETRY-40)', async () => { + // The reported symptom, with a clock that guards its input the way `defaultClock` does: the + // rejection was folded into the terminal failure by RETRY-33's catch-all, so `delayOverride: + // () => NaN` with `maxAttempts: 3` gave ONE send and surfaced a `RangeError` about `durationMs` + // in place of the transport failure being retried -- with the real error demoted to the trail. + const config: RetryConfig = { + settings: retrySettings({maxAttempts: 3, fixedDelayMs: 0}), + clock: { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => + Number.isFinite(durationMs) + ? Promise.resolve() + : Promise.reject( + new RangeError( + `Clock.sleep: durationMs must be a non-negative finite number, got ${String(durationMs)}`, + ), + ), + }, + random: () => 0.5, + delayOverride: () => Number.NaN, + }; + const dispatch = scriptedDispatch([ + failure(new TransportFailureError('first')), + failure(new TransportFailureError('second')), + failure(new TransportFailureError('third')), + ]); + + const outcome = await runWithRetry(GET, dispatch, config); + + expect(dispatch.calls).toHaveLength(3); + expect(outcome.kind).toBe('failure'); + if (outcome.kind !== 'failure') return; + expect(outcome.error).toBeInstanceOf(TransportFailureError); + expect((outcome.error as Error).message).toBe('third'); + expect(retryAttempts(outcome.error)).toHaveLength(2); + }); + + test('a finite override is honored unchanged, fractional and huge alike (RETRY-39)', async () => { + // The finiteness guard screens `NaN` and the two infinities and nothing else. A fractional or + // very large delay is still a delay, and RETRY-39 gives the caller precedence over the schedule. + const slept: number[] = []; + const clock: Clock = { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => { + slept.push(durationMs); + return Promise.resolve(); + }, + }; + + for (const override of [0.5, Number.MAX_SAFE_INTEGER]) { + await runWithRetry( + GET, + scriptedDispatch([failure(new TransportFailureError('reset'))]), + { + settings: retrySettings({maxAttempts: 2, fixedDelayMs: 5000}), + clock, + random: () => 0.5, + delayOverride: () => override, + }, + ); + } + + expect(slept).toEqual([0.5, Number.MAX_SAFE_INTEGER]); + }); }); describe('server pacing hints (RETRY-20/22)', () => { @@ -1057,4 +1168,40 @@ describe('Phase 7b retrofit: structured retry logging', () => { setGlobalLogger(NOOP_LOGGER); } }); + + test('emits delayOverrideFailed when delayOverride returns a non-finite delay', async () => { + // "Treated exactly like one that throws" (RETRY-40) is a claim about the diagnostic too: a + // silently-ignored override is a schedule the operator cannot explain. Same event, same level, + // same emit path -- only the cause differs, and it names the value that was rejected. + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map[] = []; + const testLogger = createLogger((_level, fields) => { + events.push(new Map(fields)); + }); + setGlobalLogger(testLogger); + + try { + const config: RetryConfig = { + ...configOf({maxAttempts: 2, fixedDelayMs: 0}), + delayOverride: () => Number.NaN, + }; + const dispatch = scriptedDispatch([ + failure(new IoError('first')), + success(countingResponse(200).response), + ]); + + await runWithRetry(GET, dispatch, config); + + const overrideFailed = events.filter( + e => e.get('event') === 'http.retry.delayOverrideFailed', + ); + expect(overrideFailed).toHaveLength(1); + expect(overrideFailed[0]?.get('cause')).toBe( + 'delayOverride returned a non-finite delay: NaN', + ); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + }); }); diff --git a/packages/core/src/retry/engine.ts b/packages/core/src/retry/engine.ts index edce8be..056a6f3 100644 --- a/packages/core/src/retry/engine.ts +++ b/packages/core/src/retry/engine.ts @@ -48,7 +48,7 @@ export interface RetryConfig { readonly clock: Clock; /** Injectable randomness -- jitter and the X-RateLimit-Reset spread both draw from it. */ readonly random: () => number; - /** Highest-precedence delay source (RETRY-39). A throw is non-fatal (RETRY-40). */ + /** Highest-precedence delay source (RETRY-39). A throw, or a non-finite result, is non-fatal (RETRY-40). */ readonly delayOverride?: ((attempt: number) => number | undefined) | undefined; } @@ -102,25 +102,61 @@ function overshootsBudget(delayMs: number, state: LoopState): boolean { } /** - * RETRY-40: a throwing user override is ignored, never fatal. Emits http.retry.delayOverrideFailed at warning level. + * RETRY-40's diagnostic half, shared by both ways an override can fail. An ignored override is a + * schedule the operator cannot explain from the configuration alone, so neither way is silent. + */ +function reportOverrideFailure(cause: unknown): void { + try { + getGlobalLogger() + .atLevel('warning') + .event('http.retry.delayOverrideFailed') + .cause(cause) + .emit(); + } catch { + // OBS-20: logger failure must never fail the request or retry loop + } +} + +/** + * RETRY-40: a misbehaving user override is ignored, never fatal. Emits + * http.retry.delayOverrideFailed at warning level. + * + * TWO ways to misbehave, one answer. A throw was always handled here. A non-finite RETURN was not, + * and it was the more damaging of the two: `NaN` and the infinities pass every guard downstream -- + * `overshootsBudget` and `budgetExhausted` compare false, {@link waitFor}'s `delayMs <= 0` + * short-circuit compares false -- and arrive at `Clock.sleep`, which rejects a non-finite duration + * with a `RangeError`. RETRY-33's catch-all then folds that rejection into the terminal failure, so + * a `delayOverride` returning `NaN` under `maxAttempts: 3` produced ONE send and surfaced a + * `RangeError` about `durationMs`, with the transport failure it was retrying demoted to the trail. + * Audit #67 / #78 reads the two as one case: drop the value, use the computed schedule, keep going. + * + * The screen is finiteness alone. A finite negative keeps its existing behaviour -- {@link waitFor} + * continues inline without a timer (RETRY-31), which is the same answer the budget clamp already + * produces -- and a fractional or very large delay is a delay RETRY-39 gives the caller precedence + * for. + * + * The check sits OUTSIDE the `try` on purpose: a logger that throws while reporting a non-finite + * result must not be re-reported as an override that threw. */ function callerOverride(state: LoopState): number | undefined { const {delayOverride} = state.config; if (delayOverride === undefined) return undefined; + let delayMs: number | undefined; try { - return delayOverride(state.attempt); + delayMs = delayOverride(state.attempt); } catch (error) { - try { - getGlobalLogger() - .atLevel('warning') - .event('http.retry.delayOverrideFailed') - .cause(error) - .emit(); - } catch { - // OBS-20: logger failure must never fail the request or retry loop - } + reportOverrideFailure(error); + return undefined; + } + if (delayMs !== undefined && !Number.isFinite(delayMs)) { + // A string cause, not a synthesized Error: nothing threw, and the value that was rejected is + // the whole diagnostic. + reportOverrideFailure( + `delayOverride returned a non-finite delay: ${String(delayMs)}`, + ); return undefined; } + return delayMs; } /** RETRY-39: caller override -> server pacing hint -> fixed delay -> exponential backoff. */ @@ -292,7 +328,9 @@ function attachTrail( * A non-positive delay short-circuits before `sleep` is reached: it continues inline with no timer * (RETRY-31), which is reachable after RETRY-17's past-instant hint and after the budget clamp, and * it is also what keeps a caller `delayOverride` returning a negative number out of `sleep`'s - * negative-duration rejection (RETRY-40 makes a bad override non-fatal). + * negative-duration rejection (RETRY-40 makes a bad override non-fatal). It does NOT catch a + * non-finite one -- `NaN <= 0` is false -- which is why {@link callerOverride} screens those at the + * source rather than here. * * Cancellation RESOLVES here rather than propagating: RETRY-26 wants the loop's next iteration to * observe the signal and stop through its own RETRY-32 path, so the abort rejection is the one From 1c3f46a6143721327555c7e69d5b0427579a2f5a Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:50:16 +0300 Subject: [PATCH 067/100] fix(core): scope send()'s async stores with run(), not enterWith (#80) `Runtime.send` entered both async-scoped stores -- the diagnostic fields and the active span -- with `AsyncLocalStorage.enterWith` and restored them from a closure called in its `finally`. `enterWith` installs on the async resource that runs it, which for `send`'s synchronous prefix is the CALLER's; the `finally` runs on a resource created by the first `await` inside. So the restore reached nothing the caller could observe: * after `await runtime.send(...)` the ended operation span was still the active span, so `startOperationSpan`'s `isRecording` guard suppressed the NEXT call's span and only the first operation per async context ever got an `http.client.operation` span (OBS-29's 1:1 binding); * the LOGGING pillar's OBS-23 correlation fields rode out of the call, and every later application log through any core `Logger` carried that request's `trace.id` / `span.id`. `send()` now re-runs the caller's own diagnostic store through `runWithSnapshot(captureDiagnosticSnapshot(), ...)` and activates the operation span through a new `runWithActiveSpan`, both `AsyncLocalStorage.run` forms that unwind structurally -- including over an `enterWith` a step below left open. `AsyncScopedStore` grows the matching `run(value, fn)`. The handle forms stay for callers (OBS-22 specifies one) with TSDoc that states what their restore can and cannot reach. Two hygiene fixes ride along, both from the same audit item: * `contextStore.install` and `startOperationSpan` are inside one `try`. A `tracerFactory` that throws used to skip the `finally` that evicts, so the process-wide store grew by one entry per failed send (CTX-11). * the operation span ends behind an `ended` latch. A throwing `end()` on the success path landed in the `catch`, which called `end()` a second time. Found by audit #67 / #80; four of the six new cases were red before the fix, and the other two were checked against reverted counterfactuals. Refs #80, #67 --- .../observability/diagnostic-context.test.ts | 36 ++++- .../src/observability/diagnostic-context.ts | 34 ++++- packages/core/src/observability/tracing.ts | 33 ++++- packages/core/src/pipeline/runtime.test.ts | 132 +++++++++++++++++- packages/core/src/pipeline/runtime.ts | 116 ++++++++++++--- 5 files changed, 323 insertions(+), 28 deletions(-) diff --git a/packages/core/src/observability/diagnostic-context.test.ts b/packages/core/src/observability/diagnostic-context.test.ts index 7ec1c79..ba80ff5 100644 --- a/packages/core/src/observability/diagnostic-context.test.ts +++ b/packages/core/src/observability/diagnostic-context.test.ts @@ -1,7 +1,9 @@ // SPDX-License-Identifier: MIT // packages/core/src/observability/diagnostic-context.test.ts // Exercises: OBS-10 (default allow-list {trace.id, span.id}, null allow-list folds all, null values skipped), -// OBS-24 (immutable snapshot bridge: capture, reinstall, restore including on throw). +// OBS-24 (immutable snapshot bridge: capture, reinstall, restore including on throw), OBS-22/OBS-23's +// scoping mechanics (AsyncScopedStore.run unwinds across an await and over an enter() its callback made, +// where the enter() handle alone does not). import {describe, expect, test} from 'bun:test'; import { captureDiagnosticSnapshot, @@ -156,6 +158,38 @@ describe('pushDiagnosticFields', () => { }); describe('createAsyncScopedStore', () => { + test('run installs the value for the callback and restores across an await', async () => { + const store = createAsyncScopedStore(); + expect(store.get()).toBeUndefined(); + + await store.run('scoped', async () => { + expect(store.get()).toBe('scoped'); + await Promise.resolve(); + expect(store.get()).toBe('scoped'); + }); + + expect(store.get()).toBeUndefined(); + }); + + test('run unwinds an enter() its callback left open, and restores on a throw', () => { + const store = createAsyncScopedStore(); + store.run('outer', () => { + // The handle form, deliberately never closed -- what the LOGGING pillar's correlation scope + // effectively does once its close() lands in a later continuation. + store.enter('inner'); + expect(store.get()).toBe('inner'); + }); + expect(store.get()).toBeUndefined(); + + expect(() => { + store.run('outer', () => { + store.enter('inner'); + throw new Error('boom'); + }); + }).toThrow('boom'); + expect(store.get()).toBeUndefined(); + }); + test('enter installs value and returned restore function resets prior value', () => { const store = createAsyncScopedStore(); expect(store.get()).toBeUndefined(); diff --git a/packages/core/src/observability/diagnostic-context.ts b/packages/core/src/observability/diagnostic-context.ts index 9cfa324..4c0ecf5 100644 --- a/packages/core/src/observability/diagnostic-context.ts +++ b/packages/core/src/observability/diagnostic-context.ts @@ -77,6 +77,15 @@ export function runWithSnapshot( * The scope-handle form of `withDiagnosticFields`, for callers that cannot express their scope as a single * callback -- OBS-23's span-correlation scope is one: a pipeline step pushes before `await next(...)` and * restores after, with the two halves in different statements. Returns the restore function. + * + * **The restore reaches only the continuation that called it.** `enterWith` installs the store on the + * *current* async resource and every resource created from it; the returned function does the same with + * the previous store. Call it after an `await` and it runs on a different resource, so the caller that + * pushed keeps the pushed fields for the rest of its own continuation -- which is a leak when that caller + * is a library entry point and the continuation is the application's. Use `withDiagnosticFields` whenever + * the scope CAN be written as one callback; that form is `AsyncLocalStorage.run`, which restores on exit + * by construction. `Runtime.send` used this handle until 2026-09-05 and leaked `trace.id`/`span.id` into + * every subsequent application log (audit #67 / #80). */ export function pushDiagnosticFields( fields: Readonly>, @@ -105,14 +114,37 @@ export function pushDiagnosticFields( */ export interface AsyncScopedStore { get(): T | undefined; - /** Installs `value` for the rest of this async context; the returned function restores the prior value. */ + /** + * Installs `value` for the rest of this async context; the returned function restores the prior value. + * + * Carries `pushDiagnosticFields`' caveat verbatim: the restore is an `enterWith` of its own, so it takes + * effect only on the async resource that runs it. A handle closed after an `await` leaves `value` + * installed on the resource that entered it. Prefer {@link AsyncScopedStore.run} for any scope that can + * be written as one callback. + */ enter(value: T): () => void; + /** + * Runs `fn` with `value` installed, restoring whatever was installed before when `fn` returns -- + * including on a throw, and including for anything `fn` itself entered with the handle form. This is + * `AsyncLocalStorage.run`, so the restore is structural rather than a call a later continuation has to + * remember to make, and it is what a library entry point must use if the caller's context is to survive + * the call. + * + * `fn`'s return value is passed through untouched: an `async` callback hands back its promise, and the + * store is restored when the callback's synchronous prefix returns, not when the promise settles. That + * is the intended scoping -- everything the promise chain does inherits the store from the resource it + * was created on. + */ + run(value: T, fn: () => R): R; } export function createAsyncScopedStore(): AsyncScopedStore { const scoped = new AsyncLocalStorage(); return { get: () => scoped.getStore(), + run(value: T, fn: () => R): R { + return scoped.run(value, fn); + }, enter(value: T): () => void { const previous = scoped.getStore(); scoped.enterWith(value); diff --git a/packages/core/src/observability/tracing.ts b/packages/core/src/observability/tracing.ts index c0d9bcb..c98d692 100644 --- a/packages/core/src/observability/tracing.ts +++ b/packages/core/src/observability/tracing.ts @@ -84,17 +84,42 @@ export function getActiveSpan(): Span { * @public */ export function activateSpan(span: Span): Scope { + requireSpan(span, 'activateSpan'); + + const restore = spanStorage.enter(span); + return {close: restore}; +} + +function requireSpan(span: Span, caller: string): void { invariant( (span as unknown) !== null && (span as unknown) !== undefined, - 'activateSpan: span is required', + `${caller}: span is required`, ); invariant( typeof span.end === 'function', - 'activateSpan: span must implement end()', + `${caller}: span must implement end()`, ); +} - const restore = spanStorage.enter(span); - return {close: restore}; +/** + * The callback form of {@link activateSpan}, for a scope that can be written as one function: `span` is + * active for the whole of `fn`, and whatever was active before is active again the moment `fn` returns. + * + * `activateSpan`'s handle cannot make that promise across an `await`. Its `close()` is an `enterWith` on + * whichever async resource happens to run it, so a scope opened before an `await` and closed after one + * leaves the span installed on the resource that opened it -- the caller's, when the opener is + * `Runtime.send`. This form is `AsyncLocalStorage.run`, which unwinds structurally instead. The handle + * stays because OBS-22 specifies one; this is what the runtime uses. + * + * @param span - the span to activate for the extent of `fn`. + * @param fn - the work to run with `span` active. Its result is passed through untouched. + * @returns whatever `fn` returned. + * + * @internal + */ +export function runWithActiveSpan(span: Span, fn: () => T): T { + requireSpan(span, 'runWithActiveSpan'); + return spanStorage.run(span, fn); } /** Extracts trace.id and span.id from OpenTelemetry-compatible spanContext() if present. */ diff --git a/packages/core/src/pipeline/runtime.test.ts b/packages/core/src/pipeline/runtime.test.ts index 3db8ef9..6844ab4 100644 --- a/packages/core/src/pipeline/runtime.test.ts +++ b/packages/core/src/pipeline/runtime.test.ts @@ -8,7 +8,10 @@ // with one send() method, and nests inside another pipeline with the caller's options intact), PIPE-27 // (close() never touches the wrapped transport), CTX-17's positive half (the first store entry is installed // by the first promotion), CTX-1/2/3/6 (exchangeSource pins the call key and instrumentation when it -// rebuilds) +// rebuilds), OBS-22/OBS-23 (the caller's active span and diagnostic fields are what they were once +// send() settles, either way), OBS-29 (one operation span per send, ended exactly once even when end() +// throws, and a second send gets its own), CTX-11 (a throwing tracerFactory leaks no store entry), +// CTX-16 (the pipeline's operation name reaches the request context) import {describe, expect, test} from 'bun:test'; import { createRequestContext, @@ -22,7 +25,13 @@ import {Response} from '../http/response.js'; import {Status} from '../http/status.js'; import {invariant} from '../invariant.js'; import { + getDiagnosticContext, + pushDiagnosticFields, +} from '../observability/diagnostic-context.js'; +import { + NOOP_SPAN, createInstrumentationBundle, + getActiveSpan, type Span, type Tracer, } from '../observability/tracing.js'; @@ -460,3 +469,124 @@ describe('the per-operation span: 1:1 with a logical operation (OBS-29)', () => expect(response.status.code).toBe(200); }); }); + +describe('async-context hygiene across send() (OBS-22, OBS-23, OBS-29)', () => { + test('the caller observes no active span after `await send()` resolves', async () => { + const {tracer} = recordingTracer(); + expect(getActiveSpan()).toBe(NOOP_SPAN); + + await runtimeWith(tracer, new RecordingTransport(aResponse(200))).send( + aRequest('https://example.com'), + ); + + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); + + test('the caller observes no active span after `await send()` rejects', async () => { + const {tracer} = recordingTracer(); + const failing: StepDescriptor = { + type: Symbol('failing'), + stage: 'PRE_REDIRECT', + fn: () => Promise.reject(new Error('boom')), + }; + + await runtimeWith(tracer, new RecordingTransport(aResponse(200)), [failing]) + .send(aRequest('https://example.com')) + .then( + () => undefined, + () => undefined, + ); + + expect(getActiveSpan()).toBe(NOOP_SPAN); + }); + + test('diagnostic fields a step pushed do not outlive the call (OBS-23)', async () => { + const pushing: StepDescriptor = { + type: Symbol('pushing'), + stage: 'PRE_REDIRECT', + fn: (request, ctx) => { + // What `activateSpanForCorrelation` does inside the LOGGING pillar step: a handle-based push + // whose restore runs in a later continuation and therefore never reaches this caller. + pushDiagnosticFields({'trace.id': 't-leaked', 'span.id': 's-leaked'}); + return ctx.next(request); + }, + }; + + await runtimeWith( + recordingTracer().tracer, + new RecordingTransport(aResponse(200)), + [pushing], + ).send(aRequest('https://example.com')); + + expect(getDiagnosticContext(null)).toEqual({}); + }); + + test('a second send() on the same runtime opens its own operation span (OBS-29)', async () => { + const {tracer, spans} = recordingTracer(); + const runtime = runtimeWith(tracer, new RecordingTransport(aResponse(200))); + + await runtime.send(aRequest('https://example.com/one')); + await runtime.send(aRequest('https://example.com/two')); + + expect(spans.length).toBe(2); + expect(spans.map(span => span.ended)).toEqual([1, 1]); + }); +}); + +describe('store and span hygiene on a failing tracer (CTX-11, OBS-29)', () => { + test('a throwing tracerFactory leaves the context store the size it found it', async () => { + const boom = new Error('tracer down'); + const runtime = createRuntime( + [passthroughStep()], + new RecordingTransport(aResponse(200)), + { + instrumentation: createInstrumentationBundle(() => { + throw boom; + }), + }, + ); + + const before = contextStore.size; + const thrown = await runtime.send(aRequest('https://example.com')).then( + () => undefined, + (error: unknown) => error, + ); + + expect(thrown).toBe(boom); + expect(contextStore.size).toBe(before); + }); + + test('an end() that throws on the success path is not called a second time', async () => { + let ends = 0; + const endFailed = new Error('end failed'); + const exceptions: unknown[] = []; + const span: Span = { + isRecording: true, + setAttribute(): Span { + return span; + }, + recordException(error: unknown): Span { + exceptions.push(error); + return span; + }, + end(): void { + ends += 1; + throw endFailed; + }, + }; + + const thrown = await runtimeWith( + {startSpan: () => span}, + new RecordingTransport(aResponse(200)), + ) + .send(aRequest('https://example.com')) + .then( + () => undefined, + (error: unknown) => error, + ); + + expect(thrown).toBe(endFailed); + expect(ends).toBe(1); + expect(exceptions).toEqual([endFailed]); + }); +}); diff --git a/packages/core/src/pipeline/runtime.ts b/packages/core/src/pipeline/runtime.ts index 03c3ad6..8a94263 100644 --- a/packages/core/src/pipeline/runtime.ts +++ b/packages/core/src/pipeline/runtime.ts @@ -11,9 +11,13 @@ import { } from '../context/context.js'; import {contextStore} from '../context/store.js'; import { - activateSpan, + captureDiagnosticSnapshot, + runWithSnapshot, +} from '../observability/diagnostic-context.js'; +import { getActiveSpan, NOOP_TRACER, + runWithActiveSpan, type Span, type Tracer, } from '../observability/tracing.js'; @@ -24,8 +28,12 @@ import type {Transport} from '../seams/transport.js'; import {Cursor} from './cursor.js'; import type {StepDescriptor} from './step.js'; -/** What `Runtime.send()` passes to `createDispatchContext`; `operationName` is not a dispatch-stage concept. */ -type RuntimeContextInit = Omit; +/** + * What a built pipeline carries into every drive. `createDispatchContext` takes the `instrumentation` + * and `key` halves -- `operationName` is not a dispatch-stage concept (CTX-16 introduces it at the + * request stage) -- and `send()` hands the name to `promoteToRequest` itself, one promotion later. + */ +type RuntimeContextInit = ContextInit; /** The advisory span name, matching what the LOGGING pillar step uses for its per-attempt spans. */ const OPERATION_SPAN_NAME = 'http.client.operation'; @@ -58,6 +66,36 @@ function startOperationSpan(context: RequestContext): Span | undefined { return span.isRecording ? span : undefined; } +/** + * Runs `drive` as the body of `span` and ends that span EXACTLY once, whichever way it finishes + * (OBS-29: `operationSucceeded` and `operationFailed` are mutually exclusive and happen once each). + * + * The `ended` latch is not belt-and-braces. `end()` is caller-supplied through `tracerFactory`, and + * OBS-20 deliberately does not wrap tracer calls -- so a throwing `end()` on the success path lands + * in the `catch` below, which is obliged to `recordException` the failure it now has to surface. + * Without the latch that path called `end()` a second time on a span the tracer already closed. + */ +async function driveWithSpan( + span: Span, + drive: () => Promise, +): Promise { + let ended = false; + const endOnce = (): void => { + if (ended) return; + ended = true; + span.end(); + }; + try { + const response = await drive(); + endOnce(); + return response; + } catch (error: unknown) { + span.recordException(error); + endOnce(); + throw error; + } +} + /** * The request context to promote from once the drive finishes: the original, unless a step substituted the * outbound request (PIPE-14), in which case an off-chain rebuild around the request that was actually sent, @@ -165,15 +203,45 @@ export class Runtime implements Transport { // PIPE-9: an empty pipeline dispatches directly to the terminal transport, no cursor allocated. return this.#transport.send(request, options, signal); } + // Every async-scoped store this call touches is RE-RUN around the drive rather than entered in + // place. `runWithSnapshot(captureDiagnosticSnapshot())` re-enters the caller's OWN diagnostic + // store under `AsyncLocalStorage.run`, which changes nothing a step can observe and everything + // about what survives the call: a `pushDiagnosticFields` below -- the LOGGING pillar's OBS-23 + // correlation scope is the shipped one -- now unwinds when `send()` returns. `#drive` does the + // same for the span slot with `runWithActiveSpan`. + // + // Until 2026-09-05 both slots were `enterWith` plus a restore closure called from a `finally`. + // `enterWith` installs on the async resource running it, and that resource is the CALLER's -- + // `send`'s synchronous prefix runs there -- while the `finally` runs on a resource created by + // the first `await` inside. So the restore reached nothing the caller could see: after + // `await send()` the ended operation span was still "active", suppressing the next call's span + // (OBS-29's 1:1 binding), and this call's `trace.id`/`span.id` rode into every subsequent + // application log through any core `Logger` (audit #67 / #80). + return runWithSnapshot(captureDiagnosticSnapshot(), () => + this.#drive(request, options, signal), + ); + } + + /** + * One drive, inside the re-entered stores `send()` established. Split out so `send()` is the + * scoping statement and nothing else: the whole body has to sit inside the `run` callback for the + * unwind to cover it, and a body that long inline reads as if the callback were optional. + */ + async #drive( + request: Request, + options: RequestOptions | undefined, + signal: AbortSignal | undefined, + ): Promise { const dispatchContext = createDispatchContext(this.#contextInit); - const requestContext = promoteToRequest(dispatchContext, request); - contextStore.install(requestContext); // CTX-17's positive half: the first store entry, at the first promotion. + // CTX-16: the operation name this pipeline was built with enters at the request stage and is + // carried unchanged by every promotion after it. + const requestContext = promoteToRequest( + dispatchContext, + request, + this.#contextInit.operationName, + ); let currentContext: ExecutionContext = requestContext; // tracks the latest install for the finally below. - // OBS-29's 1:1 binding. Started before the drive and outside every pillar, so a retry's second - // attempt and a redirect's second hop are the same operation as the first. - const span = startOperationSpan(requestContext); - const scope = span === undefined ? undefined : activateSpan(span); - try { + const drive = async (): Promise => { const cursor = new Cursor({ steps: this.#steps, transport: this.#transport, @@ -190,16 +258,20 @@ export class Runtime implements Transport { ); contextStore.install(exchangeContext); // install-or-replace under the same key (CTX-8). currentContext = exchangeContext; - span?.end(); return response; - } catch (error: unknown) { - // OBS-29: `operationFailed` and `operationSucceeded` are mutually exclusive and happen once - // each. `end()` is reached from exactly one of these two paths, never both. - span?.recordException(error); - span?.end(); - throw error; + }; + // CTX-11/CTX-17: the install and everything that can throw after it are inside ONE try, so the + // `finally` evicts on every path. `startOperationSpan` calls a caller-supplied `tracerFactory`, + // which OBS-30 says must not throw and nothing enforces; installed outside the try, one throwing + // factory left an entry in the process-wide store per failed send. + try { + contextStore.install(requestContext); // CTX-17's positive half: the first store entry, at the first promotion. + // OBS-29's 1:1 binding. Started before the drive and outside every pillar, so a retry's second + // attempt and a redirect's second hop are the same operation as the first. + const span = startOperationSpan(requestContext); + if (span === undefined) return await drive(); + return await runWithActiveSpan(span, () => driveWithSpan(span, drive)); } finally { - scope?.close(); contextStore.close(currentContext); // always the most recently installed context for this call. } } @@ -253,9 +325,11 @@ export class Runtime implements Transport { * * @param steps - the flattened, stage-ordered step array. Copied and frozen. * @param transport - the terminal transport. Never closed by the pipeline (PIPE-27). - * @param contextInit - what each drive's dispatch context is built from: the `instrumentation` - * bundle whose `tracerFactory` supplies `OBS-29`'s per-operation span, and an optional `key` - * pinning two contexts to one store slot (CTX-5). Defaults to the no-op bundle and a fresh key. + * @param contextInit - what each drive's context chain is built from: the `instrumentation` bundle + * whose `tracerFactory` supplies `OBS-29`'s per-operation span, the advisory `operationName` + * every promotion carries (CTX-16), and an optional `key` pinning two contexts to one store slot + * (CTX-5). Defaults to the no-op bundle, no operation name, and a fresh key. `PipelineBuilder`'s + * second constructor argument is the public way to supply the first two. * @returns the built, immutable runtime. * * @internal From d8d3ad5a9ec579ccf4462f85e7b92088198836d2 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:51:12 +0300 Subject: [PATCH 068/100] docs: item 17 states what the cause-walk matches, and why MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Audit #67 / #68 re-anchored `deviations.md` item 17 and recorded that its rationale said something false — "the cause-walk returns retryable for any `IoError`" reads as covering all five classes `isIoError` names, and the branch matches two. It left the decision to #78. #78 keeps `instanceof IoError` and writes the rule down in the three places that carry it: - `deviations.md` item 17: the rationale now says the branch is a boundary, not a category, and a second paragraph says what the four flat leaves are and why re-sending them is pointless. `classify.ts:73` re-anchored to `:90`, and the anchor-correction block records both dates. - `packages/core/src/io/index.ts`: "load-bearing on it" is qualified with what it actually matches, and with the instruction not to re-parent a leaf under `IoError` to tidy the tree. `packages/core/src/index.ts:39-47` re-anchored to `:39-48`. No behaviour change. `probe.mjs --only=citations` clean. Refs #78, #67 --- docs/deviations.md | 40 ++++++++++++++++++++++++----------- packages/core/src/io/index.ts | 12 ++++++++++- 2 files changed, 39 insertions(+), 13 deletions(-) diff --git a/docs/deviations.md b/docs/deviations.md index 6cd1179..2140ea2 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -433,22 +433,38 @@ extend `DexpaceError` **directly** (lines 29, 51, 67, 83) and are grouped by the three-level branch. **Why it cannot be corrected.** `TRANSPORT-20` requires `TransportFailureError` to *be* an `IoError` — the -subtyping is the requirement, not an artifact of modelling. It is also load-bearing: `classify.ts`'s -cause-walk tests `current instanceof IoError` (`packages/core/src/retry/classify.ts:73`), so the `extends` -is what makes a no-response transport failure retryable with zero edits to the retry layer. A flat sibling +subtyping is the requirement, not an artifact of modelling. It is also load-bearing, and **what it bears is +a boundary, not a category**: `classify.ts`'s cause-walk tests `current instanceof IoError` +(`packages/core/src/retry/classify.ts:90`), and because the tree is flat that branch matches exactly two of +the six classes in `io/errors.ts` — `IoError` itself and `TransportFailureError`. It does **not** match the +four leaves `isIoError` groups. That is the intended reading, decided by audit #67 / #78 and stated here +because it is this deviation's own consequence: the branch means "the wire failed". A send that produced no +response is `RETRY-4`'s unconditionally-retryable condition, `TRANSPORT-20` names `TransportFailureError` as +what carries it, and the `extends` is what routes it to the retry layer with no edit there. A flat sibling would have to be enumerated by hand in the retry classifier, and again for every transport added later — trading one level of depth for an open-ended maintenance obligation that the styleguide's own rule exists to prevent. Held at exactly three; a fourth level is not sanctioned by this entry. -> **Anchor correction 2026-09-04 (audit #67 / #68); the rule itself is not decided here.** The line -> numbers above were stale and are re-derived. The substantive point this paragraph used to make — "the -> cause-walk returns retryable for any `IoError`" — reads as covering all five I/O classes, and it does -> not: because the tree is flat, `instanceof IoError` at `classify.ts:73` matches `IoError` itself and -> `TransportFailureError` only. `EndOfStreamError`, `SourceContractViolationError`, -> `ClosedResourceError` and `AllocationLimitError` are **not** retryable through that branch. Whether -> they should be — and therefore whether the classifier tests `instanceof IoError` or the `isIoError` -> predicate — is decided by audit subtask #78, and this row's rationale is rewritten there. Nothing in -> the deviation itself (the three-level branch, and why it stays) turns on that answer. +**What the other four leaves are, and why they stay outside the branch.** `EndOfStreamError`, +`SourceContractViolationError`, `ClosedResourceError` and `AllocationLimitError` are this package's own +contract and lifecycle failures, and every one of them is deterministic on re-send: a closed resource +(`IO-42`) and a source that returned zero bytes for a positive read (`IO-17`) are caller programming errors, +an allocation cap (`IO-9`) is a limit the same request hits again, and `EndOfStreamError` is the +exact-length-copy contract inside `io/` — a *wire* truncation is the transport's to report, as a +`TransportFailureError`, which is the layer that can tell one from a complete short body. `RETRY-2`'s "an +I/O error" is read as that boundary. Widening the branch to `isIoError` would retry all four; only +`EndOfStreamError` was ever a candidate, and `io/` is the wrong layer to decide whether a stream ended early +because the wire broke. Six cases in `packages/core/src/retry/classify.test.ts` pin one answer per class, +each asserting both what `isIoError` says and what the classifier says, so the disagreement is recorded as a +decision rather than an oversight — and re-parenting any leaf under `IoError` turns four of them red. + +> **Anchor correction 2026-09-04 (audit #67 / #68), rule supplied 2026-09-05 (audit #67 / #78).** The line +> numbers in this section were stale on 2026-09-04 and were re-derived then; `classify.ts`'s branch moved +> from `:73` to `:90` on 2026-09-05 when the paragraph above it was written. The substantive point this +> section used to make — "the cause-walk returns retryable for any `IoError`" — read as covering all five +> classes `isIoError` names, and it never did. #68 recorded the gap and left the answer to #78; #78 chose to +> keep `instanceof IoError` and to say what it means, which is the two paragraphs above. Nothing in the +> deviation itself (the three-level branch, and why it stays) ever turned on that answer. ## Deviations recorded outside a phase diff --git a/packages/core/src/io/index.ts b/packages/core/src/io/index.ts index e099471..1358a5a 100644 --- a/packages/core/src/io/index.ts +++ b/packages/core/src/io/index.ts @@ -13,10 +13,20 @@ // U9 pass promoted `EndOfStreamError` — it was the subject of four `@throws` tags on public symbols // with no class a caller could catch. `1f48926` finished the set: `isIoError`, // `AllocationLimitError`, `ClosedResourceError` and `SourceContractViolationError` are exported as -// well, so every error symbol re-exported below is also on `packages/core/src/index.ts:39-47` and in +// well, so every error symbol re-exported below is also on `packages/core/src/index.ts:39-48` and in // `packages/core/etc/core.api.md`. Nothing in this file's error block is internal any more // (docs/work/mvp/2026-09-04-open-items-dissolution.md H8, whose remaining sub-item was the category // catch `isIoError` now provides). +// +// "Load-bearing on it" is narrower than it sounds, and the difference is a decision rather than an +// accident. The cause-walk at `../retry/classify.ts:90` tests `instanceof IoError`, so it matches +// `IoError` and `TransportFailureError` — and NOT the four leaves below, which extend `DexpaceError` +// directly and are grouped only by `isIoError`. That branch means "the wire failed": a send that +// produced no response is retryable (RETRY-4, TRANSPORT-20), while a violated source contract, a +// closed resource, an allocation cap and a short exact-length copy are this package's own failures +// and repeat identically on the next attempt. Audit #67 / #78 decided it; `docs/deviations.md` +// item 17 carries the rationale and `../retry/classify.test.ts` pins one answer per class. Do not +// re-parent a leaf under `IoError` to tidy the tree — that silently makes it retryable. export {BufferedSink} from './buffered-sink.js'; export {BufferedSource} from './buffered-source.js'; export {ByteQueue, copyBytes} from './byte-queue.js'; From 1254707de1582169598f64b476fec80861d6d171 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:53:50 +0300 Subject: [PATCH 069/100] fix(codec-json): race the abort signal against a pending read or write (SERDE-3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `deserializeFrom` checked `signal.throwIfAborted()` before the lock and after each chunk, and `serializeTo` checked once before the lock. Neither can interrupt an operation that is already pending: a source that enqueues `{"a":` and then stalls parks the drain inside `reader.read()`, so an abort 20 ms later reaches nothing, the call never settles, and `source.locked` stays `true` for the rest of the process. `seams/serde.ts:210-212` and `write-a-serde.md` both promise that "an aborted call never leaves the caller's source locked"; the promise was aspirational. Both methods now race each pending operation against the signal through a module-private `abortRace()` — one listener for the whole drive rather than one per chunk, removed in the same `finally` that releases the lock. Core exports no abort-race utility and codec-json has no dependencies (SEAM-1), so the helper lives here. Three measured facts the code and the guide now record: - Releasing a reader with a read outstanding is legal on Bun 1.3.14 and Node 20.3/26, does not throw, and does unlock the stream. The outstanding read then rejects — `AbortError` on Bun, `TypeError: Invalid state: Releasing reader` on Node — which is why the caller must see the signal's own `reason` instead of whichever the runtime picked. - That rejection is not an unhandled one: `Promise.race` keeps a handler on the loser. - The abort promise itself needs a no-op `catch`, because an abort landing between two reads has nothing racing it and Node's default policy would take the process down. The signal's `reason` is surfaced verbatim, which is what `throwIfAborted()` did and what the seam documents; a bare `abort()` still yields the platform `AbortError`. Tests: four cases in `json-serde.test.ts` and three in `tests/node-conformance/serde.test.mjs` — runtime-divergent twice over (Web Streams and `AbortSignal`), and the Node tree is where the release-rejection difference shows. All were red before the change: the two pending-abort cases hung to their deadline on both runtimes. `write-a-serde.md`'s reference implementation races too, and gains rule 7 for it. Found by: audit #67 / #79. D17. --- docs/sdk-documentation/write-a-serde.md | 50 +++++- packages/codec-json/src/abort-race.ts | 84 +++++++++++ packages/codec-json/src/json-serde.test.ts | 168 +++++++++++++++++++++ packages/codec-json/src/json-serde.ts | 34 +++-- tests/node-conformance/serde.test.mjs | 119 +++++++++++++++ 5 files changed, 441 insertions(+), 14 deletions(-) create mode 100644 packages/codec-json/src/abort-race.ts diff --git a/docs/sdk-documentation/write-a-serde.md b/docs/sdk-documentation/write-a-serde.md index daea858..5893836 100644 --- a/docs/sdk-documentation/write-a-serde.md +++ b/docs/sdk-documentation/write-a-serde.md @@ -61,6 +61,33 @@ import { const TEXT = new TextEncoder(); +/** + * Settle when `operation` settles, or as soon as `signal` aborts — whichever comes first, with the + * caller's own `reason` surfaced verbatim. `throwIfAborted()` alone cannot interrupt a `read()` or + * `write()` that never resolves, which is the case that leaves a caller's stream locked forever. + */ +const raceAbort = async ( + operation: Promise, + signal: AbortSignal | undefined, +): Promise => { + if (signal === undefined) return operation; + signal.throwIfAborted(); + let onAbort = (): void => undefined; + const aborted = new Promise((_resolve, reject) => { + onAbort = (): void => { + reject(signal.reason as unknown); + }; + signal.addEventListener('abort', onAbort, {once: true}); + }); + try { + // The loser of the race keeps `Promise.race`'s own handler, so a `read()` that rejects after + // the lock is released never becomes an unhandled rejection. + return await Promise.race([operation, aborted]); + } finally { + signal.removeEventListener('abort', onAbort); + } +}; + export function csvSerde(): Serde { const encode = (value: unknown): string => { if (!Array.isArray(value)) { @@ -106,7 +133,8 @@ export function csvSerde(): Serde { options?.signal?.throwIfAborted(); // before the lock: an aborted call leaves the sink free const writer = sink.getWriter(); // TypeError if contended — a programmer error, not re-typed try { - await writer.write(TEXT.encode(encode(value))); + // Raced, not just checked: a slow sink parks this write, and the abort must reach it. + await raceAbort(writer.write(TEXT.encode(encode(value))), options?.signal); } finally { writer.releaseLock(); // never close or abort: the caller owns the sink (SERDE-3) } @@ -115,14 +143,15 @@ export function csvSerde(): Serde { deserializer: { deserialize: (data, target) => decode(new TextDecoder().decode(data), target), async deserializeFrom(source, target, options) { - options?.signal?.throwIfAborted(); // before the lock, and again after every read below + options?.signal?.throwIfAborted(); // before the lock: an aborted call never takes one const reader = source.getReader(); const chunks: Uint8Array[] = []; try { for (;;) { - const {done, value} = await reader.read(); + // Raced, not checked between chunks: a source that stalls mid-body parks the loop + // inside `read()`, where a between-chunks check never runs again. + const {done, value} = await raceAbort(reader.read(), options?.signal); if (done) break; - options?.signal?.throwIfAborted(); chunks.push(value); } } finally { @@ -141,7 +170,7 @@ export function csvSerde(): Serde { } ``` -Six rules, all visible above: +Seven rules, all visible above: 1. **Raise `SerializationError` / `DeserializationError`, never a raw error** — with one stated exception: `serializeInto`'s out-of-range or does-not-fit case is a plain `RangeError` with no @@ -160,6 +189,17 @@ Six rules, all visible above: target, on **every** entry point (`SERDE-13`), never return a `null` that detonates at a later field access. The fallback label is the literal `'the target type'`; each codec repeats it, because `SEAM-1` leaves core with no exported constant to share. +7. **An abort must race the pending operation, not sit between two of them** (`SERDE-3`). The seam + promises that "an aborted call never leaves the caller's source locked", and a + `throwIfAborted()` between chunks cannot keep it: a source that stalls mid-body parks the drain + inside `read()`, so the call never settles and the lock is never released. Race each pending + `read()`/`write()` against the signal, remove the listener in a `finally`, then release the lock + as usual. Releasing a reader with a read still outstanding is legal on every supported runtime + and does unlock the stream — the outstanding read rejects, differently per runtime + (`AbortError` on Bun 1.3.14, `TypeError: Invalid state: Releasing reader` on Node 20.3 and 26, + measured 2026-09-05), which is why the caller must see the signal's `reason` instead. + `@dexpace/codec-json` holds **one** listener for the whole drive rather than one per chunk; the + example above takes the simpler per-operation form. `mediaType` is the default `Content-Type` — `serdeBody(value, serde)` reads it, and a caller may override per body. diff --git a/packages/codec-json/src/abort-race.ts b/packages/codec-json/src/abort-race.ts new file mode 100644 index 0000000..357158b --- /dev/null +++ b/packages/codec-json/src/abort-race.ts @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: MIT +// packages/codec-json/src/abort-race.ts + +/** + * One abort listener held for the length of a whole stream drive, plus the race that lets it settle + * an operation that is already *pending*. + * + * @internal + */ +export interface AbortRace { + /** + * Settle with `operation`, or reject with the signal's `reason` the moment it aborts — whichever + * happens first. + * + * Also rejects before `operation` is even consulted when the signal is already aborted, which is + * the between-chunks check the loop used to make for itself. + */ + race(operation: Promise): Promise; + + /** Drop the abort listener. Call from the `finally` that releases the stream lock. */ + release(): void; +} + +/** The no-signal case: no listener to install, no race to run, no allocation per chunk. */ +const UNRACED: AbortRace = Object.freeze({ + race: (operation: Promise): Promise => operation, + release: (): void => undefined, +}); + +/** + * Bind `signal` to a single listener that can interrupt any number of pending operations + * (SERDE-3, audit #67 / #79). + * + * `throwIfAborted()` between chunks is not enough on its own: a `reader.read()` that never resolves + * is never raced against anything, so the drain parks inside it, the call never settles, and + * `source.locked` stays `true` for the rest of the process — the opposite of the seam's promise that + * "an aborted call never leaves the caller's source locked". Racing the pending operation is what + * makes that promise true rather than aspirational. + * + * The signal's `reason` is surfaced verbatim, never re-typed: a caller aborting with its own error + * gets that error back, and a bare `abort()` gets the platform's `AbortError` `DOMException`, which + * is exactly what `throwIfAborted()` would have thrown. + * + * One listener per call, not one per chunk — a 10 000-chunk body would otherwise register and remove + * 10 000 listeners on a signal the caller may hold for the life of a request. + * + * @param signal - the caller's signal, or `undefined` when the call took none. + * @returns a race bound to `signal`, whose `release()` removes the listener. + * @internal + */ +export function abortRace(signal: AbortSignal | undefined): AbortRace { + if (signal === undefined) return UNRACED; + + let onAbort = (): void => undefined; + const aborted = new Promise((_resolve, reject) => { + onAbort = (): void => { + // The seam documents that a caller sees its own abort `reason` verbatim, and a caller may + // abort with any value at all — `controller.abort('gone')` is legal. Re-typing it here would + // break that contract, and it is also exactly what `throwIfAborted()` throws. + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- see above; re-enable if the seam ever narrows `reason` to an Error + reject(signal.reason as unknown); + }; + signal.addEventListener('abort', onAbort, {once: true}); + }); + // An abort can land while nothing is racing this promise — between two reads, or after the last + // one and before `release()`. That rejection would be an unhandled one, which takes the process + // down under Node's default policy (`docs/knowledge/harvested/cancellation-and-timeouts.md:26`). + // A no-op handler marks it handled without stopping `Promise.race` below from seeing it. + void aborted.catch(() => undefined); + + return Object.freeze({ + async race(operation: Promise): Promise { + signal.throwIfAborted(); + // A pending `operation` that rejects after losing the race is still settled through + // `Promise.race`'s own handler, so it never becomes an unhandled rejection either — measured + // on Bun 1.3.14 and Node 20.3/26, where releasing a reader with a read outstanding rejects + // that read (`AbortError` on Bun, `TypeError` on Node). + return Promise.race([operation, aborted]); + }, + release(): void { + signal.removeEventListener('abort', onAbort); + }, + }); +} diff --git a/packages/codec-json/src/json-serde.test.ts b/packages/codec-json/src/json-serde.test.ts index 2de27f1..db85038 100644 --- a/packages/codec-json/src/json-serde.test.ts +++ b/packages/codec-json/src/json-serde.test.ts @@ -32,6 +32,37 @@ async function rejection(promise: Promise): Promise { } } +/** How long a raced abort is given to settle a parked read or write before the case fails. */ +const SETTLE_MS = 250; + +/** + * Fails with a named error instead of letting the runner time out, so a regression reads as "the + * abort never settled the call" rather than as a five-second stall with no diagnosis. + * + * `Promise.race` keeps a handler on `promise`, so a later rejection of the losing side is never an + * unhandled one. + */ +async function settleWithin(promise: Promise, ms: number): Promise { + let timer: ReturnType | undefined; + const deadline = new Promise((_resolve, reject) => { + timer = setTimeout(() => { + reject(new Error(`did not settle within ${String(ms)}ms`)); + }, ms); + }); + try { + return await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + +/** One macrotask, which is long enough for a drain loop to reach its second, parked read. */ +function untilParked(): Promise { + return new Promise(resolve => { + setTimeout(resolve, 5); + }); +} + test('declares application/json as its wire media type', () => { expect(jsonSerde().mediaType).toBe('application/json'); }); @@ -673,3 +704,140 @@ describe('the options argument stays optional on both stream methods', () => { ).toEqual({a: 1}); }); }); + +// Module-scope, not describe-local: the pending-abort suite is split across sibling describes to +// stay inside `max-lines-per-function`, and both halves need these. +const PENDING_ABORT_SERDE = jsonSerde(); +const passthroughSchema: Schema = {parse: (i: unknown) => i}; + +/** + * Hands over one chunk and then never produces another, so the drain parks *inside* + * `reader.read()` — the state a between-chunks signal check structurally cannot observe. + */ +function stallingSource( + first: string, + onCancel?: () => void, +): ReadableStream { + return new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(first)); + }, + pull() { + return new Promise(() => undefined); + }, + cancel() { + onCancel?.(); + }, + }); +} + +describe('an abort that lands while a READ is pending (audit #67 / #79)', () => { + test('deserializeFrom settles with the caller reason and unlocks the source (SERDE-3)', async () => { + let cancelled = false; + const source = stallingSource('{"a":', () => { + cancelled = true; + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-drain'); + + const settled = rejection( + PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ); + await untilParked(); + controller.abort(reason); + + expect(await settleWithin(settled, SETTLE_MS)).toBe(reason); + // The whole point of the fix: the caller gets its stream back, still usable. + expect(source.locked).toBe(false); + expect(cancelled).toBe(false); + }); + + test('an abort with no reason surfaces the platform AbortError the seam documents', async () => { + const source = stallingSource('{"a":'); + const controller = new AbortController(); + + const settled = rejection( + PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ); + await untilParked(); + controller.abort(); + + expect(await settleWithin(settled, SETTLE_MS)).toMatchObject({ + name: 'AbortError', + }); + expect(source.locked).toBe(false); + }); +}); + +describe('an abort that lands while a WRITE is pending (audit #67 / #79)', () => { + test('serializeTo settles with the caller reason and unlocks the sink (SERDE-3)', async () => { + let closed = false; + let aborted = false; + const sink = new WritableStream({ + write() { + return new Promise(() => undefined); + }, + close() { + closed = true; + }, + abort() { + aborted = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-write'); + + const settled = rejection( + PENDING_ABORT_SERDE.serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }), + ); + await untilParked(); + controller.abort(reason); + + expect(await settleWithin(settled, SETTLE_MS)).toBe(reason); + expect(sink.locked).toBe(false); + expect(closed).toBe(false); + expect(aborted).toBe(false); + }); + + test('a signal that never fires leaves both directions unchanged', async () => { + const controller = new AbortController(); + const source = new ReadableStream({ + start(streamController) { + streamController.enqueue(new TextEncoder().encode('{"a":')); + streamController.enqueue(new TextEncoder().encode('1}')); + streamController.close(); + }, + }); + const written: string[] = []; + const sink = new WritableStream({ + write(chunk) { + written.push(new TextDecoder().decode(chunk)); + }, + }); + + expect( + await PENDING_ABORT_SERDE.deserializer.deserializeFrom( + source, + {schema: passthroughSchema}, + {signal: controller.signal}, + ), + ).toEqual({a: 1}); + await PENDING_ABORT_SERDE.serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }); + + expect(written.join('')).toBe('{"a":1}'); + expect(source.locked).toBe(false); + expect(sink.locked).toBe(false); + }); +}); diff --git a/packages/codec-json/src/json-serde.ts b/packages/codec-json/src/json-serde.ts index cd7086d..d33cf67 100644 --- a/packages/codec-json/src/json-serde.ts +++ b/packages/codec-json/src/json-serde.ts @@ -8,6 +8,7 @@ import { type Serde, type Serializer, } from '@dexpace/core'; +import {abortRace} from './abort-race.js'; import { degradeTopLevelTristate, tristateReplacer, @@ -128,14 +129,22 @@ function makeSerializer(wiring: TristateWiring): Serializer { // Encoded before the lock is taken: a failed encode then leaves the caller's sink untouched // and still usable, rather than locked-and-released around a write that never happened. const bytes = encodeToBytes(value, wiring); - // Checked after the encode and before the lock, so an aborted call leaves the sink neither - // locked nor closed (SERDE-3). One write follows, so there is no loop to check inside. - options?.signal?.throwIfAborted(); + // Checked after the encode and before the lock, so an aborted call never takes the lock at + // all (SERDE-3). + const signal = options?.signal; + signal?.throwIfAborted(); const writer = sink.getWriter(); + // After `getWriter()`, so a contended sink leaves no listener behind: the `TypeError` is + // thrown before there is one to remove. + const race = abortRace(signal); try { - await writer.write(bytes); + // Raced, not merely checked before: one write is still one operation that can park + // indefinitely against a slow sink, and the abort has to reach it (audit #67 / #79). + await race.race(writer.write(bytes)); } finally { - // Release the lock, never close: the sink is caller-owned (SERDE-3). + race.release(); + // Release the lock, never close: the sink is caller-owned (SERDE-3). A write still + // outstanding at this point stays outstanding — aborting it is the owner's call, not ours. writer.releaseLock(); } }, @@ -209,24 +218,31 @@ function makeDeserializer(): Deserializer { // A streaming TextDecoder keeps multi-byte characters intact across chunk boundaries; decoding // each chunk independently would corrupt any character split across two reads. // Checked before the lock so an aborted call leaves the source neither locked nor cancelled - // (SERDE-3), and again after every read so a long drain stops promptly. + // (SERDE-3), and raced against every read so a stalled one stops too. const signal = options?.signal; signal?.throwIfAborted(); const decoder = new TextDecoder('utf-8'); const reader = source.getReader(); + // After `getReader()`, so a contended source leaves no listener behind: the `TypeError` is + // thrown before there is one to remove. + const race = abortRace(signal); let text = ''; try { for (;;) { - // Serial by necessity: each read depends on the previous one advancing the cursor. - const {done, value} = await reader.read(); + // Serial by necessity: each read depends on the previous one advancing the cursor. Raced + // rather than checked between reads: a source that stalls mid-body parks the loop inside + // `read()`, where a between-chunks check never runs again (audit #67 / #79). + const {done, value} = await race.race(reader.read()); if (done) break; - signal?.throwIfAborted(); text += decoder.decode(value, {stream: true}); } text += decoder.decode(); } finally { + race.release(); // Release the lock, never cancel: the source is caller-owned (SERDE-3). A stream failure // surfaces from `read()` and propagates unwrapped (SERDE-12) — it is not caught here. + // Releasing with a read outstanding is legal on every supported runtime and unlocks the + // stream; the outstanding read rejects, and `Promise.race` above still owns that rejection. reader.releaseLock(); } return decodeText(text, target); diff --git a/tests/node-conformance/serde.test.mjs b/tests/node-conformance/serde.test.mjs index cdf48f1..a10141b 100644 --- a/tests/node-conformance/serde.test.mjs +++ b/tests/node-conformance/serde.test.mjs @@ -261,3 +261,122 @@ describe("decodeResponse's close-failure path on the declared Node floor (SERDE- ); }); }); + +/** + * Fails the case instead of hanging it. `node --test` has no default per-test timeout, so a + * regression in the abort race would park the runner for as long as CI allows rather than reporting + * anything. The timer is ref'd, which also holds the loop open while the abort is in flight. + */ +async function settleWithin(promise, ms) { + let timer; + const deadline = new Promise((_resolve, reject) => { + timer = setTimeout( + () => reject(new Error(`did not settle within ${ms}ms`)), + ms, + ); + }); + try { + return await Promise.race([promise, deadline]); + } finally { + clearTimeout(timer); + } +} + +/** Whatever `promise` rejected with, or the string marker when it resolved. */ +async function rejection(promise) { + try { + await promise; + return 'RESOLVED'; + } catch (e) { + return e; + } +} + +describe('an abort landing on a PENDING read or write (SERDE-3, audit #67 / #79)', () => { + // Runtime-divergent twice over. `AbortSignal` and Web Streams are independent implementations + // here, and the two disagree on what a reader release does to an outstanding read: measured + // 2026-09-05, Bun 1.3.14 rejects it with an `AbortError` and Node 20.3/26 with + // `TypeError: Invalid state: Releasing reader`. Neither may reach the caller in place of its own + // abort reason, and neither may escape as an unhandled rejection — which `node --test` would + // report as a failure of this file even if every assertion below passed. + it('settles deserializeFrom with the caller reason and unlocks the source', async () => { + let cancelled = false; + const source = new ReadableStream({ + start(controller) { + controller.enqueue(Buffer.from('{"id"')); + }, + // Parks the drain inside its second `read()`: the state a between-chunks check cannot see. + pull() { + return new Promise(() => {}); + }, + cancel() { + cancelled = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-drain'); + const settled = rejection( + jsonSerde().deserializer.deserializeFrom( + source, + {schema: identity, typeName: 'Dto'}, + {signal: controller.signal}, + ), + ); + const abortAt = setTimeout(() => controller.abort(reason), 5); + + try { + assert.equal(await settleWithin(settled, 2000), reason); + } finally { + clearTimeout(abortAt); + } + assert.equal(source.locked, false, 'the caller must get its source back'); + assert.equal(cancelled, false, 'the source is caller-owned (SERDE-3)'); + }); + + it('settles serializeTo with the caller reason and unlocks the sink', async () => { + let closed = false; + let aborted = false; + const sink = new WritableStream({ + write() { + return new Promise(() => {}); + }, + close() { + closed = true; + }, + abort() { + aborted = true; + }, + }); + const controller = new AbortController(); + const reason = new Error('the caller gave up mid-write'); + const settled = rejection( + jsonSerde().serializer.serializeTo({a: 1}, sink, { + signal: controller.signal, + }), + ); + const abortAt = setTimeout(() => controller.abort(reason), 5); + + try { + assert.equal(await settleWithin(settled, 2000), reason); + } finally { + clearTimeout(abortAt); + } + assert.equal(sink.locked, false, 'the caller must get its sink back'); + assert.equal(closed, false, 'the sink is caller-owned (SERDE-3)'); + assert.equal(aborted, false); + }); + + it('leaves a completed drain untouched when the signal never fires', async () => { + const controller = new AbortController(); + const value = await jsonSerde().deserializer.deserializeFrom( + streamOf(Buffer.from('{"id"'), Buffer.from(':42}')), + {schema: identity}, + {signal: controller.signal}, + ); + + assert.deepEqual(value, {id: 42}); + // The listener is removed on the way out, so a later abort reaches nothing at all — an + // unremoved one would reject a promise nobody is waiting on any more. + controller.abort(new Error('too late')); + }); +}); From b6b40413d0f2b13e015c04e99d764bb1a401d69c Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:54:22 +0300 Subject: [PATCH 070/100] feat(core): a public way to instrument a pipeline (#80) `createInstrumentationBundle` was public with nowhere public to pass its result. `PipelineBuilder.build()` called `createRuntime(flattened, transport)` with no `contextInit`, `createRuntime` is `@internal`, and `standardResilience()` had no slot for one -- so every pipeline a consumer could build carried `noopInstrumentationBundle`, and `OBS-29`'s `http.client.operation` span was unreachable from outside this package. `CTX-16`'s operation name had the same problem one layer down: `send()` called `promoteToRequest(dispatchContext, request)` without it, so `ctx.context.operationName` was `undefined` on every drive. `PipelineBuilder`'s constructor takes an optional second argument, the new `@public` `PipelineOptions` (`{instrumentation?, operationName?}`), threaded to `createRuntime`'s `contextInit`; `StandardResilienceOptions` extends it and forwards both. `send()` hands the name to `promoteToRequest`, and the LOGGING pillar's `resolveTracer` -- which has always read `ctx.context.operationName` -- starts seeing it. `seedFrom(runtime, 'flatten')` carries the seed runtime's options across, read through a new `@internal` friend accessor (`pipelineOptionsOf`) rather than a getter, which would publish `ContextInit`'s in-package `key` slot. Flatten produces the pipeline that replaces the one it seeded from; dropping the bundle there would silently un-trace a client that seeded from a traced preset, with no way for the derived builder to recover it. `nest` needs no carry -- the seeded runtime is still there, driving its own contexts. Additive: every existing one-argument construction compiles and behaves unchanged. `core.api.md` regenerated. Refs #80, #67 --- packages/core/etc/core.api.md | 10 +- packages/core/src/auth/preset.test.ts | 72 +++++++++++++- packages/core/src/auth/preset.ts | 17 +++- packages/core/src/index.ts | 4 + packages/core/src/pipeline/builder.test.ts | 104 ++++++++++++++++++++- packages/core/src/pipeline/builder.ts | 60 +++++++++++- packages/core/src/pipeline/runtime.ts | 24 +++++ 7 files changed, 280 insertions(+), 11 deletions(-) diff --git a/packages/core/etc/core.api.md b/packages/core/etc/core.api.md index ac916ca..f203384 100644 --- a/packages/core/etc/core.api.md +++ b/packages/core/etc/core.api.md @@ -914,7 +914,7 @@ export class PillarCollisionError extends DexpaceError { // @public export class PipelineBuilder { - constructor(transport: Transport); + constructor(transport: Transport, options?: PipelineOptions); append(descriptor: StepDescriptor): this; appendAll(descriptors: readonly StepDescriptor[]): this; build(): Runtime; @@ -928,6 +928,12 @@ export class PipelineBuilder { static seedFrom(runtime: Runtime, mode: 'flatten' | 'nest'): PipelineBuilder; } +// @public +export interface PipelineOptions { + readonly instrumentation?: InstrumentationBundle | undefined; + readonly operationName?: string | undefined; +} + // @public export class PlaintextCredentialError extends DexpaceError { constructor(stepName: string, scheme: string); @@ -1380,7 +1386,7 @@ export const STAGE_ORDER: readonly Stage[]; export function standardResilience(transport: Transport, options?: StandardResilienceOptions): Runtime; // @public -export interface StandardResilienceOptions { +export interface StandardResilienceOptions extends PipelineOptions { readonly auth?: AuthStepSettings | undefined; readonly logging?: LoggingStepSettings | undefined; readonly redirect?: Partial | undefined; diff --git a/packages/core/src/auth/preset.test.ts b/packages/core/src/auth/preset.test.ts index 1c1a614..733207b 100644 --- a/packages/core/src/auth/preset.test.ts +++ b/packages/core/src/auth/preset.test.ts @@ -3,10 +3,16 @@ // Exercises: PIPE-24 ("installs into empty pillar slots only" -- true by construction, since the preset // always starts from a fresh PipelineBuilder), PIPE-39 (installs exactly the pillars that exist), and // jointly with 5b: PIPE-2's "auth executes per redirect hop, not once for the whole call" plus -// AUTH-29's marker-CONSUMPTION side (5b produced the marker and routed consumption here). +// AUTH-29's marker-CONSUMPTION side (5b produced the marker and routed consumption here), OBS-29 + +// CTX-16 (the preset forwards an instrumentation bundle and an operation name to the built pipeline). import {describe, expect, test} from 'bun:test'; import {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; +import { + createInstrumentationBundle, + type Span, + type Tracer, +} from '../observability/tracing.js'; import {PipelineBuilder} from '../pipeline/builder.js'; import type {StepDescriptor} from '../pipeline/step.js'; import {CROSS_ORIGIN_MARKER_HEADER} from '../redirect/cross-origin.js'; @@ -216,3 +222,67 @@ describe('standardResilience logging options (Phase 7b)', () => { expect(events).toEqual(['http.request', 'http.response']); }); }); + +describe('standardResilience instrumentation options (OBS-29, CTX-16)', () => { + test('the supplied bundle opens one operation span per send, and the name reaches the step', async () => { + const spanNames: string[] = []; + const factoryNames: string[] = []; + const span: Span = { + isRecording: true, + setAttribute: (): Span => span, + recordException: (): Span => span, + end: (): void => undefined, + }; + const tracer: Tracer = { + startSpan(name: string): Span { + spanNames.push(name); + return span; + }, + }; + let seen: string | undefined; + const probe: StepDescriptor = { + type: Symbol('operation-name-probe'), + stage: 'PRE_SERDE', + fn: async (request, ctx) => { + seen = + 'operationName' in ctx.context + ? ctx.context.operationName + : undefined; + return ctx.next(request); + }, + }; + + const runtime = PipelineBuilder.seedFrom( + standardResilience( + new FakeTransport([ + countingResponse(200).response, + countingResponse(200).response, + ]), + { + instrumentation: createInstrumentationBundle(operationName => { + factoryNames.push(operationName); + return tracer; + }), + operationName: 'GetUser', + }, + ), + 'flatten', + ) + .append(probe) + .build(); + + await runtime.send(aRequest()); + await runtime.send(aRequest()); + + // `Runtime.send()` asks the factory for the operation's own tracer once per call (OBS-29's 1:1 + // binding), and the LOGGING pillar asks for one labelled with CTX-16's operation name per attempt. + expect( + factoryNames.filter(name => name === 'http.client.operation'), + ).toHaveLength(2); + expect(factoryNames.filter(name => name === 'GetUser')).toHaveLength(2); + expect( + spanNames.filter(name => name === 'http.client.operation'), + ).toHaveLength(2); + expect(seen).toBe('GetUser'); + }); +}); diff --git a/packages/core/src/auth/preset.ts b/packages/core/src/auth/preset.ts index 965ca4a..6317375 100644 --- a/packages/core/src/auth/preset.ts +++ b/packages/core/src/auth/preset.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/auth/preset.ts -import {PipelineBuilder} from '../pipeline/builder.js'; +import {PipelineBuilder, type PipelineOptions} from '../pipeline/builder.js'; import type {Runtime} from '../pipeline/runtime.js'; import type {RedirectSettings} from '../redirect/settings.js'; import {withRedirect} from '../redirect/strip-marker-step.js'; @@ -20,7 +20,7 @@ import { * * @public */ -export interface StandardResilienceOptions { +export interface StandardResilienceOptions extends PipelineOptions { /** Retry settings and injected seams; omitted yields 5a's spec defaults. */ readonly retry?: RetryStepOptions | undefined; /** Redirect policy overrides; omitted yields 5b's spec defaults. */ @@ -62,6 +62,12 @@ function noAuthSettings(): AuthStepSettings { * `LOGGING` installs {@link loggingStep} to emit telemetry and metrics around dispatches. `SERDE` remains * reserved with no shipped behavior anywhere in this roadmap's current scope. * + * The two inherited {@link PipelineOptions} fields are pipeline-wide rather than per-pillar: + * `instrumentation` is the bundle every context of every call carries, and the source of the one + * `http.client.operation` span each `send()` opens (`OBS-29`); `operationName` is `CTX-16`'s advisory + * label. Omitting `instrumentation` leaves the no-op bundle in place, which opens no span at all — it + * is the only way to switch tracing on for a preset-built pipeline. + * * This function only assembles the pipeline. The failures below surface from the returned runtime's * `send()`, and are documented here because this factory is where a caller chooses the auth * configuration that determines whether they can occur at all. @@ -104,7 +110,12 @@ export function standardResilience( transport: Transport, options: StandardResilienceOptions = {}, ): Runtime { - const builder = new PipelineBuilder(transport); + // The two `PipelineOptions` fields are pipeline-wide rather than per-pillar, so they go to the + // builder rather than into a step's settings; everything below installs one pillar each. + const builder = new PipelineBuilder(transport, { + instrumentation: options.instrumentation, + operationName: options.operationName, + }); return withRedirect(builder, options.redirect) .append(retryStep(options.retry)) .append(authStep(options.auth ?? noAuthSettings())) diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 43cc879..49eadf2 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -157,6 +157,10 @@ export type { RequestContext, } from './context/context.js'; export type {InstrumentationBundle} from './context/instrumentation.js'; +// `PipelineOptions` is what a caller passes to `new PipelineBuilder(transport, options)` and, by +// extension, to `standardResilience`; it is the only public route to `OBS-29`'s per-operation span +// and `CTX-16`'s operation name, so it is exported beside the bundle it carries. +export type {PipelineOptions} from './pipeline/builder.js'; export type {BackoffSettings} from './retry/backoff.js'; export type {RetrySettings} from './retry/settings.js'; export type {RetryStepOptions} from './retry/retry-step.js'; diff --git a/packages/core/src/pipeline/builder.test.ts b/packages/core/src/pipeline/builder.test.ts index ad22b1a..9e5b306 100644 --- a/packages/core/src/pipeline/builder.test.ts +++ b/packages/core/src/pipeline/builder.test.ts @@ -8,13 +8,20 @@ // scratch), PIPE-23 (a colliding reload leaves prior content untouched, and a same-type pillar repeat inside // one batch seats only one step), PIPE-25 (flatten order), PIPE-38 (appendAll preserves batch order; // prependAll reverses it), PIPE-1/PIPE-2 (a built pipeline, driven: entry in STAGE_ORDER, exit reversed), -// PIPE-35 (seedFrom's explicit, non-defaulted flatten-vs-nest modes) +// PIPE-35 (seedFrom's explicit, non-defaulted flatten-vs-nest modes), OBS-29 + CTX-16 (the public +// instrumentation options bag: the supplied bundle opens the operation span, the operation name reaches +// the request context, and flatten seeding carries both) import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {Protocol} from '../http/protocol.js'; import {Request} from '../http/request.js'; import {Response} from '../http/response.js'; import {Status} from '../http/status.js'; +import { + createInstrumentationBundle, + type Span, + type Tracer, +} from '../observability/tracing.js'; import type {Transport} from '../seams/transport.js'; import {PipelineBuilder} from './builder.js'; import { @@ -545,3 +552,98 @@ describe('PipelineBuilder.seedFrom nest mode (PIPE-35)', () => { expect(runtime.transport).toBe(seeded); }); }); + +/** Records the name of every span it is asked to open, so a test can count operations. */ +function countingTracer(): {tracer: Tracer; names: string[]} { + const names: string[] = []; + const span: Span = { + isRecording: true, + setAttribute: (): Span => span, + recordException: (): Span => span, + end: (): void => undefined, + }; + return { + names, + tracer: { + startSpan(name: string): Span { + names.push(name); + return span; + }, + }, + }; +} + +/** Captures what the drive's `RequestContext` says, from inside the pipeline. */ +function contextProbe(seen: { + operationName?: string | undefined; +}): StepDescriptor { + return { + type: Symbol('context-probe'), + stage: 'PRE_SERDE', + fn: async (request, ctx) => { + seen.operationName = + 'operationName' in ctx.context ? ctx.context.operationName : undefined; + return ctx.next(request); + }, + }; +} + +describe('PipelineBuilder instrumentation options (OBS-29, CTX-16)', () => { + test('the supplied bundle is what opens the per-operation span', async () => { + const {tracer, names} = countingTracer(); + const runtime = new PipelineBuilder(new RecordingTransport(), { + instrumentation: createInstrumentationBundle(() => tracer), + }) + .append(descriptor('probe', 'PRE_SERDE')) + .build(); + + await runtime.send(aRequest('https://example.com')); + await runtime.send(aRequest('https://example.com')); + + expect(names).toEqual(['http.client.operation', 'http.client.operation']); + }); + + test('operationName reaches the request context every step reads (CTX-16)', async () => { + const seen: {operationName?: string | undefined} = {}; + const runtime = new PipelineBuilder(new RecordingTransport(), { + operationName: 'GetUser', + }) + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(seen.operationName).toBe('GetUser'); + }); + + test('no options bag means the no-op bundle and no operation name', async () => { + const seen: {operationName?: string | undefined} = {}; + const runtime = new PipelineBuilder(new RecordingTransport()) + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(seen.operationName).toBeUndefined(); + }); + + test('flatten seeding carries the seed runtime’s options (PIPE-35)', async () => { + const {tracer, names} = countingTracer(); + const seen: {operationName?: string | undefined} = {}; + const seeded = new PipelineBuilder(new RecordingTransport(), { + instrumentation: createInstrumentationBundle(() => tracer), + operationName: 'GetUser', + }) + .append(descriptor('seeded', 'LOGGING')) + .build(); + + const runtime = PipelineBuilder.seedFrom(seeded, 'flatten') + .append(contextProbe(seen)) + .build(); + + await runtime.send(aRequest('https://example.com')); + + expect(names).toEqual(['http.client.operation']); + expect(seen.operationName).toBe('GetUser'); + }); +}); diff --git a/packages/core/src/pipeline/builder.ts b/packages/core/src/pipeline/builder.ts index 43d0573..83b383e 100644 --- a/packages/core/src/pipeline/builder.ts +++ b/packages/core/src/pipeline/builder.ts @@ -1,5 +1,6 @@ // SPDX-License-Identifier: MIT // packages/core/src/pipeline/builder.ts +import type {InstrumentationBundle} from '../context/instrumentation.js'; import {invariant} from '../invariant.js'; import type {Transport} from '../seams/transport.js'; import { @@ -8,7 +9,7 @@ import { PillarCollisionError, ReservedStageError, } from './errors.js'; -import {createRuntime, type Runtime} from './runtime.js'; +import {createRuntime, pipelineOptionsOf, type Runtime} from './runtime.js'; import {PILLAR_STAGES, STAGE_ORDER, type Stage} from './stage.js'; import type {StepDescriptor} from './step.js'; @@ -17,6 +18,37 @@ interface AnchorLocation { readonly index: number; } +/** + * What a pipeline carries into every call it drives, as opposed to what a single call carries + * (`RequestOptions`) or what one step is configured with (a pillar's own settings). + * + * Both fields are per-pipeline by construction: `CTX-4` gives every `send()` its own context key, and + * the bundle is shared by reference across that call's three promotions (`CTX-2`/`CTX-3`), so a client + * that needs two operation names builds two pipelines — cheap, since `seedFrom(runtime, 'flatten')` + * derives the second from the first and carries these options with it. + * + * @public + */ +export interface PipelineOptions { + /** + * The correlation bundle every context of every call carries (`CTX-14`), and — through its + * `tracerFactory` — the source of the one `http.client.operation` span `Runtime.send()` opens per + * logical operation (`OBS-29`). Build one with `createInstrumentationBundle(tracerFactory)`. + * + * @defaultValue the disabled-tracing no-op bundle (`CTX-15`), which opens no span at all + */ + readonly instrumentation?: InstrumentationBundle | undefined; + /** + * The advisory operation label (`CTX-16`) — a schema-defined operation id such as `'GetUser'`. + * Carried unchanged from the request context through every promotion, exposed to the tracing seam + * (the LOGGING pillar step names its per-attempt span with it), and never an input to the request, + * the dispatch decision or the store key. + * + * @defaultValue `undefined` — a raw request that belongs to no named operation + */ + readonly operationName?: string | undefined; +} + /** * Assembles a stage-based pipeline via surgical edits (PIPE-7, PIPE-18..PIPE-24), flattening into an * immutable Runtime at build() time (PIPE-25). Mutable while being built; the produced Runtime is frozen. @@ -26,9 +58,19 @@ interface AnchorLocation { export class PipelineBuilder { readonly #buckets = new Map(); readonly #transport: Transport; + readonly #options: PipelineOptions; - constructor(transport: Transport) { + /** + * @param transport - the terminal transport the built pipeline dispatches to. Never closed by the + * pipeline (PIPE-27). + * @param options - what the built pipeline carries into every call: the instrumentation bundle and + * the advisory operation name. Optional, and optional in the second position deliberately — this + * is the only public way to reach `OBS-29`'s per-operation span and `CTX-16`'s operation name, + * and adding it must not break the one-argument construction every existing caller writes. + */ + constructor(transport: Transport, options: PipelineOptions = {}) { this.#transport = transport; + this.#options = options; } /** @@ -225,7 +267,15 @@ export class PipelineBuilder { */ static seedFrom(runtime: Runtime, mode: 'flatten' | 'nest'): PipelineBuilder { if (mode === 'flatten') { - return new PipelineBuilder(runtime.transport).appendAll(runtime.steps); + // Flatten produces the pipeline that REPLACES `runtime`, so it inherits `runtime`'s + // `PipelineOptions` along with its steps: dropping them would silently un-trace a client that + // seeded from a traced preset, and there is no other way for the derived builder to recover + // them. `nest` needs no such carry -- `runtime` is still there, as the terminal transport, + // driving its own contexts with its own bundle. + return new PipelineBuilder( + runtime.transport, + pipelineOptionsOf(runtime), + ).appendAll(runtime.steps); } return new PipelineBuilder(runtime); } @@ -238,7 +288,9 @@ export class PipelineBuilder { const bucket = this.#buckets.get(stage); if (bucket !== undefined) flattened.push(...bucket); } - return createRuntime(flattened, this.#transport); // Runtime copies and freezes -- PIPE-10/PIPE-25. + // Runtime copies and freezes -- PIPE-10/PIPE-25. `PipelineOptions` is structurally the public + // half of `ContextInit`; the `key` half stays in-package, because CTX-4 wants one key per call. + return createRuntime(flattened, this.#transport, this.#options); } #rejectReservedStage(stage: Stage, operation: string): void { diff --git a/packages/core/src/pipeline/runtime.ts b/packages/core/src/pipeline/runtime.ts index 8a94263..640353e 100644 --- a/packages/core/src/pipeline/runtime.ts +++ b/packages/core/src/pipeline/runtime.ts @@ -136,6 +136,14 @@ let create: ( contextInit: RuntimeContextInit, ) => Runtime; +/** + * The read half of the same friend-class hook: `PipelineBuilder.seedFrom(runtime, 'flatten')` builds + * the pipeline that replaces `runtime`, so it has to recover the options `runtime` was built with, + * and `#contextInit` is private. Surfaced as {@link pipelineOptionsOf} rather than as a getter, + * because a getter on this `@public` class would publish `ContextInit`'s in-package `key` slot. + */ +let readContextInit: (runtime: Runtime) => RuntimeContextInit; + /** * The built, immutable pipeline (PIPE-10, PIPE-25). Implements `Transport` itself (PIPE-26) -- Phase 2's * `Transport` SPI has one method (`send`), so there is no second `sendAsync` entry point to delegate through. @@ -172,6 +180,7 @@ export class Runtime implements Transport { static { create = (steps, transport, contextInit) => new Runtime(steps, transport, contextInit); + readContextInit = runtime => runtime.#contextInit; } /** @@ -341,3 +350,18 @@ export function createRuntime( ): Runtime { return create(steps, transport, contextInit); } + +/** + * What `runtime` was built to carry into every call — the instrumentation bundle and the advisory + * operation name. The `key` slot of `ContextInit` rides along in the returned object when the + * in-package caller pinned one; `PipelineBuilder` never does. + * + * @param runtime - the built pipeline to read. + * @returns its context init, by reference. Not a copy: `createDispatchContext` and + * `promoteToRequest` only read it, and the object came from a caller that already owns it. + * + * @internal + */ +export function pipelineOptionsOf(runtime: Runtime): RuntimeContextInit { + return readContextInit(runtime); +} From f08bb19dd8cae6397055237f2ae2403e8807445c Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:56:28 +0300 Subject: [PATCH 071/100] fix(core): close the response on every paginator exit, including a malformed PageInfo (PAGE-4, PAGE-27) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `#walk` clears `held` before dispatching, so between the transport call and the assignment of the next page there is a window in which nothing owns the response but the loop body. `parseOrClose` covered the part of that window where `parse` *rejects*. It did not cover the part where `parse` *resolves* to something malformed: the two invariants below it threw with `held === undefined`, so the `finally` closed nothing and the response leaked. Reproduced with a strategy returning `undefined` and one returning `{items: null}`. Both invariants also tested `!== undefined` while their messages said "never null", so: - `parse` returning `null` never reached its own invariant — it died on `info.items` with a bare `TypeError` from property access. - `{items: null}` passed both invariants and died in `Page`'s constructor, on the spread that copies the item list. `Page`'s own guard had the same mismatch, and `Page` is `@public`, so a consumer constructing one directly saw it too. The checks now reject `null` and `undefined` alike, and run inside `pageOrClose`, which applies PAGE-13's close-then-rethrow discipline to the construction path as well. `closeThenRethrow` is the shared tail of both, so the two paths cannot drift: the primary failure stays primary and a close failure rides as suppressed (PAGE-15). Tests: four malformed shapes through the page view, each asserting the invariant message and `close` called exactly once; the suppressed pairing when that close also fails; and two direct `Page` construction cases. All were red — two on the leaked response, three on the error class or message. Found by: audit #67 / #79. D17. --- .../core/src/pagination/lifecycle.test.ts | 84 +++++++++++++++++- packages/core/src/pagination/page.test.ts | 27 +++++- packages/core/src/pagination/page.ts | 7 +- packages/core/src/pagination/paginator.ts | 85 +++++++++++++------ 4 files changed, 171 insertions(+), 32 deletions(-) diff --git a/packages/core/src/pagination/lifecycle.test.ts b/packages/core/src/pagination/lifecycle.test.ts index 5ea37ab..9a5cb27 100644 --- a/packages/core/src/pagination/lifecycle.test.ts +++ b/packages/core/src/pagination/lifecycle.test.ts @@ -1,9 +1,10 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/lifecycle.test.ts -// Exercises: PAGE-11 (close BEFORE yielding items — the assertion appendix B does not make), PAGE-12 -// (close-on-abandon), PAGE-13 (parse failure closes inline, close error suppressed), PAGE-14 (single-use page -// view), PAGE-15 (close errors surface), PAGE-27 (exactly once on every path), PAGE-32 (consumer throw keeps -// consumer error primary, discarding return-phase close error). +// Exercises: PAGE-4 (a malformed parse result closes the response and names the invariant), PAGE-11 (close +// BEFORE yielding items — the assertion appendix B does not make), PAGE-12 (close-on-abandon), PAGE-13 (parse +// failure closes inline, close error suppressed), PAGE-14 (single-use page view), PAGE-15 (close errors +// surface), PAGE-27 (exactly once on every path), PAGE-32 (consumer throw keeps consumer error primary, +// discarding return-phase close error). import {expect, test} from 'bun:test'; import {FakeTransport, countingResponse} from '../testing/fake-transport.js'; import {IoError} from '../io/errors.js'; @@ -451,3 +452,78 @@ test.each([ } }, ); + +// A strategy is caller code, and `parse`'s declared return type does not survive the seam: an +// `any`-typed JSON decode, a forgotten `return`, or a server field the caller trusted all land here +// as a shape the engine's own types say cannot exist (PAGE-4). The casts below ARE the test — they +// reproduce the four values that reach `#walk` in practice. +function malformedStrategy(result: unknown): PaginationStrategy { + return {parse: () => Promise.resolve(result as PageInfo)}; +} + +test.each([ + ['undefined', undefined, /never null or undefined/], + ['null', null, /never null or undefined/], + ['{items: null}', {items: null, nextRequest: undefined}, /PageInfo\.items/], + ['{items: undefined}', {nextRequest: undefined}, /PageInfo\.items/], +])( + 'a strategy that returns %s closes the response exactly once and names the invariant (PAGE-4, PAGE-27)', + async (_name, result, message) => { + const closed: number[] = []; + const transport = transportOf(1, index => closed.push(index)); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: malformedStrategy(result), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + // Before the fix `items: null` reached the spread in `Page`'s constructor and surfaced as a + // bare `TypeError` from array iteration, which names nothing a caller can act on. + expect((caught as Error).message).toMatch(message); + expect(closed).toEqual([0]); + }, +); + +test('a close failure while rejecting a malformed PageInfo is suppressed, not masking (PAGE-4, PAGE-13)', async () => { + const closeFailure = new IoError('close failed'); + const transport = new FakeTransport([ + countingResponse({ + status: 200, + headers: {}, + body: '{}', + onCancel: () => { + throw closeFailure; + }, + }), + ]); + const paginator = new Paginator({ + transport, + initialRequest: initialRequest(), + strategy: malformedStrategy(undefined), + }); + + let caught: unknown; + try { + for await (const page of paginator.pages()) { + void page; + } + } catch (e: unknown) { + caught = e; + } + + const suppressed = caught as SuppressedErrorLike; + expect(suppressed.name).toBe('SuppressedError'); + expect((suppressed.error as Error).message).toMatch( + /never null or undefined/, + ); + expect(suppressed.suppressed).toBe(closeFailure); +}); diff --git a/packages/core/src/pagination/page.test.ts b/packages/core/src/pagination/page.test.ts index 9bf8487..3c4901a 100644 --- a/packages/core/src/pagination/page.test.ts +++ b/packages/core/src/pagination/page.test.ts @@ -1,7 +1,8 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/page.test.ts -// Exercises: PAGE-2 (items and metadata survive close; items never null), PAGE-3 (one owned response, closed -// exactly once), PAGE-4 (PageInfo shape, undefined next-request is the end signal). +// Exercises: PAGE-2 (items and metadata survive close; items never null, at construction too), PAGE-3 (one +// owned response, never null, closed exactly once), PAGE-4 (PageInfo shape, undefined next-request is the end +// signal). import {expect, test} from 'bun:test'; import {Page, pageInfo} from './page.js'; @@ -125,3 +126,25 @@ test('no "undefined" prototype key survives the guarded install (PAGE-12)', () = 'undefined', ); }); + +// `Page` is `@public`, so these guards are reachable from consumer code, not only from the walk — +// and until audit #67 / #79 their messages said "never null" while the check tested `!== undefined`. +// A `null` therefore reached the item copy and surfaced as a bare `TypeError` from spread. +test.each([ + ['null items', null, /items must never be null/], + ['undefined items', undefined, /items must never be null/], +])('a Page rejects %s at construction (PAGE-2)', (_name, items, message) => { + const {response} = fakeResponse(); + expect(() => + makePage(response, items as unknown as readonly number[]), + ).toThrow(message); +}); + +test('a Page rejects a null response at construction (PAGE-3)', () => { + expect(() => + makePage( + null as unknown as ConstructorParameters>[0], + [1], + ), + ).toThrow(/must own a response/); +}); diff --git a/packages/core/src/pagination/page.ts b/packages/core/src/pagination/page.ts index 9553812..0391257 100644 --- a/packages/core/src/pagination/page.ts +++ b/packages/core/src/pagination/page.ts @@ -67,12 +67,15 @@ export class Page { readonly #response: Response; constructor(response: Response, items: readonly T[]) { + // `!== null` as well as `!== undefined`: both messages have always said "null", and testing only + // for `undefined` let a `null` through to the item copy below, where it surfaced as a bare + // `TypeError` from spread — outside the error tree and naming neither field (audit #67 / #79). invariant( - (response as unknown) !== undefined, + (response as unknown) !== undefined && (response as unknown) !== null, 'a Page must own a response (PAGE-3)', ); invariant( - (items as unknown) !== undefined, + (items as unknown) !== undefined && (items as unknown) !== null, 'a Page’s items must never be null (PAGE-2)', ); diff --git a/packages/core/src/pagination/paginator.ts b/packages/core/src/pagination/paginator.ts index e331316..2a41f91 100644 --- a/packages/core/src/pagination/paginator.ts +++ b/packages/core/src/pagination/paginator.ts @@ -196,20 +196,7 @@ export class Paginator { response, request, ); - - // PAGE-4: parse must always return a well-formed result and must never signal termination through a - // side channel. A strategy that returns nothing is a programmer error, so it crashes at the fault - // rather than silently ending the walk as if the server had run out of pages. - invariant( - (info as unknown) !== undefined, - 'PaginationStrategy.parse must return a PageInfo, never undefined', - ); - invariant( - (info.items as unknown) !== undefined, - 'PageInfo.items must never be null or absent (PAGE-2)', - ); - - held = new Page(response, info.items); + held = await pageOrClose(response, info); request = info.nextRequest; yield held; } @@ -236,17 +223,67 @@ async function parseOrClose( try { return await strategy.parse(response, template); } catch (parseError: unknown) { - try { - await response.close(); - } catch (closeError: unknown) { - throw suppress( - parseError, - closeError, - 'pagination parse failed and releasing the response also failed', - ); - } - throw parseError; + return closeThenRethrow(response, parseError, 'pagination parse failed'); + } +} + +/** + * PAGE-4: `parse` must always return a well-formed result, and must never signal termination through a side + * channel. A strategy that returns nothing is a programmer error, so the walk crashes at the fault rather than + * silently ending as if the server had run out of pages. + * + * PAGE-27: and it crashes *after* releasing the response. `parse` returning a malformed value is the one exit + * from this loop the `finally` in `#walk` cannot cover — `held` is still `undefined` there, because assigning it + * is precisely what failed — so, like PAGE-13's parse rejection, the release happens inline (audit #67 / #79). + * + * Both checks reject `null` as well as `undefined`, which is what their messages have always claimed. Testing + * only for `undefined` let `{items: null}` through to `Page`'s constructor, where the item copy surfaced as a + * bare `TypeError` from spread — naming nothing a caller could act on, and leaking the response on the way. + * + * Not async: the only asynchrony here is the close, and only on the failure path. + */ +function pageOrClose( + response: Response, + info: PageInfo, +): Promise> { + try { + invariant( + (info as unknown) !== undefined && (info as unknown) !== null, + 'PaginationStrategy.parse must return a PageInfo, never null or undefined', + ); + invariant( + (info.items as unknown) !== undefined && (info.items as unknown) !== null, + 'PageInfo.items must never be null or absent (PAGE-2)', + ); + return Promise.resolve(new Page(response, info.items)); + } catch (buildError: unknown) { + return closeThenRethrow( + response, + buildError, + 'the pagination strategy returned a malformed PageInfo', + ); + } +} + +/** + * Release `response`, then rethrow `primary`. Shared by the two inline-close paths so they cannot drift: a close + * failure is attached as suppressed and never masks the failure that got here first (PAGE-13, PAGE-15). + */ +async function closeThenRethrow( + response: Response, + primary: unknown, + context: string, +): Promise { + try { + await response.close(); + } catch (closeError: unknown) { + throw suppress( + primary, + closeError, + `${context} and releasing the response also failed`, + ); } + throw primary; } /** PAGE-26: on an already-settled cancellation path, a close error is swallowed — nothing is left to report to. */ From 6655f9de41d6eb32bf96cd335e71c0928cae4722 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:55:46 +0300 Subject: [PATCH 072/100] test(retry): keep the new retry cases inside the lint caps `max-lines-per-function` and `max-params` count a `describe` callback and a fast-check property's arrow like any other function, and the previous three commits pushed three of them over. - `backoff.test.ts`: the two zero-base cases and the totality property move to their own `a zero initial delay (RETRY-11)` describe; the property's four arbitraries become one `fc.record`, which is also closer to what `retrySettings()` actually validates. - `engine.test.ts`: the two non-finite-override cases move to their own describe; `recordingClock`, `guardingClock`, `oneFailureThenSuccess` and `captureLogEvents` are hoisted to module scope. The logger harness was copied verbatim in two existing tests and would have been a third -- extracting it shortens all three, and the throwing-override case gains the `cause` assertion its non-finite sibling has. No assertion weakened; `bun test packages/core/src/retry` is 173 pass, 0 fail. Refs #78, #67 --- packages/core/src/retry/backoff.test.ts | 45 ++-- packages/core/src/retry/engine.test.ts | 313 ++++++++++++------------ 2 files changed, 184 insertions(+), 174 deletions(-) diff --git a/packages/core/src/retry/backoff.test.ts b/packages/core/src/retry/backoff.test.ts index 0f309ae..ed872eb 100644 --- a/packages/core/src/retry/backoff.test.ts +++ b/packages/core/src/retry/backoff.test.ts @@ -2,7 +2,8 @@ // packages/core/src/retry/backoff.test.ts // Exercises: RETRY-9 (initialDelay * multiplier^(attempt-1), 1-indexed, capped), RETRY-10 (symmetric // jitter bounds, midpoint, j=0 identity, negative floors to zero), RETRY-11 (attempt < 1 rejected, -// overflow saturates), RETRY-43 (fixed delay disables backoff AND jitter). +// overflow saturates -- INCLUDING at a zero initial delay, where `0 * Infinity` used to give NaN; +// audit #67 / #78), RETRY-43 (fixed delay disables backoff AND jitter). import {describe, expect, test} from 'bun:test'; import fc from 'fast-check'; import {computeDelay, type BackoffSettings} from './backoff.js'; @@ -39,13 +40,19 @@ describe('exponential schedule', () => { expect(() => computeDelay(0, SETTINGS, never)).toThrow(); expect(() => computeDelay(-1, SETTINGS, never)).toThrow(); }); +}); - test('a zero initial delay stays zero where the power overflows (RETRY-11)', () => { - // `0 * Infinity` is NaN, and NaN is the one value the saturation above cannot absorb: `Math.min` - // propagates it, `overshootsBudget` reads false for it, the engine's `delayMs <= 0` guard reads - // false for it, and it lands in `Clock.sleep` as a RangeError that replaces the real failure. - // `retrySettings` accepts both of these (`initialDelayMs >= 0`, finite `multiplier >= 1`), so - // "overflow-safe, saturating rather than throwing" has to hold for them too. +/** + * RETRY-11's "saturating rather than throwing" has one hole, and it is the zero base. Every other + * accepted setting overflows into `Math.min`'s cap; `0 * Infinity` overflows into `NaN`, which + * `Math.min` propagates. Audit #67 / #78. + */ +describe('a zero initial delay (RETRY-11)', () => { + test('stays zero where the power overflows', () => { + // Downstream, NaN is worse than a large number: `overshootsBudget` reads false for it, the + // engine's `delayMs <= 0` guard reads false for it, and it lands in `Clock.sleep` as a + // RangeError that replaces the failure being retried. `retrySettings()` accepts both settings + // below (`initialDelayMs >= 0`, finite `multiplier >= 1`), so RETRY-11 covers them. const hugeMultiplier: BackoffSettings = { ...SETTINGS, initialDelayMs: 0, @@ -58,7 +65,7 @@ describe('exponential schedule', () => { expect(computeDelay(1100, manyAttempts, never)).toBe(0); }); - test('a zero initial delay stays zero under jitter too (RETRY-10/11)', () => { + test('stays zero under jitter too (RETRY-10)', () => { const jitteredZero: BackoffSettings = { ...SETTINGS, initialDelayMs: 0, @@ -74,18 +81,22 @@ describe('exponential schedule', () => { // configuration a caller can build reaches the engine as a non-finite delay. `initialDelayMs` // is drawn through an explicit `constant(0)` arm: the failing region needs a zero base AND an // overflowing power together, and 100 runs of an unbiased double never produced the pair. + const accepted = fc.record({ + initialDelayMs: fc.oneof( + fc.constant(0), + fc.double({min: 0, max: 1e9, noNaN: true}), + ), + multiplier: fc.double({min: 1, max: 1e300, noNaN: true}), + maxDelayMs: fc.double({min: 0, max: 1e9, noNaN: true}), + jitter: fc.double({min: 0, max: 1, noNaN: true}), + }); + fc.assert( fc.property( fc.integer({min: 1, max: 5000}), - fc.oneof(fc.constant(0), fc.double({min: 0, max: 1e9, noNaN: true})), - fc.double({min: 1, max: 1e300, noNaN: true}), - fc.double({min: 0, max: 1, noNaN: true}), - (attempt, initialDelayMs, multiplier, jitter) => { - const delay = computeDelay( - attempt, - {initialDelayMs, multiplier, maxDelayMs: 8000, jitter}, - never, - ); + accepted, + (attempt, settings) => { + const delay = computeDelay(attempt, settings, never); expect(Number.isFinite(delay)).toBe(true); expect(delay).toBeGreaterThanOrEqual(0); }, diff --git a/packages/core/src/retry/engine.test.ts b/packages/core/src/retry/engine.test.ts index 542f1b4..b1fdd20 100644 --- a/packages/core/src/retry/engine.test.ts +++ b/packages/core/src/retry/engine.test.ts @@ -97,6 +97,66 @@ function scriptedDispatch( return Object.assign(dispatch, {calls}); } +/** A clock that records every duration it is asked to sleep for, and returns immediately. */ +function recordingClock(slept: number[]): Clock { + return { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => { + slept.push(durationMs); + return Promise.resolve(); + }, + }; +} + +/** + * `defaultClock`'s own precondition, modelled: `Clock.sleep` rejects a non-finite duration with a + * `RangeError` (`config/clock.ts:148-157`). A fake that slept for any duration at all would hide the + * bug; this guard is what surfaces it. + */ +function guardingClock(): Clock { + return { + now: () => 0, + monotonic: () => 0, + sleep: durationMs => + Number.isFinite(durationMs) + ? Promise.resolve() + : Promise.reject( + new RangeError( + `Clock.sleep: durationMs must be a non-negative finite number, got ${String(durationMs)}`, + ), + ), + }; +} + +/** One retryable failure, then a 200: the shortest script that drives exactly one delay decision. */ +function oneFailureThenSuccess(): RetryDispatch { + return scriptedDispatch([ + failure(new IoError('first')), + success(countingResponse(200).response), + ]); +} + +/** Installs a capturing global logger for the duration of `body` and returns what it emitted. */ +async function captureLogEvents( + body: () => Promise, +): Promise[]> { + const {createLogger, setGlobalLogger, NOOP_LOGGER} = + await import('../observability/logger.js'); + const events: Map[] = []; + setGlobalLogger( + createLogger((_level, fields) => { + events.push(new Map(fields)); + }), + ); + try { + await body(); + } finally { + setGlobalLogger(NOOP_LOGGER); + } + return events; +} + describe('eligibility (RETRY-7/8)', () => { test('a non-retryable failure is surfaced after exactly one attempt', async () => { const dispatch = scriptedDispatch([failure(new TypeError('bad'))]); @@ -231,41 +291,61 @@ describe('delay resolution (RETRY-39/40)', () => { expect(dispatch.calls).toHaveLength(2); }); - test('a non-finite override falls back to the schedule, like a throwing one (RETRY-40)', async () => { - // RETRY-40 makes a bad override non-fatal. A throw was handled; a non-finite RETURN was not, and - // it is the worse of the two, because `NaN` fails every comparison downstream instead of failing - // loudly here. Audit #67 / #78 reads the two as one case: drop the value, use the computed - // schedule, keep going. Asserted on the delays the clock was ASKED for -- three sends alone would - // also pass on a clock that quietly slept for `NaN`. + test('a finite override is honored unchanged, fractional and huge alike (RETRY-39)', async () => { + // The finiteness guard below screens `NaN` and the two infinities and nothing else. A fractional + // or very large delay is still a delay, and RETRY-39 gives the caller precedence over the + // schedule -- 5000 ms of `fixedDelayMs` here, which neither run waits. + const slept: number[] = []; + const clock = recordingClock(slept); + + for (const override of [0.5, Number.MAX_SAFE_INTEGER]) { + await runWithRetry( + GET, + scriptedDispatch([failure(new TransportFailureError('reset'))]), + { + settings: retrySettings({maxAttempts: 2, fixedDelayMs: 5000}), + clock, + random: () => 0.5, + delayOverride: () => override, + }, + ); + } + + expect(slept).toEqual([0.5, Number.MAX_SAFE_INTEGER]); + }); +}); + +/** + * RETRY-40 makes a bad override non-fatal. A throw was handled; a non-finite RETURN was not, and it + * is the worse of the two, because `NaN` fails every comparison downstream instead of failing loudly + * at the override. Audit #67 / #78 reads the two as one case: drop the value, use the computed + * schedule, keep going. + */ +describe('a non-finite delayOverride is the throwing case (RETRY-40)', () => { + test('every non-finite value falls back to the computed schedule', async () => { + // Asserted on the delays the clock was ASKED for. Three sends alone would also pass against a + // fake that quietly slept for `NaN`, which is most of them. for (const bad of [ Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY, ]) { const slept: number[] = []; - const config: RetryConfig = { + const dispatch = scriptedDispatch([ + failure(new TransportFailureError('connection refused')), + ]); + + const outcome = await runWithRetry(GET, dispatch, { settings: retrySettings({ maxAttempts: 3, initialDelayMs: 200, multiplier: 2, jitter: 0, }), - clock: { - now: () => 0, - monotonic: () => 0, - sleep: durationMs => { - slept.push(durationMs); - return Promise.resolve(); - }, - }, + clock: recordingClock(slept), random: () => 0.5, delayOverride: () => bad, - }; - const dispatch = scriptedDispatch([ - failure(new TransportFailureError('connection refused')), - ]); - - const outcome = await runWithRetry(GET, dispatch, config); + }); expect(dispatch.calls).toHaveLength(3); expect(slept).toEqual([200, 400]); @@ -273,35 +353,23 @@ describe('delay resolution (RETRY-39/40)', () => { } }); - test('a non-finite override never reaches Clock.sleep as a duration (RETRY-40)', async () => { - // The reported symptom, with a clock that guards its input the way `defaultClock` does: the - // rejection was folded into the terminal failure by RETRY-33's catch-all, so `delayOverride: - // () => NaN` with `maxAttempts: 3` gave ONE send and surfaced a `RangeError` about `durationMs` - // in place of the transport failure being retried -- with the real error demoted to the trail. - const config: RetryConfig = { - settings: retrySettings({maxAttempts: 3, fixedDelayMs: 0}), - clock: { - now: () => 0, - monotonic: () => 0, - sleep: durationMs => - Number.isFinite(durationMs) - ? Promise.resolve() - : Promise.reject( - new RangeError( - `Clock.sleep: durationMs must be a non-negative finite number, got ${String(durationMs)}`, - ), - ), - }, - random: () => 0.5, - delayOverride: () => Number.NaN, - }; + test('it never reaches Clock.sleep as a duration, so the real failure survives', async () => { + // The reported symptom, against a clock that guards its input the way `defaultClock` does: the + // rejection was folded into the terminal failure by RETRY-33's catch-all, so `() => NaN` with + // `maxAttempts: 3` gave ONE send and surfaced a `RangeError` about `durationMs` in place of the + // transport failure being retried -- with the real error demoted to the trail. const dispatch = scriptedDispatch([ failure(new TransportFailureError('first')), failure(new TransportFailureError('second')), failure(new TransportFailureError('third')), ]); - const outcome = await runWithRetry(GET, dispatch, config); + const outcome = await runWithRetry(GET, dispatch, { + settings: retrySettings({maxAttempts: 3, fixedDelayMs: 0}), + clock: guardingClock(), + random: () => 0.5, + delayOverride: () => Number.NaN, + }); expect(dispatch.calls).toHaveLength(3); expect(outcome.kind).toBe('failure'); @@ -310,35 +378,6 @@ describe('delay resolution (RETRY-39/40)', () => { expect((outcome.error as Error).message).toBe('third'); expect(retryAttempts(outcome.error)).toHaveLength(2); }); - - test('a finite override is honored unchanged, fractional and huge alike (RETRY-39)', async () => { - // The finiteness guard screens `NaN` and the two infinities and nothing else. A fractional or - // very large delay is still a delay, and RETRY-39 gives the caller precedence over the schedule. - const slept: number[] = []; - const clock: Clock = { - now: () => 0, - monotonic: () => 0, - sleep: durationMs => { - slept.push(durationMs); - return Promise.resolve(); - }, - }; - - for (const override of [0.5, Number.MAX_SAFE_INTEGER]) { - await runWithRetry( - GET, - scriptedDispatch([failure(new TransportFailureError('reset'))]), - { - settings: retrySettings({maxAttempts: 2, fixedDelayMs: 5000}), - clock, - random: () => 0.5, - delayOverride: () => override, - }, - ); - } - - expect(slept).toEqual([0.5, Number.MAX_SAFE_INTEGER]); - }); }); describe('server pacing hints (RETRY-20/22)', () => { @@ -1102,106 +1141,66 @@ describe('per-call state (RETRY-42, RECOV-28)', () => { describe('Phase 7b retrofit: structured retry logging', () => { test('emits attemptFailed per retry and exhausted when attempts run out', async () => { - const {createLogger, setGlobalLogger, NOOP_LOGGER} = - await import('../observability/logger.js'); - const events: Map[] = []; - const testLogger = createLogger((_level, fields) => { - events.push(new Map(fields)); + const events = await captureLogEvents(async () => { + await runWithRetry( + GET, + scriptedDispatch([ + failure(new IoError('first')), + failure(new IoError('second')), + failure(new IoError('third')), + ]), + configOf({maxAttempts: 3, fixedDelayMs: 0}), + ); }); - setGlobalLogger(testLogger); - - try { - const config = configOf({maxAttempts: 3, fixedDelayMs: 0}); - const dispatch = scriptedDispatch([ - failure(new IoError('first')), - failure(new IoError('second')), - failure(new IoError('third')), - ]); - await runWithRetry(GET, dispatch, config); - - const failedEvents = events.filter( - e => e.get('event') === 'http.retry.attemptFailed', - ); - expect(failedEvents).toHaveLength(2); - expect(failedEvents[0]?.get('attempt')).toBe(1); - expect(failedEvents[1]?.get('attempt')).toBe(2); + const failedEvents = events.filter( + e => e.get('event') === 'http.retry.attemptFailed', + ); + expect(failedEvents).toHaveLength(2); + expect(failedEvents[0]?.get('attempt')).toBe(1); + expect(failedEvents[1]?.get('attempt')).toBe(2); - const exhaustedEvents = events.filter( - e => e.get('event') === 'http.retry.exhausted', - ); - expect(exhaustedEvents).toHaveLength(1); - expect(exhaustedEvents[0]?.get('attempts')).toBe(3); - } finally { - setGlobalLogger(NOOP_LOGGER); - } + const exhaustedEvents = events.filter( + e => e.get('event') === 'http.retry.exhausted', + ); + expect(exhaustedEvents).toHaveLength(1); + expect(exhaustedEvents[0]?.get('attempts')).toBe(3); }); test('emits delayOverrideFailed when delayOverride throws', async () => { - const {createLogger, setGlobalLogger, NOOP_LOGGER} = - await import('../observability/logger.js'); - const events: Map[] = []; - const testLogger = createLogger((_level, fields) => { - events.push(new Map(fields)); - }); - setGlobalLogger(testLogger); - - try { - const config: RetryConfig = { + const events = await captureLogEvents(async () => { + await runWithRetry(GET, oneFailureThenSuccess(), { ...configOf({maxAttempts: 2, fixedDelayMs: 0}), delayOverride: () => { throw new Error('bad override'); }, - }; - const dispatch = scriptedDispatch([ - failure(new IoError('first')), - success(countingResponse(200).response), - ]); - - await runWithRetry(GET, dispatch, config); + }); + }); - const overrideFailed = events.filter( - e => e.get('event') === 'http.retry.delayOverrideFailed', - ); - expect(overrideFailed).toHaveLength(1); - } finally { - setGlobalLogger(NOOP_LOGGER); - } + const overrideFailed = events.filter( + e => e.get('event') === 'http.retry.delayOverrideFailed', + ); + expect(overrideFailed).toHaveLength(1); + expect(overrideFailed[0]?.get('cause')).toBe('Error: bad override'); }); test('emits delayOverrideFailed when delayOverride returns a non-finite delay', async () => { - // "Treated exactly like one that throws" (RETRY-40) is a claim about the diagnostic too: a - // silently-ignored override is a schedule the operator cannot explain. Same event, same level, - // same emit path -- only the cause differs, and it names the value that was rejected. - const {createLogger, setGlobalLogger, NOOP_LOGGER} = - await import('../observability/logger.js'); - const events: Map[] = []; - const testLogger = createLogger((_level, fields) => { - events.push(new Map(fields)); - }); - setGlobalLogger(testLogger); - - try { - const config: RetryConfig = { + // "Treated exactly like one that throws" (RETRY-40) is a claim about the diagnostic too: an + // override dropped in silence is a schedule the operator cannot explain from the configuration. + // Same event, same level, same emit path -- only the cause differs, and it names the value. + const events = await captureLogEvents(async () => { + await runWithRetry(GET, oneFailureThenSuccess(), { ...configOf({maxAttempts: 2, fixedDelayMs: 0}), delayOverride: () => Number.NaN, - }; - const dispatch = scriptedDispatch([ - failure(new IoError('first')), - success(countingResponse(200).response), - ]); - - await runWithRetry(GET, dispatch, config); + }); + }); - const overrideFailed = events.filter( - e => e.get('event') === 'http.retry.delayOverrideFailed', - ); - expect(overrideFailed).toHaveLength(1); - expect(overrideFailed[0]?.get('cause')).toBe( - 'delayOverride returned a non-finite delay: NaN', - ); - } finally { - setGlobalLogger(NOOP_LOGGER); - } + const overrideFailed = events.filter( + e => e.get('event') === 'http.retry.delayOverrideFailed', + ); + expect(overrideFailed).toHaveLength(1); + expect(overrideFailed[0]?.get('cause')).toBe( + 'delayOverride returned a non-finite delay: NaN', + ); }); }); From b757a55cdfe7d7059d2ed492b8d34155cacca84b Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:58:13 +0300 Subject: [PATCH 073/100] fix(codec-json): a present Tristate can never carry null (SERDE-14) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `tristate()` returned `present(inner.parse(input) as NonNullable)`. The cast is what the compiler needs — `T` is the caller's, unconstrained, so it cannot know `inner.parse` returns non-null — and it is also the hole: an inner schema that NORMALIZES a value to `null` (a Zod `.transform()`, a "" → null cleanup) produced `{kind: 'present', value: null}`, the fourth state SERDE-14's union exists to forbid. Downstream, `isPresent(t)` then narrows `t.value` to a non-null `T` that is null. Checked at run time after `inner.parse`, and reported as a `DeserializationError`: the input came off the wire, and it is the pairing of that input with that schema that has no Tristate — not a broken invariant of this package. `undefined` is rejected alongside `null`, since `NonNullable` excludes both and a Present of `undefined` is an Absent wearing the wrong label. Deliberately wider than D17's `=== null`, which named the one shape the audit reproduced. The wire-`null` and missing-key branches return before `inner.parse` runs, so a normalizing schema cannot turn a legitimate Null or Absent into a failure; a case pins that. `@throws` added to both `tristate` and `tristateObject`; `codec-json.api.md` is byte-identical, because the report carries signatures and not prose. Found by: audit #67 / #79. D17. --- .../codec-json/src/tristate-schema.test.ts | 49 +++++++++++++++++-- packages/codec-json/src/tristate-schema.ts | 25 ++++++++-- 2 files changed, 67 insertions(+), 7 deletions(-) diff --git a/packages/codec-json/src/tristate-schema.test.ts b/packages/codec-json/src/tristate-schema.test.ts index c6c248c..2c68193 100644 --- a/packages/codec-json/src/tristate-schema.test.ts +++ b/packages/codec-json/src/tristate-schema.test.ts @@ -1,10 +1,16 @@ // SPDX-License-Identifier: MIT // packages/codec-json/src/tristate-schema.test.ts -// Exercises: SERDE-16 (missing → Absent, explicit null → Null, value → Present with element type preserved), -// SERDE-17 (a missing key resolves to Absent via the combinator's own default, not a JSON.parse reviver), -// SERDE-29 (both combinators return frozen schemas, so a shared schema cannot acquire state). +// Exercises: SERDE-14 (three states and only three — a Present can never carry null), SERDE-16 (missing → +// Absent, explicit null → Null, value → Present with element type preserved), SERDE-17 (a missing key resolves +// to Absent via the combinator's own default, not a JSON.parse reviver), SERDE-29 (both combinators return +// frozen schemas, so a shared schema cannot acquire state). import {expect, test} from 'bun:test'; -import {valueOrNull, type Schema, type Tristate} from '@dexpace/core'; +import { + DeserializationError, + valueOrNull, + type Schema, + type Tristate, +} from '@dexpace/core'; import {expectTypeOf} from 'expect-type'; import {MISSING, tristate, tristateObject} from './tristate-schema.js'; @@ -219,3 +225,38 @@ test('both combinators return frozen schemas, like the bundle itself', () => { expect(Object.isFrozen(tristate(identity))).toBe(true); expect(Object.isFrozen(tristateObject({a: identity}))).toBe(true); }); + +// SERDE-14 has three states. `present(null)` is a fourth, and the type system alone cannot keep it +// out: `present` takes `NonNullable`, but `inner.parse`'s declared `T` is unconstrained, so the +// cast that satisfies the compiler is exactly where a normalizing schema slips through +// (audit #67 / #79). +const nullifying: Schema = {parse: () => null}; +const erasing: Schema = {parse: () => undefined}; + +test('an inner schema that normalizes a value to null is a decode failure (SERDE-14)', () => { + expect(() => tristate(nullifying).parse('a value')).toThrow( + DeserializationError, + ); + expect(() => tristate(nullifying).parse('a value')).toThrow( + /present Tristate cannot carry null/, + ); +}); + +test('an inner schema that normalizes a value to undefined is rejected the same way', () => { + expect(() => tristate(erasing).parse('a value')).toThrow( + DeserializationError, + ); +}); + +test('the wire null and missing-key paths still decode ahead of that check (SERDE-16)', () => { + // Neither reaches `inner.parse`, so a normalizing inner schema cannot turn a legitimate Null or + // Absent into a failure. + expect(tristate(nullifying).parse(null).kind).toBe('null'); + expect(tristate(nullifying).parse(MISSING).kind).toBe('absent'); +}); + +test('a nullifying field schema fails the whole tristateObject decode (SERDE-14)', () => { + expect(() => tristateObject({age: nullifying}).parse({age: 30})).toThrow( + DeserializationError, + ); +}); diff --git a/packages/codec-json/src/tristate-schema.ts b/packages/codec-json/src/tristate-schema.ts index d201233..1e473b1 100644 --- a/packages/codec-json/src/tristate-schema.ts +++ b/packages/codec-json/src/tristate-schema.ts @@ -2,6 +2,7 @@ // packages/codec-json/src/tristate-schema.ts import { absent, + DeserializationError, nullValue, present, type Schema, @@ -31,6 +32,8 @@ export const MISSING: unique symbol = Symbol('@dexpace/codec-json.missing'); * @param inner - the schema for the value a Present carries. * @returns a schema producing `Tristate`: a missing-key sentinel or `undefined` yields Absent, a wire * `null` yields Null, anything else runs through `inner` and yields Present. + * @throws DeserializationError when `inner` resolves a present value to `null` or `undefined`. SERDE-14 + * has three states; a Present carrying nothing would be a fourth, and the wire said the key was there. * @public */ export function tristate(inner: Schema): Schema> { @@ -40,9 +43,23 @@ export function tristate(inner: Schema): Schema> { parse(input: unknown): Tristate { if (input === MISSING || input === undefined) return absent(); if (input === null) return nullValue(); - // `as NonNullable`: the null and undefined branches returned above, so the value cannot be - // nullish — a fact the compiler cannot derive through `inner.parse`'s unconstrained `T`. - return present(inner.parse(input) as NonNullable); + const value = inner.parse(input); + // The type system alone cannot hold SERDE-14's third state to non-null values: `present` takes + // `NonNullable`, but `T` here is whatever the caller's schema declares, so a schema that + // NORMALIZES to null — a Zod `.transform()`, a "" → null cleanup — type-checks and produces + // `{kind: 'present', value: null}`, the fourth state the union exists to forbid. Checked at + // run time, and reported as a decode failure rather than an assertion: the input came off the + // wire, and the pairing of that input with that schema is what has no Tristate (audit #67 / + // #79). `undefined` is rejected with it — `NonNullable` excludes both, and a Present of + // `undefined` is Absent wearing the wrong label. + if (value === null || value === undefined) { + throw new DeserializationError( + 'a present Tristate cannot carry null or undefined; the inner schema resolved a wire value to one (SERDE-14)', + ); + } + // `as NonNullable`: the nullish cases have all returned or thrown above, a fact the + // compiler cannot derive through `inner.parse`'s unconstrained `T`. + return present(value); }, }); } @@ -62,6 +79,8 @@ export function tristate(inner: Schema): Schema> { * @param shape - a schema per Tristate-decoded field, keyed by wire name. * @returns a schema producing an object whose named keys are `Tristate`-wrapped. * @throws TypeError when the value being parsed is not a non-null object, or is an array. + * @throws DeserializationError when a field's own schema resolves a present wire value to `null` or + * `undefined`, which is {@link tristate}'s check applied per field (SERDE-14). * @public */ export function tristateObject>>( From a83c206667c13337d4278fa5418c5f899f2647cc Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 11:59:25 +0300 Subject: [PATCH 074/100] fix(core): the logging step says when a body capture fails, and names its config key (#80) Three OBS items in the LOGGING pillar step, all found by audit #67 / #80. **Body-drain failures were silent (OBS-20).** Both capture paths caught and returned an empty result, emitting nothing: a `fileBody()` over a deleted file logged `"http.request.body.preview": ""` and a response whose stream errored mid-drain logged no preview at all, in both cases with nothing anywhere to say why. OBS-20 asks for the opposite of silence -- "catch any exception and re-surface it as a best-effort `http.instrumentation.*` diagnostic". Both catches now emit `http.instrumentation.bodyCaptureFailed` with the direction and the `cause`, through the same `safeEmit` that swallows a secondary failure, and still return the empty capture as before. `captureResponseBody` and `prepareRequestBody` take the `EmitContext` instead of loose granularity/preview arguments -- they needed the logger, and the context already carries it, the cap and the granularity. **`OBS-35` says the SDK MUST NOT bake in a default config key name**, and `resolveGranularity` read `CFG_KEY_LOG_LEVEL` unconditionally. `LoggingStepSettings.configKey` names the key instead; the constant stays as its default, because a required key would mean no caller gets ambient logging without naming one first. The residue -- that the default key IS baked in -- is a `deviations.md` row in a later commit. **The per-request span ended on two paths.** If `end()` threw on the success path it landed in `executePipeline`'s own `catch`, which recorded the exception and called `end()` again. It now ends once, in `handleRequestExecution`'s `finally`, nested inside the scope close so a throwing `end()` -- uncaught by design, per OBS-20/OBS-30 -- cannot also leak the activation scope. `core.api.md` regenerated for `configKey`. Refs #80, #67 --- packages/core/etc/core.api.md | 1 + .../src/observability/logging-step.test.ts | 164 +++++++++++++++++- .../core/src/observability/logging-step.ts | 89 ++++++++-- 3 files changed, 234 insertions(+), 20 deletions(-) diff --git a/packages/core/etc/core.api.md b/packages/core/etc/core.api.md index f203384..cfcf87d 100644 --- a/packages/core/etc/core.api.md +++ b/packages/core/etc/core.api.md @@ -668,6 +668,7 @@ export function loggingStep(settings?: LoggingStepSettings): StepDescriptor; // @public export interface LoggingStepSettings { readonly clock?: Clock | undefined; + readonly configKey?: string | undefined; readonly droppedHeaderPolicy?: DroppedHeaderPolicy | undefined; readonly granularity?: LoggingGranularity | undefined; readonly logger?: Logger | undefined; diff --git a/packages/core/src/observability/logging-step.test.ts b/packages/core/src/observability/logging-step.test.ts index 82894e6..ba7e12d 100644 --- a/packages/core/src/observability/logging-step.test.ts +++ b/packages/core/src/observability/logging-step.test.ts @@ -6,7 +6,11 @@ // OBS-34 (granularity gates log events, not span/metrics), OBS-35 (level resolves from // Configuration, tolerant/case-insensitive), OBS-39 (stable http.request/http.response event names/keys, // url.full always redacted), OBS-20 (a throwing Logger is caught and re-surfaced as http.instrumentation.*; -// a throwing tracer/meter propagates, NOT caught), OBS-36, OBS-37, OBS-38 (body previews). +// a throwing tracer/meter propagates, NOT caught), OBS-36, OBS-37, OBS-38 (body previews), OBS-20's +// body-drain half (a failed request-side probe or response-side drain emits +// http.instrumentation.bodyCaptureFailed and the request still completes), OBS-35's "MUST NOT bake in a +// default config key name" (the configKey setting), OBS-21 (the per-request span ends exactly once, +// even when end() itself throws). import {afterEach, describe, expect, test} from 'bun:test'; import {Headers} from '../http/headers.js'; import {Protocol} from '../http/protocol.js'; @@ -21,6 +25,7 @@ import { ConfigurationBuilder, setGlobalConfiguration, } from '../config/configuration.js'; +import type {Body} from '../body/body.js'; import {stringBody} from '../body/simple-bodies.js'; import type {Logger, LogEvent} from './logger.js'; import type {Meter} from './metrics.js'; @@ -616,3 +621,160 @@ describe('request body preview and validation (OBS-36, OBS-38)', () => { expect(responseEvent?.['http.response.body.preview']).toBeUndefined(); }); }); + +/** A declared-length response whose body errors on the first read -- a drain that cannot finish. */ +function failingResponse(error: Error): Response { + const failingStream = new ReadableStream({ + start(controller) { + controller.error(error); + }, + }); + return Response.newBuilder() + .request(Request.newBuilder().url('https://example.com').build()) + .protocol(Protocol.HTTP_1_1) + .status(Status.of(200)) + .headers( + Headers.newBuilder() + .set('content-type', 'text/plain') + .set('content-length', '100') + .build(), + ) + .body(failingStream) + .build(); +} + +describe('body-drain diagnostics (OBS-20)', () => { + test('a failing response drain emits http.instrumentation.bodyCaptureFailed with the cause', async () => { + const {logger, events} = spyLogger(); + const broke = new Error('stream broke'); + + const response = await send( + loggingStep({logger, granularity: 'body'}), + new FakeTransport([failingResponse(broke)]), + ); + + expect(response.status.code).toBe(200); // OBS-20: the request still completes + const diagnostic = events.find( + e => e.event === 'http.instrumentation.bodyCaptureFailed', + ); + expect(diagnostic).toBeDefined(); + expect(diagnostic?.['http.message.direction']).toBe('response'); + expect(diagnostic?.cause).toBe(broke); + const responseEvent = events.find(e => e.event === 'http.response'); + expect(responseEvent?.['http.response.body.preview']).toBeUndefined(); + }); + + test('a failing request-body probe emits the same diagnostic and still sends the request', async () => { + const {logger, events} = spyLogger(); + const gone = new Error('ENOENT: the file went away'); + // What `fileBody()` over a deleted file looks like to the step: replayable, so the tap probes + // it, and the probe is the thing that fails. + const brokenBody: Body = { + kind: 'file', + mediaType: 'application/octet-stream', + contentLength: 3, + replayable: true, + writeTo: () => Promise.reject(gone), + }; + const request = Request.newBuilder() + .url('https://example.com/upload') + .method('POST') + .body(brokenBody) + .build(); + + const response = await send( + loggingStep({logger, granularity: 'body'}), + new FakeTransport([countingResponse(200).response]), + request, + ); + + expect(response.status.code).toBe(200); + const diagnostic = events.find( + e => e.event === 'http.instrumentation.bodyCaptureFailed', + ); + expect(diagnostic).toBeDefined(); + expect(diagnostic?.['http.message.direction']).toBe('request'); + expect(diagnostic?.cause).toBe(gone); + // The empty capture still ships, as it did before: the tap saw no bytes, and the preview field is + // what the tap holds. What changed is that the empty string is no longer the ONLY evidence -- the + // diagnostic above says which direction failed and why. + const requestEvent = events.find(e => e.event === 'http.request'); + expect(requestEvent?.['http.request.body.preview']).toBe(''); + }); +}); + +describe('the config key the ambient granularity is read from (OBS-35)', () => { + test('configKey names the key, so the baked-in default is not the only one', async () => { + setGlobalConfiguration( + new ConfigurationBuilder() + .put('ACME_SDK_LOG_LEVEL', 'headers') + .put(CFG_KEY_LOG_LEVEL, 'none') + .build(), + ); + const {logger, events} = spyLogger(); + + await send( + loggingStep({logger, configKey: 'ACME_SDK_LOG_LEVEL'}), + new FakeTransport([countingResponse(200).response]), + ); + + expect(events.map(e => e.event)).toEqual(['http.request', 'http.response']); + }); + + test('an explicit granularity still wins over the configured key (OBS-34)', async () => { + setGlobalConfiguration( + new ConfigurationBuilder().put('ACME_SDK_LOG_LEVEL', 'headers').build(), + ); + const {logger, events} = spyLogger(); + + await send( + loggingStep({ + logger, + configKey: 'ACME_SDK_LOG_LEVEL', + granularity: 'none', + }), + new FakeTransport([countingResponse(200).response]), + ); + + expect(events).toHaveLength(0); + }); +}); + +describe('the per-request span ends exactly once (OBS-21, OBS-29)', () => { + test('an end() that throws on the success path is not called a second time', async () => { + const {logger} = spyLogger(); + let ends = 0; + const endFailed = new Error('end failed'); + const exceptions: unknown[] = []; + const span = { + isRecording: true, + setAttribute: () => span, + recordException: (error: unknown) => { + exceptions.push(error); + return span; + }, + end: (): void => { + ends += 1; + throw endFailed; + }, + }; + + let caught: unknown; + try { + await send( + loggingStep({ + logger, + granularity: 'headers', + tracerFactory: () => ({startSpan: () => span}), + }), + new FakeTransport([countingResponse(200).response]), + ); + } catch (error) { + caught = error; + } + + expect(caught).toBe(endFailed); + expect(ends).toBe(1); + expect(exceptions).toEqual([]); + }); +}); diff --git a/packages/core/src/observability/logging-step.ts b/packages/core/src/observability/logging-step.ts index 11f60d7..dfd5cd2 100644 --- a/packages/core/src/observability/logging-step.ts +++ b/packages/core/src/observability/logging-step.ts @@ -59,8 +59,24 @@ export interface LoggingStepSettings { * Default: 'info' (failures always emit at 'error'). */ readonly severity?: LogLevel | undefined; - /** Granularity of logging (default: resolved from Configuration via CFG_KEY_LOG_LEVEL, fallback 'none'). */ + /** Granularity of logging (default: resolved from Configuration via {@link LoggingStepSettings.configKey}, fallback 'none'). */ readonly granularity?: LoggingGranularity | undefined; + /** + * The configuration key the ambient granularity is read from when `granularity` is omitted (OBS-35). + * + * OBS-35 says the SDK MUST NOT bake in a default key name, and `CFG_KEY_LOG_LEVEL` + * (`DEXPACE_LOG_LEVEL`) is exactly that — CFG-14's well-known key, kept as the default because a + * required key would mean no caller gets ambient logging without naming one first. Set this to read + * a host application's own key instead; the resolution is the same layered, tolerant one either way. + * + * Nothing installs a configuration that reads the process environment by default: the global slot + * starts empty (CFG-13), so `setGlobalConfiguration(defaultConfiguration())` is the wiring that + * makes any environment variable — this one included — reachable. See + * `docs/sdk-documentation/pipelines.md`, "Turning logging on from the environment". + * + * @defaultValue {@link CFG_KEY_LOG_LEVEL} + */ + readonly configKey?: string | undefined; /** Byte limit for request/response body previews (default: 8192). */ readonly previewSizeBytes?: number | undefined; /** Optional custom tracer factory. */ @@ -82,7 +98,7 @@ function resolveGranularity(settings: LoggingStepSettings): LoggingGranularity { return 'none'; } const raw = getGlobalConfiguration() - .getString(CFG_KEY_LOG_LEVEL, 'none') + .getString(settings.configKey ?? CFG_KEY_LOG_LEVEL, 'none') ?.trim() .toLowerCase(); if (raw === 'headers' || raw === 'body') return raw; @@ -162,6 +178,32 @@ function safeEmit(logger: Logger, build: () => void): void { } } +/** + * OBS-20's body-drain clause. A capture failure is contained -- the request completes and the caller's + * body is untouched -- but containment is not silence: the failure re-surfaces as a best-effort + * `http.instrumentation.*` diagnostic, through the same {@link safeEmit} every other emission uses, so a + * secondary failure while reporting it is swallowed in turn. + * + * `verbose`, the level its sibling `http.instrumentation.logFailure` already emits at: nothing about the + * request changed, and what was lost is a diagnostic preview. Before 2026-09-05 both catches returned an + * empty capture and emitted nothing at all, so a `fileBody()` over a deleted file logged + * `"http.request.body.preview": ""` with no trace of why (audit #67 / #80). + */ +function emitBodyCaptureFailure( + logger: Logger, + direction: 'request' | 'response', + error: unknown, +): void { + safeEmit(logger, () => { + logger + .atLevel('verbose') + .event('http.instrumentation.bodyCaptureFailed') + .field('http.message.direction', direction) + .cause(error) + .emit(); + }); +} + /** * Stable identity symbol for the LOGGING pillar step. * @@ -273,12 +315,13 @@ function resolveTracer( /** Captures response body preview safely when content-length is declared (OBS-36, OBS-37). */ async function captureResponseBody( response: Response, - previewSizeBytes: number, + context: EmitContext, ): Promise<{ readonly response: Response; readonly preview: string | undefined; readonly size: number | undefined; }> { + const previewSizeBytes = context.previewSizeBytes; try { const hasContentLength = response.headers.has('content-length'); if (response.body === null || !hasContentLength) { @@ -306,33 +349,34 @@ async function captureResponseBody( ); const size = snap.length > 0 ? snap.length : undefined; return {response: captured, preview, size}; - } catch { - // OBS-20: body-drain failure must never fail the request + } catch (error) { + // OBS-20: a body-drain failure must never fail the request -- and must not vanish either. + emitBodyCaptureFailure(context.logger, 'response', error); return {response, preview: undefined, size: undefined}; } } async function prepareRequestBody( request: Request, - granularity: LoggingGranularity, - previewSizeBytes: number, + context: EmitContext, ): Promise<{ readonly outbound: Request; readonly preview: string | undefined; readonly size: number | undefined; }> { - if (granularity !== 'body' || request.body === undefined) { + if (context.granularity !== 'body' || request.body === undefined) { return {outbound: request, preview: undefined, size: undefined}; } - const logged = withRequestLogging(request.body, previewSizeBytes); + const logged = withRequestLogging(request.body, context.previewSizeBytes); if (logged.replayable) { const probeSink = new WritableStream({ write: () => undefined, }); try { await logged.writeTo(probeSink); - } catch { - // probe error is ignored per OBS-20 + } catch (error) { + // OBS-20: the probe is diagnostic-only, so its failure is contained -- and reported. + emitBodyCaptureFailure(context.logger, 'request', error); } } const snap = logged.snapshot(); @@ -368,7 +412,7 @@ interface PipelineExecutionArgs { async function executePipeline(args: PipelineExecutionArgs): Promise { const {ctx, plan, outbound, startedAt, span} = args; - const {emitContext, instruments, previewSizeBytes} = plan; + const {emitContext, instruments} = plan; try { const response = await ctx.next(outbound); const { @@ -376,7 +420,7 @@ async function executePipeline(args: PipelineExecutionArgs): Promise { preview, size, } = emitContext.granularity === 'body' - ? await captureResponseBody(response, previewSizeBytes) + ? await captureResponseBody(response, emitContext) : {response, preview: undefined, size: undefined}; const elapsedMs = instruments.clock.monotonic() - startedAt; @@ -395,7 +439,6 @@ async function executePipeline(args: PipelineExecutionArgs): Promise { size, }); - span.end(); return captured; } catch (caught) { const error = toError(caught); @@ -412,7 +455,6 @@ async function executePipeline(args: PipelineExecutionArgs): Promise { emitFailureEvent(emitContext, {error, elapsedMs}); span.recordException(error); - span.end(); throw caught; } } @@ -422,7 +464,7 @@ async function handleRequestExecution( ctx: StepContext, plan: ExecutionPlan, ): Promise { - const {settings, emitContext, instruments, previewSizeBytes} = plan; + const {settings, emitContext, instruments} = plan; const tracer = resolveTracer(settings, ctx); const span = tracer.startSpan('http.client.request'); const scope = activateSpanForCorrelation(span); @@ -431,8 +473,7 @@ async function handleRequestExecution( try { const {outbound, preview, size} = await prepareRequestBody( request, - emitContext.granularity, - previewSizeBytes, + emitContext, ); emitRequestEvent(emitContext, outbound, {preview, size}); return await executePipeline({ @@ -443,7 +484,17 @@ async function handleRequestExecution( span, }); } finally { - scope.close(); + // ONE exit for `end()`, and it is here rather than on each path inside `executePipeline`: an + // `end()` that threw on the success path used to land in that function's own `catch`, which + // recorded the exception and called `end()` a second time on a span the tracer had already + // closed (OBS-21's idempotent-end clause is the tracer's promise, not this step's licence). + // Nested rather than sequential so a throwing `end()` -- which OBS-20 deliberately does not + // catch, because OBS-30 makes it the SPI's promise not to -- still cannot leak the scope. + try { + span.end(); + } finally { + scope.close(); + } } } From ead5a629082e3abfa1af6110b606f5413089f38c Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:00:14 +0300 Subject: [PATCH 075/100] fix(core): report an SSE release failure once, not once per channel (SSE-30) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A mid-stream read failure whose release also fails was reported twice. `#iterate`'s catch called `#releaseWithInFlightError`, which attaches the close error to the thrown one as `suppressed` — and then the `finally` ran `#releaseQuietly` anyway, which awaited the same already-rejected `#closing` promise and handed the close error to `onReleaseFailure` as well. A consumer with a logger wired into that hook saw one failure logged out-of-band and caught the same failure a second time on the error it was already handling. SSE-30's scope is explicit: an automatic clean-terminal path is "natural end-of-stream or a done-sentinel, **with no error in flight**" (`docs/product-spec/13-server-sent-events-and-streaming.md:52`). With an error in flight there is something to attach the failure to, and SSE-29 says to attach it there. The `finally` now skips the quiet release when the catch already released — a local flag on the one generator activation that reads it, rather than a field, since nothing outside `#iterate` has a question to ask of it. `#releaseQuietly` keeps reporting unconditionally, which is what its name promises; the clean-terminal case and the explicit-`close()` case are unchanged. The hook's `@public` TSDoc now states the scope it always had. `core.api.md` is unchanged: the report carries signatures, not prose. Test: mid-stream failure, failing close, hook installed — one report, on the thrown error. Found by: audit #67 / #79. D17. --- packages/core/src/sse/stream.test.ts | 41 ++++++++++++++++++++++++++++ packages/core/src/sse/stream.ts | 22 +++++++++++++-- 2 files changed, 61 insertions(+), 2 deletions(-) diff --git a/packages/core/src/sse/stream.test.ts b/packages/core/src/sse/stream.test.ts index 7f3b866..1c361a4 100644 --- a/packages/core/src/sse/stream.test.ts +++ b/packages/core/src/sse/stream.test.ts @@ -169,6 +169,47 @@ test('a mid-stream read failure releases before propagating, with the close erro expect(closeCount).toBe(1); }); +test('a release failure during an in-flight error is reported exactly once (SSE-29, SSE-30)', async () => { + const readFailure = new IoError('socket reset'); + const closeFailure = new IoError('close failed too'); + const reported: unknown[] = []; + let closeCount = 0; + const web = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('data: a\n\n')); + controller.error(readFailure); + }, + }); + const stream = new SseStream( + new SseParser(BufferedSource.overStream(web)), + { + close(): Promise { + closeCount += 1; + return Promise.reject(closeFailure); + }, + }, + {onReleaseFailure: e => reported.push(e)}, + ); + + let caught: unknown; + try { + for await (const event of stream) { + void event; + } + } catch (e: unknown) { + caught = e; + } + + const suppressed = caught as SuppressedErrorLike; + expect(suppressed.error).toBe(readFailure); + expect(suppressed.suppressed).toBe(closeFailure); + // SSE-30 scopes the hook to the automatic CLEAN terminal path. With an error already in flight the + // release failure is on the thrown error, and calling the hook as well makes one failure arrive + // twice — once in whatever logs `onReleaseFailure`, once in whatever logs the caught error. + expect(reported).toEqual([]); + expect(closeCount).toBe(1); +}); + test('sseStreamFrom binds lifecycle to the response body (SSE-32)', async () => { let responseClosed = 0; const response = { diff --git a/packages/core/src/sse/stream.ts b/packages/core/src/sse/stream.ts index 980f4ad..919827e 100644 --- a/packages/core/src/sse/stream.ts +++ b/packages/core/src/sse/stream.ts @@ -28,6 +28,11 @@ export interface SseStreamOptions { * Called when a release fails on a clean automatic terminal path, where SSE-30 requires the failure to be * reported out-of-band and swallowed rather than thrown (throwing would discard events already delivered). * + * Called on **that** path only, which is SSE-30's own scope: "no error in flight". A release that fails + * while an error is already propagating is attached to that error as `suppressed` instead, and a release + * that fails during an explicit `close()` rejects that call — reporting either one here as well would + * deliver a single failure twice. + * * Defaults to a no-op. Phase 7 wires a real `Logger` in here without reshaping this class — the same * "mechanism now, wiring later" split Phase 3b used for its logging tees. */ @@ -121,6 +126,9 @@ export class SseStream implements AsyncIterable { } async *#iterate(): AsyncGenerator { + // Whether the catch below already released and already accounted for a release failure. A local, + // not a field: it is read exactly once, by the `finally` of this one generator activation. + let releasedWithError = false; try { for (;;) { // A close observed between pulls ends iteration cleanly, without reading from a torn-down resource. @@ -132,10 +140,17 @@ export class SseStream implements AsyncIterable { } catch (e: unknown) { // SSE-29: release BEFORE the error propagates, and attach a release failure as suppressed rather than // letting it mask the real cause. + releasedWithError = true; await this.#releaseWithInFlightError(e); } finally { // Covers clean end-of-stream and early `break` (the runtime calls `.return()`, which runs this block). - await this.#releaseQuietly(); + // + // Skipped after the catch, which has already released. Running it there awaited the same rejected + // `#closing` promise and handed the close failure to `onReleaseFailure` as well — so one failure was + // reported twice, once out-of-band and once as `suppressed` on the error the consumer catches. SSE-30 + // scopes the hook to the automatic CLEAN terminal path, where there is nothing to throw to; with an + // error in flight there is (audit #67 / #79). + if (!releasedWithError) await this.#releaseQuietly(); } } @@ -176,7 +191,10 @@ export class SseStream implements AsyncIterable { } } - /** SSE-29 / SSE-36: an error is already in flight, so it stays primary and the close error is suppressed. */ + /** + * SSE-29 / SSE-36: an error is already in flight, so it stays primary and the close error is suppressed — + * and NOT also handed to `onReleaseFailure`, which is the clean-terminal path's channel. + */ async #releaseWithInFlightError(primary: unknown): Promise { this.#closed = true; const releasePromise = (this.#closing ??= this.#resource.close()); From b2e1d9b860bc22f0a64e14fb9a55c09066c4a704 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:02:20 +0300 Subject: [PATCH 076/100] docs(core): TSDoc for the reachable operation span, and for what redactUrl renders (#80) `InstrumentationBundle.tracerFactory` documented a consumer with no way to supply one. It now says what the two consumers ask it for -- `send()`'s one-per-operation span (`OBS-29`) and the LOGGING step's per-transmission children, the latter under `CTX-16`'s operation name -- and how a caller reaches either: `createInstrumentationBundle(factory)` into `PipelineOptions.instrumentation`. Its three stale `file:line` citations, and the interface note's three, are re-derived against the current tree. `redactUrl` states that its output is re-rendered from the parsed `URL`, so WHATWG normalisation (host case, a default port, an empty path, percent-encoding) shows in the result and a log line need not match the request line byte for byte. Audited and left as is: re-rendering the caller's authority by hand would be a second URL renderer maintained against WHATWG, for no gain in what OBS-11..15 asks for. One `redaction.test.ts` case pins the two normalisations the note names, so the claim is checked rather than asserted. Refs #80, #67 --- packages/core/src/context/instrumentation.ts | 23 +++++++++++++++---- .../core/src/observability/redaction.test.ts | 9 ++++++++ packages/core/src/observability/redaction.ts | 10 ++++++++ 3 files changed, 37 insertions(+), 5 deletions(-) diff --git a/packages/core/src/context/instrumentation.ts b/packages/core/src/context/instrumentation.ts index cabc79a..5eb7b08 100644 --- a/packages/core/src/context/instrumentation.ts +++ b/packages/core/src/context/instrumentation.ts @@ -10,9 +10,9 @@ import {NOOP_SPAN} from '../observability/span.js'; * `activeSpan` and `tracerFactory` are `unknown` rather than a `Span`/`Tracer`. The original reason — * "nothing in the package consumes either yet, pending Phase 7a" — expired when Phase 7a landed * (`bd37a08`) and shipped `Span` and `Tracer` in `observability/tracing.ts`. That phase did not narrow - * these two: `tracerFactory` is consumed, by `pipeline/runtime.ts:52` and - * `observability/logging-step.ts:263`, both of which reach a `Tracer` through a cast; - * `createInstrumentationBundle` fills `activeSpan` (`observability/tracing.ts:222`), and nothing in + * these two: `tracerFactory` is consumed, by `pipeline/runtime.ts:60-65` and + * `observability/logging-step.ts:305-312`, both of which reach a `Tracer` through a cast; + * `createInstrumentationBundle` fills `activeSpan` (`observability/tracing.ts:191`), and nothing in * this package reads it back. * * They stay `unknown` because narrowing a published member is a breaking change rather than a @@ -46,11 +46,24 @@ export interface InstrumentationBundle { /** * Returns the tracer to open `operationName`'s span from — a **tracer**, not a started span. Every * consumer in this package narrows the result and calls `startSpan()` on it itself - * (`packages/core/src/pipeline/runtime.ts:52-57`, `observability/logging-step.ts:263-269`), and + * (`packages/core/src/pipeline/runtime.ts:60-65`, `observability/logging-step.ts:305-312`), and * `createInstrumentationBundle` supplies a `(operationName: string) => Tracer` - * (`observability/tracing.ts:212,223`). A no-op returning `undefined` when tracing is disabled; + * (`observability/tracing.ts:180,191`). A no-op returning `undefined` when tracing is disabled; * both consumers substitute `NOOP_TRACER` for that `undefined`. * + * **It is asked twice per call, for two different scopes.** `Runtime.send()` asks for + * `'http.client.operation'`'s tracer once per call and opens `OBS-29`'s one-per-operation span from + * it, outside every pillar, so a retry attempt and a redirect hop stay inside it. The LOGGING pillar + * step asks again per transmission, under `CTX-16`'s operation name when the pipeline was built with + * one and `'http.client.request'` otherwise, and its spans are children of the first. Returning one + * shared tracer for both is fine; `OBS-29`'s 1:1 clause is about the operation *span*, which + * `send()` opens exactly once whatever this returns. + * + * A consumer reaches this by building a bundle with `createInstrumentationBundle(tracerFactory)` and + * passing it as `PipelineOptions.instrumentation` — to `new PipelineBuilder(transport, options)` or + * to `standardResilience(transport, options)`. Before 2026-09-05 there was no public route: every + * pipeline a consumer could build carried the no-op bundle (audit #67 / #80). + * * PROVISIONAL: the return type is `unknown`, and narrowing it is a version-bump decision — see this * interface's own note for why Phase 7a landing did not settle it. * diff --git a/packages/core/src/observability/redaction.test.ts b/packages/core/src/observability/redaction.test.ts index 6374a3c..8ce88ff 100644 --- a/packages/core/src/observability/redaction.test.ts +++ b/packages/core/src/observability/redaction.test.ts @@ -102,6 +102,15 @@ describe('redactUrl: delimiters and total safety (OBS-14..15)', () => { expect(redactUrl('not a url at all ###')).toBe('[malformed url]'); }); + test('the output is re-rendered from the parsed URL, so WHATWG normalisation shows (OBS-14)', () => { + // Pinned, not fixed: the result is assembled from `URL`'s components, so host case, a default + // port and an empty path normalise on the way through. Documented on `redactUrl` as inherent to + // parsing, and left as is by audit #67 / #80 -- re-rendering the caller's authority by hand would + // be a second URL renderer for no gain in what OBS-11..15 asks for. + expect(redactUrl('https://EXAMPLE.com:443')).toBe('https://example.com/'); + expect(redactUrl('http://Example.COM:80/p')).toBe('http://example.com/p'); + }); + test('property: never throws for any string', () => { fc.assert( fc.property(fc.string(), value => { diff --git a/packages/core/src/observability/redaction.ts b/packages/core/src/observability/redaction.ts index 5d8a137..b398840 100644 --- a/packages/core/src/observability/redaction.ts +++ b/packages/core/src/observability/redaction.ts @@ -52,6 +52,16 @@ function hasHashDelimiter(input: URL | string): boolean { /** * Redacts sensitive components from a URL according to spec rules (OBS-11..15). * + * **The result is a re-rendered URL, not the caller's string with holes in it.** Every input goes + * through WHATWG `URL`, and the output is assembled from its parsed components, so the normalisations + * parsing performs come with it: the host is lower-cased, a default port for the scheme + * (`https://h:443/`) is dropped, a missing path becomes `/`, and percent-encoding is canonicalised. + * A log line therefore need not match the request line byte for byte. That is inherent to parsing and + * is left as is deliberately (audit #67 / #80): re-rendering the original authority by hand would mean + * a second URL renderer in this package, maintained against WHATWG, for no gain in what OBS-11..15 + * actually asks for — that userinfo, non-allow-listed query values and fragment values do not reach a + * log. Compare identity elsewhere; this is for humans and log pipelines. + * * @param input - the URL or string to redact. * @param queryAllowList - set of allowed query parameter names (default: \{api-version\}). * @returns the redacted URL string, or '[malformed url]' if parsing fails. From 0bcc6f2646eeeb06238029e311035b21c1ad8d5f Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:02:30 +0300 Subject: [PATCH 077/100] docs(retry): RetryStepOptions.delayOverride states the failure semantics `delayOverride` is the public entry point for RETRY-40's non-fatal-override behaviour, and its TSDoc documented only the `undefined` fall-through. A caller had no way to learn from the shipped `.d.ts` that a throw is ignored, and after this branch's engine fix, that a non-finite result is ignored the same way. Backticked prose, no `{@link}` -- `http.retry.delayOverrideFailed` is an event name, not a symbol, and a link out of a `@public` block is what `api:ci` fails on where `api:local` warns. `core.api.md` regenerated: byte-identical. The report records the declaration and its release tag, not the prose body, and no signature moved. The sentence does reach the shipped artifact -- verified in `packages/core/dist/retry/retry-step.d.ts`. Round 2 of #78; the maintainer added this file to the task partition. Refs #78, #67 --- packages/core/src/retry/retry-step.ts | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/packages/core/src/retry/retry-step.ts b/packages/core/src/retry/retry-step.ts index 9a58392..2680959 100644 --- a/packages/core/src/retry/retry-step.ts +++ b/packages/core/src/retry/retry-step.ts @@ -44,6 +44,11 @@ export interface RetryStepOptions { * RETRY-39's caller override: returns the delay in milliseconds to use for `attempt`, or * `undefined` to fall through to the configured schedule for that attempt. * + * A throw, or a non-finite result, is ignored: the configured schedule is used for that attempt + * and `http.retry.delayOverrideFailed` is logged at warning level (RETRY-40). Neither aborts the + * retry loop. A finite negative is honored as a delay and continues inline without a timer + * (RETRY-31). + * * @defaultValue absent, so every attempt uses the configured schedule */ readonly delayOverride?: From 80e16d7a8c5b6adbc63c83fa3c48172395c339e3 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:03:29 +0300 Subject: [PATCH 078/100] fix(core): a pagination cursor with no UTF-8 form fails inside the error tree (PAGE-22) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The last `URIError` escaping core, handed over by #76. `spliceQueryParam` and `readQueryParam` share HTTP-29's component encoder, which is `encodeURIComponent`, which throws a bare `URIError: URI malformed` on a string carrying an unpaired surrogate. Unlike the call sites #76 closed, this one needs no caller mistake: the cursor is SERVER-supplied and `{"next":"\ud800"}` is well-formed JSON that `JSON.parse` hands back verbatim, so a hostile or merely broken server could make a walk die with a platform error naming neither the parameter nor the page. Repro from D17: `spliceQueryParam(new URL('https://h/?a=1'), 'cursor', '\uD800')`. Both functions now reject a non-well-formed name or value with `UrlConstructionError` — the class `QueryParamsBuilder.add` throws for the same input, through the same `hasLoneSurrogate` predicate, so the two cannot drift. The message names the parameter and never echoes the value: a cursor is opaque server state and can carry a session token (D8's rule, applied here). A well-formed surrogate pair stays ordinary text. The engine already closes the response on this path — a strategy's `parse` rejecting is PAGE-13's case — so nothing leaks. Tests: seven direct cases, two through `cursorStrategy` / `pageNumberStrategy` asserting the error is in the `DexpaceError` tree, and a property over surrogate-bearing names and values that admits only `UrlConstructionError`. All were red, on `URIError: String contained an illegal UTF-16 sequence`. `@throws` added to both public strategies and both internal helpers; `core.api.md` is unchanged, since the report carries signatures and not prose. Found by: audit #67 / #79. D17, D14's rule for which class. --- .../pagination/query-splice.property.test.ts | 40 +++++++++++- .../core/src/pagination/query-splice.test.ts | 61 ++++++++++++++++++- packages/core/src/pagination/query-splice.ts | 39 ++++++++++++ .../core/src/pagination/strategies.test.ts | 41 ++++++++++++- packages/core/src/pagination/strategies.ts | 7 +++ 5 files changed, 183 insertions(+), 5 deletions(-) diff --git a/packages/core/src/pagination/query-splice.property.test.ts b/packages/core/src/pagination/query-splice.property.test.ts index f6c6761..56a51af 100644 --- a/packages/core/src/pagination/query-splice.property.test.ts +++ b/packages/core/src/pagination/query-splice.property.test.ts @@ -1,7 +1,8 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/query-splice.property.test.ts -import {test} from 'bun:test'; +import {expect, test} from 'bun:test'; import fc from 'fast-check'; +import {UrlConstructionError} from '../http/errors.js'; import {readQueryParam, spliceQueryParam} from './query-splice.js'; /** @@ -51,3 +52,40 @@ test('write-then-read is the identity for any value (PAGE-22)', () => { }), ); }); + +/** + * Strings that mix ordinary query text with UNPAIRED surrogate code units — the same generator + * `http/query-params.test.ts` uses, for the same reason: `fc.string()`'s default unit is printable + * ASCII, so the `URIError` path would otherwise go ungenerated. That is exactly why the identity + * property above never caught it. + */ +const surrogateBearingString = fc.string({ + unit: fc.oneof( + fc.constantFrom('a', 'b', ' ', '=', '&', '%', '+', '\u{1F600}'), + fc + .integer({min: 0xd800, max: 0xdfff}) + .map(code => String.fromCharCode(code)), + ), + maxLength: 8, +}); + +test('no URIError escapes the splice or the read, whatever a server sent (PAGE-22)', () => { + fc.assert( + fc.property( + surrogateBearingString, + surrogateBearingString, + (name, value) => { + const url = new URL('https://h/p?a=1'); + try { + const out = spliceQueryParam(url, name, value); + expect(readQueryParam(out, name)).toBe(value); + } catch (e: unknown) { + // The one sanctioned failure: inside the error tree, from the call that was handed the + // value. A `URIError` here means the guard was bypassed. + expect(e).toBeInstanceOf(UrlConstructionError); + } + }, + ), + {numRuns: 500}, + ); +}); diff --git a/packages/core/src/pagination/query-splice.test.ts b/packages/core/src/pagination/query-splice.test.ts index dc1ca1c..ca546a7 100644 --- a/packages/core/src/pagination/query-splice.test.ts +++ b/packages/core/src/pagination/query-splice.test.ts @@ -1,9 +1,10 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/query-splice.test.ts // Exercises: PAGE-21 (verbatim splice, untargeted params byte-for-byte), PAGE-22 (RFC 3986 component encoding, -// literal + is data), PAGE-23 (replace-first / append / remove, order preserved), PAGE-24 (non-query components -// preserved exactly). -import {expect, test} from 'bun:test'; +// literal + is data; a component with no UTF-8 form is rejected as UrlConstructionError), PAGE-23 +// (replace-first / append / remove, order preserved), PAGE-24 (non-query components preserved exactly). +import {describe, expect, test} from 'bun:test'; +import {UrlConstructionError} from '../http/errors.js'; import {readQueryParam, spliceQueryParam} from './query-splice.js'; const at = (href: string): URL => new URL(href); @@ -139,3 +140,57 @@ test('stray empty segments are skipped, matching HTTP-31 query parsing', () => { query(spliceQueryParam(at('https://h/p?a=1&&b=2&page=1'), 'page', '2')), ).toBe('a=1&b=2&page=2'); }); + +describe('a cursor with no UTF-8 form is rejected here, not inside encodeURIComponent (PAGE-22)', () => { + // The splice shares `HTTP-29`'s component encoder, and `encodeURIComponent` throws a bare + // `URIError: URI malformed` on a string carrying an unpaired surrogate. A cursor is SERVER + // -supplied — `{"next":"\ud800"}` is well-formed JSON — so this is reachable without any caller + // mistake, and until audit #67 / #79 it escaped the `DexpaceError` tree entirely. #76 closed the + // same hole at `QueryParamsBuilder.add` and left this one named. + const LONE_HIGH = '\uD800'; + const LONE_LOW = '\uDFFF'; + + test.each([ + ['a lone high surrogate', LONE_HIGH], + ['a lone low surrogate', LONE_LOW], + ['a lone surrogate inside a longer cursor', `ok${LONE_HIGH}ok`], + ])('spliceQueryParam rejects %s as a value', (_label, value) => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', value), + ).toThrow(UrlConstructionError); + }); + + test('the message names the parameter and never echoes the value', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', `secret${LONE_HIGH}`), + ).toThrow(/value of query parameter "cursor"/); + expect(() => + spliceQueryParam(at('https://h/p?a=1'), 'cursor', `secret${LONE_HIGH}`), + ).not.toThrow(/secret/); + }); + + test('spliceQueryParam rejects a lone surrogate in the parameter NAME', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), LONE_HIGH, '2'), + ).toThrow(UrlConstructionError); + }); + + test('readQueryParam rejects a lone surrogate in the parameter NAME', () => { + expect(() => readQueryParam(at('https://h/p?a=1'), LONE_HIGH)).toThrow( + UrlConstructionError, + ); + }); + + test('removing a parameter still validates the name', () => { + expect(() => + spliceQueryParam(at('https://h/p?a=1'), LONE_HIGH, undefined), + ).toThrow(UrlConstructionError); + }); + + test('a well-formed surrogate PAIR is ordinary text and splices normally', () => { + // Rejecting this too would make the rule "no astral characters", which PAGE-22 does not say. + const out = spliceQueryParam(at('https://h/p?a=1'), 'cursor', '\u{1F600}'); + expect(query(out)).toBe('a=1&cursor=%F0%9F%98%80'); + expect(readQueryParam(out, 'cursor')).toBe('\u{1F600}'); + }); +}); diff --git a/packages/core/src/pagination/query-splice.ts b/packages/core/src/pagination/query-splice.ts index ba38d77..754cc6a 100644 --- a/packages/core/src/pagination/query-splice.ts +++ b/packages/core/src/pagination/query-splice.ts @@ -1,9 +1,40 @@ // SPDX-License-Identifier: MIT // packages/core/src/pagination/query-splice.ts +import {UrlConstructionError} from '../http/errors.js'; import { decodeQueryComponent, encodeQueryComponent, } from '../http/query-params.js'; +import {hasLoneSurrogate} from '../http/rfc3986.js'; + +/** + * `encodeQueryComponent` is `encodeURIComponent`, which throws a bare `URIError: URI malformed` on a string + * carrying an unpaired surrogate — such a string has no UTF-8 form, so RFC 3986 percent-encoding is undefined + * for it (PAGE-22, HTTP-29). + * + * Reachable here without any caller mistake, which is what separates this call site from the others #76 closed: + * a cursor is SERVER-supplied, and `{"next":"\ud800"}` is well-formed JSON that `JSON.parse` hands back + * verbatim. The failure was a bare `URIError` from inside a strategy's `parse`, outside the `DexpaceError` tree + * and naming neither the parameter nor the page it came from (audit #67 / #79). + * + * The same class `QueryParamsBuilder.add` throws for the same input, since it is the same defect in the same + * encoder; `hasLoneSurrogate` is the single-sourced predicate, so the two cannot drift. The value itself is + * never echoed — a cursor is opaque server state and can carry a session token. + */ +function requireEncodable( + what: 'name' | 'value', + parameterName: string, + text: string, +): void { + if (!hasLoneSurrogate(text)) return; + const subject = + what === 'name' + ? 'a query parameter name' + : `the value of query parameter "${parameterName}"`; + throw new UrlConstructionError( + `${subject} contains an unpaired surrogate and cannot be percent-encoded`, + ); +} /** * Rewrite one query parameter, splicing the raw query string rather than re-rendering it (PAGE-21–PAGE-24). @@ -19,6 +50,9 @@ import { * Passing `undefined` removes the parameter. Setting replaces the first occurrence in place and drops later * duplicates — the single-value convention paging parameters follow. Everything else is copied byte-for-byte. * + * @throws UrlConstructionError when `name` or `value` carries an unpaired surrogate, and so has no + * percent-encoded form. + * * @internal */ export function spliceQueryParam( @@ -26,6 +60,8 @@ export function spliceQueryParam( name: string, value: string | undefined, ): URL { + requireEncodable('name', name, name); + if (value !== undefined) requireEncodable('value', name, value); const encodedName = encodeQueryComponent(name); const segments = splitQuery(url.search); @@ -60,9 +96,12 @@ export function spliceQueryParam( * A literal `+` reads back as `+`, `%20` as a space, a value-less flag as the empty string, and an absent name * as `undefined`. First match wins. * + * @throws UrlConstructionError when `name` carries an unpaired surrogate, and so has no percent-encoded form. + * * @internal */ export function readQueryParam(url: URL, name: string): string | undefined { + requireEncodable('name', name, name); const encodedName = encodeQueryComponent(name); for (const segment of splitQuery(url.search)) { if (nameOf(segment) !== encodedName) continue; diff --git a/packages/core/src/pagination/strategies.test.ts b/packages/core/src/pagination/strategies.test.ts index 513f522..8eebdf1 100644 --- a/packages/core/src/pagination/strategies.test.ts +++ b/packages/core/src/pagination/strategies.test.ts @@ -5,8 +5,10 @@ // PAGE-18/19/20 (link header: rel=next, RFC 3986 reference resolution, query-only reference preserves the path, // unresolvable target ends the stream without throwing, and the spec's own `` conformance fixture // resolving as a relative reference instead -- recorded as a deliberate reading in docs/deviations.md under -// "Deviations recorded outside a phase" (2026-09-04, audit #67 / #69)). +// "Deviations recorded outside a phase" (2026-09-04, audit #67 / #69)), PAGE-22 (a server-supplied cursor with +// no UTF-8 form fails inside the error tree). import {expect, test} from 'bun:test'; +import {DexpaceError, UrlConstructionError} from '../http/errors.js'; import type {Request} from '../http/request.js'; import type {Response} from '../http/response.js'; import { @@ -294,3 +296,40 @@ test('one strategy instance is safe across two concurrent walks (PAGE-5)', async expect(first.nextRequest?.url.search).toBe('?page=2'); expect(second.nextRequest?.url.search).toBe('?page=10'); }); + +// ---- a server-supplied component with no UTF-8 form (PAGE-22, audit #67 / #79) ---- + +test('a cursor carrying an unpaired surrogate fails as UrlConstructionError, not URIError', async () => { + // `{"next":"\ud800"}` is well-formed JSON, so `extract` can hand one back without the caller + // having done anything wrong. Before the fix this surfaced as a bare `URIError: URI malformed` + // from inside `encodeURIComponent`, outside the `DexpaceError` tree. + const strategy = cursorStrategy({ + extract: () => Promise.resolve({items: ['a'], cursor: 'next\uD800'}), + }); + + let caught: unknown; + try { + await strategy.parse(response({}), template('https://api.test/items')); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(UrlConstructionError); + expect(caught).toBeInstanceOf(DexpaceError); +}); + +test('a page-number parameter name carrying an unpaired surrogate fails the same way', async () => { + const strategy = pageNumberStrategy({ + extract: () => Promise.resolve(['a']), + parameterName: 'p\uD800', + }); + + let caught: unknown; + try { + await strategy.parse(response({}), template('https://api.test/items')); + } catch (e: unknown) { + caught = e; + } + + expect(caught).toBeInstanceOf(UrlConstructionError); +}); diff --git a/packages/core/src/pagination/strategies.ts b/packages/core/src/pagination/strategies.ts index d2f0ecf..84bf409 100644 --- a/packages/core/src/pagination/strategies.ts +++ b/packages/core/src/pagination/strategies.ts @@ -23,6 +23,10 @@ function withUrl(template: Request, url: URL): Request { * A `null` **or empty** cursor ends the stream — both, because a server returning `""` for "no more pages" is * common enough that treating it as a real cursor produces an infinite walk. * + * @throws UrlConstructionError from `parse` when the cursor the server sent, or `parameterName`, carries an + * unpaired surrogate: such a string has no UTF-8 form and so no percent-encoded form either (PAGE-22). The + * engine closes the response on that path like any other parse failure (PAGE-13). + * * @public */ export function cursorStrategy(init: { @@ -62,6 +66,9 @@ export function cursorStrategy(init: { * rewrite a step applied on the way out, and that is the page number worth incrementing. An absent, empty, * or non-numeric value falls back to `startPage`; `startPage: 0` supports 0-based servers. * + * @throws UrlConstructionError from `parse` when `parameterName` carries an unpaired surrogate, which has no + * percent-encoded form (PAGE-22). + * * @public */ export function pageNumberStrategy(init: { From c6bbc3a6ed34397074db0c7dfc91fe33c3f02bb9 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:04:06 +0300 Subject: [PATCH 079/100] test(node-conformance): the caller's async context survives Runtime.send (#80) The leak this pins is Node's `AsyncLocalStorage` semantics, and it is exactly the shape `tests/node-conformance/`'s membership rule names: the Bun suite ran green over it for nine phases because every assertion sat in the continuation that opened the scope. Drives a pipeline built the public way -- `new PipelineBuilder(transport, {instrumentation})` with the LOGGING pillar inside it, so OBS-23's correlation push actually fires -- and asserts, after `await send()` resolves: the active span is `NOOP_SPAN` again, a fresh application log carries neither `trace.id` nor `span.id`, and a second send opens its own `http.client.operation` span, each ended once. Red against the `enterWith` form on Node (checked by reverting `send()`'s two `run` calls and rebuilding): the caller's active span is the ended operation span. Refs #80, #67 --- tests/node-conformance/observability.test.mjs | 95 +++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/tests/node-conformance/observability.test.mjs b/tests/node-conformance/observability.test.mjs index 27300a1..dcac9d3 100644 --- a/tests/node-conformance/observability.test.mjs +++ b/tests/node-conformance/observability.test.mjs @@ -5,15 +5,25 @@ // * AsyncLocalStorage store propagation across native Node promises, microtasks, and macrotask timers (OBS-10, OBS-24). // * activateSpan / activateSpanForCorrelation scope restoration and MDC push on Node (OBS-22, OBS-23). // * W3C trace/span identifier randomness via globalThis.crypto.getRandomValues on Node (OBS-26, OBS-27). +// * What a caller's async context holds AFTER `await runtime.send()` resolves (OBS-22, OBS-23, OBS-29, +// audit #67 / #80). Node's AsyncLocalStorage is the mechanism under test, not merely the host: the +// leak this pins was `enterWith` installing on the caller's async resource with the restore closure +// running on a later one, and Bun's suite passed over it for nine phases. import assert from 'node:assert/strict'; import {describe, it} from 'node:test'; import { NOOP_SPAN, + PipelineBuilder, + Protocol, + Request, + Response, + Status, activateSpan, activateSpanForCorrelation, createInstrumentationBundle, createLogger, getActiveSpan, + loggingStep, } from '@dexpace/core'; describe('observability on Node.js native runtime floor', () => { @@ -109,3 +119,88 @@ describe('observability on Node.js native runtime floor', () => { assert.equal(emitted[0]['span.id'], spanId); }); }); + +/** A transport literal: two methods, no socket. */ +const okTransport = { + send: async request => + Response.newBuilder() + .request(request) + .status(Status.of(200)) + .protocol(Protocol.HTTP_1_1) + .body(null) + .build(), + close: async () => undefined, +}; + +/** A recording tracer whose spans carry a spanContext, so OBS-23's correlation push actually fires. */ +function recordingTracer() { + const started = []; + return { + started, + tracer: { + startSpan(name) { + const record = {name, ended: 0}; + started.push(record); + const span = { + isRecording: true, + setAttribute: () => span, + recordException: () => span, + end: () => { + record.ended += 1; + }, + spanContext: () => ({ + traceId: '4bf92f3577b34da6a3ce929d0e0e4736', + spanId: '00f067aa0ba902b7', + }), + }; + return span; + }, + }, + }; +} + +describe('async context after Runtime.send on Node.js (OBS-22, OBS-23, OBS-29)', () => { + it('leaves the caller the active span and diagnostic fields it had before the call', async () => { + const {tracer, started} = recordingTracer(); + const emitted = []; + // A pipeline built the public way, with the LOGGING pillar inside it: that step is what pushes + // OBS-23's trace.id/span.id, and before 2026-09-05 the push outlived the call. + const runtime = new PipelineBuilder(okTransport, { + instrumentation: createInstrumentationBundle(() => tracer), + }) + .append( + loggingStep({ + granularity: 'headers', + logger: createLogger(() => undefined), + }), + ) + .build(); + const request = Request.newBuilder().url('https://example.com/one').build(); + + assert.equal(getActiveSpan(), NOOP_SPAN); + const response = await runtime.send(request); + assert.equal(response.status.code, 200); + + assert.equal(getActiveSpan(), NOOP_SPAN); + createLogger((level, fields) => { + emitted.push(Object.fromEntries(fields)); + }) + .atLevel('info') + .event('application.event.after.send') + .emit(); + assert.equal(emitted.length, 1); + assert.equal(emitted[0]['trace.id'], undefined); + assert.equal(emitted[0]['span.id'], undefined); + + // OBS-29's 1:1 binding, which the leak also broke: the second call opens its own operation span. + await runtime.send(request); + const operationSpans = started.filter( + span => span.name === 'http.client.operation', + ); + assert.equal(operationSpans.length, 2); + assert.deepEqual( + operationSpans.map(span => span.ended), + [1, 1], + ); + }); +}); From 620cf8d31c5e88be17aeaa29254ff273b1160267 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:05:27 +0300 Subject: [PATCH 080/100] docs: how to instrument a pipeline, and what makes DEXPACE_LOG_LEVEL live (#80) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `pipelines.md` gains two sections. "Instrumenting a pipeline" documents the options bag both constructors now take: what `instrumentation` buys (one `http.client.operation` span per call, the LOGGING pillar's per-transmission spans as its children), what `operationName` is for and what it deliberately does not influence, that both are per-pipeline rather than per-call, and which `seedFrom` mode carries them. "Turning logging on from the environment" closes the gap the audit found: the process-wide configuration slot starts empty (`CFG-13`), so `DEXPACE_LOG_LEVEL=headers` on its own has never done anything, and nothing in `packages/` or the guides said so. `setGlobalConfiguration(defaultConfiguration())` is the wiring; `configKey` is how a host application names its own variable instead. The same section states that a failed body capture now emits `http.instrumentation.bodyCaptureFailed` at `verbose`. The "Execution context" section states what a call leaves behind — nothing — and the pillar table lists `configKey`. Fences typecheck (`check-fences.mjs`: 49 fences, PASS); probe reports no drift. Refs #80, #67 --- docs/sdk-documentation/pipelines.md | 95 ++++++++++++++++++++++++++++- 1 file changed, 94 insertions(+), 1 deletion(-) diff --git a/docs/sdk-documentation/pipelines.md b/docs/sdk-documentation/pipelines.md index 0ff6532..93fb6e7 100644 --- a/docs/sdk-documentation/pipelines.md +++ b/docs/sdk-documentation/pipelines.md @@ -144,6 +144,58 @@ All three carry the same `InstrumentationBundle`, so trace and span identity sur `AsyncLocalStorage`-based, which is why `@dexpace/rx` installs no RxJS scheduler — adding `observeOn`/`subscribeOn` downstream makes reinstating the context the caller's job. +**What a call leaves behind is nothing.** `send()` scopes both async-scoped stores with +`AsyncLocalStorage.run`, so the moment it settles — resolved or rejected — the active span and the +diagnostic fields are what they were before it. That is a guarantee about *your* context, not only the +pipeline's: an application log emitted after `await client.send(...)` carries no `trace.id` from it. +Inside the call the usual rules apply, and a step that pushes fields with the handle form +(`activateSpanForCorrelation`) should still close its scope in the same continuation that opened it. + +## Instrumenting a pipeline + +Both constructors take an options bag that the built pipeline carries into every call: + +```typescript +import { + PipelineBuilder, + createInstrumentationBundle, + standardResilience, + type PipelineOptions, + type Runtime, + type Tracer, +} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +declare const otelTracer: Tracer; // whatever your tracing library hands you + +const instrumented: PipelineOptions = { + instrumentation: createInstrumentationBundle(() => otelTracer), + operationName: 'GetUser', +}; + +// `StandardResilienceOptions` extends `PipelineOptions`, so the preset takes the same two fields +// beside its per-pillar ones. +export const preset: Runtime = standardResilience(fetchTransport(), instrumented); +export const handBuilt: Runtime = new PipelineBuilder( + fetchTransport(), + instrumented, +).build(); +``` + +- **`instrumentation`** is the bundle every context of the call carries, and its `tracerFactory` is + what opens spans. `send()` asks it for `'http.client.operation'` once per call and opens **one** + span there (`OBS-29`), outside every pillar — a retry attempt and a redirect hop stay inside it. The + LOGGING pillar asks again per transmission, and those spans are its children. Omitted, the pipeline + carries the no-op bundle (`CTX-15`) and opens no span at all. +- **`operationName`** is `CTX-16`'s advisory label. It is carried unchanged across every promotion, + readable from a custom step as `ctx.context.operationName`, and used to name the tracer the LOGGING + pillar asks for. It never influences the request, the dispatch decision, or the store key. + +Both are per-pipeline, not per-call: build a second pipeline for a second operation name. +`PipelineBuilder.seedFrom(runtime, 'flatten')` carries them over, since the flattened builder replaces +the runtime it seeded from; `'nest'` does not need to, because the seeded runtime is still there, +driving its own contexts as the terminal transport. + ## The four shipped pillars | Pillar | Factory | Key settings | @@ -151,7 +203,7 @@ All three carry the same `InstrumentationBundle`, so trace and span identity sur | Retry | `retryStep(options?)` | `maxAttempts`, `retryableStatuses`, `totalTimeoutMs`, `attemptHeaderName`, backoff (`initialDelayMs`, `multiplier`, `maxDelayMs`, `jitter`, `fixedDelayMs`), injectable `clock`/`random` | | Redirect | `redirectStep(overrides?)` | `maxHops`, `allowedMethods`, `allow303`, `allowSchemeDowngrade`, `locationHeader`, `predicate` | | Auth | `authStep(settings)` | `credentials`, `tiers`, `challengeHook`, `bearerMarginMs` — see [`auth.md`](./auth.md) | -| Logging | `loggingStep(settings?)` | `granularity`, `severity`, `previewSizeBytes`, `droppedHeaderPolicy`, `logger`, `meter`, `tracerFactory` | +| Logging | `loggingStep(settings?)` | `granularity`, `configKey`, `severity`, `previewSizeBytes`, `droppedHeaderPolicy`, `logger`, `meter`, `tracerFactory` | Retry and redirect are worth a few notes each, because both surprise people: @@ -211,6 +263,47 @@ Retry and redirect are worth a few notes each, because both surprise people: - **A non-replayable body ends a redirect.** `PIPE-40` and `REDIR-22` disagree about what should happen; this port closes the response and throws (`docs/work/mvp/2026-09-04-open-items-dissolution.md` G1). +### Turning logging on from the environment + +`loggingStep()` with no `granularity` resolves one from layered configuration (`OBS-35`), and the +default is `'none'`. What it reads is the process-wide configuration slot — **which starts empty**. +`CFG-13` makes that slot last-write-wins with no ambient default, so nothing in this SDK reads +`process.env` until you say so, and `DEXPACE_LOG_LEVEL=headers` on its own changes nothing: + +```typescript +import { + defaultConfiguration, + loggingStep, + setGlobalConfiguration, + standardResilience, + type Runtime, +} from '@dexpace/core'; +import {fetchTransport} from '@dexpace/transport-fetch'; + +// Once, at start-up. `defaultConfiguration()` is the production wiring: its environment seam reads +// the live `process.env` on every lookup, and its property seam finds nothing (Node has no ambient +// key/value store distinct from the environment). +setGlobalConfiguration(defaultConfiguration()); + +// Now `DEXPACE_LOG_LEVEL=headers` reaches the step. `configKey` renames the variable it reads — +// `OBS-35` says an SDK must not bake a key name in, and `DEXPACE_LOG_LEVEL` is only the default. +export const client: Runtime = standardResilience(fetchTransport(), { + logging: {configKey: 'ACME_HTTP_LOG_LEVEL'}, +}); +export const step = loggingStep({configKey: 'ACME_HTTP_LOG_LEVEL'}); +``` + +Parsing is tolerant either way: `' Headers '` and `'HEADERS'` both resolve to `'headers'`, and an +absent, empty or unrecognised value falls back to `'none'`. An explicit `granularity` in the settings +wins over both, and is the right choice for a library that does not want to read a host application's +environment at all. + +Body-preview capture is best-effort and says so when it fails: a request body that cannot be probed +or a response body that errors mid-drain emits `http.instrumentation.bodyCaptureFailed`, carrying the +direction and the cause, and the request still completes (`OBS-20`). Like every +`http.instrumentation.*` diagnostic it emits at `verbose`, so a logger that filters that level will +not show it. + ## Testing a pipeline Nothing here needs a socket. A `Transport` is two methods, so the test double is a literal: From c44ce5371931c5fc6fdb889806b33d2d8f57a40d Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:07:06 +0300 Subject: [PATCH 081/100] =?UTF-8?q?docs:=20deviations=20=E2=80=94=20OBS-29?= =?UTF-8?q?=20closed,=20and=20an=20OBS-35=20row=20for=20the=20default=20ke?= =?UTF-8?q?y=20name=20(#80)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `OBS-29` row has carried "IN PROGRESS — see #80" since 2026-09-04, with caller-reachability of the operation span as the open half. That half is closed: `PipelineOptions` is `@public` and both constructors take it. The row now records what is met (ordering, under `startSpan`/`end` rather than the requirement's method names; the 1:1 binding, which was *stated* before this week and only became true when the `enterWith` leak went), what remains a deviation (the vocabulary), and that appendix C's "not yet runtime-enforced" note is out of date for this port. Anchors re-derived — every `file:line` in the row was checked against the tree by script. A new row records `OBS-35`: the SDK does bake in a default config key name, `configKey` makes it only a default, and a required key was rejected because it would mean no caller gets ambient logging without naming one first. The row also states the quieter half — the configuration slot starts empty, so no key of any name resolves until a host wires `defaultConfiguration()`. Per D0: appended at the end of "Deviations recorded outside a phase", dated, with `file:line` evidence and audit #67 / #80 in the Found-by column. No phase ledger touched. Refs #80, #67 --- docs/deviations.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/deviations.md b/docs/deviations.md index 6cd1179..60f8df4 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -466,7 +466,7 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended |---|---|---|---|---| | **`HTTP-11`'s range classifications are on `Status` only, not mirrored onto `Response`.** The spec places them on both: `docs/product-spec/04-core-http-domain-model.md:23` reads "Status MUST classify by range … **and a response MUST expose these derived from its status**", and appendix C (`appendix-c-consolidated-normative-requirement-index.md:47`) restates it as "Response MUST expose these same classifications derived from its status." The port ships them once, on `Status`, reachable as `response.status.isSuccess`. Six delegating getters on `Response` would duplicate surface that cannot drift, since there would be one implementation behind both — but the letter of the requirement does name the response, so the reading is recorded rather than assumed | the dissolved register's A3, register audit | 2026-09-02 | `packages/core/src/http/status.ts` carries all six; `packages/core/src/http/response.ts` carries `status` and no classification of its own | not yet in §10 | | **`REDIR-20`'s "fully override" is read as scoped to code/method eligibility, not to the safety mechanics that follow it.** A configured redirect predicate replaces the built-in follow decision; it does **not** bypass userinfo stripping, credential hygiene, the downgrade guard, body replayability, or loop/cap detection. Those are stated as unconditional MUSTs elsewhere in the same chapter and are not "should this kind of redirect be followed" policy — a caller predicate opting to follow a 307 with a single-use body still cannot make that body re-sendable. Genuinely ambiguous wording, decided one way and now recorded as decided | the dissolved register's G4, register audit | 2026-09-02 | `packages/core/src/redirect/decide.ts` consults `settings.predicate` at the eligibility gate and runs every later guard unconditionally; pinned by "the predicate does NOT bypass the safety mechanics" in `decide.test.ts`. Phase 9's conformance sweep was to re-confirm this and closed without doing so | not yet in §10 | -| **`OBS-29` is carried by spans rather than by the named tracer callbacks. IN PROGRESS — see [#80](https://github.com/dexpace/nodejs-sdk/issues/80) for the final disposition.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`. The **ordering half is satisfied** under different names, at both levels: `Runtime.send` opens one span before the drive and ends it on exactly one of two paths — `span.end()` for success, or `span.recordException(error)` then `span.end()` for failure (`pipeline/runtime.ts:193,198-199`) — and the per-attempt spans the LOGGING pillar step opens follow the same shape (`observability/logging-step.ts:427`, then `:398` or `:414-415`). **The 1:1-binding half was recorded here on 2026-09-02 as NOT met, and that reading is out of date.** `Runtime.send` is the one place that runs exactly once per logical operation, and it is where the operation span is opened, outside every pillar, so a retry's second attempt and a redirect's second hop stay inside the same span (`pipeline/runtime.ts:33-48,171-175`; `send`'s own `@remarks` at `:154-157` states the binding). `PIPE-2` still fixes the LOGGING step *inside* the RETRY and REDIRECT pipelines, so its spans remain per transmission — they are now children of the operation span rather than the only spans there are. What stays open is what a *caller* can reach: appendix C's entry (`appendix-c-consolidated-normative-requirement-index.md:509`) says "pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced", and the operation span is not reachable from the public API. #80 decides that and rewrites this row; anchors re-derived by audit #67 / #68 | the dissolved register's L1/V2, register audit; anchors re-derived by audit #67 / #68 | 2026-09-02, re-anchored 2026-09-04 | `packages/core/src/observability/span.ts:54-56` (`Tracer`, one method — declared in `tracing.ts` until 2026-09-04, when audit #67 / #69 moved the inert tracing declarations into their own module to break an import cycle; `tracing.ts` re-exports them, so the public path is unchanged); `packages/core/src/pipeline/runtime.ts:33-48,154-157,171-175,193,198-199` (the operation span and its 1:1 binding); `observability/logging-step.ts:398,414-415,427` (the per-attempt span); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | +| **`OBS-29` is carried by spans rather than by the named tracer callbacks; both halves are now met, and the operation span is reachable from the public API.** `OBS-29` (MUST) requires `operationStarted` once at the start, `operationSucceeded`/`operationFailed` mutually exclusive and once each at the end, and **one tracer instance per logical operation**. The port has no method of those names; it has `Tracer.startSpan(name): Span`, and the requirement is discharged under that vocabulary. **Ordering:** `Runtime.send` opens one span before the drive and ends it exactly once, behind an `ended` latch that a throwing `end()` cannot get past (`pipeline/runtime.ts:78-96`), and the per-attempt spans the LOGGING pillar step opens follow the same shape from a single `finally` (`observability/logging-step.ts:469`, then `:457,494`). **1:1 binding:** the span is opened outside every pillar, so a retry's second attempt and a redirect's second hop stay inside the first call's span (`pipeline/runtime.ts:57-67,280-282`); before 2026-09-05 a leaked `enterWith` made the *previous* call's ended span read as active and suppressed the next one, so only the first operation per async context got a span at all — the binding was stated and not delivered. **Caller reachability**, which this row carried as the open half from 2026-09-02 to 2026-09-05, is closed: `PipelineOptions` (`pipeline/builder.ts:32-50`) is `@public`, is the second constructor argument of `PipelineBuilder` (`:71`) and is extended by `StandardResilienceOptions` (`auth/preset.ts:23,115-118`), so `createInstrumentationBundle`'s result has somewhere public to go. What remains a deviation is only the **vocabulary**: a consumer sees `startSpan`/`end`/`recordException`, not `operationStarted`/`operationSucceeded`/`operationFailed`, and appendix C's own note that "pipeline/transport wiring to emit it is a follow-up, so it is not yet runtime-enforced" (`appendix-c-consolidated-normative-requirement-index.md:509`) is now out of date for this port | the dissolved register's L1/V2, register audit; anchors re-derived by audit #67 / #68; finished by audit #67 / #80 | 2026-09-02, re-anchored 2026-09-04, closed 2026-09-05 | `packages/core/src/observability/span.ts:54-56` (`Tracer`, one method — declared in `tracing.ts` until 2026-09-04, when audit #67 / #69 moved the inert tracing declarations into their own module to break an import cycle; `tracing.ts` re-exports them, so the public path is unchanged); `packages/core/src/pipeline/runtime.ts:57-67,78-96,206-233,280-282` (the operation span, its single `end()`, and the `run`-scoped stores that keep the binding true across calls); `packages/core/src/pipeline/builder.ts:32-50,71,293` and `packages/core/src/auth/preset.ts:23,115-118` (the public route to a bundle); `observability/logging-step.ts:457,469,494` (the per-attempt span); `docs/product-spec/15-instrumentation-and-observability.md:54` (the requirement) | not yet in §10 | | **`invariant()` density is not a target, project-wide.** `docs/knowledge/harvested/assertions.md:6-7` sets a 2-per-function module average. The port's position: a module gains an `invariant()` when it has an internal precondition worth asserting, and `recovery/`, `http/`, `seams/` and `generated/` have none — measured 2026-09-02, all four at zero. Adding assertions to reach an average would assert nothing. `recovery/` is the sharp case: an `invariant()` inside `ResponseRecoveryChain.apply()` throws, and `RECOV-8` forbids `apply()` from throwing, so a density rule would push that module toward a shape the specification forbids | the dissolved register's F3/H6 and the deferral register's *Assertion-density rule applied project-wide* row (retired to [the purge note](./work/mvp/2026-09-04-register-retirement-purge.md)), register audit | 2026-09-02 | `packages/core/src/recovery/`, `http/`, `seams/`: zero `invariant(` calls. `pipeline/` and `context/` both carry them, so the rule is applied where it earns its place. Counted qualitatively on purpose — the two figures this cell used to state were wrong by the time anyone read them; re-derive with `grep -rn 'invariant(' packages/core/src/ --include='*.ts' \| grep -v '\.test\.'` | not yet in §10 | | **`PIPE-40` and `REDIR-22` contradict each other on the non-replayable-body path, and the port implements `REDIR-22`.** Two MUSTs name the same trigger and prescribe opposite dispositions. `docs/product-spec/08-execution-pipelines.md:20` (`PIPE-40`): "on paths that abandon a re-drive (redirect cycle, **non-replayable body**, budget exhausted) the in-flight response MUST be returned unclosed." `docs/product-spec/10-redirect-handling.md:22` (`REDIR-22`): "if building the follow-up throws (**non-replayable body**, downgrade rejection) the current response MUST be closed before the error propagates." The port closes, then throws, on three grounds: `REDIR-6` independently fixes the control flow ("the operation MUST fail with a clear error naming replayability"), so the path throws and a response never *returned* cannot be "returned unclosed"; specific governs general, since §10 of the spec owns the redirect step's lifecycle; and closing is the safer reading, because the alternative leaks a body on an error path with no caller holding a reference to close it. `PIPE-40`'s other two named paths do genuinely return, and both return unclosed as it requires. **The erratum this needs is proposed below and is not applied here**, because `docs/product-spec/` is frozen and correcting a normative sentence is the specification owner's act, not a maintenance one | the dissolved register's G1, Phase 5b design's Deviation Ledger | 2026-09-04 | `packages/core/src/redirect/redirect-step.ts` closes before throwing, with the reasoning asserted inline in `redirect-step.test.ts`; nothing in the code waits on the erratum | not yet in §10 | | **`PIPE-37`'s outermost pre-redirect status-mapping step was never built, and until this row nothing recorded that.** `PIPE-37` (MUST) requires a step whose correctness depends on the single terminal response — status-to-typed-error mapping is its own worked example — to occupy the outermost pre-redirect slot, so it runs outside both the redirect and the retry loop. The port ships the *mapping*, but as `statusMappingStep`, a `ResponseStep` on the response-recovery chain (RECOV-15/RECOV-16), not as a pipeline `Step` carrying `stage: 'PRE_REDIRECT'`. The slot itself exists and is installable — `config/clientIdentityStep` occupies it today — so this is a wiring gap, not a missing mechanism. It has an owner in writing and the owner never took it: Phase 4's checklist marked the row ⏳ and said "the obligation lands on whichever phase wires 4b's `statusMappingStep` into a real pipeline — **Phase 5**", and Phase 5 shipped without it, with no deferral carrying the hand-off forward. That is why a code audit found it and no checklist did. **Ledgered rather than implemented.** Installing a `PRE_REDIRECT` mapping step is public pipeline surface with a real behaviour change behind it, and the petstore spike arrives at the same work from the other side: its finding 2 wants a declarative `StatusErrorMap` applied *at* the `toHttpError` call site rather than wrapped around it, so a mapped error class can see a decoded payload instead of raw bytes. Whoever does one does both | audit #67 / #69 | 2026-09-04 | `packages/core/src/recovery/status-mapping.ts:26` (`statusMappingStep(response: Response): Promise` — a `ResponseStep`, with no `StepDescriptor` and no stage); `packages/core/src/pipeline/stage.ts:39` (`PRE_REDIRECT` heads `STAGE_ORDER`) and `packages/core/src/config/client-identity-step.ts:124` (the one shipped step that occupies it); `docs/work/mvp/phase4/2026-07-26-phase4-execution-context-and-pipelines-checklist.md:141` (the dropped hand-off); `docs/product-spec/08-execution-pipelines.md:26` (PIPE-37 shares that line with PIPE-25/36/38); `examples/petstore/FINDINGS.md:96-97` | not yet in §10 | @@ -485,6 +485,7 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **`RETRY-44`'s "downstream chain" is read as everything BELOW the retry point, which in the recovery stack excludes the request chain.** The requirement has two clauses: each attempt re-executes the downstream chain with fresh per-attempt state, and "upstream steps MUST NOT mutate the shared in-flight request between attempts". The port originally read the first clause as covering the *whole* recovery chain and re-ran `RequestRecoveryChain.apply()` per attempt, with a test that said so by name. That makes `packages/core/src/recovery/idempotency-key.ts` generate a fresh key on every attempt, so three attempts of one logical request reach the server as three unrelated writes — the precise failure `RECOV-32` exists to prevent, and the opposite of what that step's own `@public` TSDoc promises. **The chain is now applied once, above the loop; each attempt re-executes transport plus response chain over `stampAttempt`'s fresh copy of the prepared request.** Under this reading both clauses hold and the second holds *by construction*: upstream steps cannot mutate the in-flight request between attempts because they no longer run between attempts. The pillar stack is untouched — there "downstream" is the forked continuation (`ctx.fork()`), and `retryStep` still re-drives it per attempt. *Rejected:* memoizing the key on the template (a `WeakMap` keyed by the `Request` instance) — a caller who deliberately sends one immutable `Request` value twice would replay the key and have the server drop a genuine second call. *Rejected:* re-running the chain over the *prepared* request each attempt — the chain would read its own output, which is clause two's mutation in different clothes, and every shipped and caller-written step would have to be proven idempotent. One consequence recorded rather than assumed: the re-send gate (`RETRY-5`/`RECOV-18`) now judges the prepared request rather than the caller's, which is what a retry would actually re-send | audit #67 / #73 | 2026-09-05 | `packages/core/src/retry/retry-dispatch.ts:83-88` (the chain applied once, then `runWithRetry`) against `:27-33` (the per-attempt half); `packages/core/src/recovery/orchestrator.ts:62` (`prepareRequest`) and `:117` (`dispatchPrepared`); `packages/core/src/retry/engine.ts:243` is the gate that now reads the prepared request. `RETRY-44`'s wording is `docs/product-spec/09-retry-and-resilience.md:35` and `docs/product-spec/appendix-c-consolidated-normative-requirement-index.md:306`. Pinned by `packages/core/src/retry/retry-dispatch.test.ts:126` (chain applied once), `:169` (one `generate()`, one key on three sends), `:201` (the `RETRY-38` ordinal varies while the key does not) and `:227` (a request-chain throw is not retried and meets the recovery phase exactly once) | not yet in §10 | | **`HTTP-35`'s timeout check is read as the FULL range `AbortSignal.timeout()` accepts, not the lower bound the requirement enumerates.** `HTTP-35` says the options builder "MUST reject a non-null timeout that is zero or negative". `RequestOptionsBuilder.timeoutMs` rejects three more classes: non-finite (shipped unledgered before this audit), non-integer, and anything above `2**32 - 1`. **Strictly stricter than the letter, and deliberately so.** The field has exactly one consumer — `composeSignal` hands it to `AbortSignal.timeout()` — so a value this setter admits and that function refuses is `HTTP-35`'s own failure mode with the seam moved: the error surfaces inside a transport, as an unwrapped platform `RangeError`, one frame away from the call that supplied it. The earlier reading accepted `1.5` and argued in TSDoc that "a timeout is a duration and a fractional millisecond is meaningful"; no consumer of the field can express one. **The range checked is Node's, and that is the point:** `AbortSignal.timeout(1.5)` and `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are ACCEPTED on Bun, and a negative delay is `RangeError` on Node against `TypeError` on Bun (measured 2026-09-05), so leaving the check to the runtime would make an SDK-level contract depend on which runtime the caller happens to be on. *Rejected:* rounding with `Math.ceil` and clamping inside `composeSignal`, which hides the caller's mistake in the one place `HTTP-35` exists to surface it. `composeSignal` is documented as still able to raise, because a transport's own `defaultTimeoutMs` construction option bypasses this setter and is not validated by core — recorded for #81/#82, not fixed here | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:48` is `HTTP-35`'s wording. `packages/core/src/http/request-options.ts:12` (`MAX_TIMEOUT_MS`) and `:204-214` (the check and the rewritten TSDoc paragraph); `packages/core/src/seams/transport.ts:86-92` is `composeSignal`'s new `@throws`, which states the two-runtime divergence rather than naming one error class. Pinned by "rejects a fractional timeout, which no transport deadline can honor" (`packages/core/src/http/request-options.test.ts:128`, the FLIPPED case — it pinned acceptance until this audit), "rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1" (`:134`), "accepts the ceiling itself" (`:143`) and the `every accepted timeout is an integer in 1..2**32 - 1` property (`:157`); the Node half is `composeSignal timeout range on Node (HTTP-35)` in `tests/node-conformance/seams.test.mjs:105`, which cannot live in `bun test` because Bun accepts both rejected values | not yet in §10 | | **`HTTP-31`'s "falls back to raw text rather than throwing" is satisfied for an unpaired surrogate by SUBSTITUTING U+FFFD, not by keeping the raw text.** `HTTP-31` (MUST) makes `QueryParams.parse` lenient and enumerates the lenient cases, ending with "malformed percent-encoding falling back to raw text rather than throwing". An unpaired surrogate is a fourth kind of malformed input the enumeration does not name, and the fallback it prescribes is not available for it: the raw text has no UTF-8 form, so keeping it produces a `QueryParams` whose `encode()` throws `URIError` — the throw merely deferred out of `parse` and into an accessor that documents no throw at all. **The port repairs instead.** `parse` runs `toWellFormed()` over each decoded name and value, so every instance it returns is encodable, which is what "parsing MUST invert encode" needs to mean. The strict half of the rule is unaffected and is where `#76` puts the rejection: `QueryParamsBuilder.add` throws `UrlConstructionError` for the same input, and `substitutePathParams` throws `OperationAssemblyError`. That asymmetry is not new to the query model — it is exactly the outbound/inbound split `Headers` already draws for `HTTP-18` against `HTTP-19`, applied to the one requirement pair that needs it here. Substitution matches the platform rather than inventing a policy: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` (measured 2026-09-05). *Rejected:* letting `parse` throw the builder's error, which breaks a MUST. *Rejected:* dropping the offending parameter, which loses a name the caller may be matching on | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:42` carries `HTTP-31`'s wording (shared with `HTTP-30`). `packages/core/src/http/rfc3986.ts:17-18` are the two patterns, `:31` `hasLoneSurrogate` (strict) and `:44` `toWellFormed` (lenient) — one rule, two entry points, so no caller can pick the wrong one; `packages/core/src/http/query-params.ts:144-150` is `parse`'s repair with the `HTTP-18`/`HTTP-19` comparison stated inline, against `:44-50` and `:240-241` for the strict `add` path; `packages/core/src/seams/operation.ts:139-144` is the path-param half. `/\p{Surrogate}/u` rather than `String.prototype.isWellFormed()` because the latter is ES2024 and `tsconfig.base.json:5-11` pins `lib: ES2023`, though the `engines.node >= 20.3` runtime has it. Pinned by the `lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)` block in `packages/core/src/http/query-params.test.ts:170` — "parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing" (`:197`) and the `no anything escapes parse()` property (`:233`) | not yet in §10 | +| **`OBS-35`'s "MUST NOT bake in a default config key name" is satisfied by making the key configurable, not by removing the default.** `OBS-35` (SHOULD) asks for a tolerant, layered log-level resolution and adds one MUST: no baked-in default key name. The port ships `CFG_KEY_LOG_LEVEL` (`DEXPACE_LOG_LEVEL`) as `CFG-14`'s well-known key and, until 2026-09-05, read it unconditionally. It is now `LoggingStepSettings.configKey`'s default: a caller names their own key and the resolution is otherwise identical. **Why the default stays.** A required key would mean no caller gets ambient granularity without naming one first, which trades a MUST about *naming* for a worse default experience, and `CFG-14` — which this port also implements — exists precisely to standardise the name. The layered resolution itself is `CFG-1`'s (override → environment → normalised property → default) and is tolerant as the requirement asks. **A second, quieter half:** the process-wide configuration slot starts empty (`CFG-13`), so no key of any name resolves until a host calls `setGlobalConfiguration(defaultConfiguration())`. That is deliberate — defaulting the slot to a configuration that reads `process.env` would make an import-time environment read the SDK's default behaviour — and it is now documented as the required wiring rather than left to be discovered | audit #67 / #80 | 2026-09-05 | `packages/core/src/observability/logging-step.ts:64-79,94-104` (the setting and the resolution); `packages/core/src/config/configuration.ts:311,354-358,368` (`CFG_KEY_LOG_LEVEL`, `defaultConfiguration`, the empty default slot); `docs/sdk-documentation/pipelines.md` "Turning logging on from the environment"; `docs/product-spec/15-instrumentation-and-observability.md:66` (the requirement) | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) From 835ac4426557a9586efb017a313aabb6e233631e Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:08:34 +0300 Subject: [PATCH 082/100] docs(core): send() states what it restores in the caller's async context (#80) Two `@remarks` on the one public method the guarantee is about: where the tracer comes from now that there is a public route (`PipelineOptions`), and that the active span and diagnostic fields are the caller's own again once `send()` settles either way. Refs #80, #67 --- packages/core/src/pipeline/runtime.ts | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/packages/core/src/pipeline/runtime.ts b/packages/core/src/pipeline/runtime.ts index 640353e..fb87486 100644 --- a/packages/core/src/pipeline/runtime.ts +++ b/packages/core/src/pipeline/runtime.ts @@ -202,6 +202,12 @@ export class Runtime implements Transport { * tracer and no span is already active — `OBS-29`'s "one tracer instance per logical operation". * It is the parent of whatever per-attempt spans the LOGGING pillar step opens inside the RETRY * and REDIRECT pipelines, which `PIPE-2` fixes there and which are therefore per *transmission*. + * Supply the tracer through `PipelineOptions.instrumentation`. + * + * @remarks The caller's async context is restored when this settles, resolved or rejected: the + * active span and the diagnostic fields (`trace.id`, `span.id`) are what they were before the + * call, so an application log emitted after `await send()` carries nothing from it. Both stores + * are scoped with `AsyncLocalStorage.run`, which also unwinds a scope a step left open. */ async send( request: Request, From 8093e8a213aedde03532ff8ce75c0e66af85b73f Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:11:11 +0300 Subject: [PATCH 083/100] docs(core): the shipped .d.ts names the splice site and the abort race MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two prose corrections in `@dexpace/core`'s public surface, both of them claims the round-1 fixes made incomplete. Round 2 of #79; these files were outside the task partition and are now in it. `UrlConstructionError`'s surrogate bullet named `QueryParamsBuilder.add` as the site that rejects a string with no UTF-8 form. Pagination's splice now rejects the same input for the same reason through the same predicate, so the bullet names it too — with the part a consumer actually needs: that path is reachable with no caller mistake, because the cursor is server-supplied, and its message names the parameter rather than echoing the value. Backticked prose, not `{@link}`: `spliceQueryParam` and `readQueryParam` are `@internal` and `api:ci` fails on the unresolved link even where `api:local` only warns. `Deserializer.deserializeFrom` and `Serializer.serializeTo` said the signal is "checked before the lock is taken and between reads". That describes what an implementation may do and not what the promise on the same line requires: only racing each pending operation makes "an aborted call never leaves the caller's source locked" true, because a stalled source parks the drain inside a read that a between-reads check can never reach again. The clause now says raced, and says which half is load-bearing — a codec author reading only this seam had no way to know. `core.api.md` is unchanged: the report carries signatures, not prose. --- packages/core/src/http/errors.ts | 4 ++++ packages/core/src/seams/serde.ts | 13 +++++++++---- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/packages/core/src/http/errors.ts b/packages/core/src/http/errors.ts index 66c0f83..bc8e01a 100644 --- a/packages/core/src/http/errors.ts +++ b/packages/core/src/http/errors.ts @@ -140,6 +140,10 @@ export class ProtocolParseError extends DexpaceError {} * fail; `QueryParamsBuilder.add` rejects it at the call that supplied it instead. `cause` is not * set on this path — nothing was caught, the input was inspected (HTTP-29, audit #67 / #76). * `QueryParams.parse` does NOT throw it: HTTP-31 makes parsing lenient, so it substitutes U+FFFD. + * Pagination's query splice — `spliceQueryParam` and `readQueryParam`, behind `cursorStrategy` + * and `pageNumberStrategy` — rejects the same input for the same reason, through the same + * predicate (PAGE-22, audit #67 / #79). That one is reachable without any caller mistake, since + * the cursor is server-supplied; its message names the parameter and never echoes the value. * * @public */ diff --git a/packages/core/src/seams/serde.ts b/packages/core/src/seams/serde.ts index 0e26146..adced9b 100644 --- a/packages/core/src/seams/serde.ts +++ b/packages/core/src/seams/serde.ts @@ -94,8 +94,10 @@ export interface Serializer { * because the caller owns it (SERDE-3). * * @throws Whatever `options.signal` was aborted with — its `reason`, or a `DOMException` named - * `'AbortError'` when none was given. Checked before the writer lock is taken, so an aborted call - * never leaves the caller's sink locked and never closes it (SERDE-3). + * `'AbortError'` when none was given. Checked before the writer lock is taken, and then raced + * against each pending write, so an aborted call never leaves the caller's sink locked and never + * closes it (SERDE-3). A write parked against a slow sink is the case the pre-check cannot cover; + * the write itself is left outstanding, because aborting it would be taking ownership. * * @remarks Takes `{signal}` because this method drives a stream it did not open, which is the * project-wide test for whether an API owes one. Buffered-bytes APIs — `serialize`, @@ -210,8 +212,11 @@ export interface Deserializer { * not re-typed, because a contended source is a programmer error rather than a decode failure. * * @throws Whatever `options.signal` was aborted with — its `reason`, or a `DOMException` named - * `'AbortError'` when none was given. Checked before the reader lock is taken and between reads, - * so an aborted call never leaves the caller's source locked and never cancels it (SERDE-3). + * `'AbortError'` when none was given. Checked before the reader lock is taken, and then raced + * against each pending read, so an aborted call never leaves the caller's source locked and never + * cancels it (SERDE-3). Racing is the load-bearing half: a source that stalls mid-body parks the + * drain inside a read that a between-reads check can never reach again, and an implementation + * that only checks between reads leaves that call unsettled and that source locked forever. * * @remarks Takes `{signal}` because this method drives a stream it did not open, which is the * project-wide test for whether an API owes one. The abort reaches the drain loop; the CPU-bound From 118d56cc7c53cd0ecb8a482ce46c75a1cd46f95d Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:11:11 +0300 Subject: [PATCH 084/100] docs: a paging strategy owes a well-formed PageInfo (PAGE-4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `write-a-paging-strategy.md` had four rules and no statement of the one obligation the engine enforces with an assertion. A strategy that returns `undefined`, `null`, or `{items: null}` now closes the response and throws, naming the invariant (#79) — and the guide should say so before an author meets it, along with the reason it is not treated as a quiet end of stream: "the strategy forgot to `return`" and "the server ran out of pages" must not look the same from the outside. The rule also draws the line the guide left implicit, and which the `PaginationError` paragraph below it only half states: ending a walk is `pageInfo(items)` with no next request, failing one is a throw. Both are supported; neither is `undefined`. Round 2 of #79; this file was outside the task partition and is now in it. Fences typecheck. --- .../sdk-documentation/write-a-paging-strategy.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/sdk-documentation/write-a-paging-strategy.md b/docs/sdk-documentation/write-a-paging-strategy.md index 40d1700..5913cbf 100644 --- a/docs/sdk-documentation/write-a-paging-strategy.md +++ b/docs/sdk-documentation/write-a-paging-strategy.md @@ -95,7 +95,7 @@ for await (const page of paginator.pages()) { /* page by page */ } independent walks, not two views of one. That is also why `@dexpace/rx`'s `pageItems$`/`pages$` are cold and repeatable while its SSE observables are not. -## The four rules +## The five rules **1. Take everything you need from the response before your promise settles** (`PAGE-5`). The response you are handed is live and single-use; the engine may close it the moment `parse` resolves. @@ -135,6 +135,20 @@ which is an erratum recorded in `docs/knowledge/notes/pagination.md` and `docs/w `maxPages` on `PaginatorInit` is the backstop, not the design. Loop detection is not the paginator's job. +**5. Always return a well-formed `PageInfo`** (`PAGE-4`). `items` must be an array — an empty one is +fine and is a perfectly valid non-terminal page — and `nextRequest === undefined` is the **single, +exclusive** end-of-stream signal. A `PageInfo` that is itself `null` or `undefined`, or whose `items` +is either, is a programmer error and the engine treats it as one: it closes the response and throws +an assertion naming the invariant you broke. It does **not** end the walk quietly, because "the +strategy forgot to `return`" and "the server ran out of pages" must not look the same from the +outside. Use `pageInfo(items, next?)` and this cannot happen; the check exists because `parse` +crosses a seam, where an `any`-typed decode or a trusted server field can produce a shape the types +say is impossible. + +Terminating and failing are different acts. To *end* the walk, return `pageInfo(items)` with no next +request. To *fail* it, throw — the engine closes the response and your error reaches the consumer +unwrapped (`PAGE-13`, `PAGE-28`). + `PaginationError` is reserved for engine misuse and precondition violations — not for "the server returned a page I did not understand", which is your `extract`'s error to raise. From a9c5b43e90af6d0b92b28d765783bc8fdb8259df Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:16:18 +0300 Subject: [PATCH 085/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20wave=205?= =?UTF-8?q?=20landed;=20pre-take=20wave=206=20decisions=20D19=E2=80=93D20?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 96 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index 56db4f4..cc132f0 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -412,6 +412,18 @@ existing. The unknown-length `pipeTo` path still forwards empty chunks; out of H Known merge seams: `core.api.md` (up to three), which the supervisor regenerates on the merged tree rather than hand-merging. No two tasks share a source file. +## Wave 5 — landed 2026-09-05 +PR #93 (#78), PR #95 (#80) and PR #94 (#79) merged into the umbrella at `4576658`, in that order. No conflicts: +#78's item 17 section edit and #80's OBS-29 row edit plus two appended rows sit in disjoint regions of +`deviations.md`. Merged tree preflighted in a throwaway worktree before the merge (byte-identical result): all +20 steps passed. Every commit on all three branches checked as `wahbehmo20@gmail.com`. + +**Traps added this wave.** `gts --fix` deletes an `eslint-disable-next-line` whose next line is another +comment (the directive binds to the comment and becomes unused): prose first, directive last. And +`packages/core/src/io/index.ts` is a dead barrel nothing imports, whose file-level comment carries the +`@internal` substring and so ships a broken `dist/io/index.d.ts` today; harmless only because the modules it +re-exports emit `export {}`. Left for the release pass (deleting it is a design call). + ## Decisions taken for wave 5 (M4: #78, #79, #80) — pre-taken 2026-09-05, before dispatch ### D16 — #78: `instanceof IoError` stays; it means "transport-layer failure", and item 17 says so @@ -493,6 +505,36 @@ comment says what the cause-walk actually matches. *Rejected:* switching to `isI `AbortSignal.timeout()`; Node rejects `1.5`/`2**32`/`-1` with `RangeError`, Bun accepts the first two. Validate at the transport factory the way `RequestOptionsBuilder.timeoutMs` now does (D14), with a conformance row. + +**D16 outcome (2026-09-05, #78, PR #93).** Rule kept and written down in `classify.ts`, `io/index.ts` and item +17. Six classify cases (five leaves plus `TransportFailureError`, each asserting `isIoError`'s answer beside the +classifier's). Non-finite `delayOverride` is screened at the source and reported through the same +`http.retry.delayOverrideFailed` path a throw uses (`reportOverrideFailure`); finite negatives unchanged. +`computeDelay` short-circuits `initialDelayMs === 0`. Round 2 put the failure semantics on the `@public` +`RetryStepOptions.delayOverride` TSDoc. `core.api.md` byte-identical. + +**D17 outcome (2026-09-05, #79, PR #94).** `codec-json/src/abort-race.ts` races each pending read/write; both +Bun and Node abort cases hung to their deadlines before it. Paginator: `pageOrClose()` runs the PAGE-4 +invariants under the same `closeThenRethrow()` as `parseOrClose`; all four invariants (paginator + page) reject +`null` as well as `undefined`. `tristate()` rejects a present `undefined` too, not only `null` +(`NonNullable` excludes both). **Correction to D17's text:** the SSE double report never came from +`#releaseWithInFlightError` — it came from `#iterate`'s `finally` running `#releaseQuietly()` after the catch +had already released; fixed there. A second double report survives deliberately: `bindAbort`'s hook call plus +the `suppressed` copy when an iterator is parked in a read at abort time (the listener cannot know). Query-splice +surrogates → `UrlConstructionError` (the strategies used no class of their own). Round 2 aligned +`UrlConstructionError`'s TSDoc, both abort clauses in `seams/serde.ts`, and added rule 5 (PAGE-4) to +`write-a-paging-strategy.md`. + +**D18 outcome (2026-09-05, #80, PR #95).** `send()` runs under `runWithSnapshot` (diagnostic store) and +`runWithActiveSpan` (span), both `AsyncLocalStorage.run`; the handle forms stay with TSDoc on their reach. New +`@public` `PipelineOptions {instrumentation?, operationName?}` as `PipelineBuilder`'s optional second argument; +`StandardResilienceOptions extends PipelineOptions`; `seedFrom('flatten')` carries the options through an +`@internal` friend accessor (beyond D18's letter, so the documented derivation path does not drop the bundle). +`install` + span start inside one `try`; `endOnce` latch; LOGGING step's span in a `finally`. +`http.instrumentation.bodyCaptureFailed` at `verbose`; `LoggingStepSettings.configKey`; `pipelines.md` section +"Turning logging on from the environment". OBS-29 row closed, OBS-35 row appended. *Left for a later pass:* +`packages/core/README.md`'s Observability bullet does not yet point at `PipelineOptions.instrumentation`. + ### Wave 5 partition | Task | Owns | |---|---| @@ -502,6 +544,57 @@ transport factory the way `RequestOptionsBuilder.timeoutMs` now does (D14), with Known merge seams: `deviations.md` (item 17 section vs OBS-29 row — disjoint regions); `core.api.md` (regenerated on the merged tree). +## Decisions taken for wave 6 (M5: #81, then #82) — pre-taken 2026-09-05 + +### D19 — #81: degrade what undici cannot carry; one short-write detector; a typed refusal for SOCKS +- **Header rejections become logged drops (TRANSPORT-11/12).** `UNDICI_FORBIDDEN_HEADERS` gains `expect`, + `keep-alive` and `upgrade`; `connection` stays forwarded (§17's note) but a value other than `close` / + `keep-alive` is dropped and logged; every name is validated against RFC 9110 `token` in `toUndiciHeaders` + and a non-token name is dropped and logged, which is the degrade `fetch-transport.ts:146-152` already does + by `try`/`catch`. Check `FETCH_FORBIDDEN_HEADERS` against the same rows on Node's undici-backed `fetch` and + widen it only if a row proves a rejection. Conformance rows both transports run: `Expect: 100-continue`, + `Upgrade: websocket`, a non-token name — the send succeeds, the header is absent on the wire, the drop is + logged by name. +- **`fileBody` goes through `writeTo` (BODY-13).** The undici file branch stops handing `createReadStream` to + undici and takes the same `pumpBody` path the streamed case uses, so `transferred === count` runs on both + transports and a truncate-after-stat fails with `TransportFailureError` naming transferred-of-total. + *Rejected:* a byte-counting wrapper around the read stream (a second BODY-13 implementation, which is the + drift the shared pump exists to prevent). Conformance row: truncate a 1 MB file to 10 bytes after `stat`, + both transports reject identically. If the pump path changes the framing (`content-length` vs chunked), + say so in the PR and keep whatever the streamed case does today. +- **SOCKS:** `undiciTransport()` refuses `proxy.type !== 'http'` at the factory with the same `TypeError` + shape `toDispatchError` uses for a terminal misconfiguration (deliberately outside the `IoError` tree), + `@throws` documented. `ProxyType` keeps `socks4`/`socks5` — narrowing it is a public-shape decision for the + release pass (D1) — and one `deviations.md` row records "SOCKS is resolved by core and supported by neither + transport". `fetchTransport()` gets the same check if it accepts a proxy at all. +- Rows go in `run-suite.ts` + `fixtures.ts`; #82 lands on top of them (D2). `api:local` on `transport-undici` + (and `transport-fetch` if touched). No changesets (D1). +*Partition:* `packages/transport-undici/**`, `packages/transport-fetch/src/fetch-transport.ts` (forbidden set +only, if a row proves it), `packages/transport-conformance/src/**`, `docs/sdk-documentation/write-a-transport.md`, +`docs/deviations.md` (one appended row), the two transports' `api.md`. + +### D20 — #82 (after #81 merges): one classification table; `body === null` when there is none; abort the fork +- **Permanent errors on fetch:** the fetch transport classifies a `TypeError` from an unsupported scheme, an + invalid URL, a forbidden method or an invalid header exactly as the undici transport classifies its + `TERMINAL_ARGUMENT_CODES` — a bare `TypeError` with `{cause}`, outside the `IoError` tree — and the table + that decides it moves to `@dexpace/transport-shared` so the two cannot drift (the precedent is + `abort-mapping.ts`). `fetch failed` with a network `cause` stays `TransportFailureError`. Conformance row: + `ftp://` (or another scheme the runtime refuses) is non-retryable on both — assert `isIoError(e) === false`. +- **Body-less responses (204, 304, HEAD):** `Response.body` is `null` — the WHATWG shape, and what core's + model already types (`http/response.ts:18`); undici stops wrapping `result.body` unconditionally and + hands `null` for those three. Rows for 204, 304 and HEAD assert `body === null`, `contentLength`, and that + `reasonPhrase` is `undefined` or a string (the divergence is D7's row beside §10 item 13; the row does not + re-ledger it). *Rejected:* an empty stream on both (a consumer would have to read to learn there is nothing). +- **`CONTROL_BYTE`** becomes `/[\x00-\x08\x0A-\x1F\x7F]/u`, with a test in `transport-shared` for LF. +- **Producer-failure race:** the producer-failure branch aborts the forked signal before rethrowing, on both + transports, so a native call that resolves afterwards is cancelled rather than stranded with an unread body. + Verify with an instrumented transport whose native call resolves after the producer fails. +- `api:local` on `transport-shared`, `transport-fetch`, `transport-undici` as touched. No changesets (D1). +- **Carried from #76:** validate `defaultTimeoutMs` at both transport factories with the same integer + `1..2**32-1` rule `RequestOptionsBuilder.timeoutMs` uses (D14), typed error, `@throws`, a conformance row. +*Partition:* the three transport packages, `packages/transport-conformance/src/**`, +`docs/sdk-documentation/write-a-transport.md`, the three `api.md`s. + ## Deferred — release machinery (recoverable list) | Issue / PR | Deferred item | |---|---| @@ -516,3 +609,6 @@ Known merge seams: `deviations.md` (item 17 section vs OBS-29 row — disjoint r | #73 / PR #90 | patch changeset for `@dexpace/core`: one idempotency key per logical request across retry attempts; `.changeset/2026-08-26-recovery-chain-primitives.md:14`'s "single `try`/`catch`" description of `dispatchWithRecovery` is now two functions (guarantee unchanged) | | #76 / PR #92 | patch changeset for `@dexpace/core`: `timeoutMs` rejects non-integers and values above `2**32-1` (was accepted, threw at send); lone surrogates rejected at `QueryParamsBuilder.add` and path substitution; `getAll` frozen on every path | | #77 / PR #91 | patch changeset for `@dexpace/core`: empty chunk in `streamBody` is a contract violation; multipart boundary quoted when not a token; logging tap safe after `close()`; `.d.ts` `@throws` prose changed | +| #78 / PR #93 | patch changeset for `@dexpace/core`: non-finite `delayOverride` ignored with a warning (was a `RangeError` after one send); `computeDelay` returns `0` not `NaN` for `initialDelayMs: 0` at overflow | +| #79 / PR #94 | patch changesets for `@dexpace/core` (paginator closes on malformed `PageInfo`; SSE release failure reported once; splice surrogate typed) and `@dexpace/codec-json` (abortable reads/writes; `tristate` rejects present null) | +| #80 / PR #95 | minor changeset for `@dexpace/core`: `PipelineOptions` (new public shape), `LoggingStepSettings.configKey`, context restored after `send()`; `.changeset/2026-09-04-per-operation-span.md`'s `createRuntime(...)` example is stale | From f02dd8e62621bcb4828ba4df180b2373f0239d1e Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:30:57 +0300 Subject: [PATCH 086/100] fix(transport): degrade the headers a native client refuses, on both adapters `undiciTransport().send()` failed outright for four model-valid headers, where TRANSPORT-12 requires the header to give way and the request to dispatch. undici 6.28.0 rejects `expect` (`NotSupportedError`), `keep-alive` and `upgrade` (`InvalidArgumentError`) unconditionally, `connection` with any value but `close`/`keep-alive`, and any name outside RFC 9110 `token` -- while `@dexpace/core` admits every printable ASCII byte in a name, so `X Custom` is model-valid and unsendable. `toDispatchError` mapped all five to a bare non-retryable `TypeError`; nothing reached the wire. The fetch twin was no better on the runtime that ships. Node's global `fetch` is undici-backed and undici's `Headers` deliberately does not implement the WHATWG forbidden-name list, so `expect`/`keep-alive`/`upgrade` reach the same validation and `fetch()` rejects with a bare `TypeError: fetch failed` -- which this transport can only classify as the RETRYABLE `TransportFailureError`, so a permanent misconfiguration spends the caller's whole retry budget re-proving itself. Bun 1.3.14 diverges a third way: it forwards `expect` and `keep-alive` to the wire and hangs indefinitely on `upgrade`. Measured on both, 2026-09-05. - `UNDICI_FORBIDDEN_HEADERS` gains `expect`, `keep-alive`, `upgrade`. - `FETCH_FORBIDDEN_HEADERS` gains the same three (`connection` was already in it), which is the widening D19 left conditional on a row proving it. - `toUndiciHeaders` validates each name against RFC 9110 `token` and each `connection` value against undici's own two, dropping and logging what fails -- the degrade transport-fetch gets for free from `try`/`catch` around `Headers.append`. Value grammar is not re-checked: undici's admits obs-text, which is strictly wider than the outbound rule `mapOutboundHeaders` applied. Five conformance rows both transports run (`Expect`, `Keep-Alive`, `Upgrade`, a non-token name, `Connection: upgrade`); all five were red on undici and three on fetch beforehand. Plus two unit rows against a recording dispatcher, and one Node-conformance case -- the Bun rows prove a weaker claim than Node's, because only on Node does an undropped name reject the send. Refs #81, #67. Deviation ledger: none -- TRANSPORT-11's drop set is transport-specific by its own text, and TRANSPORT-12 is now satisfied rather than departed from. --- .../transport-conformance/src/run-suite.ts | 81 +++++++++++- .../transport-fetch/src/fetch-transport.ts | 20 ++- packages/transport-undici/README.md | 9 +- .../src/undici-transport.test.ts | 119 +++++++++++++++--- .../transport-undici/src/undici-transport.ts | 74 ++++++++++- tests/node-conformance/transport.test.mjs | 38 ++++++ 6 files changed, 313 insertions(+), 28 deletions(-) diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index e7a416f..476bf77 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -1,11 +1,13 @@ // SPDX-License-Identifier: MIT // packages/transport-conformance/src/run-suite.ts // The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot -// drift. Exercises: TRANSPORT-1..9, TRANSPORT-14..17, TRANSPORT-20..21, TRANSPORT-23..27, +// drift. Exercises: TRANSPORT-1..9, TRANSPORT-11..17, TRANSPORT-20..21, TRANSPORT-23..27, // TRANSPORT-29, SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is // answerable for them (the repeated-challenge-header row). TRANSPORT-10..13's SHARED half -- the one // outbound header pass both adapters call -- is asserted at its source in -// @dexpace/transport-shared, and the rows here cover only what each adapter decides for itself. +// @dexpace/transport-shared; the rows here cover what each adapter decides for itself, which since +// audit #67 / #81 includes TRANSPORT-11/12's per-header degrade, because the two adapters had four +// different answers for the same model-valid header and only a shared row could say so. // TRANSPORT-18/28's collapses are Deviation Ledger rows; TRANSPORT-30's // full flow is transport-undici's challenge-handler.test.ts. TRANSPORT-22 is NOT driven from here -- // forcing an adaptation throw needs a per-transport hook into the native response, so each adapter @@ -538,6 +540,80 @@ function registerHeaderRows(ctx: SuiteContext): void { }); } +/** + * Headers every model layer in this SDK accepts and at least one shipped native client refuses + * outright, paired with the value that provokes the refusal. + * + * `expect`, `keep-alive` and `upgrade` are `undici`'s three unconditional rejections + * (`lib/core/request.js:398,409` — `InvalidArgumentError` for the first two, `NotSupportedError` + * for `expect`), and Node's global `fetch` is undici-backed, so both adapters meet them. `X Custom` + * is the non-token name: `@dexpace/core` admits any printable ASCII byte in a header name + * (`http/ascii-validation.ts:29-33`), while both native layers require RFC 9110 `token`. + * + * TRANSPORT-12 is what makes these one table rather than four transport-specific quirks: whatever + * the native client refuses, the transport drops *that header only* and still dispatches. + */ +const NATIVE_REJECTED_HEADERS: readonly (readonly [string, string])[] = [ + ['Expect', '100-continue'], + ['Keep-Alive', 'timeout=5'], + ['Upgrade', 'websocket'], + ['X Custom', 'model-valid, non-token'], +]; + +function registerNativeRejectionRows(ctx: SuiteContext): void { + describe('TRANSPORT-11/12/13: a header the native client refuses degrades to a logged drop', () => { + for (const [name, value] of NATIVE_REJECTED_HEADERS) { + test(`${name} is dropped and logged, and the rest of the request still dispatches`, async () => { + let echoed: Record = {}; + const dropped = await captureDroppedHeaders(async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers( + Headers.newBuilder() + .set(name, value) + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(); + echoed = await readEchoedHeaders(transport, request); + }); + }); + // The send resolved at all, which is TRANSPORT-12's "the resulting native exception MUST NOT + // escape the send contract"; the two header assertions are its "the bad header is absent, + // the normal header present". + expect(echoed[name.toLowerCase()]).toBeUndefined(); + expect(echoed['x-pass-through']).toBe('survives'); + // TRANSPORT-13: the drop is discoverable by name, never silent. + expect(dropped).toContain(name.toLowerCase()); + }); + } + + test('a Connection value the native client cannot carry is dropped on both transports', async () => { + // `Connection` is the one name whose drop set legitimately differs (TRANSPORT-11's own note, + // and `registerDropSetRows` below), so this row asserts the intersection: `upgrade` is neither + // `close` nor `keep-alive`, undici rejects it outright (`request.js:400-404`), and the WHATWG + // layer forbids the name entirely. Both must drop it, whichever reason applies. + // + // Asserted through the log rather than the echo for the same reason `registerDropSetRows` + // does: both clients set a `Connection` header of their own, so the wire cannot tell a + // forwarded caller header from the client's. + const dropped = await captureDroppedHeaders(async () => { + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(ctx.url('/echo-headers')) + .headers(Headers.newBuilder().set('Connection', 'upgrade').build()) + .build(); + const response = await transport.send(request); + expect(response.status.code).toBe(200); + await response.close(); + }); + }); + expect(dropped).toContain('connection'); + }); + }); +} + /** * The fixture's challenge list, recovered from however this transport chose to split it. * @@ -687,6 +763,7 @@ export function runTransportConformanceSuite( registerCancellationRows(ctx); registerLifecycleRows(ctx); registerHeaderRows(ctx); + registerNativeRejectionRows(ctx); registerInboundHeaderRows(ctx); registerDropSetRows(ctx); registerScopedRows(ctx); diff --git a/packages/transport-fetch/src/fetch-transport.ts b/packages/transport-fetch/src/fetch-transport.ts index e76236d..2cae39e 100644 --- a/packages/transport-fetch/src/fetch-transport.ts +++ b/packages/transport-fetch/src/fetch-transport.ts @@ -38,15 +38,29 @@ import { const REDIRECT_MODE = 'manual' as const; /** - * TRANSPORT-11's outbound drop set for this transport. `connection` is in it because WHATWG `fetch` - * treats it as a forbidden request header and would strip it silently — dropping it here makes the - * removal observable through the drop log instead. + * TRANSPORT-11's outbound drop set for this transport. + * + * `connection` is in it because WHATWG `fetch` treats it as a forbidden request header and would + * strip it silently — dropping it here makes the removal observable through the drop log instead. + * + * `expect`, `keep-alive` and `upgrade` are in it because the *implementations* do not honour the + * WHATWG forbidden-header list at all. Node's global `fetch` is undici-backed and undici's `Headers` + * deliberately does not implement forbidden names in a non-browser environment, so the three reach + * `lib/core/request.js:398,409` and reject the dispatch; `fetch()` then rejects with a bare + * `TypeError: fetch failed`, which this transport can only classify as the **retryable** + * `TransportFailureError` — a permanent misconfiguration that spends the caller's whole retry budget + * re-proving itself. Bun 1.3.14 diverges again: it forwards `expect` and `keep-alive` to the wire + * and hangs indefinitely on `upgrade`. Measured on both, 2026-09-05, audit #67 / #81. Neither + * outcome is TRANSPORT-12's, and a name in the drop set is the one behaviour that is. */ const FETCH_FORBIDDEN_HEADERS = [ 'content-length', 'host', 'transfer-encoding', 'connection', + 'expect', + 'keep-alive', + 'upgrade', ] as const; /** diff --git a/packages/transport-undici/README.md b/packages/transport-undici/README.md index 5858f98..43c7a40 100644 --- a/packages/transport-undici/README.md +++ b/packages/transport-undici/README.md @@ -87,7 +87,14 @@ silently misbehaving (`TRANSPORT-30`): - Redirects are pinned off (`maxRedirections: 0`) even behind a bring-your-own dispatcher that may carry a redirect interceptor. The pipeline is the single redirect authority. - `Connection` is **not** dropped outbound — §17's own note is that an undici-class transport - forwards it. `Content-Length`, `Host`, and `Transfer-Encoding` are. + forwards it — but only with a value undici will carry (`close` or `keep-alive`, matched + case-insensitively). Any other value is dropped, because undici rejects it outright. +- `Content-Length`, `Host` and `Transfer-Encoding` are dropped because undici computes them; + `Expect`, `Keep-Alive` and `Upgrade` because undici refuses them + (`InvalidArgumentError`/`NotSupportedError` out of its own argument validation, before anything + reaches the wire). So is any header name outside RFC 9110 `token` — `@dexpace/core` admits every + printable ASCII byte in a name, undici does not. Every one of these is a drop logged by name + (`TRANSPORT-12`), never a failed send. - Destroying the dispatcher mid-flight surfaces as the terminal `CancellationError`, while a timeout on the same path stays the retryable `TransportFailureError` (`TRANSPORT-8`). - `Response.protocol` is always `HTTP_1_1`: undici's `ResponseData` does not surface the negotiated diff --git a/packages/transport-undici/src/undici-transport.test.ts b/packages/transport-undici/src/undici-transport.test.ts index 862a718..d3dd2b8 100644 --- a/packages/transport-undici/src/undici-transport.test.ts +++ b/packages/transport-undici/src/undici-transport.test.ts @@ -348,25 +348,41 @@ describe('undiciTransport disposal (TRANSPORT-15/16)', () => { }); }); +/** + * A `Dispatcher` that records every `request()` it is handed and answers 200 with an empty body, so + * a row can assert what reached undici's argument validation rather than what reached the wire. + * Returns the recording array; the caller builds the transport around it. + */ +function recordingDispatcher(sink: Dispatcher.RequestOptions[]): Dispatcher { + return { + request: (options: Dispatcher.RequestOptions) => { + sink.push(options); + return Promise.resolve({ + statusCode: 200, + headers: {}, + body: { + destroy: () => undefined, + [Symbol.asyncIterator]: () => ({ + next: () => Promise.resolve({done: true, value: undefined}), + }), + }, + } as unknown as Dispatcher.ResponseData); + }, + close: () => Promise.resolve(), + } as unknown as Dispatcher; +} + +/** The flat `[name, value, ...]` array one recorded dispatch carried. */ +function dispatchedHeaders( + sent: Dispatcher.RequestOptions | undefined, +): string[] { + return (sent?.headers ?? []) as string[]; +} + describe('undiciTransport dispatch', () => { test('TRANSPORT-2/11: redirects are pinned off and Connection is forwarded, not dropped', async () => { const dispatched: Dispatcher.RequestOptions[] = []; - const recorder = { - request: (options: Dispatcher.RequestOptions) => { - dispatched.push(options); - return Promise.resolve({ - statusCode: 200, - headers: {}, - body: { - destroy: () => undefined, - [Symbol.asyncIterator]: () => ({ - next: () => Promise.resolve({done: true, value: undefined}), - }), - }, - } as unknown as Dispatcher.ResponseData); - }, - close: () => Promise.resolve(), - } as unknown as Dispatcher; + const recorder = recordingDispatcher(dispatched); const transport = undiciTransport({dispatcher: recorder}); const request = Request.newBuilder() @@ -383,12 +399,81 @@ describe('undiciTransport dispatch', () => { const sent = dispatched[0]; expect(sent?.maxRedirections).toBe(0); expect(sent?.path).toBe('/anything?q=1'); - const headers = sent?.headers as string[]; + const headers = dispatchedHeaders(sent); expect(headers).toContain('Connection'); expect(headers).not.toContain('Content-Length'); }); }); +describe('undiciTransport outbound header degradation (TRANSPORT-11/12)', () => { + test('every header undici refuses is dropped before dispatch, and logged by name', async () => { + const dispatched: Dispatcher.RequestOptions[] = []; + const transport = undiciTransport({ + dispatcher: recordingDispatcher(dispatched), + headerDropLogging: 'all', + }); + const capture = captureDroppedHeaders(); + try { + const request = Request.newBuilder() + .url(`${origin}/anything`) + .headers( + Headers.newBuilder() + .set('Expect', '100-continue') + .set('Keep-Alive', 'timeout=5') + .set('Upgrade', 'websocket') + .set('X Custom', 'model-valid, non-token') + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(); + await (await transport.send(request)).close(); + await transport.close(); + const headers = dispatchedHeaders(dispatched[0]); + // Asserted against the argument array, not the wire: undici validates in `new Request(...)`, + // so a name that reaches this array is a name that would have thrown (audit #67 / #81). + expect(headers).toEqual(['X-Pass-Through', 'survives']); + for (const name of ['expect', 'keep-alive', 'upgrade', 'x custom']) { + expect(capture.dropped).toContain(name); + } + } finally { + capture.restore(); + } + }); + + test('a Connection value undici cannot carry is dropped, close and keep-alive are not', async () => { + const carried: string[] = []; + const dropped: string[] = []; + for (const value of ['close', 'Keep-Alive', 'upgrade']) { + const dispatched: Dispatcher.RequestOptions[] = []; + const transport = undiciTransport({ + dispatcher: recordingDispatcher(dispatched), + headerDropLogging: 'all', + }); + const capture = captureDroppedHeaders(); + try { + const request = Request.newBuilder() + .url(`${origin}/anything`) + .headers(Headers.newBuilder().set('Connection', value).build()) + .build(); + await (await transport.send(request)).close(); + await transport.close(); + if (dispatchedHeaders(dispatched[0]).includes('Connection')) { + carried.push(value); + } + // `createDropLogger` lower-cases the name it logs, so the field is `connection`. + if (capture.dropped.includes('connection')) dropped.push(value); + } finally { + capture.restore(); + } + } + // `Keep-Alive` mixed-cased on purpose: undici lower-cases the value before comparing + // (`lib/core/request.js:401`), so this transport must too or it would drop a header undici + // would have carried. + expect(carried).toEqual(['close', 'Keep-Alive']); + expect(dropped).toEqual(['upgrade']); + }); +}); + describe('undiciTransport body and adaptation paths', () => { test('TRANSPORT-28: a file body dispatches exactly its declared byte range', async () => { const dir = await mkdtemp(join(tmpdir(), 'undici-file-body-')); diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index 0853169..e5fb83a 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -52,13 +52,26 @@ const require = createRequire(import.meta.url); const undici = require('undici/index.js') as typeof import('undici'); /** - * TRANSPORT-11's outbound drop set for this transport. `connection` is deliberately absent — §17's - * own note is that an undici-class transport forwards it rather than dropping it. + * TRANSPORT-11's outbound drop set for this transport: the three framing headers undici computes + * itself, plus the three it refuses outright. + * + * `expect`, `keep-alive` and `upgrade` are undici's unconditional rejections + * (`lib/core/request.js:398,409` in 6.28.0 — `InvalidArgumentError` for the first two, + * `NotSupportedError` for `expect`). Until 2026-09-05 they were absent, so a caller who set any of + * them got a failed send out of `undiciTransport().send()` where the fetch twin dispatched + * (audit #67 / #81). TRANSPORT-12 says the header is what gives way, not the request. + * + * `connection` is deliberately absent — §17's own note is that an undici-class transport forwards + * it rather than dropping it. undici carries only `close` and `keep-alive`, so a third value is + * dropped per-request by {@link isUnsendableHeader} rather than by name here. */ const UNDICI_FORBIDDEN_HEADERS: readonly string[] = [ 'content-length', 'host', 'transfer-encoding', + 'expect', + 'keep-alive', + 'upgrade', ]; /** @@ -146,7 +159,52 @@ function selectDispatchers(options: UndiciTransportOptions): DispatcherSet { return {proxied, direct, owned: [proxied, direct]}; } -/** undici's flat `[name, value, name, value, ...]` form -- the only shape that keeps a repeated name repeated (HTTP-14). */ +/** + * RFC 9110 `token`, which is what undici's `isValidHTTPToken` enforces on every header name + * (`lib/core/util.js:547-587`): VCHAR minus the delimiters `"(),/:;<=>?@[\]{}`. Anchored and + * one-or-more, so the empty name undici also rejects fails here too. + * + * `@dexpace/core` is deliberately laxer: `hasForbiddenNameByte` admits every printable ASCII byte + * (`http/ascii-validation.ts:29-33`), so `X Custom` is a model-valid name that no native client on + * this platform will carry. + */ +const RFC9110_TOKEN = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/u; + +/** The only two `Connection` values undici carries; anything else is `InvalidArgumentError`. */ +const FORWARDABLE_CONNECTION_VALUES: ReadonlySet = new Set([ + 'close', + 'keep-alive', +]); + +/** + * Whether undici's own request validation would reject this header, which for TRANSPORT-12 means it + * is dropped rather than allowed to fail the send. + * + * Only the *name* grammar and the `connection` value are checked. undici's value grammar + * (`headerCharRegex`, `lib/core/util.js:598`) admits HTAB, 0x20-0x7E **and** obs-text 0x80-0xFF, + * which is strictly wider than the outbound value rule `mapOutboundHeaders` has already applied + * (HTTP-18), so no value that reaches here can fail there. + * + * @param name - the header name, as the caller spelled it. + * @param value - the header value. + * @returns `true` when the header must be dropped instead of dispatched. + */ +function isUnsendableHeader(name: string, value: string): boolean { + if (!RFC9110_TOKEN.test(name)) return true; + return ( + name.toLowerCase() === 'connection' && + !FORWARDABLE_CONNECTION_VALUES.has(value.toLowerCase()) + ); +} + +/** + * undici's flat `[name, value, name, value, ...]` form -- the only shape that keeps a repeated name + * repeated (HTTP-14) -- with every header undici would refuse degraded to a logged drop. + * + * The per-header guard is the same degrade `@dexpace/transport-fetch` gets for free from wrapping + * `Headers.append` in a `try`/`catch` (`fetch-transport.ts:146-152`): undici validates at dispatch + * rather than at construction, so this transport has to ask the question itself (TRANSPORT-12). + */ function toUndiciHeaders( request: Request, forbidden: readonly string[], @@ -155,8 +213,14 @@ function toUndiciHeaders( const {sent, dropped} = mapOutboundHeaders(request.headers, forbidden, { bodyDerivedMediaType: request.body?.mediaType, }); - logDrops(dropped); - return [...sent.entries()].flat(); + const degraded = [...dropped]; + const flat: string[] = []; + for (const [name, value] of sent.entries()) { + if (isUnsendableHeader(name, value)) degraded.push(name); + else flat.push(name, value); + } + logDrops(degraded); + return flat; } /** diff --git a/tests/node-conformance/transport.test.mjs b/tests/node-conformance/transport.test.mjs index c6e16f6..23e4caa 100644 --- a/tests/node-conformance/transport.test.mjs +++ b/tests/node-conformance/transport.test.mjs @@ -12,6 +12,8 @@ // // Exercises: TRANSPORT-1 (redirects not followed), TRANSPORT-4/20 (timeout and no-response classification), // TRANSPORT-17 (a single-use body written once, its bytes on the wire), TRANSPORT-24 (vendor status codes), +// TRANSPORT-11/12 (a header the native layer refuses is dropped, not a failed send -- Node's undici-backed +// `fetch` rejects three names Bun's forwards), // TRANSPORT-28/BODY-11 (a real fileBody() over the wire, whole and ranged), // TRANSPORT-25 (the response body is a lazily-read stream and close releases it), TRANSPORT-29/SEAM-12 // (concurrent sends), SEAM-16 (an abort after delivery must not close the delivered body). @@ -140,6 +142,42 @@ describe('the transport adapters on the Node runtime', () => { } }); + it('drops the headers the native layer refuses rather than failing the send (TRANSPORT-11/12)', async () => { + // The one case in this file where Bun and Node disagree about the *outcome*, not merely the + // implementation. `Expect`, `Keep-Alive` and `Upgrade` are undici's three unconditional + // rejections, and Node's global `fetch` is undici-backed, so on this runtime an undropped + // one rejects the send: `TypeError: fetch failed` through transport-fetch, which can only + // be classified as the RETRYABLE TransportFailureError, and a terminal TypeError through + // transport-undici. Bun's `fetch` instead forwards `Expect`/`Keep-Alive` to the wire and + // hangs on `Upgrade`, so the Bun conformance rows for these names prove a weaker claim than + // this one does (audit #67 / #81, measured 2026-09-05). + const transport = makeTransport(); + try { + const response = await transport.send( + Request.newBuilder() + .url(`${origin}/echo`) + .headers( + Headers.newBuilder() + .set('Expect', '100-continue') + .set('Keep-Alive', 'timeout=5') + .set('Upgrade', 'websocket') + .set('X Custom', 'model-valid, non-token') + .set('X-Pass-Through', 'survives') + .build(), + ) + .build(), + RequestOptions.newBuilder().timeoutMs(2_000).build(), + ); + const echoed = JSON.parse(await response.text()); + for (const name of ['expect', 'keep-alive', 'upgrade', 'x custom']) { + assert.equal(echoed.headers[name], undefined, name); + } + assert.equal(echoed.headers['x-pass-through'], 'survives'); + } finally { + await transport.close(); + } + }); + it('surfaces a vendor status with a readable body (TRANSPORT-24)', async () => { const transport = makeTransport(); try { From 61fce6d08c38f15654016216fcab842966d1abf1 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:38:32 +0300 Subject: [PATCH 087/100] fix(transport-undici): write a file body through its own writeTo (BODY-13) The file branch of `prepareBody` handed `createReadStream(path, {start, end})` straight to undici. That is one fewer userspace copy, and it skipped the descriptor's `writeTo` entirely, so `@dexpace/body-file`'s `transferred === count` invariant never ran. `content-length` is dropped outbound, so undici framed the body chunked and the wire could not detect a short write either: a file truncated between `stat` and `send` POSTed its surviving bytes and resolved 200, where `@dexpace/transport-fetch` raised `TransportFailureError`. The Phase 8a checklist marked BODY-13 done while recording the bypass. The branch is gone rather than rerouted, so `prepareBody` is now the same function on both transports: buffered at or below 1,000,000 declared bytes, streamed above. D19 said "takes the same `pumpBody` path the streamed case uses"; deleting the branch is that for a large file and the buffered sibling for a small one, and it is the only version in which the two adapters cannot drift again. Framing changes for a small file body -- `content-length` where it used to be chunked, matching the fetch twin. TRANSPORT-28's zero-copy clause is a SHOULD no user-space path in either client can honour (`docs/deviations.md` item 13, recorded since Phase 8a). Its MUSTs -- a file body is replayable, and exactly its declared byte range reaches the wire -- are honoured by the descriptor, on both transports, by one code path. The zero-count case needs no branch of its own: `isMaterializable` admits `contentLength === 0` and `materializeBody` never opens a read stream. Rows: a shared conformance row on the buffered path (intact ranged body through `writeTo`, and truncate-after-stat), red on undici and green on fetch before this; the streamed leg in `tests/node-conformance/transport.test.mjs` with a real `fileBody()`, likewise red on undici only. The streamed leg cannot live in the Bun suite -- Bun 1.3.14's `Readable.fromWeb` leaks the abort reason as unhandled rejections when the web readable behind it is aborted mid-pull, on both transports and with no SDK code involved (isolated; `node --test` is clean). `run-suite.ts`'s constant says so, so nobody raises it back. Refs #81, #67. Deviation ledger: none -- BODY-13 is now satisfied rather than departed from, and TRANSPORT-28's SHOULD already has item 13. --- .../transport-conformance/src/fixtures.ts | 76 +++++++++++ .../transport-conformance/src/run-suite.ts | 123 +++++++++++++++++- packages/transport-undici/README.md | 16 ++- .../src/undici-transport.test.ts | 114 +++++++++------- .../transport-undici/src/undici-transport.ts | 45 +++---- tests/node-conformance/transport.test.mjs | 61 ++++++++- 6 files changed, 355 insertions(+), 80 deletions(-) diff --git a/packages/transport-conformance/src/fixtures.ts b/packages/transport-conformance/src/fixtures.ts index fe20b7f..6088a21 100644 --- a/packages/transport-conformance/src/fixtures.ts +++ b/packages/transport-conformance/src/fixtures.ts @@ -1,11 +1,13 @@ // SPDX-License-Identifier: MIT // packages/transport-conformance/src/fixtures.ts +import {createReadStream} from 'node:fs'; import { createServer, type IncomingMessage, type Server, type ServerResponse, } from 'node:http'; +import type {FileBodyDescriptor} from '@dexpace/core'; /** A running fixture server, addressable by URL and shut down through {@link TestServer.close}. */ export interface TestServer { @@ -168,3 +170,77 @@ export function startFixtureServer(): Promise { }); }); } + +/** What {@link fileBodyFixture} needs beyond the path; mirrors `fileBody()`'s own option bag. */ +export interface FileBodyFixtureOptions { + /** The byte offset the descriptor declares; defaults to 0. */ + readonly start?: number; + /** The byte count the descriptor declares, captured as `fileBody()` captures it from `stat`. */ + readonly count: number; + /** Incremented on every `writeTo` call, so a row can assert the transport used it. */ + readonly writes?: {count: number}; +} + +/** + * A `kind: 'file'` request body over a real path, carrying BODY-13's `transferred === count` check + * itself — the shape `@dexpace/body-file`'s `fileBody()` produces, minus the construction-time + * validation no row here needs. + * + * **Deliberately a stand-in, not the real factory.** `@dexpace/transport-conformance` is `private`, + * resolves unbuilt, and depends on `@dexpace/core` alone; taking `@dexpace/body-file` would put a + * ninth entry in the root `build:deps` chain for one row. A real `fileBody()` crossing a real + * transport already has a home — `tests/node-conformance/transport.test.mjs`, which is the only + * layer that can host it. What a *transport* is answerable for is narrower, and is exactly what this + * exercises: TRANSPORT-28's structural recognition on `kind` alone, and that the declared length is + * honoured by calling the descriptor's own `writeTo` rather than by reading `path` behind its back. + * + * @param path - the file to stream; read fresh on every `writeTo`, as BODY-11 requires. + * @param options - the declared range, and an optional write counter. + * @returns a frozen descriptor a transport must recognise structurally. + */ +export function fileBodyFixture( + path: string, + options: FileBodyFixtureOptions, +): FileBodyDescriptor { + const start = options.start ?? 0; + const {count} = options; + return Object.freeze({ + kind: 'file' as const, + mediaType: 'application/octet-stream', + contentLength: count, + replayable: true, + path, + start, + count, + async writeTo(sink: WritableStream): Promise { + if (options.writes !== undefined) options.writes.count += 1; + const writer = sink.getWriter(); + if (count === 0) { + writer.releaseLock(); + return; + } + const stream = createReadStream(path, {start, end: start + count - 1}); + let transferred = 0; + try { + for await (const chunk of stream) { + const bytes = chunk as Buffer; + await writer.write(new Uint8Array(bytes)); + transferred += bytes.byteLength; + } + if (transferred !== count) { + // BODY-13's exact sentence, and its exact wording in `@dexpace/body-file`: the error names + // transferred-of-total, so a row can assert the numbers rather than only the class. + throw new Error( + `short write: transferred ${String(transferred)} of ${String(count)} bytes`, + ); + } + } catch (error) { + await writer.abort(error); + throw error; + } finally { + stream.destroy(); + writer.releaseLock(); + } + }, + }); +} diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 476bf77..9d6a318 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -1,18 +1,22 @@ // SPDX-License-Identifier: MIT // packages/transport-conformance/src/run-suite.ts // The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot -// drift. Exercises: TRANSPORT-1..9, TRANSPORT-11..17, TRANSPORT-20..21, TRANSPORT-23..27, -// TRANSPORT-29, SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is +// drift. Exercises: TRANSPORT-1..9, TRANSPORT-11..17, TRANSPORT-20..21, TRANSPORT-23..29, BODY-13, +// SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is // answerable for them (the repeated-challenge-header row). TRANSPORT-10..13's SHARED half -- the one // outbound header pass both adapters call -- is asserted at its source in // @dexpace/transport-shared; the rows here cover what each adapter decides for itself, which since // audit #67 / #81 includes TRANSPORT-11/12's per-header degrade, because the two adapters had four // different answers for the same model-valid header and only a shared row could say so. -// TRANSPORT-18/28's collapses are Deviation Ledger rows; TRANSPORT-30's +// TRANSPORT-18's collapse is a Deviation Ledger row, as is TRANSPORT-28's zero-copy SHOULD, whose two +// MUSTs the file-body rows do assert; TRANSPORT-30's // full flow is transport-undici's challenge-handler.test.ts. TRANSPORT-22 is NOT driven from here -- // forcing an adaptation throw needs a per-transport hook into the native response, so each adapter // asserts it against its own (transport-fetch's fetch-transport.test.ts:118, transport-undici's // undici-transport.test.ts:503). +import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; import {afterAll, beforeAll, describe, expect, test} from 'bun:test'; import { getBuildInfo, @@ -26,6 +30,7 @@ import { type Transport, } from '@dexpace/core'; import { + fileBodyFixture, REPEATED_CHALLENGES, startFixtureServer, type TestServer, @@ -614,6 +619,117 @@ function registerNativeRejectionRows(ctx: SuiteContext): void { }); } +/** + * The declared length of the truncate-after-stat file body, deliberately **below** both shipped + * adapters' 1,000,000-byte materialize bound, so this row drives the buffered path. + * + * The streamed path is asserted in `tests/node-conformance/transport.test.mjs` instead, and that is + * not a preference. Bun 1.3.14's `Readable.fromWeb` leaks the abort reason as two or three unhandled + * rejections when the web readable behind it is aborted mid-pull, which is exactly what a producer + * failure on the streamed path does; `bun:test` then fails whichever row happens to be running. + * Isolated to sixteen lines with no SDK code in them, and clean under `node --test` on both + * transports (measured 2026-09-05, audit #67 / #81). Raising this constant past 1,000,000 will make + * the row red for that reason and no other. + */ +const TRUNCATED_FILE_BYTES = 64; + +/** How far a truncate-after-stat cuts the file back; small enough that no read can be a full one. */ +const TRUNCATED_TO_BYTES = 10; + +/** Distinguishable bytes, so a misaligned send fails on content and not merely on length. */ +function fileFixtureBytes(size: number): Uint8Array { + const bytes = new Uint8Array(size); + for (let index = 0; index < size; index += 1) { + bytes[index] = 33 + ((index * 7) % 94); + } + return bytes; +} + +/** Writes `size` fixture bytes to a fresh temporary file, runs `body`, and removes the directory. */ +async function withFixtureFile( + size: number, + body: (path: string) => Promise, +): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dexpace-conformance-file-')); + try { + const path = join(dir, 'payload.bin'); + await writeFile(path, fileFixtureBytes(size)); + return await body(path); + } finally { + await rm(dir, {recursive: true, force: true}); + } +} + +/** + * Every message on `error` and its `cause` chain, joined. + * + * BODY-13's transferred-of-total text surfaces at a different depth per path: the streamed path + * rethrows the producer's own failure as the message, while the buffered path wraps it as a cause + * under a fixed "request body could not be written". Both satisfy the requirement; asserting on the + * chain is what lets one row cover both without pinning either transport's wrapper wording. + */ +function messageChain(error: unknown): string { + const parts: string[] = []; + let current: unknown = error; + for (let depth = 0; depth < 8 && current instanceof Error; depth += 1) { + parts.push(current.message); + current = current.cause; + } + return parts.join(' <- '); +} + +function registerFileBodyRows(ctx: SuiteContext): void { + describe('TRANSPORT-28, BODY-13: a file body is dispatched through its own writeTo', () => { + test('an intact ranged file body puts exactly its declared bytes on the wire', async () => { + await withFixtureFile(64, async path => { + const writes = {count: 0}; + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder() + .method('POST') + .url(ctx.url('/echo-body')) + .body(fileBodyFixture(path, {start: 10, count: 20, writes})) + .build(), + ); + expect([...(await response.bytes())]).toEqual([ + ...fileFixtureBytes(64).slice(10, 30), + ]); + }); + // TRANSPORT-17's counterpart for a replayable body: a transport that reads `path` itself + // rather than calling `writeTo` would put the same bytes on the wire and leave this at 0. + expect(writes.count).toBe(1); + }); + }); + + test('a file body truncated after its length was captured fails the send', async () => { + await withFixtureFile(TRUNCATED_FILE_BYTES, async path => { + // The descriptor is built first, exactly as `fileBody()` captures `count` from `stat`, and + // the file shrinks underneath it afterwards. `content-length` is dropped outbound, so the + // wire cannot detect this either — BODY-13's check inside `writeTo` is the only thing that + // can, and a transport that hands the path to its native client never runs it + // (audit #67 / #81, where undici POSTed the ten surviving bytes and resolved 200). + const body = fileBodyFixture(path, {count: TRUNCATED_FILE_BYTES}); + await truncate(path, TRUNCATED_TO_BYTES); + const error = await withTransport(ctx.makeTransport, transport => + rejection( + transport.send( + Request.newBuilder() + .method('POST') + .url(ctx.isolatedUrl('/echo-body')) + .body(body) + .build(), + ), + ), + ); + expect(error).toMatchObject({name: 'TransportFailureError'}); + expect(messageChain(error)).toContain( + `transferred ${String(TRUNCATED_TO_BYTES)} of ${String(TRUNCATED_FILE_BYTES)} bytes`, + ); + }); + }); + }); +} + /** * The fixture's challenge list, recovered from however this transport chose to split it. * @@ -764,6 +880,7 @@ export function runTransportConformanceSuite( registerLifecycleRows(ctx); registerHeaderRows(ctx); registerNativeRejectionRows(ctx); + registerFileBodyRows(ctx); registerInboundHeaderRows(ctx); registerDropSetRows(ctx); registerScopedRows(ctx); diff --git a/packages/transport-undici/README.md b/packages/transport-undici/README.md index 43c7a40..6cfd739 100644 --- a/packages/transport-undici/README.md +++ b/packages/transport-undici/README.md @@ -79,11 +79,17 @@ silently misbehaving (`TRANSPORT-30`): ## Behavior worth knowing -- File bodies (`body.kind === 'file'`, e.g. `@dexpace/body-file`'s `fileBody()`) dispatch straight - off the file honoring `start`/`count`, one fewer userspace copy than the `fetch` transport - (`TRANSPORT-28`; a literal kernel zero-copy path does not exist on Node — see the Deviation - Ledger). Recognition is structural, on `kind` alone: this package does not depend on - `@dexpace/body-file`. +- File bodies (`body.kind === 'file'`, e.g. `@dexpace/body-file`'s `fileBody()`) are written + through the descriptor's own `writeTo`, exactly as in the `fetch` transport — buffered below + 1 MB, streamed above it. Until 2026-09-05 this transport instead handed + `createReadStream(path, {start, end})` to undici: one fewer userspace copy, and no `writeTo`, so + `BODY-13`'s `transferred === count` check never ran and a file truncated between `stat` and + `send` uploaded its remaining bytes and returned 200. `content-length` is dropped outbound, so + the framing could not catch it either. `TRANSPORT-28`'s zero-copy clause is a SHOULD that no + user-space path in either client can honour anyway (Deviation Ledger item 13); its MUSTs — a file + body is replayable, and exactly its declared byte range reaches the wire — are honoured by the + descriptor. Recognition, where it is still needed, stays structural: this package does not depend + on `@dexpace/body-file`. - Redirects are pinned off (`maxRedirections: 0`) even behind a bring-your-own dispatcher that may carry a redirect interceptor. The pipeline is the single redirect authority. - `Connection` is **not** dropped outbound — §17's own note is that an undici-class transport diff --git a/packages/transport-undici/src/undici-transport.test.ts b/packages/transport-undici/src/undici-transport.test.ts index d3dd2b8..4d970a5 100644 --- a/packages/transport-undici/src/undici-transport.test.ts +++ b/packages/transport-undici/src/undici-transport.test.ts @@ -11,7 +11,7 @@ // TRANSPORT-19 (a header-mapping throw leaves no started body producer stranded), SEAM-30 (so no // producer rejection reaches Node's default unhandledRejection policy) import {createRequire} from 'node:module'; -import {mkdtemp, rm, writeFile} from 'node:fs/promises'; +import {mkdtemp, readFile, rm, writeFile} from 'node:fs/promises'; import {createServer, type Server} from 'node:http'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; @@ -474,73 +474,99 @@ describe('undiciTransport outbound header degradation (TRANSPORT-11/12)', () => }); }); +/** + * A `kind: 'file'` descriptor over a real path whose `writeTo` streams its own declared range and + * counts its calls -- the shape `@dexpace/body-file`'s `fileBody()` produces. Built by hand because + * TRANSPORT-28's recognition contract is structural: this package narrows on `kind` alone and must + * never `instanceof` against a package it does not depend on. + */ +function fileDescriptor( + path: string, + range: {start: number; count: number}, + writes: {count: number}, +): FileBodyDescriptor { + return { + kind: 'file', + mediaType: 'application/octet-stream', + contentLength: range.count, + replayable: true, + path, + start: range.start, + count: range.count, + async writeTo(sink: WritableStream): Promise { + writes.count += 1; + const writer = sink.getWriter(); + if (range.count === 0) { + writer.releaseLock(); + return; + } + const bytes = (await readFile(path)).subarray( + range.start, + range.start + range.count, + ); + await writer.write(new Uint8Array(bytes)); + writer.releaseLock(); + }, + }; +} + +/** Runs `body` against a fresh temporary file holding `ABCDEFGH`, and removes the directory after. */ +async function withPayloadFile( + run: (path: string) => Promise, +): Promise { + const dir = await mkdtemp(join(tmpdir(), 'undici-file-body-')); + try { + const path = join(dir, 'payload.bin'); + await writeFile(path, 'ABCDEFGH'); + return await run(path); + } finally { + await rm(dir, {recursive: true, force: true}); + } +} + describe('undiciTransport body and adaptation paths', () => { - test('TRANSPORT-28: a file body dispatches exactly its declared byte range', async () => { - const dir = await mkdtemp(join(tmpdir(), 'undici-file-body-')); - try { - const path = join(dir, 'payload.bin'); - await writeFile(path, 'ABCDEFGH'); - // The structural recognition contract, built by hand: this package must narrow on - // `kind === 'file'` alone, never on an instanceof against @dexpace/body-file, which it - // deliberately does not depend on. - const descriptor: FileBodyDescriptor = { - kind: 'file', - mediaType: 'application/octet-stream', - contentLength: 4, - replayable: true, - path, - start: 2, - count: 4, - writeTo: () => - Promise.reject(new Error('the transport must not call writeTo here')), - }; + test('TRANSPORT-28/BODY-13: a file body dispatches its declared range THROUGH its own writeTo', async () => { + await withPayloadFile(async path => { + const writes = {count: 0}; const transport = undiciTransport(); const request = Request.newBuilder() .method('POST') .url(`${origin}/upload`) - .body(descriptor) + .body(fileDescriptor(path, {start: 2, count: 4}, writes)) .build(); received.length = 0; await (await transport.send(request)).close(); await transport.close(); expect(received[0]).toBe('CDEF'); - } finally { - await rm(dir, {recursive: true, force: true}); - } + // The `writes` count is the whole point of the row. Until audit #67 / #81 this transport + // handed `createReadStream(path, {start, end})` to undici and left `writeTo` uncalled, so the + // same four bytes reached the wire with BODY-13's `transferred === count` invariant -- + // the only thing that can see a file truncated after its length was captured -- never run. + expect(writes.count).toBe(1); + }); }); test('a zero-count file body dispatches as an empty body, not a stream error', async () => { - const dir = await mkdtemp(join(tmpdir(), 'undici-empty-file-body-')); - try { - const path = join(dir, 'payload.bin'); - await writeFile(path, 'ABCDEFGH'); - // createReadStream throws ERR_OUT_OF_RANGE the moment `end` falls below `start`, which is what - // `start + count - 1` computes for count 0 -- the empty range needs its own branch. - const descriptor: FileBodyDescriptor = { - kind: 'file', - mediaType: 'application/octet-stream', - contentLength: 0, - replayable: true, - path, - start: 4, - count: 0, - writeTo: () => Promise.resolve(), - }; + await withPayloadFile(async path => { + // `isMaterializable` admits `contentLength === 0`, and `materializeBody` returns an empty + // buffer without opening anything, so the empty range needs no branch of its own. It did when + // the file body went to `createReadStream`, which throws ERR_OUT_OF_RANGE the moment `end` + // (start + count - 1) falls below `start`. + const writes = {count: 0}; const transport = undiciTransport(); received.length = 0; const response = await transport.send( Request.newBuilder() .method('POST') .url(`${origin}/upload`) - .body(descriptor) + .body(fileDescriptor(path, {start: 4, count: 0}, writes)) .build(), ); await response.close(); await transport.close(); expect(received[0]).toBe(''); - } finally { - await rm(dir, {recursive: true, force: true}); - } + expect(writes.count).toBe(1); + }); }); }); diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index e5fb83a..bac2eb0 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -1,6 +1,5 @@ // SPDX-License-Identifier: MIT // packages/transport-undici/src/undici-transport.ts -import {createReadStream} from 'node:fs'; import {createRequire} from 'node:module'; import {Readable} from 'node:stream'; import type {ReadableStream as NodeReadableStream} from 'node:stream/web'; @@ -13,7 +12,6 @@ import { Status, TransportFailureError, type Body, - type FileBodyDescriptor, type ProxyOptions, type Request, type RequestOptions, @@ -309,33 +307,28 @@ interface PreparedBody { } /** - * TRANSPORT-28's recognition contract, in one named place: a plain string-literal check, never a - * cross-package `instanceof` against `@dexpace/body-file` (which this package does not depend on). - * `Body.kind` is a union on one interface rather than a discriminated union of interfaces, so the - * narrowing has to be spelled out as a predicate. + * Prepares one request body for dispatch. Identical in shape to the fetch twin's, and deliberately + * so: there is **no** `kind === 'file'` branch here. + * + * There was one until 2026-09-05. It handed `createReadStream(path, {start, end})` to undici, which + * is a genuinely shorter path to the wire — one fewer userspace copy — and it bypassed the + * descriptor's own `writeTo`, so `@dexpace/body-file`'s `transferred === count` invariant never ran + * (BODY-13). `content-length` is dropped outbound, so undici framed the body chunked and the wire + * could not detect the short write either: a file truncated between `stat` and `send` POSTed its + * remaining bytes and resolved 200, where the fetch transport raised `TransportFailureError` + * (audit #67 / #81). A file body is now written through `writeTo` like any other, which is what + * makes BODY-13 hold on both transports and what the shared conformance row asserts. + * + * TRANSPORT-28's SHOULD is not thereby abandoned so much as re-described: no user-space path in + * either client reaches `sendfile(2)`, which `docs/deviations.md` item 13 has recorded since Phase + * 8a, and the clause's MUST — a file body is replayable, and exactly its declared byte range reaches + * the wire — is honoured by the descriptor itself, on both transports, by the same code. + * + * The zero-count case needs no branch either: `isMaterializable` admits `contentLength === 0`, and + * `materializeBody` returns an empty buffer without ever opening a read stream. */ -function isFileBody(body: Body): body is FileBodyDescriptor { - return body.kind === 'file'; -} - async function prepareBody(body: Body | undefined): Promise { if (body === undefined) return {init: null, pump: undefined}; - if (isFileBody(body)) { - // An empty range is not a degenerate read stream: `createReadStream` throws ERR_OUT_OF_RANGE the - // moment `end` (start + count - 1) falls below `start`, so a zero-count file body has to become - // an explicit empty body rather than a stream nobody can open. - if (body.count === 0) return {init: new Uint8Array(0), pump: undefined}; - // TRANSPORT-28: dispatch straight off the file, honoring start/count, rather than routing the - // bytes through a userspace TransformStream first. The closest available approximation of the - // reference's zero-copy path -- see the Deviation Ledger for why a literal one does not exist. - return { - init: createReadStream(body.path, { - start: body.start, - end: body.start + body.count - 1, - }), - pump: undefined, - }; - } if (isMaterializable(body, MAX_MATERIALIZED_BODY_BYTES)) { try { return {init: await materializeBody(body), pump: undefined}; diff --git a/tests/node-conformance/transport.test.mjs b/tests/node-conformance/transport.test.mjs index 23e4caa..b481f13 100644 --- a/tests/node-conformance/transport.test.mjs +++ b/tests/node-conformance/transport.test.mjs @@ -14,13 +14,14 @@ // TRANSPORT-17 (a single-use body written once, its bytes on the wire), TRANSPORT-24 (vendor status codes), // TRANSPORT-11/12 (a header the native layer refuses is dropped, not a failed send -- Node's undici-backed // `fetch` rejects three names Bun's forwards), -// TRANSPORT-28/BODY-11 (a real fileBody() over the wire, whole and ranged), +// TRANSPORT-28/BODY-11 (a real fileBody() over the wire, whole and ranged), BODY-13 (a truncate-after-stat +// short write fails the send on the streamed path, which only this runtime can assert), // TRANSPORT-25 (the response body is a lazily-read stream and close releases it), TRANSPORT-29/SEAM-12 // (concurrent sends), SEAM-16 (an abort after delivery must not close the delivered body). import assert from 'node:assert/strict'; import {createServer} from 'node:http'; import {after, before, describe, it} from 'node:test'; -import {mkdtemp, rm, writeFile} from 'node:fs/promises'; +import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; import {createHash} from 'node:crypto'; @@ -383,6 +384,62 @@ describe('the transport adapters on the Node runtime', () => { await transport.close(); } }); + + // BODY-13's short-write clause on the STREAMED request-body path, which is this tree's to + // hold for two independent reasons. The shared conformance row + // (`run-suite.ts`'s "a file body truncated after its length was captured") drives the + // buffered path only: above the adapters' 1,000,000-byte materialize bound the producer + // failure aborts a `TransformStream` mid-pull, and Bun 1.3.14's `Readable.fromWeb` leaks + // that abort reason as unhandled rejections, which `bun:test` scores against an unrelated + // row. Node's bridge does not. And this is the only layer where a real `fileBody()` — whose + // `transferred === count` invariant is the thing under test — meets a real transport. + // + // Until audit #67 / #81 the undici transport handed `createReadStream(path, …)` to undici + // and never called `writeTo` at all, so this case reported 200 with ten bytes on the wire + // while transport-fetch raised. `content-length` is dropped outbound, so the framing cannot + // catch it either. + it('fails a send whose file was truncated after its length was captured (BODY-13)', async () => { + const declared = 1_100_000; + const shortDir = await mkdtemp( + join(tmpdir(), 'dexpace-filebody-short-'), + ); + const shortPath = join(shortDir, 'payload.bin'); + const transport = makeTransport(); + try { + await writeFile(shortPath, fixtureBytes(declared)); + const body = fileBody(shortPath); + assert.equal(body.contentLength, declared); + await truncate(shortPath, 10); + await assert.rejects( + transport.send( + Request.newBuilder() + .method('POST') + .url(`${origin}/echo`) + .body(body) + .build(), + ), + error => { + assert.equal(error.name, 'TransportFailureError'); + // BODY-13 names transferred-of-total. The streamed path rethrows the producer's own + // message; the buffered one carries it as a cause, so both are searched. + const chain = []; + for (let at = error; at instanceof Error; at = at.cause) { + chain.push(at.message); + } + assert.ok( + chain.some(message => + message.includes(`transferred 10 of ${declared} bytes`), + ), + `no transferred-of-total in ${JSON.stringify(chain)}`, + ); + return true; + }, + ); + } finally { + await transport.close(); + await rm(shortDir, {recursive: true, force: true}); + } + }); }); }); } From 2a75c5bb33bf02eae867cb32afedf595ffba6af7 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:42:33 +0300 Subject: [PATCH 088/100] feat(transport-undici): refuse a proxy type undici cannot honour, at the factory MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `toProxyAgentOptions` used `proxy.type` as the URI scheme with no check, so `undiciTransport({proxy: createProxyOptions({type: 'socks5', …})})` reached `new ProxyAgent({uri: 'socks5://…'})` and threw undici's `InvalidArgumentError('Invalid URL protocol: socks5:')` straight out of a public factory -- untyped, undocumented, and not in the SDK's error vocabulary. The configuration can express it perfectly legitimately: core maps `ALL_PROXY`'s `socks:`, `socks4:`, `socks4a:`, `socks5:` and `socks5h:` schemes onto `ProxyType` (CFG-22, `config/proxy.ts:372-380`). `selectDispatchers` now refuses anything but `http` with a `TypeError` naming the type, matching its existing dispatcher-plus-proxy refusal and deliberately outside the `IoError` tree, so `retry/classify.ts`'s allow-list makes it non-retryable for free (RETRY-2). The check runs before `new undici.Agent(...)`, so a refused construction allocates nothing -- there would be no transport left to close it through. `@throws` on `undiciTransport` and on `UndiciTransportOptions.proxy` say so. `ProxyType` keeps `socks4`/`socks5`: narrowing a `@public` union is a release-pass decision (D1/D19), and the gap is recorded in `docs/deviations.md` instead. `fetchTransport()` needs no equivalent -- it ships no `proxy` option at all, deliberately (design doc §6). The conformance row runs on both transports. `TransportCapabilities` gains an optional `unsupportedProxy: {type, build()}`, because only the adapter knows which of `ProxyType`'s values its client refuses; where it is absent the row asserts `supportsProxy === false` rather than skipping, so "no proxy surface" and "an unasserted gap" cannot look the same. Red before this change: undici's raw error is neither a `TypeError` nor names `socks5`. Plus a unit row covering both SOCKS values and asserting no `Agent` was constructed on the way out. Refs #81, #67. --- .../transport-conformance/src/run-suite.ts | 57 ++++++++++++++++++- packages/transport-undici/README.md | 9 ++- .../src/undici-transport.conformance.test.ts | 14 +++++ .../src/undici-transport.test.ts | 29 ++++++++++ .../transport-undici/src/undici-transport.ts | 48 +++++++++++++++- 5 files changed, 153 insertions(+), 4 deletions(-) diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 9d6a318..8550577 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -10,7 +10,8 @@ // different answers for the same model-valid header and only a shared row could say so. // TRANSPORT-18's collapse is a Deviation Ledger row, as is TRANSPORT-28's zero-copy SHOULD, whose two // MUSTs the file-body rows do assert; TRANSPORT-30's -// full flow is transport-undici's challenge-handler.test.ts. TRANSPORT-22 is NOT driven from here -- +// full proxy-challenge flow is transport-undici's challenge-handler.test.ts, and only its +// unsupported-type refusal is a row here. TRANSPORT-22 is NOT driven from here -- // forcing an adaptation throw needs a per-transport hook into the native response, so each adapter // asserts it against its own (transport-fetch's fetch-transport.test.ts:118, transport-undici's // undici-transport.test.ts:503). @@ -22,6 +23,7 @@ import { getBuildInfo, getGlobalLogger, Headers, + isIoError, Request, RequestOptions, setGlobalLogger, @@ -36,6 +38,21 @@ import { type TestServer, } from './fixtures.js'; +/** + * A proxy configuration the transport under test cannot honour, so TRANSPORT-30's + * "discoverable rather than silently misbehaving" clause has something to be asserted against. + * + * Supplied by the adapter because only the adapter knows which of `ProxyType`'s values its native + * client refuses — `@dexpace/core` resolves `socks4`/`socks5` from `ALL_PROXY` (CFG-22) and neither + * shipped transport can carry either. + */ +export interface UnsupportedProxy { + /** The `ProxyOptions.type` the native client cannot honour; the refusal must name it. */ + readonly type: string; + /** Builds a transport configured with that type. Expected to throw rather than return one. */ + build(): Transport; +} + /** * The clauses `docs/product-spec/17-transport-adapter-conformance-contract.md` scopes to only one * reference transport, plus the one drop-set entry that legitimately differs between the two. @@ -50,6 +67,12 @@ export interface TransportCapabilities { readonly supportsProxy: boolean; /** TRANSPORT-11: whether `Connection` is in this transport's outbound drop set. */ readonly dropsConnectionHeader: boolean; + /** + * TRANSPORT-30: a proxy type this transport's configuration can express and its native client + * cannot honour. Omit it when the transport takes no proxy at all, or honours every type it can + * be handed — the row then asserts that omission is the truth rather than skipping. + */ + readonly unsupportedProxy?: UnsupportedProxy; } /** What every row below needs: a transport factory, the live fixture origin, and the capability flags. */ @@ -795,6 +818,37 @@ function registerDropSetRows(ctx: SuiteContext): void { }); } +function registerProxyRefusalRows(ctx: SuiteContext): void { + describe('TRANSPORT-30: a proxy the native client cannot honour is refused, not attempted', () => { + test('an unsupported proxy type fails at construction with a typed, naming error', () => { + const unsupported = ctx.capabilities.unsupportedProxy; + if (unsupported === undefined) { + // The row still runs here rather than being skipped, and asserts the only thing left to + // assert: that there is genuinely no proxy surface to mis-set. `@dexpace/transport-fetch` + // is this leg — it ships no `proxy` option at all, deliberately (design doc §6), so + // TRANSPORT-30 has no subject on it and a silent skip would look the same as a gap. + expect(ctx.capabilities.supportsProxy).toBe(false); + return; + } + let thrown: unknown; + try { + // A transport that returns instead of throwing has deferred the failure to the first send, + // where it arrives as whatever the native client raises — the shape TRANSPORT-30 rules out. + void unsupported.build(); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // Not an `IoError`: `retry/classify.ts` is an allow-list, so a misconfiguration no retry can + // fix is non-retryable for free (RETRY-2). An undici `InvalidArgumentError` escaping raw + // would fail this too, being neither a `TypeError` nor documented. + expect(isIoError(thrown)).toBe(false); + // "Discoverable": the message names the type that was refused, not merely that something was. + expect((thrown as Error).message).toContain(unsupported.type); + }); + }); +} + function registerScopedRows(ctx: SuiteContext): void { if (ctx.capabilities.supportsInternalCancel) { describe('TRANSPORT-8: an internal cancel is told apart from a timeout', () => { @@ -883,6 +937,7 @@ export function runTransportConformanceSuite( registerFileBodyRows(ctx); registerInboundHeaderRows(ctx); registerDropSetRows(ctx); + registerProxyRefusalRows(ctx); registerScopedRows(ctx); }); } diff --git a/packages/transport-undici/README.md b/packages/transport-undici/README.md index 6cfd739..c67c411 100644 --- a/packages/transport-undici/README.md +++ b/packages/transport-undici/README.md @@ -55,11 +55,18 @@ terminal `CancellationError`, and so does a `send()` issued after `close()` — documented `SEAM-15` post-close mode. It cannot succeed over a dispatcher that no longer exists, so it is not reported as a retryable failure. -## Proxy support and its one real limit +## Proxy support and its two real limits `ProxyOptions` routes here in full: address, Basic credentials, and `NO_PROXY`/`nonProxyHosts` bypass globs, which route over a separate direct `Agent` rather than being tunnelled anyway. +**`type` must be `http`.** undici's `ProxyAgent` is an HTTP `CONNECT` tunnel and reads its `uri` as +a URL, so `socks4`/`socks5` — which `ProxyType` admits and core resolves from `ALL_PROXY` +(`CFG-22`) — are refused at construction with a `TypeError` naming the type. Before 2026-09-05 they +reached `new ProxyAgent({uri: 'socks5://…'})` and escaped this factory as an undici +`InvalidArgumentError`, untyped and undocumented. Neither shipped transport can carry SOCKS; +`docs/deviations.md` records the gap. + **A custom `challengeHandler` cannot be dispatched**, and the limitation is surfaced rather than silently misbehaving (`TRANSPORT-30`): diff --git a/packages/transport-undici/src/undici-transport.conformance.test.ts b/packages/transport-undici/src/undici-transport.conformance.test.ts index 866b18f..1f5106a 100644 --- a/packages/transport-undici/src/undici-transport.conformance.test.ts +++ b/packages/transport-undici/src/undici-transport.conformance.test.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/transport-undici/src/undici-transport.conformance.test.ts // Runs the shared TRANSPORT-N suite (@dexpace/transport-conformance) against undiciTransport(). +import {createProxyOptions} from '@dexpace/core'; import {runTransportConformanceSuite} from '@dexpace/transport-conformance'; import {undiciTransport} from './undici-transport.js'; @@ -9,4 +10,17 @@ runTransportConformanceSuite('undiciTransport', () => undiciTransport(), { supportsProxy: true, // TRANSPORT-11's own note: an undici-class transport forwards `Connection` rather than dropping it. dropsConnectionHeader: false, + // TRANSPORT-30: core resolves `ALL_PROXY=socks5://host:1080` to this type (CFG-22), and undici's + // `ProxyAgent` is an HTTP CONNECT tunnel that cannot carry it. + unsupportedProxy: { + type: 'socks5', + build: () => + undiciTransport({ + proxy: createProxyOptions({ + type: 'socks5', + host: '127.0.0.1', + port: 1080, + }), + }), + }, }); diff --git a/packages/transport-undici/src/undici-transport.test.ts b/packages/transport-undici/src/undici-transport.test.ts index 4d970a5..e43390c 100644 --- a/packages/transport-undici/src/undici-transport.test.ts +++ b/packages/transport-undici/src/undici-transport.test.ts @@ -687,6 +687,35 @@ describe('undiciTransport failure classification (TRANSPORT-20)', () => { }); describe('undiciTransport proxy dispatch (TRANSPORT-30)', () => { + test('TRANSPORT-30: a SOCKS proxy is refused at the factory, before any Agent is built', () => { + // Both SOCKS values `ProxyType` admits, because core resolves both from the environment + // (`socks4`/`socks4a` and `socks:`/`socks5`/`socks5h`, `config/proxy.ts:372-380`) and undici's + // `ProxyAgent` carries neither -- it is an HTTP CONNECT tunnel and reads its `uri` as a URL. + const {agents, restore} = captureOwnedAgents(); + try { + for (const type of ['socks4', 'socks5'] as const) { + let thrown: unknown; + try { + undiciTransport({ + proxy: createProxyOptions({type, host: '127.0.0.1', port: 1080}), + }); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // TRANSPORT-30's "discoverable": the refusal names the type, and it is not the raw + // `InvalidArgumentError: Invalid URL protocol: socks5:` undici used to let escape. + expect((thrown as Error).message).toContain(type); + expect(thrown).not.toBeInstanceOf(IoError); + } + // The check runs before `new undici.Agent(...)`, so a refused construction leaks nothing -- + // there is no transport for the caller to have called `close()` on. + expect(agents.length).toBe(0); + } finally { + restore(); + } + }); + test('a per-request Proxy-Authorization is dropped when a proxy is configured', async () => { // ProxyAgent.dispatch throws InvalidArgumentError on ANY per-request Proxy-Authorization -- a // deliberate undici security fix -- so forwarding one would turn every proxied send into a hard diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index bac2eb0..9761d28 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -13,6 +13,7 @@ import { TransportFailureError, type Body, type ProxyOptions, + type ProxyType, type Request, type RequestOptions, type Transport, @@ -98,7 +99,13 @@ export interface UndiciTransportOptions { * (SEAM-14); supplying it together with `proxy` is a construction-time error. */ readonly dispatcher?: Dispatcher; - /** Proxy configuration; the transport constructs and owns the resulting `ProxyAgent`. */ + /** + * Proxy configuration; the transport constructs and owns the resulting `ProxyAgent`. + * + * `type` must be `http`. undici's `ProxyAgent` is an HTTP `CONNECT` tunnel, so the `socks4` and + * `socks5` values `ProxyType` also admits — which core resolves from `ALL_PROXY` — are refused at + * construction rather than at the first send (TRANSPORT-30). + */ readonly proxy?: ProxyOptions; /** How dropped header names are logged (TRANSPORT-13); defaults to `'first-per-name'`. */ readonly headerDropLogging?: HeaderDropLogging; @@ -118,6 +125,38 @@ interface DispatcherSet { readonly owned: readonly Dispatcher[]; } +/** + * The one `ProxyOptions.type` this transport can build a dispatcher for. + * + * undici's `ProxyAgent` is an HTTP `CONNECT` tunnel and takes its `uri` as a URL, so a `socks5://` + * or `socks4://` one throws `InvalidArgumentError('Invalid URL protocol: …')` out of its + * constructor. Core resolves `ALL_PROXY=socks5://host:1080` to `type: 'socks5'` quite legitimately + * (`config/proxy.ts:372-380`, CFG-22), so the configuration can express a proxy neither shipped + * transport can honour, and until 2026-09-05 the way you found out was an undici error escaping a + * public factory untyped and undocumented (audit #67 / #81, `docs/deviations.md`). + */ +const SUPPORTED_PROXY_TYPE: ProxyType = 'http'; + +/** + * TRANSPORT-30's "make the limitation discoverable rather than silently misbehaving", applied at + * the earliest point that can: construction, before any dispatcher is allocated. + * + * A `TypeError`, matching {@link selectDispatchers}' other construction-time refusal and + * deliberately outside the `IoError` tree — `retry/classify.ts` is an allow-list, so a + * misconfiguration that no retry can fix is non-retryable for free (RETRY-2). + * + * @param proxy - the configured proxy. + * @throws `TypeError` when `proxy.type` is anything but `http`. + */ +function requireSupportedProxyType(proxy: ProxyOptions): void { + if (proxy.type === SUPPORTED_PROXY_TYPE) return; + throw new TypeError( + `unsupported proxy type \`${proxy.type}\`: undici's ProxyAgent is an HTTP CONNECT tunnel and ` + + 'cannot carry a SOCKS proxy, and @dexpace/transport-fetch has no proxy support at all. ' + + 'Configure an http proxy, or route SOCKS outside this SDK.', + ); +} + /** * The proxy URI plus its Basic credential, kept apart. `formatProxyOptions` is deliberately *not* * used here: it masks credentials as `***:***` for logging, and feeding that to `ProxyAgent` would @@ -148,6 +187,9 @@ function selectDispatchers(options: UndiciTransportOptions): DispatcherSet { const byo = options.dispatcher; return {proxied: byo, direct: byo, owned: []}; } + // Before anything is allocated: a refusal after `new undici.Agent(...)` would leak the direct + // agent on the way out, with no transport for the caller to close it through. + if (options.proxy !== undefined) requireSupportedProxyType(options.proxy); // Agent, not Pool: a Pool is bound to one origin at construction, but a general-purpose Transport // must reach whatever origin each Request names. const direct = new undici.Agent(options.agentOptions); @@ -648,7 +690,9 @@ if (typeof Symbol.asyncDispose === 'symbol') { * * @param options - optional transport settings. * @returns a transport ready to send; release it with `close()`. - * @throws `TypeError` when both `dispatcher` and `proxy` are supplied. + * @throws `TypeError` when both `dispatcher` and `proxy` are supplied, or when `proxy.type` is + * anything but `http` — undici's `ProxyAgent` cannot carry a SOCKS proxy, and neither can + * `@dexpace/transport-fetch`, which has no proxy option at all. * * @public */ From 6df96453a1a70aaff28465bed74936a04589caff Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:44:43 +0300 Subject: [PATCH 089/100] docs: record the SOCKS gap, and the two transport rules #81 was about `docs/deviations.md` gains one row at the end of "Deviations recorded outside a phase" (D0): `CFG-22`'s proxy model carries SOCKS4/SOCKS5 in full, core resolves both from `ALL_PROXY`'s five schemes, and neither shipped transport can send over one -- undici's `ProxyAgent` is an HTTP CONNECT tunnel and `@dexpace/transport-fetch` has no proxy option at all. The row records why `ProxyType` keeps the two values (narrowing a `@public` union is a release-pass decision, and `CFG-22`'s MUST is about the model, which would then stop satisfying it) and where the refusal now happens instead. `write-a-transport.md` grows from nine rules to eleven, both of them things #81 found the shipped transports getting wrong: - Rule 3, split out of rule 2: whatever your native client refuses, drop that header, never the request -- and find out *where* it decides, because WHATWG `Headers.append` throws at construction while undici validates inside `dispatch`. Getting it wrong does not look like a transport bug; it looks like a retryable network failure. - Rule 9, rewritten: recognise a file body structurally and still write it through `writeTo`, because reading `path` yourself skips BODY-13's `transferred === count` and `Content-Length` is dropped by rule 2, so nothing else can see a short write. - Rule 10, new: refuse a proxy you cannot honour at construction, typed, named, and outside the `IoError` tree. Plus the new optional `unsupportedProxy` capability in the "Prove it" example. Fence check passes. Refs #81, #67. --- docs/deviations.md | 1 + docs/sdk-documentation/write-a-transport.md | 62 ++++++++++++++++----- 2 files changed, 48 insertions(+), 15 deletions(-) diff --git a/docs/deviations.md b/docs/deviations.md index d0fab78..ea1d822 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -502,6 +502,7 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **`HTTP-35`'s timeout check is read as the FULL range `AbortSignal.timeout()` accepts, not the lower bound the requirement enumerates.** `HTTP-35` says the options builder "MUST reject a non-null timeout that is zero or negative". `RequestOptionsBuilder.timeoutMs` rejects three more classes: non-finite (shipped unledgered before this audit), non-integer, and anything above `2**32 - 1`. **Strictly stricter than the letter, and deliberately so.** The field has exactly one consumer — `composeSignal` hands it to `AbortSignal.timeout()` — so a value this setter admits and that function refuses is `HTTP-35`'s own failure mode with the seam moved: the error surfaces inside a transport, as an unwrapped platform `RangeError`, one frame away from the call that supplied it. The earlier reading accepted `1.5` and argued in TSDoc that "a timeout is a duration and a fractional millisecond is meaningful"; no consumer of the field can express one. **The range checked is Node's, and that is the point:** `AbortSignal.timeout(1.5)` and `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are ACCEPTED on Bun, and a negative delay is `RangeError` on Node against `TypeError` on Bun (measured 2026-09-05), so leaving the check to the runtime would make an SDK-level contract depend on which runtime the caller happens to be on. *Rejected:* rounding with `Math.ceil` and clamping inside `composeSignal`, which hides the caller's mistake in the one place `HTTP-35` exists to surface it. `composeSignal` is documented as still able to raise, because a transport's own `defaultTimeoutMs` construction option bypasses this setter and is not validated by core — recorded for #81/#82, not fixed here | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:48` is `HTTP-35`'s wording. `packages/core/src/http/request-options.ts:12` (`MAX_TIMEOUT_MS`) and `:204-214` (the check and the rewritten TSDoc paragraph); `packages/core/src/seams/transport.ts:86-92` is `composeSignal`'s new `@throws`, which states the two-runtime divergence rather than naming one error class. Pinned by "rejects a fractional timeout, which no transport deadline can honor" (`packages/core/src/http/request-options.test.ts:128`, the FLIPPED case — it pinned acceptance until this audit), "rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1" (`:134`), "accepts the ceiling itself" (`:143`) and the `every accepted timeout is an integer in 1..2**32 - 1` property (`:157`); the Node half is `composeSignal timeout range on Node (HTTP-35)` in `tests/node-conformance/seams.test.mjs:105`, which cannot live in `bun test` because Bun accepts both rejected values | not yet in §10 | | **`HTTP-31`'s "falls back to raw text rather than throwing" is satisfied for an unpaired surrogate by SUBSTITUTING U+FFFD, not by keeping the raw text.** `HTTP-31` (MUST) makes `QueryParams.parse` lenient and enumerates the lenient cases, ending with "malformed percent-encoding falling back to raw text rather than throwing". An unpaired surrogate is a fourth kind of malformed input the enumeration does not name, and the fallback it prescribes is not available for it: the raw text has no UTF-8 form, so keeping it produces a `QueryParams` whose `encode()` throws `URIError` — the throw merely deferred out of `parse` and into an accessor that documents no throw at all. **The port repairs instead.** `parse` runs `toWellFormed()` over each decoded name and value, so every instance it returns is encodable, which is what "parsing MUST invert encode" needs to mean. The strict half of the rule is unaffected and is where `#76` puts the rejection: `QueryParamsBuilder.add` throws `UrlConstructionError` for the same input, and `substitutePathParams` throws `OperationAssemblyError`. That asymmetry is not new to the query model — it is exactly the outbound/inbound split `Headers` already draws for `HTTP-18` against `HTTP-19`, applied to the one requirement pair that needs it here. Substitution matches the platform rather than inventing a policy: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` (measured 2026-09-05). *Rejected:* letting `parse` throw the builder's error, which breaks a MUST. *Rejected:* dropping the offending parameter, which loses a name the caller may be matching on | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:42` carries `HTTP-31`'s wording (shared with `HTTP-30`). `packages/core/src/http/rfc3986.ts:17-18` are the two patterns, `:31` `hasLoneSurrogate` (strict) and `:44` `toWellFormed` (lenient) — one rule, two entry points, so no caller can pick the wrong one; `packages/core/src/http/query-params.ts:144-150` is `parse`'s repair with the `HTTP-18`/`HTTP-19` comparison stated inline, against `:44-50` and `:240-241` for the strict `add` path; `packages/core/src/seams/operation.ts:139-144` is the path-param half. `/\p{Surrogate}/u` rather than `String.prototype.isWellFormed()` because the latter is ES2024 and `tsconfig.base.json:5-11` pins `lib: ES2023`, though the `engines.node >= 20.3` runtime has it. Pinned by the `lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)` block in `packages/core/src/http/query-params.test.ts:170` — "parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing" (`:197`) and the `no anything escapes parse()` property (`:233`) | not yet in §10 | | **`OBS-35`'s "MUST NOT bake in a default config key name" is satisfied by making the key configurable, not by removing the default.** `OBS-35` (SHOULD) asks for a tolerant, layered log-level resolution and adds one MUST: no baked-in default key name. The port ships `CFG_KEY_LOG_LEVEL` (`DEXPACE_LOG_LEVEL`) as `CFG-14`'s well-known key and, until 2026-09-05, read it unconditionally. It is now `LoggingStepSettings.configKey`'s default: a caller names their own key and the resolution is otherwise identical. **Why the default stays.** A required key would mean no caller gets ambient granularity without naming one first, which trades a MUST about *naming* for a worse default experience, and `CFG-14` — which this port also implements — exists precisely to standardise the name. The layered resolution itself is `CFG-1`'s (override → environment → normalised property → default) and is tolerant as the requirement asks. **A second, quieter half:** the process-wide configuration slot starts empty (`CFG-13`), so no key of any name resolves until a host calls `setGlobalConfiguration(defaultConfiguration())`. That is deliberate — defaulting the slot to a configuration that reads `process.env` would make an import-time environment read the SDK's default behaviour — and it is now documented as the required wiring rather than left to be discovered | audit #67 / #80 | 2026-09-05 | `packages/core/src/observability/logging-step.ts:64-79,94-104` (the setting and the resolution); `packages/core/src/config/configuration.ts:311,354-358,368` (`CFG_KEY_LOG_LEVEL`, `defaultConfiguration`, the empty default slot); `docs/sdk-documentation/pipelines.md` "Turning logging on from the environment"; `docs/product-spec/15-instrumentation-and-observability.md:66` (the requirement) | not yet in §10 | +| **`CFG-22`'s SOCKS proxy types are resolved by the configuration layer and supported by neither shipped transport; the refusal is at the transport factory, and `ProxyType` keeps them.** `CFG-22` (MUST) requires the proxy model to carry "the proxy protocol type (HTTP, SOCKS4, SOCKS5)", and the port implements it in full: `ProxyType` is `'http' \| 'socks4' \| 'socks5'`, and `resolveProxyOptions` maps `ALL_PROXY`/`HTTPS_PROXY`'s `socks:`, `socks4:`, `socks4a:`, `socks5:` and `socks5h:` schemes onto it. Nothing can then send over one. `@dexpace/transport-undici` builds undici's `ProxyAgent`, which is an HTTP `CONNECT` tunnel reading its `uri` as a URL, and `@dexpace/transport-fetch` ships no `proxy` option at all because Node's bare global `fetch` exposes no proxy hook outside undici internals. So a configuration that resolves cleanly has no transport that can honour it. **What changed on 2026-09-05 (audit #67 / #81).** Until then the discovery was `new ProxyAgent({uri: 'socks5://…'})` throwing undici's `InvalidArgumentError('Invalid URL protocol: socks5:')` out of a public factory — untyped, undocumented, and outside the SDK's error vocabulary. `undiciTransport()` now refuses `proxy.type !== 'http'` at construction with a `TypeError` naming the type, before any dispatcher is allocated, deliberately outside the `IoError` tree so `retry/classify.ts`'s allow-list makes it non-retryable (RETRY-2). That is `TRANSPORT-30`'s "make the limitation discoverable rather than silently misbehaving" applied at the earliest point that can. **Why `ProxyType` still admits `socks4`/`socks5`.** Narrowing a `@public` union is a breaking change and therefore a release-pass decision, which this run is not taking (the run's release machinery is suspended); and `CFG-22`'s MUST is about the *model*, which would then no longer satisfy it. The honest state is a configuration layer that is complete and a transport layer that is not, which is what this row records. A future transport — a `node:net` SOCKS dialer, or a `ProxyAgent` replacement — closes the gap without a model change. | audit #67 / #81 | 2026-09-05 | `packages/core/src/config/proxy.ts:34` (`ProxyType`), `:372-380` (the scheme map); `packages/transport-undici/src/undici-transport.ts:138,151-158,192` (the supported type, the refusal, and where it runs); `packages/transport-fetch/src/fetch-transport.ts:76-79` (no `proxy` option, and why); `docs/product-spec/16-configuration.md:42` (`CFG-22`); `docs/product-spec/17-transport-adapter-conformance-contract.md:48` (`TRANSPORT-30`) | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) diff --git a/docs/sdk-documentation/write-a-transport.md b/docs/sdk-documentation/write-a-transport.md index dbf2648..39f88c5 100644 --- a/docs/sdk-documentation/write-a-transport.md +++ b/docs/sdk-documentation/write-a-transport.md @@ -44,7 +44,7 @@ export function echoTransport(): Transport { Note `setInbound`, not `set`: values a server sent are accepted leniently. Using the strict setter on a real server's headers means a response with an obs-text byte in it becomes unreadable. -## Nine rules a real transport must follow +## Eleven rules a real transport must follow The full contract is `docs/product-spec/17-transport-adapter-conformance-contract.md`, thirty `TRANSPORT-N` clauses. These are the ones that are easy to get wrong. @@ -61,24 +61,36 @@ client, so forwarding a caller's copy corrupts framing. `Connection` is in the d `fetch`-class transport and not for an undici-class one — §17 says so explicitly. Log the **name**, never the value, and dedupe per name by default. -**3. Map aborts to exactly two errors** (`TRANSPORT-3`/`TRANSPORT-4`/`TRANSPORT-8`). A timeout is the +**3. Whatever your native client refuses, drop that header — never the request** (`TRANSPORT-12`). +This is the half that is easy to miss, because the refusal happens somewhere you are not looking. +`@dexpace/core` admits every printable ASCII byte in a header name (`http/ascii-validation.ts`), so +`X Custom` is a model-valid name no HTTP client on this platform will carry. Both shipped clients +also refuse `Expect`, `Keep-Alive` and `Upgrade` outright, and undici refuses `Connection` with any +value but `close`/`keep-alive`. Find out *where* your client decides: WHATWG `Headers.append` throws +at construction, which a `try`/`catch` degrades for free; undici validates inside `dispatch`, so +that transport has to ask the question itself before handing the array over. Getting this wrong +does not look like a bug in your transport — it looks like a retryable network failure that burns +the caller's whole retry budget re-proving a permanent misconfiguration. The shared suite has a row +per name. + +**4. Map aborts to exactly two errors** (`TRANSPORT-3`/`TRANSPORT-4`/`TRANSPORT-8`). A timeout is the retryable `TransportFailureError`; a caller abort is the terminal `CancellationError`. A raw `DOMException` must never surface. `isTimeoutSignal(signal)` is how you tell them apart. -**4. An abort after delivery must not close the delivered body** (`SEAM-16`). Both native clients tie +**5. An abort after delivery must not close the delivered body** (`SEAM-16`). Both native clients tie a response body's lifetime to the signal they were given, so dispatch over a **fork** of the signal and detach it at delivery. Get this wrong and a caller who aborts a moment after `send()` resolves finds the body they already own torn out from under them. -**5. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do +**6. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do not close it. -**6. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is +**7. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is never touched by your `close()`. One you constructed is yours to close. Make that decision once, at construction, and make supplying both a caller-owned client *and* an option that would build one a construction-time `TypeError` rather than a silent win for one of them. -**7. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). +**8. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). No unbounded await — a graceful drain would stall teardown for as long as one in-flight send against a slow peer takes. Destroying is the sanctioned choice; in-flight sends then reject with `CancellationError`, and so does a `send()` issued after `close()`, because it cannot succeed over a @@ -86,11 +98,28 @@ dispatcher that no longer exists and so is not a retryable failure. Declare your (`SEAM-15`) either way: `@dexpace/transport-fetch`'s `close()` is a documented no-op over a runtime global it does not own, and `send()` keeps working after it. -**8. Recognize a file body structurally** (`TRANSPORT-28`). `body.kind === 'file'` widens the body to -`FileBodyDescriptor` — `path`, `start`, `count` — and lets you dispatch straight off the file. Never -`instanceof` against `@dexpace/body-file`: a transport must not depend on it. - -**9. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. +**9. Recognize a file body structurally, and still write it through `writeTo`** +(`TRANSPORT-28`, `BODY-13`). `body.kind === 'file'` widens the body to `FileBodyDescriptor` — +`path`, `start`, `count`. Never `instanceof` against `@dexpace/body-file`: a transport must not +depend on it. + +Reading `path` yourself is the trap. It is a shorter path to the wire, and it skips the +descriptor's own `writeTo`, which is where `BODY-13`'s `transferred === count` check lives — the +only thing that can notice a file truncated between `stat` and `send`, because `Content-Length` is +dropped outbound (rule 2) so the framing cannot. `@dexpace/transport-undici` did exactly this until +2026-09-05 and uploaded short files with a 200. Unless your client has a genuine kernel `sendfile` +path, treat a file body as an ordinary `Body` and let `writeTo` produce the bytes; `TRANSPORT-28`'s +zero-copy clause is a SHOULD, and its MUSTs — replayable, and exactly the declared range on the +wire — are the descriptor's to keep, not yours. + +**10. Refuse a proxy you cannot honour, at construction** (`TRANSPORT-30`). `ProxyType` admits +`socks4` and `socks5`, and core resolves both from `ALL_PROXY`, so a configuration can hand you a +proxy your client cannot build. Reject it in the factory with a typed error that names the type, +before you allocate anything — not on the first send, where it arrives as whatever the native +client raises. Keep it outside the `IoError` tree: `retry/classify.ts` is an allow-list, so a +misconfiguration no retry can fix is then non-retryable for free. Declare it in `@throws`. + +**11. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. ## Prove it @@ -105,10 +134,13 @@ runTransportConformanceSuite('my-transport', () => myTransport(), { supportsInternalCancel: false, // TRANSPORT-8: a cancel path distinct from a caller abort supportsProxy: false, // TRANSPORT-30 dropsConnectionHeader: true, // TRANSPORT-11: is `Connection` in your drop set? + // TRANSPORT-30, optional: a proxy type your configuration can express and your client cannot + // honour. Omit it and the row asserts `supportsProxy` is false, rather than skipping. + // unsupportedProxy: {type: 'socks5', build: () => myTransport({proxy: socks5Proxy})}, }); ``` -The three capability flags are the only clauses §17 scopes to a subset of transports; everything else +Those capability entries are the only clauses §17 scopes to a subset of transports; everything else runs unconditionally. The suite starts its own fixture server, and a second one on a separate origin for the rows that deliberately leave a connection unusable — a client that reuses a poisoned connection otherwise fails thirty rows downstream, which is a debugging problem of a different order. @@ -120,15 +152,15 @@ The package is `private` and its `exports` name `./src/index.ts`, so it resolves `@dexpace/transport-shared` exists so the algorithm both adapters need exists once. Its exports are `@internal` and it is not a package to install directly, but reading it is the fastest way to see -what a correct implementation of rules 2, 3, 4 and 6 looks like: +what a correct implementation of rules 2, 3, 4, 5 and 7 looks like: | Module | Concern | |---|---| -| `header-mapping.ts` | The outbound drop-and-degrade pass and the lenient inbound copy | +| `header-mapping.ts` | Rules 2 and 3: the outbound drop-and-degrade pass, and the lenient inbound copy | | `drop-log.ts` | Bounded, case-insensitive, drain-to-cap dedup of already-logged drop names | | `abort-mapping.ts` | The single mapping from an aborted signal to `TransportFailureError` or `CancellationError` | | `body-pump.ts` | Turning a `Body` into a request stream the transport owns, plus idempotent teardown for an abandoned producer | -| `signal-fork.ts` | Rule 4's fork-and-detach | +| `signal-fork.ts` | Rule 5's fork-and-detach | ## Package it From 9c6f145e719da6a8c0c488456ffae5998f84dd71 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:48:51 +0300 Subject: [PATCH 090/100] docs: re-anchor two citations the same change moved `fetch-transport.ts:146-152` for the `Headers.append` degrade became `:159-166` when this branch widened the fetch drop set above it. And `prepareBody` is not "identical in shape" to the fetch twin's -- the two return different structures; what is identical is the two decisions it makes and their order, which is what the comment meant and now says. Plus the undici version behind the `lib/core/request.js` line numbers, in the one of the three places that omitted it. Refs #81, #67. --- packages/transport-conformance/src/run-suite.ts | 9 +++++---- packages/transport-undici/src/undici-transport.ts | 7 ++++--- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 8550577..5e6a668 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -573,10 +573,11 @@ function registerHeaderRows(ctx: SuiteContext): void { * outright, paired with the value that provokes the refusal. * * `expect`, `keep-alive` and `upgrade` are `undici`'s three unconditional rejections - * (`lib/core/request.js:398,409` — `InvalidArgumentError` for the first two, `NotSupportedError` - * for `expect`), and Node's global `fetch` is undici-backed, so both adapters meet them. `X Custom` - * is the non-token name: `@dexpace/core` admits any printable ASCII byte in a header name - * (`http/ascii-validation.ts:29-33`), while both native layers require RFC 9110 `token`. + * (`lib/core/request.js:398,409` in 6.28.0 — `InvalidArgumentError` for the first two, + * `NotSupportedError` for `expect`), and Node's global `fetch` is undici-backed, so both adapters + * meet them. `X Custom` is the non-token name: `@dexpace/core` admits any printable ASCII byte in + * a header name (`http/ascii-validation.ts:29-33`), while both native layers require RFC 9110 + * `token`. * * TRANSPORT-12 is what makes these one table rather than four transport-specific quirks: whatever * the native client refuses, the transport drops *that header only* and still dispatches. diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index 9761d28..46ec7bc 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -242,7 +242,7 @@ function isUnsendableHeader(name: string, value: string): boolean { * repeated (HTTP-14) -- with every header undici would refuse degraded to a logged drop. * * The per-header guard is the same degrade `@dexpace/transport-fetch` gets for free from wrapping - * `Headers.append` in a `try`/`catch` (`fetch-transport.ts:146-152`): undici validates at dispatch + * `Headers.append` in a `try`/`catch` (`fetch-transport.ts:159-166`): undici validates at dispatch * rather than at construction, so this transport has to ask the question itself (TRANSPORT-12). */ function toUndiciHeaders( @@ -349,8 +349,9 @@ interface PreparedBody { } /** - * Prepares one request body for dispatch. Identical in shape to the fetch twin's, and deliberately - * so: there is **no** `kind === 'file'` branch here. + * Prepares one request body for dispatch: buffer it if it is small and replayable, stream it + * otherwise. Exactly the two decisions the fetch twin makes, in the same order, and deliberately so + * — there is **no** `kind === 'file'` branch here. * * There was one until 2026-09-05. It handed `createReadStream(path, {start, end})` to undici, which * is a genuinely shorter path to the wire — one fewer userspace copy — and it bypassed the From ca682c519b42484f2cb9f4d14b93f3c6ef5c8545 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:49:38 +0300 Subject: [PATCH 091/100] test(transport): name the intact file-body fixture's size and range Two bare `64`s and a `.slice(10, 30)` that had to be read together to see they agreed. Also re-anchors the header comment's TRANSPORT-22 pointer, which this branch moved from `undici-transport.test.ts:503` to `:614`. Refs #81, #67. --- packages/transport-conformance/src/run-suite.ts | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 5e6a668..50d66bc 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -14,7 +14,7 @@ // unsupported-type refusal is a row here. TRANSPORT-22 is NOT driven from here -- // forcing an adaptation throw needs a per-transport hook into the native response, so each adapter // asserts it against its own (transport-fetch's fetch-transport.test.ts:118, transport-undici's -// undici-transport.test.ts:503). +// undici-transport.test.ts:614). import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; @@ -660,6 +660,10 @@ const TRUNCATED_FILE_BYTES = 64; /** How far a truncate-after-stat cuts the file back; small enough that no read can be a full one. */ const TRUNCATED_TO_BYTES = 10; +/** The intact file-body fixture's size, and the byte range the ranged row asks for inside it. */ +const INTACT_FILE_BYTES = 64; +const INTACT_RANGE = {start: 10, count: 20} as const; + /** Distinguishable bytes, so a misaligned send fails on content and not merely on length. */ function fileFixtureBytes(size: number): Uint8Array { const bytes = new Uint8Array(size); @@ -705,18 +709,22 @@ function messageChain(error: unknown): string { function registerFileBodyRows(ctx: SuiteContext): void { describe('TRANSPORT-28, BODY-13: a file body is dispatched through its own writeTo', () => { test('an intact ranged file body puts exactly its declared bytes on the wire', async () => { - await withFixtureFile(64, async path => { + await withFixtureFile(INTACT_FILE_BYTES, async path => { const writes = {count: 0}; await withTransport(ctx.makeTransport, async transport => { const response = await transport.send( Request.newBuilder() .method('POST') .url(ctx.url('/echo-body')) - .body(fileBodyFixture(path, {start: 10, count: 20, writes})) + .body(fileBodyFixture(path, {...INTACT_RANGE, writes})) .build(), ); + // TRANSPORT-28's "assert exactly that byte range reaches the wire". expect([...(await response.bytes())]).toEqual([ - ...fileFixtureBytes(64).slice(10, 30), + ...fileFixtureBytes(INTACT_FILE_BYTES).slice( + INTACT_RANGE.start, + INTACT_RANGE.start + INTACT_RANGE.count, + ), ]); }); // TRANSPORT-17's counterpart for a replayable body: a transport that reads `path` itself From 0eebc23e1b2c992b0a11c1e9e1fbf0a251268009 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:54:57 +0300 Subject: [PATCH 092/100] docs(transport-fetch): the README's drop set was three names short `README.md:53` still listed `Content-Length`, `Host`, `Transfer-Encoding` and `Connection`. `FETCH_FORBIDDEN_HEADERS` gained `Expect`, `Keep-Alive` and `Upgrade` earlier on this branch, so the sentence has been false since f02dd8e. Rewritten as three bullets rather than one: which names the client computes and which the layer underneath refuses; why the refused three are dropped rather than forwarded (WHATWG names all four forbidden, the implementations enforce none of it, and they disagree about what happens instead -- Node's undici-backed `fetch` fails the send with the RETRYABLE `TransportFailureError`, Bun 1.3.14 forwards two to the wire and hangs on `Upgrade`); and that a non-token header name degrades to the same drop, which the README had never said at all. `packages/transport-undici/README.md` needed nothing: 6df9645 already widened its enumeration, including the `Connection`-value split undici alone has. Refs #81, #67. --- packages/transport-fetch/README.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/packages/transport-fetch/README.md b/packages/transport-fetch/README.md index 3ef2b5b..cf97160 100644 --- a/packages/transport-fetch/README.md +++ b/packages/transport-fetch/README.md @@ -50,9 +50,21 @@ records the decision and the four reasons the floor does not move instead. - Redirects are **never** followed (`redirect: 'manual'`). The SDK pipeline is the redirect authority (`TRANSPORT-1`/`TRANSPORT-2`). -- `Content-Length`, `Host`, `Transfer-Encoding`, and `Connection` are dropped outbound — the client - computes its own framing — and every drop is logged by name (never by value) through the global - logger, deduped per name by default (`TRANSPORT-11`/`TRANSPORT-13`). +- `Content-Length`, `Host` and `Transfer-Encoding` are dropped outbound because the client computes + its own framing; `Connection`, `Expect`, `Keep-Alive` and `Upgrade` because the layer underneath + refuses them. WHATWG names all four forbidden request headers, but the implementations do not + enforce that list and disagree about what happens instead: on Node the global `fetch` is + undici-backed, so an undropped `Expect`/`Keep-Alive`/`Upgrade` reaches undici's own validation and + fails the send with the **retryable** `TransportFailureError` — a permanent misconfiguration + spending the caller's whole retry budget — while Bun 1.3.14 forwards the first two to the wire and + hangs on the third until something else times the call out. Dropping the name is the one behaviour + `TRANSPORT-12` asks for, + and it matches `@dexpace/transport-undici` (measured 2026-09-05; audit #67 / #81). +- A header name the WHATWG `Headers` layer rejects — `@dexpace/core` admits every printable ASCII + byte in a name, so `X Custom` is model-valid and unsendable — degrades to the same drop, never a + failed send (`TRANSPORT-12`). +- Every drop is logged by name (never by value) through the global logger, deduped per name by + default (`TRANSPORT-11`/`TRANSPORT-13`). - An abort that fires **after** `send()` resolved does not close the delivered body: the caller owns it (`SEAM-16`). Cancellation stays live for the whole in-flight window. - A timeout surfaces as the retryable `TransportFailureError`; a caller abort as the terminal From fe042493074600a8246cc975626da6a5817d2226 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 12:55:39 +0300 Subject: [PATCH 093/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20wave=206a?= =?UTF-8?q?=20landed=20(#81)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index cc132f0..c839cd6 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -544,6 +544,10 @@ surrogates → `UrlConstructionError` (the strategies used no class of their own Known merge seams: `deviations.md` (item 17 section vs OBS-29 row — disjoint regions); `core.api.md` (regenerated on the merged tree). +## Wave 6a — landed 2026-09-05 +PR #96 (#81) merged into the umbrella at `808f6b0`. Single branch on the umbrella tip, so its own preflight (all 20 +steps passed, at `ca682c5`; round 2 changed a README only) is the merged tree's. Identity checked. + ## Decisions taken for wave 6 (M5: #81, then #82) — pre-taken 2026-09-05 ### D19 — #81: degrade what undici cannot carry; one short-write detector; a typed refusal for SOCKS @@ -573,6 +577,19 @@ Known merge seams: `deviations.md` (item 17 section vs OBS-29 row — disjoint r only, if a row proves it), `packages/transport-conformance/src/**`, `docs/sdk-documentation/write-a-transport.md`, `docs/deviations.md` (one appended row), the two transports' `api.md`. + +**D19 outcome (2026-09-05, #81, PR #96).** `expect`/`keep-alive`/`upgrade` added to BOTH forbidden sets — a row +proved Node's undici-backed `fetch` rejects them with a *retryable* `TransportFailureError`, and Bun 1.3.14 +diverges a third way (forwards two, hangs on `upgrade`). undici gains a per-header RFC 9110 token guard and a +`connection`-value guard, all degrading to logged drops. **The undici file branch was deleted outright** rather +than rerouted, so `prepareBody` is the same two decisions on both transports; framing for a file ≤ 1,000,000 +bytes changed from chunked to `content-length`, matching fetch. SOCKS refused at the factory with a `TypeError`; +`ProxyType` keeps the union (narrowing it would breach `CFG-22`'s MUST for the model); one row. The streamed leg +of the truncate row lives in `tests/node-conformance/transport.test.mjs` because Bun's `Readable.fromWeb` leaks +the abort reason as an unhandled rejection. *Constrains #82:* new registrars `registerNativeRejectionRows`, +`registerFileBodyRows`, `registerProxyRefusalRows`, fixture `fileBodyFixture`, `TransportCapabilities.unsupportedProxy`; +the `fromWeb` leak sits in the producer-race code D20 edits; `ftp://` still reaches `dispatcher.request` unchecked. + ### D20 — #82 (after #81 merges): one classification table; `body === null` when there is none; abort the fork - **Permanent errors on fetch:** the fetch transport classifies a `TypeError` from an unsupported scheme, an invalid URL, a forbidden method or an invalid header exactly as the undici transport classifies its @@ -612,3 +629,4 @@ only, if a row proves it), `packages/transport-conformance/src/**`, `docs/sdk-do | #78 / PR #93 | patch changeset for `@dexpace/core`: non-finite `delayOverride` ignored with a warning (was a `RangeError` after one send); `computeDelay` returns `0` not `NaN` for `initialDelayMs: 0` at overflow | | #79 / PR #94 | patch changesets for `@dexpace/core` (paginator closes on malformed `PageInfo`; SSE release failure reported once; splice surrogate typed) and `@dexpace/codec-json` (abortable reads/writes; `tristate` rejects present null) | | #80 / PR #95 | minor changeset for `@dexpace/core`: `PipelineOptions` (new public shape), `LoggingStepSettings.configKey`, context restored after `send()`; `.changeset/2026-09-04-per-operation-span.md`'s `createRuntime(...)` example is stale | +| #81 / PR #96 | patch changesets for `@dexpace/transport-undici` (header degrade, file body via `writeTo`, SOCKS refusal, file framing change) and `@dexpace/transport-fetch` (three names added to the drop set); `ProxyType` narrowing declined — keep the union | From 3c72ef0617ce69d3d9aa754bbd7008acc2e32b89 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 13:10:11 +0300 Subject: [PATCH 094/100] fix(transport-shared): CONTROL_BYTE was excepting LF as well as HTAB MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `degradeInboundHeaders`' inbound-value gate read `/[\x00-\x08\x0B-\x1F\x7F]/u`, which skips `\x0A` along with the intended `\x09`. Nothing observable changed — `Headers.addInbound` applies core's own `hasForbiddenInboundValueByte`, which does reject LF, and the `try`/`catch` two lines down records the same drop — so the two gates were redundant and only one of them was right. The class is now `/[\x00-\x08\x0A-\x1F\x7F]/u`, identical to core's. The constant is exported from the module (not from the barrel) so its test can read the character class directly: a test that went through `degradeInboundHeaders` would have passed against the broken class, which is how this survived from Phase 8a to audit #67 / #82. Found by: audit #67 / #82. --- .../src/header-mapping.test.ts | 46 ++++++++++++++++++- .../transport-shared/src/header-mapping.ts | 19 +++++++- 2 files changed, 63 insertions(+), 2 deletions(-) diff --git a/packages/transport-shared/src/header-mapping.test.ts b/packages/transport-shared/src/header-mapping.test.ts index 4ee6219..6d02976 100644 --- a/packages/transport-shared/src/header-mapping.test.ts +++ b/packages/transport-shared/src/header-mapping.test.ts @@ -4,7 +4,11 @@ // TRANSPORT-12 (per-header graceful degradation), TRANSPORT-14 (lenient inbound copy, obs-text preserved, control-byte header dropped) import {describe, expect, test} from 'bun:test'; import {Headers} from '@dexpace/core'; -import {degradeInboundHeaders, mapOutboundHeaders} from './header-mapping.js'; +import { + CONTROL_BYTE, + degradeInboundHeaders, + mapOutboundHeaders, +} from './header-mapping.js'; describe('mapOutboundHeaders', () => { test('drops framing headers the native client computes', () => { @@ -67,7 +71,47 @@ describe('mapOutboundHeaders graceful degradation (TRANSPORT-12)', () => { }); }); +describe('CONTROL_BYTE (TRANSPORT-14)', () => { + // Asserted on the character class itself, not through `degradeInboundHeaders`, because the two + // gates that reject an inbound value are redundant by construction: the regex below and + // `Headers.addInbound`'s own `hasForbiddenInboundValueByte`, whose throw the `try`/`catch` turns + // into the same drop. A value only the second one caught looked identical from outside, which is + // how `\x0A` stayed out of this class from Phase 8a to audit #67 / #82. + test('every C0 control byte except HTAB is refused, LF included', () => { + for (let code = 0x00; code <= 0x1f; code += 1) { + const value = `v${String.fromCharCode(code)}alue`; + expect([code, CONTROL_BYTE.test(value)]).toEqual([code, code !== 0x09]); + } + expect(CONTROL_BYTE.test('v\x7falue')).toBe(true); + }); + + test('LF is refused whether or not it is preceded by CR', () => { + // The obs-fold shape (`\r\n ` continuation) and a bare LF are both header injection on the + // inbound path; RFC 9110 5.5 forbids either from reaching a field value. + expect(CONTROL_BYTE.test('one\nvalue')).toBe(true); + expect(CONTROL_BYTE.test('one\r\n two')).toBe(true); + }); + + test('HTAB and obs-text are carried, not refused', () => { + // TRANSPORT-14's own SHOULD: a non-ASCII byte in a value is preserved rather than stripped, and + // HTAB is legal whitespace inside a field value (RFC 9110 5.5). + expect(CONTROL_BYTE.test('one\ttwo')).toBe(false); + expect(CONTROL_BYTE.test('café')).toBe(false); + }); +}); + describe('degradeInboundHeaders', () => { + test('drops a header whose value carries a line feed, keeps the rest', () => { + const {headers, dropped} = degradeInboundHeaders([ + ['x-injected', 'value\nx-forged: yes'], + ['x-good', 'value'], + ]); + expect(headers.get('x-injected')).toBeUndefined(); + expect(headers.get('x-forged')).toBeUndefined(); + expect(headers.get('x-good')).toBe('value'); + expect(dropped).toEqual(['x-injected']); + }); + test('drops a header whose value carries a control byte, keeps the rest', () => { const {headers, dropped} = degradeInboundHeaders([ ['x-bad', 'v\x01alue'], diff --git a/packages/transport-shared/src/header-mapping.ts b/packages/transport-shared/src/header-mapping.ts index fbf72db..d570e4c 100644 --- a/packages/transport-shared/src/header-mapping.ts +++ b/packages/transport-shared/src/header-mapping.ts @@ -2,8 +2,25 @@ // packages/transport-shared/src/header-mapping.ts import {Headers} from '@dexpace/core'; +/** + * Every byte TRANSPORT-14 refuses in an *inbound* header value: the C0 controls except HTAB, plus + * DEL. Deliberately the same character class as `@dexpace/core`'s `hasForbiddenInboundValueByte`, + * which `Headers.addInbound` applies a few lines later \u2014 obs-text (\u2265 0x80) is carried, HTAB is + * carried, everything else below 0x20 is not. + * + * `\x0A` was missing from Phase 8a until audit #67 / #82: the class read `\x0B-\x1F`, excepting LF + * alongside the intended HTAB. Nothing observable changed, because `addInbound` rejected the value + * anyway and the `try`/`catch` in {@link degradeInboundHeaders} recorded the same drop \u2014 which is + * exactly why it survived, and why the test for this constant reads the class directly rather than + * going through that function. + * + * Exported for that test only. The package barrel deliberately does not re-export it: it is one + * half of a redundant pair, not plumbing another transport should reach for. + * + * @internal + */ /* eslint-disable no-control-regex -- RFC 9110 requires testing for ASCII control characters */ -const CONTROL_BYTE = /[\x00-\x08\x0B-\x1F\x7F]/u; +export const CONTROL_BYTE = /[\x00-\x08\x0A-\x1F\x7F]/u; const NON_ASCII_OR_CONTROL = /[\x00-\x1F\x7F-\uFFFF]/u; /* eslint-enable no-control-regex -- re-enable */ From 73af7c5010426d9875483cba2bef794ba7a782a4 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 13:17:45 +0300 Subject: [PATCH 095/100] fix(transport): one classification table for a permanent native failure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `@dexpace/transport-fetch` wrapped every native rejection as `TransportFailureError`, which `retry/classify.ts` reports retryable for being an `IoError`. `Request` accepts any absolute URL, so `ftp://example.com` reached `fetch`, was refused permanently, and spent the caller's whole retry budget re-proving it. `@dexpace/transport-undici` refused the same condition through `TERMINAL_ARGUMENT_CODES`: the two adapters classified one condition oppositely. The decision moves to `@dexpace/transport-shared`'s new `dispatch-classification.ts` — the precedent is `abort-mapping.ts` — as `isPermanentDispatchFailure` plus `toDispatchFailure`, and both adapters call it. It is an allow-list of three positive recognitions, so an unrecognised rejection stays the retryable `TransportFailureError` TRANSPORT-20 makes a MUST: - a terminal argument code on the error or its immediate cause, which is undici's whole dispatcher leg (`UND_ERR_INVALID_ARG`, `UND_ERR_NOT_SUPPORTED`) and Bun's `fetch` (`ERR_INVALID_ARG_VALUE` and friends); - a `TypeError` with no `cause`, which is how undici's `fetch` — Node's global `fetch` — reports argument validation, network failures always carrying one; - a cause naming one of three WHATWG scheme refusals, which is the only way that same `fetch` can report `ftp://` at all. `bad port` is deliberately excluded: port 1 is on WHATWG's blocked list, so TRANSPORT-20's own dead-port probe arrives with that reason and must stay retryable. A `DexpaceError` is passed through unchanged — it was classified at its source. Rows added, red against the unfixed fetch transport and already green against undici: `an unsupported URL scheme fails outside the IoError tree` in the shared suite (asserting `isIoError(e) === false`, which is exactly what the retry engine asks), the Node-runtime twin in `tests/node-conformance/transport.test.mjs` because the two runtimes use entirely different error shapes for it, five unit rows in `transport-fetch`, and twelve in `transport-shared`. Found by: audit #67 / #82. --- docs/sdk-documentation/write-a-transport.md | 44 ++++-- .../transport-conformance/src/run-suite.ts | 37 +++++ .../src/fetch-transport.test.ts | 64 ++++++++- .../transport-fetch/src/fetch-transport.ts | 11 +- .../etc/transport-shared.api.md | 10 ++ .../src/dispatch-classification.test.ts | 128 +++++++++++++++++ .../src/dispatch-classification.ts | 135 ++++++++++++++++++ packages/transport-shared/src/index.ts | 4 + .../transport-undici/src/undici-transport.ts | 51 ++----- tests/node-conformance/transport.test.mjs | 36 ++++- 10 files changed, 463 insertions(+), 57 deletions(-) create mode 100644 packages/transport-shared/src/dispatch-classification.test.ts create mode 100644 packages/transport-shared/src/dispatch-classification.ts diff --git a/docs/sdk-documentation/write-a-transport.md b/docs/sdk-documentation/write-a-transport.md index 39f88c5..076d649 100644 --- a/docs/sdk-documentation/write-a-transport.md +++ b/docs/sdk-documentation/write-a-transport.md @@ -44,7 +44,7 @@ export function echoTransport(): Transport { Note `setInbound`, not `set`: values a server sent are accepted leniently. Using the strict setter on a real server's headers means a response with an obs-text byte in it becomes unreadable. -## Eleven rules a real transport must follow +## Twelve rules a real transport must follow The full contract is `docs/product-spec/17-transport-adapter-conformance-contract.md`, thirty `TRANSPORT-N` clauses. These are the ones that are easy to get wrong. @@ -73,24 +73,43 @@ does not look like a bug in your transport — it looks like a retryable network the caller's whole retry budget re-proving a permanent misconfiguration. The shared suite has a row per name. -**4. Map aborts to exactly two errors** (`TRANSPORT-3`/`TRANSPORT-4`/`TRANSPORT-8`). A timeout is the +**4. Classify a native rejection with the shared table, not by hand** (`TRANSPORT-20`, `RETRY-2`). +There are two kinds, and they are not the same kind of thing. An *exchange* that failed — connection +refused, DNS, TLS, peer reset, read timeout — is the retryable `TransportFailureError`, which +`TRANSPORT-20` makes a MUST. A *request* the client refused before dispatching — an unsupported +scheme, a forbidden method, an argument its own validation rejects — can never succeed on a retry, +and `retry/classify.ts` is an allow-list over `IoError`, so reporting it as anything outside that +tree makes it non-retryable for free. Both shipped transports report it as a bare `TypeError` with +the native error as `cause`, matching the `TypeError` they already raise for a misconfiguration +caught at construction. + +Telling the two apart is runtime-specific enough that you should not: call +`toDispatchFailure(error, fallbackMessage)` from `@dexpace/transport-shared`. Node's global `fetch` +reports an unsupported scheme as `TypeError: fetch failed` with an `unknown scheme` *cause* — the +same top-level shape as a DNS failure — while Bun 1.3.14 reports it as +`TypeError [ERR_INVALID_ARG_VALUE]` with no cause, and undici's dispatcher as +`UND_ERR_INVALID_ARG`. The two shipped transports disagreed about `ftp://` until audit #67 / #82 for +exactly that reason. The default is retryable, so a shape the table does not recognize keeps +`TRANSPORT-20`'s MUST. + +**5. Map aborts to exactly two errors** (`TRANSPORT-3`/`TRANSPORT-4`/`TRANSPORT-8`). A timeout is the retryable `TransportFailureError`; a caller abort is the terminal `CancellationError`. A raw `DOMException` must never surface. `isTimeoutSignal(signal)` is how you tell them apart. -**5. An abort after delivery must not close the delivered body** (`SEAM-16`). Both native clients tie +**6. An abort after delivery must not close the delivered body** (`SEAM-16`). Both native clients tie a response body's lifetime to the signal they were given, so dispatch over a **fork** of the signal and detach it at delivery. Get this wrong and a caller who aborts a moment after `send()` resolves finds the body they already own torn out from under them. -**6. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do +**7. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do not close it. -**7. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is +**8. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is never touched by your `close()`. One you constructed is yours to close. Make that decision once, at construction, and make supplying both a caller-owned client *and* an option that would build one a construction-time `TypeError` rather than a silent win for one of them. -**8. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). +**9. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). No unbounded await — a graceful drain would stall teardown for as long as one in-flight send against a slow peer takes. Destroying is the sanctioned choice; in-flight sends then reject with `CancellationError`, and so does a `send()` issued after `close()`, because it cannot succeed over a @@ -98,7 +117,7 @@ dispatcher that no longer exists and so is not a retryable failure. Declare your (`SEAM-15`) either way: `@dexpace/transport-fetch`'s `close()` is a documented no-op over a runtime global it does not own, and `send()` keeps working after it. -**9. Recognize a file body structurally, and still write it through `writeTo`** +**10. Recognize a file body structurally, and still write it through `writeTo`** (`TRANSPORT-28`, `BODY-13`). `body.kind === 'file'` widens the body to `FileBodyDescriptor` — `path`, `start`, `count`. Never `instanceof` against `@dexpace/body-file`: a transport must not depend on it. @@ -112,14 +131,14 @@ path, treat a file body as an ordinary `Body` and let `writeTo` produce the byte zero-copy clause is a SHOULD, and its MUSTs — replayable, and exactly the declared range on the wire — are the descriptor's to keep, not yours. -**10. Refuse a proxy you cannot honour, at construction** (`TRANSPORT-30`). `ProxyType` admits +**11. Refuse a proxy you cannot honour, at construction** (`TRANSPORT-30`). `ProxyType` admits `socks4` and `socks5`, and core resolves both from `ALL_PROXY`, so a configuration can hand you a proxy your client cannot build. Reject it in the factory with a typed error that names the type, before you allocate anything — not on the first send, where it arrives as whatever the native client raises. Keep it outside the `IoError` tree: `retry/classify.ts` is an allow-list, so a misconfiguration no retry can fix is then non-retryable for free. Declare it in `@throws`. -**11. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. +**12. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. ## Prove it @@ -152,15 +171,16 @@ The package is `private` and its `exports` name `./src/index.ts`, so it resolves `@dexpace/transport-shared` exists so the algorithm both adapters need exists once. Its exports are `@internal` and it is not a package to install directly, but reading it is the fastest way to see -what a correct implementation of rules 2, 3, 4, 5 and 7 looks like: +what a correct implementation of rules 2, 3, 4, 5, 6 and 8 looks like: | Module | Concern | |---|---| | `header-mapping.ts` | Rules 2 and 3: the outbound drop-and-degrade pass, and the lenient inbound copy | | `drop-log.ts` | Bounded, case-insensitive, drain-to-cap dedup of already-logged drop names | -| `abort-mapping.ts` | The single mapping from an aborted signal to `TransportFailureError` or `CancellationError` | +| `dispatch-classification.ts` | Rule 4: the one table deciding permanent-versus-retryable for a native rejection | +| `abort-mapping.ts` | Rule 5's single mapping from an aborted signal to `TransportFailureError` or `CancellationError` | | `body-pump.ts` | Turning a `Body` into a request stream the transport owns, plus idempotent teardown for an abandoned producer | -| `signal-fork.ts` | Rule 5's fork-and-detach | +| `signal-fork.ts` | Rule 6's fork-and-detach | ## Package it diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 50d66bc..9e050ce 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -2,6 +2,7 @@ // packages/transport-conformance/src/run-suite.ts // The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot // drift. Exercises: TRANSPORT-1..9, TRANSPORT-11..17, TRANSPORT-20..21, TRANSPORT-23..29, BODY-13, +// RETRY-2 (a permanent misconfiguration is outside the retryable IoError tree), // SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is // answerable for them (the repeated-challenge-header row). TRANSPORT-10..13's SHARED half -- the one // outbound header pass both adapters call -- is asserted at its source in @@ -350,6 +351,41 @@ function registerProducerRows(ctx: SuiteContext): void { }); } +/** + * A scheme every runtime under test refuses and `@dexpace/core` accepts. + * + * `Request` validates a URL by handing it to WHATWG `URL`, which parses `ftp:` perfectly well and + * gives it a real origin — so an `ftp://` request reaches the native client, which refuses it. + * `foo://` would be refused too, but its `origin` is the string `"null"`, which changes what undici + * is even asked; `ftp:` keeps the two adapters answering the same question. + */ +const UNSUPPORTED_SCHEME_URL = 'ftp://example.com/anything'; + +function registerPermanentFailureRows(ctx: SuiteContext): void { + describe('TRANSPORT-20, RETRY-2: a permanent misconfiguration is not a retryable failure', () => { + test('an unsupported URL scheme fails outside the IoError tree', async () => { + // The two adapters answered this oppositely until audit #67 / #82. undici's dispatcher + // rejects `ftp://` with `UND_ERR_INVALID_ARG`, which `transport-undici` already mapped to a + // TypeError; `fetch` rejects with a TypeError whose shape depends on the runtime -- Node's + // undici-backed one says `fetch failed` with an `unknown scheme` cause, Bun 1.3.14 says + // `protocol must be http:, https: or s3:` with `code: ERR_INVALID_ARG_VALUE` -- and + // `transport-fetch` classified all of it as the RETRYABLE TransportFailureError. + // + // `classify.ts` returns true for every IoError, so that verdict spends the caller's entire + // retry budget re-proving a URL no retry can fix. `isIoError(e) === false` is the assertion + // because it is exactly what the retry engine asks. + await withTransport(ctx.makeTransport, async transport => { + const request = Request.newBuilder() + .url(UNSUPPORTED_SCHEME_URL) + .build(); + const error = await rejection(transport.send(request)); + expect(isIoError(error)).toBe(false); + expect(error).toBeInstanceOf(TypeError); + }); + }); + }); +} + function registerFailureRows(ctx: SuiteContext): void { describe('TRANSPORT-4/5/6/20: failure classification and per-call timeouts', () => { test('a dead port surfaces the retryable TransportFailureError', async () => { @@ -939,6 +975,7 @@ export function runTransportConformanceSuite( registerBodyRows(ctx); registerProducerRows(ctx); registerFailureRows(ctx); + registerPermanentFailureRows(ctx); registerCancellationRows(ctx); registerLifecycleRows(ctx); registerHeaderRows(ctx); diff --git a/packages/transport-fetch/src/fetch-transport.test.ts b/packages/transport-fetch/src/fetch-transport.test.ts index 8eea329..3cb39fb 100644 --- a/packages/transport-fetch/src/fetch-transport.test.ts +++ b/packages/transport-fetch/src/fetch-transport.test.ts @@ -5,15 +5,19 @@ // its close owns nothing to release), // TRANSPORT-2 (no retrying/redirecting dispatcher is ever composed), TRANSPORT-15/16 // (close is a documented no-op), TRANSPORT-17/19 (single-use body written once, abandoned producer -// unblocked), TRANSPORT-22 (an adaptation throw still closes the native response), TRANSPORT-30 +// unblocked), TRANSPORT-22 (an adaptation throw still closes the native response), +// TRANSPORT-20 with RETRY-2 (a permanent misconfiguration is classified outside the IoError tree, +// a failed exchange inside it), TRANSPORT-30 // (no proxy option exists at all), SEAM-30 (no producer is left running for its rejection to reach // Node's default unhandledRejection policy) import {describe, expect, test} from 'bun:test'; import { byteArrayBody, Headers, + isIoError, Request, streamBody, + TransportFailureError, type Body, } from '@dexpace/core'; import {fetchTransport} from './fetch-transport.js'; @@ -237,6 +241,64 @@ describe('fetchTransport request-body failures', () => { }); }); +describe('fetchTransport failure classification (TRANSPORT-20, RETRY-2)', () => { + /** Every shape a runtime's `fetch` uses to say "these arguments can never work". */ + const permanent: readonly (readonly [string, Error])[] = [ + // Node's undici-backed `fetch`, thrown out of the `Request` constructor: no cause, because no + // dispatch was ever attempted. + [ + 'a forbidden method', + new TypeError("'CONNECT' HTTP method is unsupported."), + ], + [ + 'a non-token method', + new TypeError("'BAD METHOD' is not a valid HTTP method."), + ], + // The same runtime's scheme refusal, which it can only report as a network error. + [ + 'an unsupported scheme', + new TypeError('fetch failed', {cause: new Error('unknown scheme')}), + ], + // Bun 1.3.14's shape for the same scheme refusal: a code, and no cause at all. + [ + "Bun's coded scheme refusal", + Object.assign(new TypeError('protocol must be http:, https: or s3:'), { + code: 'ERR_INVALID_ARG_VALUE', + }), + ], + ]; + + for (const [what, cause] of permanent) { + test(`${what} is terminal, outside the IoError tree`, async () => { + // A permanent misconfiguration classified as TransportFailureError is an IoError, and + // `classify.ts` returns true for every IoError -- so the caller's whole retry budget goes on + // re-proving it. The undici twin has refused its own equivalents since Phase 8a; this + // transport refused none of them until audit #67 / #82. + const transport = fetchTransport({fetch: () => Promise.reject(cause)}); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TypeError); + expect(isIoError(error)).toBe(false); + expect((error as Error).cause).toBe(cause); + }); + } + + test('a network failure reported the same way stays retryable', async () => { + // The twin of the rows above: `fetch failed` is also how every genuine connect/DNS/TLS failure + // arrives, so the cause is the only discriminator and narrowing must not swallow this. + const cause = new TypeError('fetch failed', { + cause: Object.assign(new Error('getaddrinfo ENOTFOUND h.invalid'), { + code: 'ENOTFOUND', + }), + }); + const transport = fetchTransport({fetch: () => Promise.reject(cause)}); + const request = Request.newBuilder().url('http://h.invalid/x').build(); + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TransportFailureError); + expect(isIoError(error)).toBe(true); + }); +}); + describe('fetchTransport lifecycle', () => { test('TRANSPORT-15/16: close is a no-op and send still works afterwards (SEAM-15)', async () => { const recorder = recordingFetch(); diff --git a/packages/transport-fetch/src/fetch-transport.ts b/packages/transport-fetch/src/fetch-transport.ts index 2cae39e..56c6c03 100644 --- a/packages/transport-fetch/src/fetch-transport.ts +++ b/packages/transport-fetch/src/fetch-transport.ts @@ -21,6 +21,7 @@ import { materializeBody, producerFailure, pumpBody, + toDispatchFailure, type ForkedSignal, type HeaderDropLogging, } from '@dexpace/transport-shared'; @@ -296,10 +297,12 @@ class FetchTransport implements Transport { } catch (error) { await prepared.abandon(error); if (signal?.aborted) throw abortToSdkError(signal, error); - throw new TransportFailureError( - error instanceof Error ? error.message : 'fetch failed', - {cause: error}, - ); + // TRANSPORT-20 versus RETRY-2, decided by the table in `@dexpace/transport-shared` rather + // than here: until audit #67 / #82 every native rejection became `TransportFailureError`, + // which `classify.ts` reports retryable for being an `IoError`, so an `ftp://` URL or a + // `CONNECT` method spent the caller's whole retry budget re-proving a permanent + // misconfiguration. The undici twin already refused those; the two must not disagree. + throw toDispatchFailure(error, 'fetch failed'); } } diff --git a/packages/transport-shared/etc/transport-shared.api.md b/packages/transport-shared/etc/transport-shared.api.md index 2ee005e..de04799 100644 --- a/packages/transport-shared/etc/transport-shared.api.md +++ b/packages/transport-shared/etc/transport-shared.api.md @@ -58,6 +58,11 @@ export type HeaderDropLogging = 'all' | 'first-per-name' | 'quiet'; // @internal export function isMaterializable(body: Body_2, maxBytes: number): boolean; +// Warning: (ae-internal-missing-underscore) The name "isPermanentDispatchFailure" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function isPermanentDispatchFailure(error: unknown): boolean; + // Warning: (ae-internal-missing-underscore) The name "mapOutboundHeaders" should be prefixed with an underscore because the declaration is marked as @internal // // @internal @@ -88,6 +93,11 @@ export function producerFailure(done: Promise | undefined): Promise // @internal export function pumpBody(body: Body_2): BodyPump; +// Warning: (ae-internal-missing-underscore) The name "toDispatchFailure" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function toDispatchFailure(error: unknown, fallbackMessage: string): Error; + // (No @packageDocumentation comment for this package) ``` diff --git a/packages/transport-shared/src/dispatch-classification.test.ts b/packages/transport-shared/src/dispatch-classification.test.ts new file mode 100644 index 0000000..9a61ef1 --- /dev/null +++ b/packages/transport-shared/src/dispatch-classification.test.ts @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/dispatch-classification.test.ts +// Exercises: TRANSPORT-20 (a failure that produced no response is the retryable transport failure), +// RETRY-2 (the retryable set is an allow-list, so a permanent misconfiguration outside the IoError +// tree is non-retryable for free), TRANSPORT-8 (an argument the native client can never accept is +// told apart from an exchange that failed) +import {describe, expect, test} from 'bun:test'; +import {isIoError, TransportFailureError} from '@dexpace/core'; +import { + isPermanentDispatchFailure, + toDispatchFailure, +} from './dispatch-classification.js'; + +/** An error carrying a native `code`, the shape undici and Bun both attach one to. */ +function coded(message: string, code: string): Error { + return Object.assign(new Error(message), {code}); +} + +describe('isPermanentDispatchFailure', () => { + test("undici's two argument-validation codes are permanent", () => { + expect( + isPermanentDispatchFailure( + coded( + 'Invalid URL protocol: the URL must start with `http:` or `https:`.', + 'UND_ERR_INVALID_ARG', + ), + ), + ).toBe(true); + expect( + isPermanentDispatchFailure(coded('expect', 'UND_ERR_NOT_SUPPORTED')), + ).toBe(true); + }); + + test("Bun's coded fetch refusal for an unsupported scheme is permanent", () => { + // Bun 1.3.14, measured: `fetch('ftp://…')` rejects with this exact shape, where Node's + // undici-backed `fetch` rejects with `fetch failed` and an `unknown scheme` cause instead. + const error = Object.assign( + new TypeError('protocol must be http:, https: or s3:'), + {code: 'ERR_INVALID_ARG_VALUE'}, + ); + expect(isPermanentDispatchFailure(error)).toBe(true); + }); + + test("a causeless TypeError is undici's argument validation, so it is permanent", () => { + // Node's global `fetch` throws these out of the `Request` constructor, before any dispatch. + for (const message of [ + "'CONNECT' HTTP method is unsupported.", + "'BAD METHOD' is not a valid HTTP method.", + 'Request with GET/HEAD method cannot have body.', + ]) { + expect(isPermanentDispatchFailure(new TypeError(message))).toBe(true); + } + }); + + test('a scheme refusal reported through `fetch failed` is permanent', () => { + const error = new TypeError('fetch failed', { + cause: new Error('unknown scheme'), + }); + expect(isPermanentDispatchFailure(error)).toBe(true); + }); + + test('a network failure reported through `fetch failed` is NOT permanent', () => { + const error = new TypeError('fetch failed', { + cause: coded('getaddrinfo ENOTFOUND example.invalid', 'ENOTFOUND'), + }); + expect(isPermanentDispatchFailure(error)).toBe(false); + }); + + test('a blocked port stays retryable (TRANSPORT-20 probes one by name)', () => { + // `http://127.0.0.1:1` is the dead-port probe §17 names for TRANSPORT-20, and port 1 is on + // WHATWG's blocked list, so Node's `fetch` refuses it before connecting and says so in the + // cause. Classifying that reason as permanent would turn the SDK's headline retryable case + // terminal, which is why the reason table excludes it explicitly. + const error = new TypeError('fetch failed', {cause: new Error('bad port')}); + expect(isPermanentDispatchFailure(error)).toBe(false); + }); + + test('a plain connection failure and a non-Error rejection stay retryable', () => { + expect( + isPermanentDispatchFailure( + coded('connect ECONNREFUSED 127.0.0.1:1', 'ECONNREFUSED'), + ), + ).toBe(false); + expect(isPermanentDispatchFailure('a string nobody typed')).toBe(false); + }); +}); + +describe('toDispatchFailure', () => { + test('a permanent misconfiguration is a TypeError outside the IoError tree', () => { + const cause = coded('invalid request method', 'UND_ERR_INVALID_ARG'); + const mapped = toDispatchFailure(cause, 'dispatch failed'); + expect(mapped).toBeInstanceOf(TypeError); + // RETRY-2's allow-list is what makes this non-retryable; the class is how it stays outside it. + expect(isIoError(mapped)).toBe(false); + expect(mapped.cause).toBe(cause); + }); + + test('an exchange failure is the retryable TransportFailureError, cause intact', () => { + const cause = coded('connect ECONNREFUSED 127.0.0.1:1', 'ECONNREFUSED'); + const mapped = toDispatchFailure(cause, 'dispatch failed'); + expect(mapped).toBeInstanceOf(TransportFailureError); + expect(isIoError(mapped)).toBe(true); + expect(mapped.message).toBe('connect ECONNREFUSED 127.0.0.1:1'); + expect(mapped.cause).toBe(cause); + }); + + test('a permanent verdict taken from the cause names the cause in its message', () => { + // `fetch failed` names nothing; the reason that made the verdict is the useful half. + const mapped = toDispatchFailure( + new TypeError('fetch failed', {cause: new Error('unknown scheme')}), + 'fetch failed', + ); + expect(mapped.message).toBe('fetch failed: unknown scheme'); + }); + + test('an error already in the SDK vocabulary is passed through untouched', () => { + // A request-body producer failure racing the dispatch arrives here already classified; the + // table knows nothing about the producer and must not re-answer for it. + const already = new TransportFailureError('producer exploded'); + expect(toDispatchFailure(already, 'fetch failed')).toBe(already); + }); + + test('a non-Error rejection falls back to the caller-supplied message', () => { + const mapped = toDispatchFailure(Symbol('nope'), 'fetch failed'); + expect(mapped).toBeInstanceOf(TransportFailureError); + expect(mapped.message).toBe('fetch failed'); + }); +}); diff --git a/packages/transport-shared/src/dispatch-classification.ts b/packages/transport-shared/src/dispatch-classification.ts new file mode 100644 index 0000000..cd838de --- /dev/null +++ b/packages/transport-shared/src/dispatch-classification.ts @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/dispatch-classification.ts +import {DexpaceError, TransportFailureError} from '@dexpace/core'; + +/** + * Error codes a native client uses for "these arguments can never work", as opposed to "this + * exchange failed". + * + * `UND_ERR_INVALID_ARG` and `UND_ERR_NOT_SUPPORTED` are undici's two argument-validation codes, + * raised by `Dispatcher.request` before a socket is touched (`lib/core/errors.js` in 6.28.0): a + * non-`http(s)` origin, `CONNECT` as a method, a non-token method, a per-request + * `Proxy-Authorization` on a `ProxyAgent`. `ERR_INVALID_ARG_VALUE`, `ERR_INVALID_ARG_TYPE` and + * `ERR_INVALID_URL` are the Node-style codes Bun's `fetch` sets on the same class of refusal — Bun + * 1.3.14 rejects an `ftp://` URL with a `TypeError` carrying `ERR_INVALID_ARG_VALUE`, where Node's + * undici-backed `fetch` rejects with a causing network error instead. Measured on both, 2026-09-05. + */ +const TERMINAL_ARGUMENT_CODES: ReadonlySet = new Set([ + 'UND_ERR_INVALID_ARG', + 'UND_ERR_NOT_SUPPORTED', + 'ERR_INVALID_ARG_VALUE', + 'ERR_INVALID_ARG_TYPE', + 'ERR_INVALID_URL', +]); + +/** + * WHATWG network-error reasons that describe the *request* rather than the exchange. + * + * undici's `fetch` funnels every failure into one `TypeError('fetch failed', {cause})` + * (`lib/web/fetch/index.js:230`), so the top-level error cannot tell a refused scheme from a + * refused connection — the cause's message is the only discriminator the runtime offers. These + * three are `makeNetworkError` reasons raised before any dispatch (`:620`, `:793`, `:962` in + * 6.28.0); a scheme this SDK cannot speak is the same permanent misconfiguration undici's + * *dispatcher* reports as `UND_ERR_INVALID_ARG`. + * + * `'bad port'` is deliberately **not** here. WHATWG blocks a fixed list of ports, `1` among them, + * so on Node's `fetch` the canonical dead-port probe (`http://127.0.0.1:1`) arrives with that + * reason — and TRANSPORT-20's own conformance sentence is "connect to a dead port; assert the + * retryable type". Adding it would turn that row, and the SDK's headline retryable case, terminal. + */ +const TERMINAL_NETWORK_REASONS: ReadonlySet = new Set([ + 'unknown scheme', + 'URL scheme must be a HTTP(S) scheme', + 'about scheme is not supported', +]); + +function errorCode(error: unknown): string | undefined { + const code = (error as {code?: unknown} | null | undefined)?.code; + return typeof code === 'string' ? code : undefined; +} + +function hasTerminalCode(error: unknown): boolean { + const code = errorCode(error); + return code !== undefined && TERMINAL_ARGUMENT_CODES.has(code); +} + +/** + * Whether a native rejection is a permanent misconfiguration rather than a failed exchange. + * + * Three positive recognitions, in one place so the two shipped adapters cannot answer differently + * for the same condition (the `ftp://` row asserts they do not). Everything else falls through to + * retryable, which is both the safe default and the behaviour every adapter had before audit #67 / + * #82 — TRANSPORT-20 makes "no response was produced" a MUST-retryable, so a rejection this table + * does not recognise must stay one. + * + * 1. A **terminal argument code** on the error or its immediate cause, per the + * `TERMINAL_ARGUMENT_CODES` table above. This is the whole undici-dispatcher leg, and Bun's + * `fetch`. + * 2. A **`TypeError` with no `cause`**. undici's `fetch` — which is also Node's global `fetch` — + * builds every *network* rejection with a cause, and every argument rejection as a bare + * `TypeError` thrown out of the `Request`/`Headers` constructors before a dispatch is attempted: + * an unsupported method, a non-token method, a body on a GET. The presence of a cause is + * therefore the runtime's own line between the two, and it needs no message matching. + * 3. A cause whose message is one of the `TERMINAL_NETWORK_REASONS` above — the scheme refusals + * that undici's `fetch` can only report through its fixed `fetch failed` message. + * + * @param error - whatever the native call rejected with. + * @returns `true` when no retry of the same request could succeed. + * + * @internal + */ +export function isPermanentDispatchFailure(error: unknown): boolean { + if (hasTerminalCode(error)) return true; + if (!(error instanceof Error)) return false; + const {cause} = error; + if (error instanceof TypeError && cause === undefined) return true; + if (hasTerminalCode(cause)) return true; + return cause instanceof Error && TERMINAL_NETWORK_REASONS.has(cause.message); +} + +/** + * The message to put on the mapped error: the native message, plus the cause's when the native + * layer's own message is a fixed placeholder. `fetch failed` names nothing on its own, and the + * reason that made the verdict permanent is the only useful thing to say. + */ +function describe(error: unknown, fallbackMessage: string): string { + if (!(error instanceof Error)) return fallbackMessage; + const {cause} = error; + if (!(cause instanceof Error) || error.message.includes(cause.message)) { + return error.message; + } + return `${error.message}: ${cause.message}`; +} + +/** + * Maps one native dispatch rejection onto the SDK's error vocabulary. + * + * A permanent misconfiguration becomes a bare `TypeError` carrying the native error as `cause`, + * deliberately **outside** the `IoError` tree: `retry/classify.ts` is an allow-list that returns + * `true` for every `IoError`, so a condition no retry can fix is non-retryable for free (RETRY-2), + * and `TypeError` is already what both transports raise for a caller misconfiguration caught at + * construction. Anything else becomes the retryable `TransportFailureError` TRANSPORT-20 requires. + * + * An error that already descends from `DexpaceError` is returned unchanged: it was classified at + * its own source — a request-body producer failure racing the dispatch is the live case — and + * re-classifying it here would answer for a layer this table knows nothing about. + * + * @param error - whatever the native call rejected with. + * @param fallbackMessage - the message to use when the rejection is not an `Error` at all. + * @returns the error to throw; the caller always throws it. + * + * @internal + */ +export function toDispatchFailure( + error: unknown, + fallbackMessage: string, +): Error { + if (error instanceof DexpaceError) return error; + if (isPermanentDispatchFailure(error)) { + return new TypeError(describe(error, fallbackMessage), {cause: error}); + } + return new TransportFailureError( + error instanceof Error ? error.message : fallbackMessage, + {cause: error}, + ); +} diff --git a/packages/transport-shared/src/index.ts b/packages/transport-shared/src/index.ts index 9c88c43..71b6812 100644 --- a/packages/transport-shared/src/index.ts +++ b/packages/transport-shared/src/index.ts @@ -8,6 +8,10 @@ export { pumpBody, type BodyPump, } from './body-pump.js'; +export { + isPermanentDispatchFailure, + toDispatchFailure, +} from './dispatch-classification.js'; export {createDropLogger, type HeaderDropLogging} from './drop-log.js'; export { degradeInboundHeaders, diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index 46ec7bc..d1b3c9c 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -28,6 +28,7 @@ import { materializeBody, producerFailure, pumpBody, + toDispatchFailure, type BodyPump, type ForkedSignal, type HeaderDropLogging, @@ -276,35 +277,22 @@ const NATIVE_CANCEL_CODES: ReadonlySet = new Set([ 'UND_ERR_CLOSED', ]); -/** - * undici's codes for "these arguments can never work", as opposed to "this exchange failed". Both are - * raised by argument validation and are perfectly reproducible, so classifying them as - * `TransportFailureError` would hand `classify.ts` an always-retryable verdict (it returns `true` for - * every `IoError`) and spend a caller's whole retry budget re-proving a permanent misconfiguration. - * The commonest way to reach one is a bring-your-own `ProxyAgent` plus a per-request - * `Proxy-Authorization`: `UNDICI_PROXIED_FORBIDDEN_HEADERS` only drops that header when this - * transport constructed the proxy itself, so with a BYO dispatcher it reaches `dispatch` and is - * rejected outright. - */ -const TERMINAL_ARGUMENT_CODES: ReadonlySet = new Set([ - 'UND_ERR_INVALID_ARG', - 'UND_ERR_NOT_SUPPORTED', -]); - -function errorCode(error: unknown): string | undefined { - const code = (error as {code?: unknown} | null | undefined)?.code; - return typeof code === 'string' ? code : undefined; -} - function isNativeCancel(error: unknown): boolean { - const code = errorCode(error); - return code !== undefined && NATIVE_CANCEL_CODES.has(code); + const code = (error as {code?: unknown} | null | undefined)?.code; + return typeof code === 'string' && NATIVE_CANCEL_CODES.has(code); } /** - * Maps one dispatch failure onto the SDK's error vocabulary. Extracted from `#dispatch` so the four + * Maps one dispatch failure onto the SDK's error vocabulary. Extracted from `#dispatch` so the * branches read as one classification table rather than as control flow wrapped around a call. * + * Only the first two branches are this transport's own. The permanent-versus-retryable question the + * third asks is `@dexpace/transport-shared`'s {@link toDispatchFailure}, because the two shipped + * adapters answered it oppositely for the same condition until audit #67 / #82: `ftp://` is + * `UND_ERR_INVALID_ARG` here and a `fetch failed` with an `unknown scheme` cause over there, and + * only this transport treated it as permanent. A shared table is what keeps that from recurring — + * the same reason `abort-mapping.ts` exists. + * * @param error - whatever the dispatch rejected with. * @param signal - the forked signal the dispatch was given, if any. * @returns the error to throw; never returns normally without one. @@ -321,22 +309,7 @@ function toDispatchError( cause: error, }); } - const code = errorCode(error); - if (code !== undefined && TERMINAL_ARGUMENT_CODES.has(code)) { - // Deliberately outside the IoError tree: `classify.ts` is an allow-list, so anything that is not - // an IoError, a timeout, or a retryable status is non-retryable for free (RETRY-2). `TypeError` - // matches `selectDispatchers`, which already reports a caller misconfiguration that way. - return new TypeError( - error instanceof Error - ? error.message - : 'undici rejected the request arguments', - {cause: error}, - ); - } - return new TransportFailureError( - error instanceof Error ? error.message : 'undici dispatch failed', - {cause: error}, - ); + return toDispatchFailure(error, 'undici dispatch failed'); } /** What undici accepts as a request body; `undefined` is not one of them, `null` is. */ diff --git a/tests/node-conformance/transport.test.mjs b/tests/node-conformance/transport.test.mjs index b481f13..7360281 100644 --- a/tests/node-conformance/transport.test.mjs +++ b/tests/node-conformance/transport.test.mjs @@ -11,6 +11,8 @@ // `fileBody()` crossing a real transport has no home inside either package's own suite. // // Exercises: TRANSPORT-1 (redirects not followed), TRANSPORT-4/20 (timeout and no-response classification), +// TRANSPORT-20 with RETRY-2 (an unsupported URL scheme is a permanent misconfiguration outside the IoError +// tree -- Node and Bun report it with entirely different error shapes), // TRANSPORT-17 (a single-use body written once, its bytes on the wire), TRANSPORT-24 (vendor status codes), // TRANSPORT-11/12 (a header the native layer refuses is dropped, not a failed send -- Node's undici-backed // `fetch` rejects three names Bun's forwards), @@ -25,7 +27,7 @@ import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; import {createHash} from 'node:crypto'; -import {Headers, Request, RequestOptions} from '@dexpace/core'; +import {Headers, isIoError, Request, RequestOptions} from '@dexpace/core'; import {fileBody} from '@dexpace/body-file'; import {fetchTransport} from '@dexpace/transport-fetch'; import {undiciTransport} from '@dexpace/transport-undici'; @@ -245,6 +247,38 @@ describe('the transport adapters on the Node runtime', () => { } }); + it('classifies an unsupported URL scheme as permanent, not retryable (TRANSPORT-20, RETRY-2)', async () => { + // Runtime-divergent in the strongest sense: the two runtimes do not merely word this + // differently, they use different error shapes. Node's undici-backed `fetch` rejects + // `ftp://` with `TypeError: fetch failed` carrying `Error: unknown scheme` as its cause -- + // byte-identical, at the top level, to a DNS or connect failure -- while Bun 1.3.14 rejects + // with `TypeError [ERR_INVALID_ARG_VALUE]: protocol must be http:, https: or s3:` and no + // cause at all. The Bun conformance row therefore proves nothing about this runtime, which + // is the runtime the SDK ships to. undici's dispatcher agrees with itself on both + // (`UND_ERR_INVALID_ARG`) and is here for the pairing (audit #67 / #82). + const transport = makeTransport(); + try { + await assert.rejects( + transport.send( + Request.newBuilder().url('ftp://example.com/anything').build(), + ), + error => { + // `classify.ts` is an allow-list over `IoError`, so the class IS the retry verdict: + // a `TransportFailureError` here would spend the caller's whole budget re-proving a + // URL no retry can fix. + assert.ok( + error instanceof TypeError, + `expected a TypeError, got ${error?.constructor?.name}`, + ); + assert.equal(isIoError(error), false); + return true; + }, + ); + } finally { + await transport.close(); + } + }); + it('classifies a dead port as a retryable transport failure (TRANSPORT-20)', async () => { const transport = makeTransport(); try { From 55f06ac34b714a30e7abde3aaab4631486902571 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 13:22:29 +0300 Subject: [PATCH 096/100] fix(transport): a body-less response reports body === null on both adapters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 204, 304, 205, 101, 103, every HEAD and a 2xx CONNECT can carry no body, and three of the four native combinations the two adapters meet disagreed about how to say so. undici's dispatcher always hands back a `BodyReadable`, so `@dexpace/transport-undici` wrapped an empty stream; Node's `fetch` returns `null` per the spec; Bun 1.3.14's `fetch` returns a live `ReadableStream` for all three (measured 2026-09-05), so `@dexpace/transport-fetch` was reporting the runtime's answer rather than the contract's. `hasNoResponseBody(method, status)` in `@dexpace/transport-shared` is now the rule and both adapters apply it, so `body === null` is a property of the SDK on every runtime. It is the WHATWG shape and the one `http/response.ts:18` already types; the rejected alternative, an empty stream on both, makes a consumer read to learn there is nothing to read. Each adapter releases the native handle it declines to expose — `cancel()` on fetch's, `dump()` on undici's. `Response.close()` is a no-op on a null body, so nobody else would, and an undrained `BodyReadable` holds the pooled connection open until the dispatcher times it out (TRANSPORT-25, SEAM-30). Rows: 204, 304 and HEAD in the shared suite, each asserting `body === null`, the `content-length` the case does or does not justify, and `reasonPhrase` as `undefined`-or-string — the fetch/undici divergence there is D7's ledger row beside §10 item 13 and is not re-ledgered. A GET over the same route is the twin, so nulling a body-less response cannot quietly null an ordinary one. All six were red on both adapters. `tests/node-conformance/transport.test.mjs` gets the runtime-divergent case, red on undici and green on fetch there, which is the asymmetry the Bun rows cannot show. `fixtures.ts`'s `route` passed the 70-line cap, so the three body-less fixtures are their own function. Found by: audit #67 / #82. --- docs/sdk-documentation/write-a-transport.md | 30 +++++-- .../transport-conformance/src/fixtures.ts | 54 ++++++++++++ .../transport-conformance/src/run-suite.ts | 83 +++++++++++++++++++ .../transport-fetch/src/fetch-transport.ts | 20 ++++- .../etc/transport-shared.api.md | 6 ++ .../transport-shared/src/body-less.test.ts | 66 +++++++++++++++ packages/transport-shared/src/body-less.ts | 49 +++++++++++ packages/transport-shared/src/index.ts | 1 + .../transport-undici/src/undici-transport.ts | 17 +++- tests/node-conformance/transport.test.mjs | 61 ++++++++++++++ 10 files changed, 377 insertions(+), 10 deletions(-) create mode 100644 packages/transport-shared/src/body-less.test.ts create mode 100644 packages/transport-shared/src/body-less.ts diff --git a/docs/sdk-documentation/write-a-transport.md b/docs/sdk-documentation/write-a-transport.md index 076d649..8c2ce2e 100644 --- a/docs/sdk-documentation/write-a-transport.md +++ b/docs/sdk-documentation/write-a-transport.md @@ -44,7 +44,7 @@ export function echoTransport(): Transport { Note `setInbound`, not `set`: values a server sent are accepted leniently. Using the strict setter on a real server's headers means a response with an obs-text byte in it becomes unreadable. -## Twelve rules a real transport must follow +## Thirteen rules a real transport must follow The full contract is `docs/product-spec/17-transport-adapter-conformance-contract.md`, thirty `TRANSPORT-N` clauses. These are the ones that are easy to get wrong. @@ -104,12 +104,27 @@ finds the body they already own torn out from under them. **7. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do not close it. -**8. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is +**8. A response that can carry no body must report `body === null`** (`TRANSPORT-24`, +`TRANSPORT-25`). The WHATWG null-body statuses — `101`, `103`, `204`, `205`, `304` — plus every +`HEAD` response and a 2xx `CONNECT`. Do not forward whatever your native client produced: three of +the four combinations the two shipped adapters meet disagree. undici's dispatcher always hands back +a `BodyReadable`; Node's `fetch` returns `null`; Bun 1.3.14's `fetch` returns a live +`ReadableStream`. `hasNoResponseBody(method, status)` in `@dexpace/transport-shared` is the rule, so +that a consumer can branch on `null` instead of reading to discover there is nothing there. + +Whatever handle you then decline to expose is yours to release — `cancel()` it, `dump()` it — before +you return. `Response.close()` is a no-op on a null body, so nobody else will, and an undrained +`BodyReadable` holds a pooled connection open until the dispatcher times it out. + +A `Content-Length` on a body-less response is not a lie to correct: on a `HEAD` it describes the +body a `GET` would have returned, and it must survive verbatim. Only the body is absent. + +**9. Ownership decides who closes what** (`SEAM-14`). A dispatcher or client the caller supplied is never touched by your `close()`. One you constructed is yours to close. Make that decision once, at construction, and make supplying both a caller-owned client *and* an option that would build one a construction-time `TypeError` rather than a silent win for one of them. -**9. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). +**10. `close()` must be idempotent, concurrent-safe, and non-blocking** (`TRANSPORT-15`/`TRANSPORT-16`). No unbounded await — a graceful drain would stall teardown for as long as one in-flight send against a slow peer takes. Destroying is the sanctioned choice; in-flight sends then reject with `CancellationError`, and so does a `send()` issued after `close()`, because it cannot succeed over a @@ -117,7 +132,7 @@ dispatcher that no longer exists and so is not a retryable failure. Declare your (`SEAM-15`) either way: `@dexpace/transport-fetch`'s `close()` is a documented no-op over a runtime global it does not own, and `send()` keeps working after it. -**10. Recognize a file body structurally, and still write it through `writeTo`** +**11. Recognize a file body structurally, and still write it through `writeTo`** (`TRANSPORT-28`, `BODY-13`). `body.kind === 'file'` widens the body to `FileBodyDescriptor` — `path`, `start`, `count`. Never `instanceof` against `@dexpace/body-file`: a transport must not depend on it. @@ -131,14 +146,14 @@ path, treat a file body as an ordinary `Body` and let `writeTo` produce the byte zero-copy clause is a SHOULD, and its MUSTs — replayable, and exactly the declared range on the wire — are the descriptor's to keep, not yours. -**11. Refuse a proxy you cannot honour, at construction** (`TRANSPORT-30`). `ProxyType` admits +**12. Refuse a proxy you cannot honour, at construction** (`TRANSPORT-30`). `ProxyType` admits `socks4` and `socks5`, and core resolves both from `ALL_PROXY`, so a configuration can hand you a proxy your client cannot build. Reject it in the factory with a typed error that names the type, before you allocate anything — not on the first send, where it arrives as whatever the native client raises. Keep it outside the `IoError` tree: `retry/classify.ts` is an allow-list, so a misconfiguration no retry can fix is then non-retryable for free. Declare it in `@throws`. -**12. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. +**13. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. ## Prove it @@ -171,13 +186,14 @@ The package is `private` and its `exports` name `./src/index.ts`, so it resolves `@dexpace/transport-shared` exists so the algorithm both adapters need exists once. Its exports are `@internal` and it is not a package to install directly, but reading it is the fastest way to see -what a correct implementation of rules 2, 3, 4, 5, 6 and 8 looks like: +what a correct implementation of rules 2, 3, 4, 5, 6, 8 and 9 looks like: | Module | Concern | |---|---| | `header-mapping.ts` | Rules 2 and 3: the outbound drop-and-degrade pass, and the lenient inbound copy | | `drop-log.ts` | Bounded, case-insensitive, drain-to-cap dedup of already-logged drop names | | `dispatch-classification.ts` | Rule 4: the one table deciding permanent-versus-retryable for a native rejection | +| `body-less.ts` | Rule 8: which method/status pairs can carry no response body at all | | `abort-mapping.ts` | Rule 5's single mapping from an aborted signal to `TransportFailureError` or `CancellationError` | | `body-pump.ts` | Turning a `Body` into a request stream the transport owns, plus idempotent teardown for an abandoned producer | | `signal-fork.ts` | Rule 6's fork-and-detach | diff --git a/packages/transport-conformance/src/fixtures.ts b/packages/transport-conformance/src/fixtures.ts index 6088a21..c4a1bf7 100644 --- a/packages/transport-conformance/src/fixtures.ts +++ b/packages/transport-conformance/src/fixtures.ts @@ -35,11 +35,65 @@ export const REPEATED_CHALLENGES: readonly string[] = [ 'Digest realm="conformance", nonce="n1", algorithm=SHA-256, qop="auth"', ]; +/** + * `/fixed-length`'s payload. Its length is what a HEAD response advertises and does not deliver, so + * it is exported: the row asserts the header survived the body-less decision rather than asserting + * a number written twice. + */ +export const FIXED_LENGTH_BODY = 'seventeen-bytes!!'; + +/** `/not-modified`'s validator, the one header a 304 exists to carry. */ +export const NOT_MODIFIED_ETAG = '"conformance-v1"'; + +/** + * The three fixtures whose responses can carry no body at all, in their own function because the + * main switch is at the 70-line lint cap -- and because they are one topic (TRANSPORT-24/25). + * `req` is not needed: `node:http` suppresses the body of a HEAD response by itself. + * + * @param pathname - the requested path. + * @param res - the response to write. + * @returns `true` when this function answered, `false` to fall through to {@link route}. + */ +function routeBodyless(pathname: string, res: ServerResponse): boolean { + switch (pathname) { + case '/no-content': + // TRANSPORT-24 with the WHATWG null-body rule: a 204 has no body and no framing to describe + // one. Node's `node:http` sends no `Content-Length` here at all; Bun 1.3.14's sends `0`. The + // row therefore asserts the header is absent-or-zero, never a positive length -- what a + // transport is answerable for is the body SHAPE, which is the same on both. + res.writeHead(204); + res.end(); + return true; + case '/not-modified': + // Deliberately WITHOUT a `Content-Length`, though RFC 9110 15.4.5 permits a 304 to carry the + // one a 200 would have had. undici 6.28.0 believes it: a 304 declaring 17 bytes leaves the + // dispatcher waiting for a body that cannot come, and the exchange dies with + // `UND_ERR_SOCKET: other side closed` (measured 2026-09-05, Node and Bun alike). That is + // undici's bug to have, not this suite's to provoke -- the row is about the ETag surviving. + res.writeHead(304, {etag: NOT_MODIFIED_ETAG}); + res.end(); + return true; + case '/fixed-length': + // The HEAD row's target. The declared length describes the body a GET would return, so the + // header promises bytes the HEAD response will not deliver: a transport that framed a stream + // from it hands the caller a read that never completes. + res.writeHead(200, { + 'content-type': 'text/plain', + 'content-length': String(FIXED_LENGTH_BODY.length), + }); + res.end(FIXED_LENGTH_BODY); + return true; + default: + return false; + } +} + function route( pathname: string, req: IncomingMessage, res: ServerResponse, ): void { + if (routeBodyless(pathname, res)) return; switch (pathname) { case '/echo-headers': res.writeHead(200, {'content-type': 'application/json'}); diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 9e050ce..12a498e 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -34,6 +34,8 @@ import { } from '@dexpace/core'; import { fileBodyFixture, + FIXED_LENGTH_BODY, + NOT_MODIFIED_ETAG, REPEATED_CHALLENGES, startFixtureServer, type TestServer, @@ -811,6 +813,86 @@ function challengeList(values: readonly string[]): readonly string[] { return values.join(', ').split(/,\s+(?=Digest )/u); } +function registerBodylessRows(ctx: SuiteContext): void { + describe('TRANSPORT-24/25/27: a response that can carry no body reports none', () => { + // `body === null` is the WHATWG shape and the one `@dexpace/core` already types + // (`http/response.ts:18`); it is also the only shape a consumer can branch on without reading. + // Three of the four native combinations disagreed until audit #67 / #82 -- undici's dispatcher + // always hands back a `BodyReadable`, Node's `fetch` returns `null`, Bun 1.3.14's `fetch` + // returns a live `ReadableStream` -- so each adapter decides for itself now and these rows are + // what say so. The alternative, an empty stream on both, makes a consumer read to learn there + // is nothing there. + // + // `reasonPhrase` is `undefined`-or-string on purpose: `fetch` surfaces `statusText` and undici's + // `ResponseData` has no such field, a divergence recorded beside §10 item 13. Asserting the + // union is what keeps this row about the body shape. + test('a 204 carries a null body and no positive length', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/no-content')).build(), + ); + expect(response.status.code).toBe(204); + expect(response.body).toBeNull(); + // Absent on Node's `node:http`, `'0'` on Bun 1.3.14's -- what a transport is answerable for + // is that it never invented a length for a body that does not exist. + expect([undefined, '0']).toContain( + response.headers.get('content-length'), + ); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + // Idempotent and non-blocking over a response the transport already released. + await response.close(); + await response.close(); + }); + }); + + test('a 304 carries a null body and still delivers its validator', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/not-modified')).build(), + ); + expect(response.status.code).toBe(304); + expect(response.body).toBeNull(); + // A 304 exists to carry validators; dropping the body must not drop them. + expect(response.headers.get('etag')).toBe(NOT_MODIFIED_ETAG); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + await response.close(); + }); + }); + + test('a HEAD carries a null body and keeps the length it advertises', async () => { + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder() + .method('HEAD') + .url(ctx.url('/fixed-length')) + .build(), + ); + expect(response.status.code).toBe(200); + expect(response.body).toBeNull(); + // The header describes the body a GET would have returned and must survive verbatim -- + // this is the one body-less case where a length is meaningful (TRANSPORT-27). + expect(response.headers.get('content-length')).toBe( + String(FIXED_LENGTH_BODY.length), + ); + expect(['undefined', 'string']).toContain(typeof response.reasonPhrase); + await response.close(); + }); + }); + + test('the same resource over GET does carry its body', async () => { + // The twin of the three rows above: nulling a body-less response must not null an ordinary + // one, and `/fixed-length` is the same route the HEAD row just read nothing from. + await withTransport(ctx.makeTransport, async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/fixed-length')).build(), + ); + expect(response.body).toBeInstanceOf(ReadableStream); + expect(await response.text()).toBe(FIXED_LENGTH_BODY); + }); + }); + }); +} + function registerInboundHeaderRows(ctx: SuiteContext): void { describe('TRANSPORT-14, AUTH-12/AUTH-25: a repeated inbound header keeps every value', () => { test('two WWW-Authenticate lines reach the pipeline as the same challenge list', async () => { @@ -981,6 +1063,7 @@ export function runTransportConformanceSuite( registerHeaderRows(ctx); registerNativeRejectionRows(ctx); registerFileBodyRows(ctx); + registerBodylessRows(ctx); registerInboundHeaderRows(ctx); registerDropSetRows(ctx); registerProxyRefusalRows(ctx); diff --git a/packages/transport-fetch/src/fetch-transport.ts b/packages/transport-fetch/src/fetch-transport.ts index 56c6c03..e934412 100644 --- a/packages/transport-fetch/src/fetch-transport.ts +++ b/packages/transport-fetch/src/fetch-transport.ts @@ -16,6 +16,7 @@ import { createDropLogger, degradeInboundHeaders, forkSignal, + hasNoResponseBody, isMaterializable, mapOutboundHeaders, materializeBody, @@ -194,7 +195,16 @@ function adaptResponse( .status(Status.of(fetchResponse.status)) .reasonPhrase(fetchResponse.statusText || undefined) .headers(headers) - .body(fetchResponse.body) + // Decided here, not inherited from the runtime. Node's `fetch` returns `null` for 204, 304 + // and HEAD as the spec requires, and Bun 1.3.14's returns a live `ReadableStream` for all + // three (measured 2026-09-05) -- so forwarding `fetchResponse.body` made the SHAPE of a + // body-less response a property of the runtime rather than of this SDK. `#exchange` releases + // whatever handle this declines (audit #67 / #82). + .body( + hasNoResponseBody(request.method, fetchResponse.status) + ? null + : fetchResponse.body, + ) .build() ); } @@ -261,7 +271,13 @@ class FetchTransport implements Transport { try { // TRANSPORT-22: a live socket is in hand, so any throw here must release it before propagating. - return adaptResponse(request, fetchResponse, this.#logDrops); + const response = adaptResponse(request, fetchResponse, this.#logDrops); + if (response.body === null && fetchResponse.body !== null) { + // A runtime handed a body for a response that cannot have one. Nothing references it any + // more, so releasing it is this transport's, not the caller's (TRANSPORT-25, SEAM-30). + await fetchResponse.body.cancel().catch(() => undefined); + } + return response; } catch (error) { await fetchResponse.body?.cancel().catch(() => undefined); // TRANSPORT-19: nothing is delivered on this path either, so the producer is owed its teardown diff --git a/packages/transport-shared/etc/transport-shared.api.md b/packages/transport-shared/etc/transport-shared.api.md index de04799..d838409 100644 --- a/packages/transport-shared/etc/transport-shared.api.md +++ b/packages/transport-shared/etc/transport-shared.api.md @@ -7,6 +7,7 @@ import type { Body as Body_2 } from '@dexpace/core'; import { DexpaceError } from '@dexpace/core'; import { Headers as Headers_2 } from '@dexpace/core'; +import type { Method } from '@dexpace/core'; // Warning: (ae-internal-missing-underscore) The name "abortToSdkError" should be prefixed with an underscore because the declaration is marked as @internal // @@ -48,6 +49,11 @@ export interface ForkedSignal { // @internal export function forkSignal(source: AbortSignal | undefined): ForkedSignal; +// Warning: (ae-internal-missing-underscore) The name "hasNoResponseBody" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function hasNoResponseBody(method: Method, status: number): boolean; + // Warning: (ae-internal-missing-underscore) The name "HeaderDropLogging" should be prefixed with an underscore because the declaration is marked as @internal // // @internal diff --git a/packages/transport-shared/src/body-less.test.ts b/packages/transport-shared/src/body-less.test.ts new file mode 100644 index 0000000..a7ff678 --- /dev/null +++ b/packages/transport-shared/src/body-less.test.ts @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/body-less.test.ts +// Exercises: TRANSPORT-24 (every status is surfaced faithfully, including the ones that carry no +// body), TRANSPORT-25 (a response whose body a transport declines to expose still has its native +// handle released), TRANSPORT-27 (an absent length is the unknown-length case, not a failure) +import {describe, expect, test} from 'bun:test'; +import type {Method} from '@dexpace/core'; +import {hasNoResponseBody} from './body-less.js'; + +describe('hasNoResponseBody', () => { + test('the WHATWG null-body statuses carry none, whatever the method', () => { + for (const status of [101, 103, 204, 205, 304]) { + expect([status, hasNoResponseBody('GET', status)]).toEqual([ + status, + true, + ]); + expect([status, hasNoResponseBody('POST', status)]).toEqual([ + status, + true, + ]); + } + }); + + test('an ordinary status carries one', () => { + for (const status of [200, 201, 206, 302, 400, 404, 500, 520]) { + expect([status, hasNoResponseBody('GET', status)]).toEqual([ + status, + false, + ]); + } + }); + + test('HEAD never carries one, whatever the status', () => { + // The Content-Length of a HEAD response describes the body a GET would have returned, so a + // transport that framed a stream from it would hand the caller a read that never completes. + for (const status of [200, 206, 404, 500]) { + expect([status, hasNoResponseBody('HEAD', status)]).toEqual([ + status, + true, + ]); + } + }); + + test('a 2xx CONNECT is a tunnel, a failed CONNECT is an ordinary error response', () => { + expect(hasNoResponseBody('CONNECT', 200)).toBe(true); + expect(hasNoResponseBody('CONNECT', 299)).toBe(true); + expect(hasNoResponseBody('CONNECT', 407)).toBe(false); + expect(hasNoResponseBody('CONNECT', 502)).toBe(false); + }); + + test('every other method the model admits is decided by the status alone', () => { + const methods: readonly Method[] = [ + 'GET', + 'POST', + 'PUT', + 'DELETE', + 'OPTIONS', + 'TRACE', + 'PATCH', + ]; + for (const method of methods) { + expect([method, hasNoResponseBody(method, 204)]).toEqual([method, true]); + expect([method, hasNoResponseBody(method, 200)]).toEqual([method, false]); + } + }); +}); diff --git a/packages/transport-shared/src/body-less.ts b/packages/transport-shared/src/body-less.ts new file mode 100644 index 0000000..6cbdb5a --- /dev/null +++ b/packages/transport-shared/src/body-less.ts @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/body-less.ts +import type {Method} from '@dexpace/core'; + +/** + * The statuses that can never carry a body, whatever the request was: WHATWG fetch's null-body + * status set, which is RFC 9110's own list of body-less statuses (`101`, `103`, `204`, `205`, `304`). + * A `Content-Length` on one of them describes the body a `200` would have had and frames nothing. + */ +const NULL_BODY_STATUSES: ReadonlySet = new Set([ + 101, 103, 204, 205, 304, +]); + +/** The lower and upper bounds of the 2xx range, inside which a `CONNECT` response is body-less. */ +const OK_MIN = 200; +const OK_MAX = 299; + +/** + * Whether the response to `method` with `status` can carry a body at all. + * + * The WHATWG rule, and what `@dexpace/core`'s model already types: `Response.body` is + * `ReadableStream | null` (`http/response.ts:18`), and `null` is what a consumer can + * branch on without reading. Both shipped adapters apply this rather than forwarding whatever their + * native client happened to produce, because three of the four combinations disagreed until audit + * #67 / #82: + * + * - undici's dispatcher always hands back a `BodyReadable`, so `@dexpace/transport-undici` wrapped + * an empty stream for 204, 304 and HEAD alike; + * - Node's global `fetch` returns `null` for all three, per the spec; + * - Bun 1.3.14's `fetch` returns a live `ReadableStream` for all three (measured 2026-09-05), so + * `@dexpace/transport-fetch` inherited the runtime's answer rather than the contract's. + * + * A transport that decides here instead reports the same shape on every runtime, which is what a + * conformance row can assert. Whatever native handle it then declines to expose is its own to + * release — an undrained `BodyReadable` holds the pooled connection open (TRANSPORT-25, SEAM-30). + * + * @param method - the request method; always a canonical uppercase token (HTTP-9). + * @param status - the response status code as the server sent it. + * @returns `true` when the adapted response must carry `body === null`. + * + * @internal + */ +export function hasNoResponseBody(method: Method, status: number): boolean { + if (method === 'HEAD') return true; + // A 2xx CONNECT switches the connection to a tunnel; anything after the blank line is tunnelled + // bytes, not a body. A non-2xx CONNECT is an ordinary error response and may carry one. + if (method === 'CONNECT') return status >= OK_MIN && status <= OK_MAX; + return NULL_BODY_STATUSES.has(status); +} diff --git a/packages/transport-shared/src/index.ts b/packages/transport-shared/src/index.ts index 71b6812..3712363 100644 --- a/packages/transport-shared/src/index.ts +++ b/packages/transport-shared/src/index.ts @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT // packages/transport-shared/src/index.ts export {abortToSdkError} from './abort-mapping.js'; +export {hasNoResponseBody} from './body-less.js'; export { isMaterializable, materializeBody, diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index d1b3c9c..e882dee 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -23,6 +23,7 @@ import { createDropLogger, degradeInboundHeaders, forkSignal, + hasNoResponseBody, isMaterializable, mapOutboundHeaders, materializeBody, @@ -424,7 +425,15 @@ function adaptResponse( .protocol(Protocol.HTTP_1_1) .status(Status.of(result.statusCode)) .headers(headers) - .body(toDemandDrivenStream(result.body)) + // undici's dispatcher always hands back a `BodyReadable`, even for a 204, a 304 or a HEAD -- + // so wrapping it unconditionally gave a caller an empty stream it had to read to discover + // was empty, where the fetch twin on Node gave `null`. The two adapters now decide by the + // same rule; `#exchange` dumps whatever this declines (audit #67 / #82). + .body( + hasNoResponseBody(request.method, result.statusCode) + ? null + : toDemandDrivenStream(result.body), + ) .build() ); } @@ -538,6 +547,12 @@ class UndiciTransport implements Transport { try { // TRANSPORT-22: a live socket is in hand, so any throw here must release it before propagating. const response = adaptResponse(request, result, this.#logDrops); + if (response.body === null) { + // Nothing references the `BodyReadable` any more, and an undrained one holds the pooled + // connection open until the dispatcher times it out (TRANSPORT-25, SEAM-30). `dump` reads + // and discards, which is what returns the socket to the pool. + await result.body.dump().catch(() => undefined); + } this.#reportProxyChallenge(response); return response; } catch (error) { diff --git a/tests/node-conformance/transport.test.mjs b/tests/node-conformance/transport.test.mjs index 7360281..ff3fe94 100644 --- a/tests/node-conformance/transport.test.mjs +++ b/tests/node-conformance/transport.test.mjs @@ -18,6 +18,8 @@ // `fetch` rejects three names Bun's forwards), // TRANSPORT-28/BODY-11 (a real fileBody() over the wire, whole and ranged), BODY-13 (a truncate-after-stat // short write fails the send on the streamed path, which only this runtime can assert), +// TRANSPORT-24/25 (a 204 and a HEAD carry a null body on this runtime as well -- Node's `fetch` +// returns null where Bun's returns a stream, and undici's dispatcher always returns a readable), // TRANSPORT-25 (the response body is a lazily-read stream and close releases it), TRANSPORT-29/SEAM-12 // (concurrent sends), SEAM-16 (an abort after delivery must not close the delivered body). import assert from 'node:assert/strict'; @@ -71,6 +73,9 @@ function fixtureBytes(size) { const sha = bytes => createHash('sha256').update(bytes).digest('hex'); +/** `/fixed-length`'s payload; its length is what the HEAD response advertises and never delivers. */ +const FIXED_LENGTH_BODY = 'seventeen-bytes!!'; + // Every hook and test lives inside this suite rather than at the file root, and that is // load-bearing on the declared floor. Under Node 20.3.0 -- `engines.node`, and the floor leg of // CI's node-conformance matrix -- an async ROOT-level `before` does not finish before subtests @@ -101,6 +106,21 @@ describe('the transport adapters on the Node runtime', () => { res.end('vendor status body'); return; } + if (pathname === '/no-content') { + res.writeHead(204); + res.end(); + return; + } + if (pathname === '/fixed-length') { + // `node:http` suppresses the body for a HEAD request by itself and keeps the declared + // length, which is the trap: the header promises bytes no response will deliver. + res.writeHead(200, { + 'content-type': 'text/plain', + 'content-length': String(FIXED_LENGTH_BODY.length), + }); + res.end(FIXED_LENGTH_BODY); + return; + } const chunks = []; req.on('data', chunk => chunks.push(chunk)); req.on('end', () => { @@ -215,6 +235,47 @@ describe('the transport adapters on the Node runtime', () => { } }); + it('reports a null body for a 204 and a HEAD, on this runtime too (TRANSPORT-24/25)', async () => { + // The one place the WHATWG null-body rule can be checked against the runtime the SDK ships + // to. Node's `fetch` returns `null` for 204/304/HEAD by itself, Bun 1.3.14's returns a live + // `ReadableStream` for all three, and undici's dispatcher always hands back a + // `BodyReadable` -- so the Bun conformance rows prove the adapters normalise Bun's answers + // and this proves they did not normalise into Bun's shape (audit #67 / #82). + const transport = makeTransport(); + try { + const empty = await transport.send( + Request.newBuilder().url(`${origin}/no-content`).build(), + ); + assert.equal(empty.status.code, 204); + assert.equal(empty.body, null); + await empty.close(); + + const head = await transport.send( + Request.newBuilder() + .method('HEAD') + .url(`${origin}/fixed-length`) + .build(), + ); + assert.equal(head.status.code, 200); + assert.equal(head.body, null); + // The advertised length survives; only the body a GET would have returned is absent. + assert.equal( + head.headers.get('content-length'), + String(FIXED_LENGTH_BODY.length), + ); + await head.close(); + + // A body-less decision that also nulled an ordinary response would pass every assertion + // above, so the same route is read once more over GET. + const full = await transport.send( + Request.newBuilder().url(`${origin}/fixed-length`).build(), + ); + assert.equal(await full.text(), FIXED_LENGTH_BODY); + } finally { + await transport.close(); + } + }); + it('exposes the response body as a stream that close() releases (TRANSPORT-25)', async () => { const transport = makeTransport(); try { From deb32e093b0a83474fe1bb6605a1f12a9d18712d Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 13:28:51 +0300 Subject: [PATCH 097/100] fix(transport): a producer failure aborts the native call it raced MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When a streaming request-body producer lost the race in `#dispatch`, `send()` rejected while the native call was still pending, and nothing cancelled it. A response arriving afterwards was dropped with its body neither read nor released — TRANSPORT-9's leak, from the request side. `abandon` unwound the producer; it could not reach the fork. It could not reach the fork because for a send with no caller signal and no composed timeout there was none: `forkSignal(undefined)` returned `{signal: undefined}` and both transports dispatched with no signal at all, which is exactly the case with nothing left to cancel with. `ForkedSignal.signal` is now always a live `AbortSignal` — one controller nobody may ever abort, and indistinguishable to the native client from no signal — and the interface gains `abort(reason)`. `detach()` latches it, so the new direction cannot become the SEAM-16 violation the fork's original direction exists to prevent. Both transports read whether the *caller* aborted before pulling the fork themselves; reading it after would surface every producer failure as a `CancellationError`. `producerFailure` now classifies its own rejection as the retryable `TransportFailureError`, which is what both catches already produced for it. That is not cosmetic: the same catch now runs native rejections through a table that reads a bare `TypeError` as a permanent misconfiguration, and a producer that threw one would have been mistaken for the wire refusing the request. `prepareBody`'s buffered branch has classified the same failure at its source since Phase 8a. Instrumented rows on both transports — a `FetchLike` and a bring-your-own `Dispatcher` whose native call resolves 30ms after the producer fails — assert the dispatched signal is aborted with the producer's error and that the late response never settles into the send. Both were red. Their twins assert a delivered response leaves the fork unaborted. `signal-fork.test.ts` gains four rows for the two-way fork and the latch. The fetch `defaultTimeoutMs` row is rewritten: "a signal was handed over" no longer discriminates anything, so it asserts the deadline is honoured instead. Found by: audit #67 / #82. --- docs/sdk-documentation/write-a-transport.md | 19 +++- .../src/fetch-transport.test.ts | 87 +++++++++++++++++-- .../transport-fetch/src/fetch-transport.ts | 13 ++- .../etc/transport-shared.api.md | 5 +- packages/transport-shared/src/body-pump.ts | 27 ++++-- .../transport-shared/src/signal-fork.test.ts | 49 +++++++++-- packages/transport-shared/src/signal-fork.ts | 64 ++++++++++---- .../src/undici-transport.test.ts | 84 +++++++++++++++++- .../transport-undici/src/undici-transport.ts | 27 +++--- 9 files changed, 320 insertions(+), 55 deletions(-) diff --git a/docs/sdk-documentation/write-a-transport.md b/docs/sdk-documentation/write-a-transport.md index 8c2ce2e..f8177c1 100644 --- a/docs/sdk-documentation/write-a-transport.md +++ b/docs/sdk-documentation/write-a-transport.md @@ -96,10 +96,21 @@ exactly that reason. The default is retryable, so a shape the table does not rec retryable `TransportFailureError`; a caller abort is the terminal `CancellationError`. A raw `DOMException` must never surface. `isTimeoutSignal(signal)` is how you tell them apart. -**6. An abort after delivery must not close the delivered body** (`SEAM-16`). Both native clients tie -a response body's lifetime to the signal they were given, so dispatch over a **fork** of the signal -and detach it at delivery. Get this wrong and a caller who aborts a moment after `send()` resolves -finds the body they already own torn out from under them. +**6. Dispatch over a fork of the caller's signal — and keep the fork even when there is no signal** +(`SEAM-16`, `TRANSPORT-9`). Both native clients tie a response body's lifetime to the signal they +were given, so a caller who aborts a moment after `send()` resolves would find the body they already +own torn out from under them. Fork the signal, forward the caller's abort through it, and detach at +delivery. + +The fork runs the other way too, and that half is easy to miss. When a streaming request-body +producer fails while the native call is still pending, your `send()` rejects and nothing is left +awaiting that call: a response arriving afterwards is dropped with its body neither read nor +released, which is the leak `TRANSPORT-9` names. Abort the fork before you rethrow. That is why +`forkSignal()` hands back a live signal even when the caller supplied none and no timeout was +composed — a send with no signal at all is precisely the case where nothing could cancel it. Read +whether the *caller* aborted before you pull the fork yourself, or every producer failure surfaces +as a `CancellationError`; and let `detach()` latch the abort, so the second direction cannot become +the `SEAM-16` violation the first one exists to prevent. **7. The caller owns the response body** (`BODY-15`). Return it live and unread. Do not buffer it, do not close it. diff --git a/packages/transport-fetch/src/fetch-transport.test.ts b/packages/transport-fetch/src/fetch-transport.test.ts index 3cb39fb..6b3c036 100644 --- a/packages/transport-fetch/src/fetch-transport.test.ts +++ b/packages/transport-fetch/src/fetch-transport.test.ts @@ -7,7 +7,8 @@ // (close is a documented no-op), TRANSPORT-17/19 (single-use body written once, abandoned producer // unblocked), TRANSPORT-22 (an adaptation throw still closes the native response), // TRANSPORT-20 with RETRY-2 (a permanent misconfiguration is classified outside the IoError tree, -// a failed exchange inside it), TRANSPORT-30 +// a failed exchange inside it), TRANSPORT-9 (a producer that loses the race cancels the native call +// it raced, so no response is stranded), TRANSPORT-30 // (no proxy option exists at all), SEAM-30 (no producer is left running for its rejection to reach // Node's default unhandledRejection policy) import {describe, expect, test} from 'bun:test'; @@ -345,14 +346,86 @@ describe('fetchTransport lifecycle', () => { expect(recorder.calls.length).toBe(0); }); - test('defaultTimeoutMs applies when the call supplies no timeout of its own', async () => { - const recorder = recordingFetch(); + test('defaultTimeoutMs bounds a call that supplies no timeout of its own', async () => { + // Asserted through the outcome, not through "a signal was handed over": every send dispatches + // with a forked signal since audit #67 / #82, so the presence of one no longer discriminates. + // The double honours its signal the way a real `fetch` does, which is what makes the composed + // deadline observable. const transport = fetchTransport({ - fetch: recorder.fetch, - defaultTimeoutMs: 5_000, + fetch: (_input, init) => + new Promise((_resolve, reject) => { + init.signal?.addEventListener('abort', () => { + reject(init.signal?.reason as Error); + }); + }), + defaultTimeoutMs: 20, }); const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); - await (await transport.send(request)).close(); - expect(recorder.calls[0]?.signal).toBeInstanceOf(AbortSignal); + // TRANSPORT-4: a timeout is the retryable failure, never a cancellation, and the fork carries + // the source's `TimeoutError` reason through for `isTimeoutSignal` to read. + expect(await rejection(transport.send(request))).toMatchObject({ + name: 'TransportFailureError', + message: 'request timed out', + }); + }); +}); + +describe('fetchTransport producer-failure race (TRANSPORT-9, SEAM-30)', () => { + test('a producer that loses the race takes the pending fetch down with it', async () => { + // The send rejects the moment `writeTo` fails, while `fetch` is still pending -- and until + // audit #67 / #82 nothing then cancelled it. With no caller signal and no timeout the transport + // dispatched with NO signal at all, so a response arriving afterwards was dropped with its body + // neither read nor cancelled: a connection held for as long as the pool would keep it. + let dispatched: AbortSignal | undefined; + let settled = false; + const transport = fetchTransport({ + fetch: (_input, init) => { + dispatched = init.signal ?? undefined; + return new Promise(resolve => { + setTimeout(() => { + settled = true; + resolve(new globalThis.Response('late', {status: 200})); + }, 30); + }); + }, + }); + const failing: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + writeTo: () => Promise.reject(new Error('producer exploded')), + }; + const request = Request.newBuilder() + .method('POST') + .url('http://127.0.0.1:1/x') + .body(failing) + .build(); + + const error = await rejection(transport.send(request)); + // The producer's own classification, not the native table's: `writeTo` failing is a transport + // failure, and the send must not be waiting on `fetch` to say so. + expect(error).toBeInstanceOf(TransportFailureError); + expect(settled).toBe(false); + expect(dispatched?.aborted).toBe(true); + expect((dispatched?.reason as Error | undefined)?.message).toBe( + 'producer exploded', + ); + }); + + test('a delivered response is never aborted by the same handle (SEAM-16)', async () => { + // The twin: `abort` is latched by `detach`, so the fork's second direction cannot become the + // very violation its first direction exists to prevent. + let dispatched: AbortSignal | undefined; + const transport = fetchTransport({ + fetch: (_input, init) => { + dispatched = init.signal ?? undefined; + return Promise.resolve(new globalThis.Response('ok', {status: 200})); + }, + }); + const request = Request.newBuilder().url('http://127.0.0.1:1/x').build(); + const response = await transport.send(request); + expect(await response.text()).toBe('ok'); + expect(dispatched?.aborted).toBe(false); }); }); diff --git a/packages/transport-fetch/src/fetch-transport.ts b/packages/transport-fetch/src/fetch-transport.ts index e934412..f63c19b 100644 --- a/packages/transport-fetch/src/fetch-transport.ts +++ b/packages/transport-fetch/src/fetch-transport.ts @@ -301,7 +301,7 @@ class FetchTransport implements Transport { }; if (prepared.init !== undefined) init.body = prepared.init; if (prepared.duplex !== undefined) init.duplex = prepared.duplex; - if (signal !== undefined) init.signal = signal; + init.signal = signal; try { // Raced, not sequenced: a producer failure must surface even while `fetch` is still pending, @@ -311,8 +311,17 @@ class FetchTransport implements Transport { producerFailure(prepared.done), ]); } catch (error) { + // Read BEFORE the fork is pulled below, or every producer failure would look like a caller + // abort and surface as a CancellationError. + const abortedByCaller = signal.aborted; await prepared.abandon(error); - if (signal?.aborted) throw abortToSdkError(signal, error); + if (abortedByCaller) throw abortToSdkError(signal, error); + // TRANSPORT-9: when the producer lost the race, `fetch` is still pending. Nothing awaits it + // any more, so a response that arrives later would be dropped with its body neither read nor + // cancelled -- a leaked connection for as long as the pool keeps it. Pulling the fork takes + // the native call down instead. On the path where `fetch` itself rejected there is nothing + // left to cancel and this is inert (audit #67 / #82). + plan.fork.abort(error); // TRANSPORT-20 versus RETRY-2, decided by the table in `@dexpace/transport-shared` rather // than here: until audit #67 / #82 every native rejection became `TransportFailureError`, // which `classify.ts` reports retryable for being an `IoError`, so an `ftp://` URL or a diff --git a/packages/transport-shared/etc/transport-shared.api.md b/packages/transport-shared/etc/transport-shared.api.md index d838409..1db54b9 100644 --- a/packages/transport-shared/etc/transport-shared.api.md +++ b/packages/transport-shared/etc/transport-shared.api.md @@ -4,7 +4,7 @@ ```ts -import type { Body as Body_2 } from '@dexpace/core'; +import { Body as Body_2 } from '@dexpace/core'; import { DexpaceError } from '@dexpace/core'; import { Headers as Headers_2 } from '@dexpace/core'; import type { Method } from '@dexpace/core'; @@ -40,8 +40,9 @@ export function degradeInboundHeaders(raw: Iterable): // // @internal export interface ForkedSignal { + abort(reason: unknown): void; detach(): void; - readonly signal: AbortSignal | undefined; + readonly signal: AbortSignal; } // Warning: (ae-internal-missing-underscore) The name "forkSignal" should be prefixed with an underscore because the declaration is marked as @internal diff --git a/packages/transport-shared/src/body-pump.ts b/packages/transport-shared/src/body-pump.ts index a7e7f6d..f657484 100644 --- a/packages/transport-shared/src/body-pump.ts +++ b/packages/transport-shared/src/body-pump.ts @@ -1,6 +1,6 @@ // SPDX-License-Identifier: MIT // packages/transport-shared/src/body-pump.ts -import type {Body} from '@dexpace/core'; +import {TransportFailureError, type Body} from '@dexpace/core'; /** * A streaming request body in flight: the stream to hand the native client, the producer's own @@ -81,9 +81,17 @@ export function pumpBody(body: Body): BodyPump { * unhandled one. Without it that late rejection reaches Node's default `unhandledRejection` policy * and takes the process down — the exact hazard SEAM-30 names, arriving from the request side. * + * The rejection is classified **here**, as the retryable `TransportFailureError` both transports + * already reported for it, and not left raw for the caller's catch to guess at. A body that could + * not be written is a failure of this layer, and the catch that receives it is the one that also + * receives the *native* client's rejections — which since audit #67 / #82 go through a table that + * can call a bare `TypeError` a permanent misconfiguration. A producer that happened to throw one + * would have been read as the wire refusing the request. Classifying at the source is what + * `prepareBody`'s buffered branch already does with the same failure. + * * @param done - the producer settlement from {@link pumpBody}, or `undefined` when the body was not * streamed. - * @returns a promise that rejects with the producer's failure and never resolves. + * @returns a promise that rejects with the producer's failure, wrapped, and never resolves. * * @internal */ @@ -91,9 +99,18 @@ export function producerFailure( done: Promise | undefined, ): Promise { if (done === undefined) return new Promise(() => undefined); - // `then` with no rejection handler: a producer *success* says nothing about the response, so the - // derived promise only ever carries the failure onward. - return done.then(() => new Promise(() => undefined)); + return done.then( + // A producer *success* says nothing about the response, so only the failure is carried onward. + () => new Promise(() => undefined), + (cause: unknown) => { + throw new TransportFailureError( + cause instanceof Error + ? cause.message + : 'request body could not be written', + {cause}, + ); + }, + ); } /** diff --git a/packages/transport-shared/src/signal-fork.test.ts b/packages/transport-shared/src/signal-fork.test.ts index 6fa63e8..37ff026 100644 --- a/packages/transport-shared/src/signal-fork.test.ts +++ b/packages/transport-shared/src/signal-fork.test.ts @@ -1,14 +1,19 @@ // SPDX-License-Identifier: MIT // packages/transport-shared/src/signal-fork.test.ts // Exercises: SEAM-16 (an abort after delivery must not reach the native client), SEAM-13/TRANSPORT-7 -// (an abort before delivery must) +// (an abort before delivery must), TRANSPORT-9 (the transport can cancel a native call it abandons, +// with or without a caller signal) import {describe, expect, test} from 'bun:test'; import {forkSignal} from './signal-fork.js'; describe('forkSignal', () => { - test('returns no signal when the caller supplied none', () => { + test('still yields a live signal when the caller supplied none', () => { + // The transport's own cancellation handle. Until audit #67 / #82 this returned `undefined`, so + // a send with no caller signal and no timeout dispatched with none -- and a request-body + // producer that failed mid-flight had no way to take the native call down with it. const fork = forkSignal(undefined); - expect(fork.signal).toBeUndefined(); + expect(fork.signal).toBeInstanceOf(AbortSignal); + expect(fork.signal.aborted).toBe(false); expect(() => { fork.detach(); }).not.toThrow(); @@ -19,15 +24,17 @@ describe('forkSignal', () => { const fork = forkSignal(controller.signal); const reason = new Error('caller changed their mind'); controller.abort(reason); - expect(fork.signal?.aborted).toBe(true); - expect(fork.signal?.reason).toBe(reason); + expect(fork.signal.aborted).toBe(true); + // Carried verbatim because `isTimeoutSignal` reads `reason.name`: a fork that invented its own + // reason would turn every per-call timeout into a CancellationError (TRANSPORT-4). + expect(fork.signal.reason).toBe(reason); }); test('an already-aborted source forks as already aborted', () => { const controller = new AbortController(); controller.abort(new Error('too late')); const fork = forkSignal(controller.signal); - expect(fork.signal?.aborted).toBe(true); + expect(fork.signal.aborted).toBe(true); }); test('an abort after detach never reaches the fork (SEAM-16)', () => { @@ -36,6 +43,34 @@ describe('forkSignal', () => { fork.detach(); fork.detach(); // idempotent controller.abort(new Error('after delivery')); - expect(fork.signal?.aborted).toBe(false); + expect(fork.signal.aborted).toBe(false); + }); +}); + +describe('forkSignal.abort (TRANSPORT-9)', () => { + test('cancels the native call with the reason the transport gave up for', () => { + const fork = forkSignal(undefined); + const reason = new Error('producer exploded'); + fork.abort(reason); + expect(fork.signal.aborted).toBe(true); + expect(fork.signal.reason).toBe(reason); + }); + + test('cancels a fork that has a source too, without touching the source', () => { + const controller = new AbortController(); + const fork = forkSignal(controller.signal); + fork.abort(new Error('producer exploded')); + expect(fork.signal.aborted).toBe(true); + // The caller's own signal is not the transport's to abort; only the fork it dispatched with. + expect(controller.signal.aborted).toBe(false); + }); + + test('is a no-op after detach, so a delivered body is never torn out (SEAM-16)', () => { + // The latch is what keeps the second direction of the fork from becoming the very violation + // the first direction exists to prevent. + const fork = forkSignal(undefined); + fork.detach(); + fork.abort(new Error('too late to matter')); + expect(fork.signal.aborted).toBe(false); }); }); diff --git a/packages/transport-shared/src/signal-fork.ts b/packages/transport-shared/src/signal-fork.ts index 5561906..55c84d3 100644 --- a/packages/transport-shared/src/signal-fork.ts +++ b/packages/transport-shared/src/signal-fork.ts @@ -2,15 +2,39 @@ // packages/transport-shared/src/signal-fork.ts /** - * A caller signal, forwarded to the native client only for as long as the transport wants it. + * A caller signal, forwarded to the native client only for as long as the transport wants it — and + * a handle the transport can pull itself. * * @internal */ export interface ForkedSignal { - /** Hand this to the native client instead of the caller's own signal. */ - readonly signal: AbortSignal | undefined; - /** Stops forwarding. Idempotent; later aborts of the source no longer reach the native client. */ + /** + * Hand this to the native client instead of the caller's own signal. + * + * Always present, even when the caller supplied no signal and no timeout was composed. That is + * not symmetry for its own sake: {@link ForkedSignal.abort} is the only way a transport can + * cancel a native call it has decided to abandon, and a send with no caller signal is exactly + * the case where a failed request-body producer would otherwise leave one running forever + * (TRANSPORT-9, SEAM-30). A controller nobody ever aborts costs one allocation and is + * indistinguishable, to the native client, from no signal at all. + */ + readonly signal: AbortSignal; + /** + * Stops forwarding, and latches the fork: a later {@link ForkedSignal.abort} is a no-op too. + * Idempotent. Called at delivery, which is the moment the response stops being the transport's. + */ detach(): void; + /** + * Cancels the in-flight native call, so a response that arrives afterwards is refused rather + * than stranded with its body neither read nor released (TRANSPORT-9). + * + * A no-op after {@link ForkedSignal.detach}, which is what keeps this from becoming the SEAM-16 + * violation the fork exists to prevent: once a body has been handed to the caller, nothing in + * this transport may close it. + * + * @param reason - the abort reason; the failure that made the transport give up. + */ + abort(reason: unknown): void; } /** @@ -24,29 +48,37 @@ export interface ForkedSignal { * delivery keeps cancellation live for the whole in-flight window (SEAM-13, TRANSPORT-7) and inert * afterwards. * + * The fork is two-way. It carries the caller's abort *in*, and it lets the transport cancel the + * native call *out* — the second direction added by audit #67 / #82, because a request-body + * producer that fails while the native call is still pending has to take that call down with it. + * * @param source - the composed caller/timeout signal, if any. - * @returns the signal to dispatch with, plus the detach the transport calls on delivery. + * @returns the signal to dispatch with, the detach the transport calls on delivery, and the abort + * it calls when it abandons the exchange. * * @internal */ export function forkSignal(source: AbortSignal | undefined): ForkedSignal { - if (source === undefined) { - return {signal: undefined, detach: () => undefined}; - } const controller = new AbortController(); - if (source.aborted) { - controller.abort(source.reason); - return {signal: controller.signal, detach: () => undefined}; - } + let detached = false; const forward = (): void => { - controller.abort(source.reason); + controller.abort(source?.reason); }; - source.addEventListener('abort', forward, {once: true}); + if (source !== undefined) { + if (source.aborted) forward(); + else source.addEventListener('abort', forward, {once: true}); + } return { signal: controller.signal, - // removeEventListener is idempotent, so a detach on both the success and failure path is safe. detach: () => { - source.removeEventListener('abort', forward); + detached = true; + // removeEventListener is idempotent and a no-op for a listener never added, so a detach on + // both the success and failure path, with or without a source, is safe. + source?.removeEventListener('abort', forward); + }, + abort: (reason: unknown) => { + if (detached) return; + controller.abort(reason); }, }; } diff --git a/packages/transport-undici/src/undici-transport.test.ts b/packages/transport-undici/src/undici-transport.test.ts index e43390c..c8fe3b5 100644 --- a/packages/transport-undici/src/undici-transport.test.ts +++ b/packages/transport-undici/src/undici-transport.test.ts @@ -9,7 +9,8 @@ // native body), TRANSPORT-20 (a permanent argument error is terminal, a no-response failure is // retryable), TRANSPORT-28 (a file body dispatches its declared byte range), SEAM-14, // TRANSPORT-19 (a header-mapping throw leaves no started body producer stranded), SEAM-30 (so no -// producer rejection reaches Node's default unhandledRejection policy) +// producer rejection reaches Node's default unhandledRejection policy), TRANSPORT-9 (a producer that +// loses the race cancels the dispatch it raced, so no response is stranded) import {createRequire} from 'node:module'; import {mkdtemp, readFile, rm, writeFile} from 'node:fs/promises'; import {createServer, type Server} from 'node:http'; @@ -686,6 +687,87 @@ describe('undiciTransport failure classification (TRANSPORT-20)', () => { }); }); +describe('undiciTransport producer-failure race (TRANSPORT-9, SEAM-30)', () => { + /** + * A `Dispatcher` whose `request()` resolves only after `delayMs`, recording the signal it was + * handed. Nothing awaits that promise once the producer has lost the race, so the signal is the + * only thing that can still stop the exchange. + */ + function lateDispatcher( + seen: {signal?: AbortSignal | null; settled: boolean}, + delayMs: number, + ): Dispatcher { + return { + request: (options: Dispatcher.RequestOptions) => { + seen.signal = options.signal as AbortSignal | null; + return new Promise(resolve => { + setTimeout(() => { + seen.settled = true; + resolve({ + statusCode: 200, + headers: {}, + body: { + destroy: () => undefined, + dump: () => Promise.resolve(), + [Symbol.asyncIterator]: () => ({ + next: () => Promise.resolve({done: true, value: undefined}), + }), + }, + } as unknown as Dispatcher.ResponseData); + }, delayMs); + }); + }, + close: () => Promise.resolve(), + } as unknown as Dispatcher; + } + + test('a producer that loses the race takes the pending dispatch down with it', async () => { + // Until audit #67 / #82 this send dispatched with `signal: null` -- the fork only existed when + // the caller supplied a signal or a timeout was composed -- so undici kept dispatching after + // `send()` rejected and whatever came back was dropped with its `BodyReadable` neither read nor + // destroyed, holding the pooled connection open. + const seen: {signal?: AbortSignal | null; settled: boolean} = { + settled: false, + }; + const transport = undiciTransport({dispatcher: lateDispatcher(seen, 30)}); + const failing: Body = { + kind: 'stream', + mediaType: undefined, + contentLength: -1, + replayable: false, + writeTo: () => Promise.reject(new Error('producer exploded')), + }; + const request = Request.newBuilder() + .method('POST') + .url(`${origin}/upload`) + .body(failing) + .build(); + + const error = await rejection(transport.send(request)); + expect(error).toBeInstanceOf(TransportFailureError); + expect(seen.settled).toBe(false); + expect(seen.signal?.aborted).toBe(true); + expect((seen.signal?.reason as Error | undefined)?.message).toBe( + 'producer exploded', + ); + await transport.close(); + }); + + test('a delivered response is never aborted by the same handle (SEAM-16)', async () => { + const seen: {signal?: AbortSignal | null; settled: boolean} = { + settled: false, + }; + const transport = undiciTransport({dispatcher: lateDispatcher(seen, 0)}); + const request = Request.newBuilder().url(`${origin}/anything`).build(); + const response = await transport.send(request); + await response.close(); + // The fork is latched at delivery, so its abort direction can no longer reach a body the + // caller now owns. + expect(seen.signal?.aborted).toBe(false); + await transport.close(); + }); +}); + describe('undiciTransport proxy dispatch (TRANSPORT-30)', () => { test('TRANSPORT-30: a SOCKS proxy is refused at the factory, before any Agent is built', () => { // Both SOCKS values `ProxyType` admits, because core resolves both from the environment diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index e882dee..7241146 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -295,14 +295,11 @@ function isNativeCancel(error: unknown): boolean { * the same reason `abort-mapping.ts` exists. * * @param error - whatever the dispatch rejected with. - * @param signal - the forked signal the dispatch was given, if any. + * @param signal - the forked signal the dispatch was given. * @returns the error to throw; never returns normally without one. */ -function toDispatchError( - error: unknown, - signal: AbortSignal | undefined, -): Error { - if (signal?.aborted) return abortToSdkError(signal, error); +function toDispatchError(error: unknown, signal: AbortSignal): Error { + if (signal.aborted) return abortToSdkError(signal, error); if (isNativeCancel(error)) { // TRANSPORT-8: terminal, never retryable -- the dispatcher this send was routed over no longer // exists, so a retry over it cannot succeed. @@ -537,7 +534,7 @@ class UndiciTransport implements Transport { // which is exactly the in-flight window this check is about. const dispatched = context.fork.signal; - if (dispatched?.aborted) { + if (dispatched.aborted) { // TRANSPORT-9 / SEAM-30: this response will never reach a caller, so this producer closes it. await result.body.dump().catch(() => undefined); await pump?.abandon(dispatched.reason); @@ -586,9 +583,9 @@ class UndiciTransport implements Transport { method: request.method, headers: context.headers, body: context.body, - // `?? null` rather than an omitted key: `exactOptionalPropertyTypes` makes an explicit - // `undefined` a distinct, rejected value here, and undici reads `null` as "no signal". - signal: context.fork.signal ?? null, + // Always a real signal since audit #67 / #82: the fork is this transport's own + // cancellation handle, not merely a relay for the caller's. + signal: context.fork.signal, // TRANSPORT-1: pinned explicitly rather than inherited -- a BYO dispatcher may carry a // redirect interceptor, and the pipeline is the single redirect authority. maxRedirections: 0, @@ -596,8 +593,16 @@ class UndiciTransport implements Transport { producerFailure(pump?.done), ]); } catch (error) { + // Read BEFORE the fork is pulled below, or every producer failure would look like a caller + // abort and surface as a CancellationError. + const mapped = toDispatchError(error, context.fork.signal); await pump?.abandon(error); - throw toDispatchError(error, context.fork.signal); + // TRANSPORT-9: when the producer lost the race, undici is still dispatching. Nothing awaits + // it any more, so a response that arrives later would be dropped with its `BodyReadable` + // neither read nor destroyed, holding the pooled connection. Pulling the fork takes the + // dispatch down instead; on the path where undici itself rejected it is inert. + context.fork.abort(error); + throw mapped; } } From ac26cb05fb52abb1e06bb079ea4e177bb4b3d45a Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 13:33:17 +0300 Subject: [PATCH 098/100] fix(transport): validate defaultTimeoutMs at both factories MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `defaultTimeoutMs` was unchecked on both transports and reached `AbortSignal.timeout()` untouched. Node throws `RangeError` on `1.5`, `2**32` and `-1`; Bun 1.3.14 accepts the first two. The same misconfigured transport therefore failed every send on one runtime and used a deadline nobody asked for on the other. It is also the last such path. `RequestOptionsBuilder.timeoutMs` has enforced the integer `1 .. 2**32 - 1` range at its setter since audit #67 / #76, on HTTP-35's reading that a timeout a setter accepted and a transport then refused belongs at the call site. `requireValidDefaultTimeoutMs` in `@dexpace/transport-shared` applies the identical rule with the identical wording, and both factories call it first thing — before `selectDispatchers` allocates, so a refusal cannot leak an `Agent` with no transport to close it through. A `TypeError`, matching the two construction-time refusals `undiciTransport` already raises and asserted the same way. `@throws` on both factories, and the `defaultTimeoutMs` TSDoc now states the range. `TransportCapabilities` gains a required `buildWithDefaultTimeoutMs(value)`: required rather than a flag because §17 assumes every transport has a default (TRANSPORT-5 is written against one), and typed `number` because every value the rows supply legitimately is one. Twelve rows, red on both adapters, plus the in-range twin that proves narrowing did not reject a legitimate default, plus the Node-runtime case — that one matters because Node is the runtime that used to fail late and loudly where Bun failed silently. Carried from audit #67 / #76 (D14's hand-off). Found by: audit #67 / #82. --- docs/sdk-documentation/write-a-transport.md | 27 +++++-- .../transport-conformance/src/run-suite.ts | 70 +++++++++++++++++++ .../src/fetch-transport.conformance.test.ts | 2 + .../transport-fetch/src/fetch-transport.ts | 15 +++- .../etc/transport-shared.api.md | 5 ++ .../src/default-timeout.test.ts | 51 ++++++++++++++ .../transport-shared/src/default-timeout.ts | 46 ++++++++++++ packages/transport-shared/src/index.ts | 1 + .../src/undici-transport.conformance.test.ts | 3 + .../transport-undici/src/undici-transport.ts | 17 ++++- tests/node-conformance/transport.test.mjs | 44 +++++++++++- 11 files changed, 268 insertions(+), 13 deletions(-) create mode 100644 packages/transport-shared/src/default-timeout.test.ts create mode 100644 packages/transport-shared/src/default-timeout.ts diff --git a/docs/sdk-documentation/write-a-transport.md b/docs/sdk-documentation/write-a-transport.md index f8177c1..27c2576 100644 --- a/docs/sdk-documentation/write-a-transport.md +++ b/docs/sdk-documentation/write-a-transport.md @@ -157,13 +157,21 @@ path, treat a file body as an ordinary `Body` and let `writeTo` produce the byte zero-copy clause is a SHOULD, and its MUSTs — replayable, and exactly the declared range on the wire — are the descriptor's to keep, not yours. -**12. Refuse a proxy you cannot honour, at construction** (`TRANSPORT-30`). `ProxyType` admits -`socks4` and `socks5`, and core resolves both from `ALL_PROXY`, so a configuration can hand you a -proxy your client cannot build. Reject it in the factory with a typed error that names the type, -before you allocate anything — not on the first send, where it arrives as whatever the native +**12. Refuse at construction what you cannot honour** (`TRANSPORT-30`, `HTTP-35`). `ProxyType` +admits `socks4` and `socks5`, and core resolves both from `ALL_PROXY`, so a configuration can hand +you a proxy your client cannot build. Reject it in the factory with a typed error that names the +type, before you allocate anything — not on the first send, where it arrives as whatever the native client raises. Keep it outside the `IoError` tree: `retry/classify.ts` is an allow-list, so a misconfiguration no retry can fix is then non-retryable for free. Declare it in `@throws`. +A transport-wide default timeout is the same shape of decision. It ends up in +`AbortSignal.timeout()`, whose range is an integer in `1 .. 2**32 - 1`, and nothing downstream will +check it for you: `RequestOptions.timeoutMs` is validated at its setter, so an unchecked +`defaultTimeoutMs` is the last path by which `1.5` or `2**32` reaches a deadline — where Node throws +a `RangeError` on the first send and Bun 1.3.14 quietly accepts it. Call +`requireValidDefaultTimeoutMs(value)` from `@dexpace/transport-shared` first thing in your +constructor, before anything is allocated. + **13. Send a real `User-Agent`** (`NFR-15`), never a placeholder. `getBuildInfo()` supplies the tokens. ## Prove it @@ -179,14 +187,18 @@ runTransportConformanceSuite('my-transport', () => myTransport(), { supportsInternalCancel: false, // TRANSPORT-8: a cancel path distinct from a caller abort supportsProxy: false, // TRANSPORT-30 dropsConnectionHeader: true, // TRANSPORT-11: is `Connection` in your drop set? + // HTTP-35, required: the rows hand this values `AbortSignal.timeout()` refuses and expect your + // factory to refuse them too, rather than deferring the failure to the first send. + buildWithDefaultTimeoutMs: value => myTransport({defaultTimeoutMs: value}), // TRANSPORT-30, optional: a proxy type your configuration can express and your client cannot // honour. Omit it and the row asserts `supportsProxy` is false, rather than skipping. // unsupportedProxy: {type: 'socks5', build: () => myTransport({proxy: socks5Proxy})}, }); ``` -Those capability entries are the only clauses §17 scopes to a subset of transports; everything else -runs unconditionally. The suite starts its own fixture server, and a second one on a separate origin +Those capability entries are the clauses §17 scopes to a subset of transports, plus the one builder +the suite needs to construct a deliberately misconfigured transport; everything else runs +unconditionally. The suite starts its own fixture server, and a second one on a separate origin for the rows that deliberately leave a connection unusable — a client that reuses a poisoned connection otherwise fails thirty rows downstream, which is a debugging problem of a different order. @@ -197,12 +209,13 @@ The package is `private` and its `exports` name `./src/index.ts`, so it resolves `@dexpace/transport-shared` exists so the algorithm both adapters need exists once. Its exports are `@internal` and it is not a package to install directly, but reading it is the fastest way to see -what a correct implementation of rules 2, 3, 4, 5, 6, 8 and 9 looks like: +what a correct implementation of rules 2, 3, 4, 5, 6, 8, 9 and 12 looks like: | Module | Concern | |---|---| | `header-mapping.ts` | Rules 2 and 3: the outbound drop-and-degrade pass, and the lenient inbound copy | | `drop-log.ts` | Bounded, case-insensitive, drain-to-cap dedup of already-logged drop names | +| `default-timeout.ts` | Rule 12: the range check a transport-wide default timeout has to pass | | `dispatch-classification.ts` | Rule 4: the one table deciding permanent-versus-retryable for a native rejection | | `body-less.ts` | Rule 8: which method/status pairs can carry no response body at all | | `abort-mapping.ts` | Rule 5's single mapping from an aborted signal to `TransportFailureError` or `CancellationError` | diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index 12a498e..bbe2402 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -3,6 +3,8 @@ // The single TRANSPORT-N conformance suite, run once per transport package so the two adapters cannot // drift. Exercises: TRANSPORT-1..9, TRANSPORT-11..17, TRANSPORT-20..21, TRANSPORT-23..29, BODY-13, // RETRY-2 (a permanent misconfiguration is outside the retryable IoError tree), +// HTTP-35 (a transport-wide default timeout outside AbortSignal.timeout()'s range is refused at the +// factory), // SEAM-12, SEAM-16, SEAM-30, NFR-15, and AUTH-12/AUTH-25 to the extent a transport is // answerable for them (the repeated-challenge-header row). TRANSPORT-10..13's SHARED half -- the one // outbound header pass both adapters call -- is asserted at its source in @@ -76,6 +78,19 @@ export interface TransportCapabilities { * be handed — the row then asserts that omission is the truth rather than skipping. */ readonly unsupportedProxy?: UnsupportedProxy; + /** + * HTTP-35: builds a transport whose transport-wide default timeout is `value`. + * + * Required, not a capability flag, because §17 assumes every transport has one — TRANSPORT-5 is + * written as "a per-call override … overriding the configured default for that call only". The + * rows hand it values `AbortSignal.timeout()` cannot take and expect the factory to refuse them, + * because a default nobody checked is the last path by which such a value reaches a deadline + * (`RequestOptions.timeoutMs` has been checked at its setter since audit #67 / #76). + * + * Typed `number` on purpose: `0`, `-1`, `1.5`, `2**32` and `NaN` are all legitimately `number`, + * so the row needs no cast to express what it is testing. + */ + buildWithDefaultTimeoutMs(value: number): Transport; } /** What every row below needs: a transport factory, the live fixture origin, and the capability flags. */ @@ -976,6 +991,60 @@ function registerProxyRefusalRows(ctx: SuiteContext): void { }); } +/** + * Defaults `AbortSignal.timeout()` refuses. `1.5` and `2**32` are the two Bun 1.3.14 accepts and + * Node rejects with a `RangeError`, which is what made an unvalidated default a per-runtime + * behaviour rather than a per-caller error. + */ +const UNHONOURABLE_TIMEOUTS: readonly number[] = [ + 0, + -1, + 1.5, + 2 ** 32, + Number.NaN, + Number.POSITIVE_INFINITY, +]; + +function registerDefaultTimeoutRows(ctx: SuiteContext): void { + describe('HTTP-35, TRANSPORT-5: an unhonourable default timeout is refused at construction', () => { + for (const value of UNHONOURABLE_TIMEOUTS) { + test(`a default of ${String(value)} fails the factory, not the first send`, () => { + let thrown: unknown; + try { + // A transport that returns instead of throwing has deferred the failure to the first + // send, where it arrives as a raw `RangeError` out of `AbortSignal.timeout()` on Node -- + // or, on Bun, as no failure at all and a deadline nobody asked for. + void ctx.capabilities.buildWithDefaultTimeoutMs(value); + } catch (error) { + thrown = error; + } + expect(thrown).toBeInstanceOf(TypeError); + // The same shape every other construction-time refusal in these transports has, and + // outside the IoError tree for the same reason (RETRY-2). + expect(isIoError(thrown)).toBe(false); + // "Discoverable": the message names the value that was refused. + expect((thrown as Error).message).toContain(String(value)); + }); + } + + test('a default inside the range builds a transport that still sends', async () => { + // The twin: narrowing the accepted range must not reject a legitimate default. 30s is the + // shape a caller actually configures, and the send proves the value reached `composeSignal` + // without tripping it. + await withTransport( + () => ctx.capabilities.buildWithDefaultTimeoutMs(30_000), + async transport => { + const response = await transport.send( + Request.newBuilder().url(ctx.url('/echo-headers')).build(), + ); + expect(response.status.code).toBe(200); + await response.close(); + }, + ); + }); + }); +} + function registerScopedRows(ctx: SuiteContext): void { if (ctx.capabilities.supportsInternalCancel) { describe('TRANSPORT-8: an internal cancel is told apart from a timeout', () => { @@ -1067,6 +1136,7 @@ export function runTransportConformanceSuite( registerInboundHeaderRows(ctx); registerDropSetRows(ctx); registerProxyRefusalRows(ctx); + registerDefaultTimeoutRows(ctx); registerScopedRows(ctx); }); } diff --git a/packages/transport-fetch/src/fetch-transport.conformance.test.ts b/packages/transport-fetch/src/fetch-transport.conformance.test.ts index 4ab7aa1..260693b 100644 --- a/packages/transport-fetch/src/fetch-transport.conformance.test.ts +++ b/packages/transport-fetch/src/fetch-transport.conformance.test.ts @@ -11,4 +11,6 @@ runTransportConformanceSuite('fetchTransport', () => fetchTransport(), { supportsProxy: false, // TRANSPORT-11: `Connection` is a WHATWG forbidden request header, so fetch drops it either way. dropsConnectionHeader: true, + // HTTP-35: the factory is where a default `AbortSignal.timeout()` could not take is refused. + buildWithDefaultTimeoutMs: value => fetchTransport({defaultTimeoutMs: value}), }); diff --git a/packages/transport-fetch/src/fetch-transport.ts b/packages/transport-fetch/src/fetch-transport.ts index f63c19b..784c33b 100644 --- a/packages/transport-fetch/src/fetch-transport.ts +++ b/packages/transport-fetch/src/fetch-transport.ts @@ -22,6 +22,7 @@ import { materializeBody, producerFailure, pumpBody, + requireValidDefaultTimeoutMs, toDispatchFailure, type ForkedSignal, type HeaderDropLogging, @@ -85,7 +86,13 @@ const MAX_MATERIALIZED_BODY_BYTES = 1_000_000; export interface FetchTransportOptions { /** How dropped header names are logged (TRANSPORT-13); defaults to `'first-per-name'`. */ readonly headerDropLogging?: HeaderDropLogging; - /** A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. */ + /** + * A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. + * + * An integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s + * range and so the only one any transport can honour; anything else is refused by + * {@link fetchTransport} rather than by the first send (HTTP-35). + */ readonly defaultTimeoutMs?: number; /** A custom `fetch` implementation; defaults to `globalThis.fetch`. */ readonly fetch?: FetchLike; @@ -223,6 +230,9 @@ class FetchTransport implements Transport { readonly #defaultTimeoutMs: number | undefined; constructor(options: FetchTransportOptions) { + // Before anything else: a default no `AbortSignal.timeout()` can take is a caller error, and + // HTTP-35 puts it where it was supplied rather than at the first send (audit #67 / #82). + requireValidDefaultTimeoutMs(options.defaultTimeoutMs); this.#logDrops = createDropLogger( options.headerDropLogging ?? 'first-per-name', ); @@ -377,6 +387,9 @@ if (typeof Symbol.asyncDispose === 'symbol') { * * @param options - optional transport settings. * @returns a transport ready to send; release it with `close()`. + * @throws `TypeError` when `defaultTimeoutMs` is not an integer number of milliseconds in + * `1 .. 2**32 - 1` — `AbortSignal.timeout()`'s range, and therefore the only one a per-call + * deadline can be built from. * * @public */ diff --git a/packages/transport-shared/etc/transport-shared.api.md b/packages/transport-shared/etc/transport-shared.api.md index 1db54b9..4ac3d3d 100644 --- a/packages/transport-shared/etc/transport-shared.api.md +++ b/packages/transport-shared/etc/transport-shared.api.md @@ -100,6 +100,11 @@ export function producerFailure(done: Promise | undefined): Promise // @internal export function pumpBody(body: Body_2): BodyPump; +// Warning: (ae-internal-missing-underscore) The name "requireValidDefaultTimeoutMs" should be prefixed with an underscore because the declaration is marked as @internal +// +// @internal +export function requireValidDefaultTimeoutMs(value: number | undefined): void; + // Warning: (ae-internal-missing-underscore) The name "toDispatchFailure" should be prefixed with an underscore because the declaration is marked as @internal // // @internal diff --git a/packages/transport-shared/src/default-timeout.test.ts b/packages/transport-shared/src/default-timeout.test.ts new file mode 100644 index 0000000..a4637ef --- /dev/null +++ b/packages/transport-shared/src/default-timeout.test.ts @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/default-timeout.test.ts +// Exercises: HTTP-35 (a timeout outside the range a transport can honour is refused where it was +// supplied, not where it is used), TRANSPORT-5 (a per-call override replaces a transport default, +// so the default is a real configuration value and answerable for its own range) +import {describe, expect, test} from 'bun:test'; +import {isIoError} from '@dexpace/core'; +import {requireValidDefaultTimeoutMs} from './default-timeout.js'; + +describe('requireValidDefaultTimeoutMs', () => { + test('accepts undefined and every integer in the honourable range', () => { + for (const value of [undefined, 1, 50, 30_000, 2 ** 32 - 1]) { + expect(() => { + requireValidDefaultTimeoutMs(value); + }).not.toThrow(); + } + }); + + test('refuses everything AbortSignal.timeout() cannot take', () => { + // The full range, not merely its lower bound. `1.5` and `2**32` are the two Bun 1.3.14 accepts + // and Node rejects with a `RangeError`, which is the divergence that made an unvalidated + // default a per-runtime behaviour rather than a per-caller error. + for (const value of [ + 0, + -1, + 1.5, + 2 ** 32, + Number.NaN, + Number.POSITIVE_INFINITY, + ]) { + let thrown: unknown; + try { + requireValidDefaultTimeoutMs(value); + } catch (error) { + thrown = error; + } + expect([value, thrown instanceof TypeError]).toEqual([value, true]); + // Outside the `IoError` tree, like every other construction-time refusal these transports + // raise: nothing retries a factory, and one class for all of them is easier to catch. + expect([value, isIoError(thrown)]).toEqual([value, false]); + // "Discoverable": the message names the value that was refused, not merely that one was. + expect((thrown as Error).message).toContain(String(value)); + } + }); + + test('the message names the range as well as the value', () => { + expect(() => { + requireValidDefaultTimeoutMs(0); + }).toThrow('1..4294967295'); + }); +}); diff --git a/packages/transport-shared/src/default-timeout.ts b/packages/transport-shared/src/default-timeout.ts new file mode 100644 index 0000000..afa6907 --- /dev/null +++ b/packages/transport-shared/src/default-timeout.ts @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: MIT +// packages/transport-shared/src/default-timeout.ts + +/** + * `AbortSignal.timeout()`'s upper bound, and therefore every transport's. Duplicated from + * `@dexpace/core`'s `http/request-options.ts:12`, which is not exported: the two must agree, and + * `RequestOptionsBuilder.timeoutMs`'s own rejection message is the wording copied below so a caller + * who trips either one reads the same sentence. + */ +const MAX_TIMEOUT_MS = 2 ** 32 - 1; + +/** + * Rejects a transport-wide default timeout that no transport could honour. + * + * The range is `AbortSignal.timeout()`'s — an integer in `1 .. 2**32 - 1` — because that is the + * only range the thing this value ends up in accepts. `RequestOptionsBuilder.timeoutMs` has checked + * exactly this since audit #67 / #76, on HTTP-35's reading that a timeout a setter accepted and a + * transport then refused is a failure belonging at the call site. `defaultTimeoutMs` was left + * unchecked on both transports and so became the last path by which `1.5`, `0` or `2**32` reached + * `composeSignal` — where Node throws `RangeError` and Bun 1.3.14 accepts the first two, so the same + * misconfiguration was a failed send on one runtime and a silently different deadline on the other + * (audit #67 / #82). + * + * A `TypeError`, matching the construction-time refusals both transports already raise for a + * caller misconfiguration (`undiciTransport`'s two) and deliberately outside the `IoError` tree — + * though nothing retries a factory, the conformance row asserts the same shape for both, and a + * transport is easier to reason about when every construction-time refusal is one class. + * + * @param value - the configured default, or `undefined` for none. + * @throws `TypeError` when a defined value is zero, negative, not finite, not an integer, or + * greater than `2**32 - 1`. + * + * @internal + */ +export function requireValidDefaultTimeoutMs(value: number | undefined): void { + if ( + value === undefined || + (Number.isInteger(value) && value > 0 && value <= MAX_TIMEOUT_MS) + ) { + return; + } + throw new TypeError( + `defaultTimeoutMs must be an integer number of milliseconds in 1..${String(MAX_TIMEOUT_MS)}, ` + + `got ${String(value)}: it is handed to AbortSignal.timeout(), which accepts nothing else`, + ); +} diff --git a/packages/transport-shared/src/index.ts b/packages/transport-shared/src/index.ts index 3712363..ebfcf58 100644 --- a/packages/transport-shared/src/index.ts +++ b/packages/transport-shared/src/index.ts @@ -9,6 +9,7 @@ export { pumpBody, type BodyPump, } from './body-pump.js'; +export {requireValidDefaultTimeoutMs} from './default-timeout.js'; export { isPermanentDispatchFailure, toDispatchFailure, diff --git a/packages/transport-undici/src/undici-transport.conformance.test.ts b/packages/transport-undici/src/undici-transport.conformance.test.ts index 1f5106a..24f0ef1 100644 --- a/packages/transport-undici/src/undici-transport.conformance.test.ts +++ b/packages/transport-undici/src/undici-transport.conformance.test.ts @@ -12,6 +12,9 @@ runTransportConformanceSuite('undiciTransport', () => undiciTransport(), { dropsConnectionHeader: false, // TRANSPORT-30: core resolves `ALL_PROXY=socks5://host:1080` to this type (CFG-22), and undici's // `ProxyAgent` is an HTTP CONNECT tunnel that cannot carry it. + // HTTP-35: the factory is where a default `AbortSignal.timeout()` could not take is refused. + buildWithDefaultTimeoutMs: value => + undiciTransport({defaultTimeoutMs: value}), unsupportedProxy: { type: 'socks5', build: () => diff --git a/packages/transport-undici/src/undici-transport.ts b/packages/transport-undici/src/undici-transport.ts index 7241146..0fc67a2 100644 --- a/packages/transport-undici/src/undici-transport.ts +++ b/packages/transport-undici/src/undici-transport.ts @@ -29,6 +29,7 @@ import { materializeBody, producerFailure, pumpBody, + requireValidDefaultTimeoutMs, toDispatchFailure, type BodyPump, type ForkedSignal, @@ -111,7 +112,13 @@ export interface UndiciTransportOptions { readonly proxy?: ProxyOptions; /** How dropped header names are logged (TRANSPORT-13); defaults to `'first-per-name'`. */ readonly headerDropLogging?: HeaderDropLogging; - /** A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. */ + /** + * A timeout applied to every call that supplies no `RequestOptions.timeoutMs` of its own. + * + * An integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s + * range and so the only one any transport can honour; anything else is refused by + * {@link undiciTransport} rather than by the first send (HTTP-35). + */ readonly defaultTimeoutMs?: number; /** `Agent` options, used only when no `dispatcher` is supplied. */ readonly agentOptions?: Agent.Options; @@ -472,6 +479,9 @@ class UndiciTransport implements Transport { #closing: Promise | undefined; constructor(options: UndiciTransportOptions) { + // Before `selectDispatchers` allocates anything: a refusal afterwards would leak the agents it + // built, with no transport for the caller to close them through (audit #67 / #82). + requireValidDefaultTimeoutMs(options.defaultTimeoutMs); this.#dispatchers = selectDispatchers(options); this.#proxy = options.proxy; this.#logDrops = createDropLogger( @@ -684,9 +694,10 @@ if (typeof Symbol.asyncDispose === 'symbol') { * * @param options - optional transport settings. * @returns a transport ready to send; release it with `close()`. - * @throws `TypeError` when both `dispatcher` and `proxy` are supplied, or when `proxy.type` is + * @throws `TypeError` when both `dispatcher` and `proxy` are supplied; when `proxy.type` is * anything but `http` — undici's `ProxyAgent` cannot carry a SOCKS proxy, and neither can - * `@dexpace/transport-fetch`, which has no proxy option at all. + * `@dexpace/transport-fetch`, which has no proxy option at all; or when `defaultTimeoutMs` is not + * an integer number of milliseconds in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s range. * * @public */ diff --git a/tests/node-conformance/transport.test.mjs b/tests/node-conformance/transport.test.mjs index ff3fe94..92541c0 100644 --- a/tests/node-conformance/transport.test.mjs +++ b/tests/node-conformance/transport.test.mjs @@ -20,6 +20,8 @@ // short write fails the send on the streamed path, which only this runtime can assert), // TRANSPORT-24/25 (a 204 and a HEAD carry a null body on this runtime as well -- Node's `fetch` // returns null where Bun's returns a stream, and undici's dispatcher always returns a readable), +// HTTP-35 (a defaultTimeoutMs AbortSignal.timeout() cannot take is refused at the factory -- Node +// throws RangeError for two of the values Bun accepts), // TRANSPORT-25 (the response body is a lazily-read stream and close releases it), TRANSPORT-29/SEAM-12 // (concurrent sends), SEAM-16 (an abort after delivery must not close the delivered body). import assert from 'node:assert/strict'; @@ -146,9 +148,14 @@ describe('the transport adapters on the Node runtime', () => { }); }); + // `makeTransport` takes the transport-wide default timeout so the HTTP-35 case can build a + // misconfigured transport; every other case passes nothing and gets today's shape. for (const [name, makeTransport] of [ - ['transport-fetch', () => fetchTransport()], - ['transport-undici', () => undiciTransport()], + ['transport-fetch', defaultTimeoutMs => fetchTransport({defaultTimeoutMs})], + [ + 'transport-undici', + defaultTimeoutMs => undiciTransport({defaultTimeoutMs}), + ], ]) { describe(`${name} on the Node runtime`, () => { it('returns a 302 raw and never follows it (TRANSPORT-1)', async () => { @@ -290,6 +297,39 @@ describe('the transport adapters on the Node runtime', () => { } }); + it('refuses an unhonourable defaultTimeoutMs at the factory (HTTP-35)', async () => { + // Runtime-divergent, and the reason the check exists at all: `AbortSignal.timeout(1.5)` and + // `AbortSignal.timeout(2**32)` throw `RangeError` on Node and are accepted by Bun 1.3.14, + // so before audit #67 / #82 the same misconfigured transport failed every send here and + // silently used a different deadline there. The factory now answers identically on both, + // which is what this pins on the runtime that used to be the strict one. + for (const value of [0, -1, 1.5, 2 ** 32, Number.NaN]) { + assert.throws( + () => makeTransport(value), + error => { + assert.ok( + error instanceof TypeError, + `expected a TypeError for ${value}, got ${error?.constructor?.name}`, + ); + assert.equal(isIoError(error), false); + assert.ok(error.message.includes(String(value)), error.message); + return true; + }, + ); + } + // And a legitimate default still builds something that sends. + const transport = makeTransport(30_000); + try { + const response = await transport.send( + Request.newBuilder().url(`${origin}/echo`).build(), + ); + assert.equal(response.status.code, 200); + await response.close(); + } finally { + await transport.close(); + } + }); + it('classifies a per-call timeout as retryable, not cancellation (TRANSPORT-4)', async () => { const transport = makeTransport(); try { From a889d5d34c0b99f44552ed892b24729ab9b95494 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 13:39:12 +0300 Subject: [PATCH 099/100] docs(transport): record the parity work, and the TRANSPORT-20 reading MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `docs/deviations.md` gains one row for the reading the classification table rests on: `TRANSPORT-20`'s "any transport failure that produced no HTTP response" is read as an exchange that failed, not as a request the native client refused to make, and such a refusal is reported outside the `IoError` tree so `retry/classify.ts`'s allow-list makes it non-retryable. The reading is not new — undici has applied it since Phase 8a — but it lived only in that phase's checklist, and until this run `@dexpace/transport-fetch` did the opposite for the identical condition. The row records that the MUST is still the default: the table is an allow-list, and `bad port` is excluded by name because TRANSPORT-20's own dead-port probe arrives with that reason on Node. `reasonPhrase` is deliberately not re-ledgered — it is already a row beside §10 item 13. Two citations this branch moved are re-anchored: the SOCKS row's `undici-transport.ts:138,151-158,192` -> `:147,160-167,201` and `fetch-transport.ts:76-79` -> `:79-82`. `run-suite.ts`'s header re-anchors the two TRANSPORT-22 test citations for the same reason, and says why TRANSPORT-9's producer race is not a shared row: only an instrumented native client can show that a pending call was cancelled. READMEs: `transport-shared`'s module table gains the three new modules and the fork's second direction; both adapters' behaviour lists gain the classification rule, the body-less contract and the `defaultTimeoutMs` range. --- docs/deviations.md | 3 ++- packages/transport-conformance/src/run-suite.ts | 8 ++++++-- packages/transport-fetch/README.md | 12 ++++++++++++ packages/transport-shared/README.md | 7 +++++-- packages/transport-undici/README.md | 13 +++++++++++++ 5 files changed, 38 insertions(+), 5 deletions(-) diff --git a/docs/deviations.md b/docs/deviations.md index ea1d822..dc59fb7 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -502,7 +502,8 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **`HTTP-35`'s timeout check is read as the FULL range `AbortSignal.timeout()` accepts, not the lower bound the requirement enumerates.** `HTTP-35` says the options builder "MUST reject a non-null timeout that is zero or negative". `RequestOptionsBuilder.timeoutMs` rejects three more classes: non-finite (shipped unledgered before this audit), non-integer, and anything above `2**32 - 1`. **Strictly stricter than the letter, and deliberately so.** The field has exactly one consumer — `composeSignal` hands it to `AbortSignal.timeout()` — so a value this setter admits and that function refuses is `HTTP-35`'s own failure mode with the seam moved: the error surfaces inside a transport, as an unwrapped platform `RangeError`, one frame away from the call that supplied it. The earlier reading accepted `1.5` and argued in TSDoc that "a timeout is a duration and a fractional millisecond is meaningful"; no consumer of the field can express one. **The range checked is Node's, and that is the point:** `AbortSignal.timeout(1.5)` and `AbortSignal.timeout(2 ** 32)` raise `RangeError` on Node and are ACCEPTED on Bun, and a negative delay is `RangeError` on Node against `TypeError` on Bun (measured 2026-09-05), so leaving the check to the runtime would make an SDK-level contract depend on which runtime the caller happens to be on. *Rejected:* rounding with `Math.ceil` and clamping inside `composeSignal`, which hides the caller's mistake in the one place `HTTP-35` exists to surface it. `composeSignal` is documented as still able to raise, because a transport's own `defaultTimeoutMs` construction option bypasses this setter and is not validated by core — recorded for #81/#82, not fixed here | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:48` is `HTTP-35`'s wording. `packages/core/src/http/request-options.ts:12` (`MAX_TIMEOUT_MS`) and `:204-214` (the check and the rewritten TSDoc paragraph); `packages/core/src/seams/transport.ts:86-92` is `composeSignal`'s new `@throws`, which states the two-runtime divergence rather than naming one error class. Pinned by "rejects a fractional timeout, which no transport deadline can honor" (`packages/core/src/http/request-options.test.ts:128`, the FLIPPED case — it pinned acceptance until this audit), "rejects a timeout above AbortSignal.timeout()'s ceiling of 2**32 - 1" (`:134`), "accepts the ceiling itself" (`:143`) and the `every accepted timeout is an integer in 1..2**32 - 1` property (`:157`); the Node half is `composeSignal timeout range on Node (HTTP-35)` in `tests/node-conformance/seams.test.mjs:105`, which cannot live in `bun test` because Bun accepts both rejected values | not yet in §10 | | **`HTTP-31`'s "falls back to raw text rather than throwing" is satisfied for an unpaired surrogate by SUBSTITUTING U+FFFD, not by keeping the raw text.** `HTTP-31` (MUST) makes `QueryParams.parse` lenient and enumerates the lenient cases, ending with "malformed percent-encoding falling back to raw text rather than throwing". An unpaired surrogate is a fourth kind of malformed input the enumeration does not name, and the fallback it prescribes is not available for it: the raw text has no UTF-8 form, so keeping it produces a `QueryParams` whose `encode()` throws `URIError` — the throw merely deferred out of `parse` and into an accessor that documents no throw at all. **The port repairs instead.** `parse` runs `toWellFormed()` over each decoded name and value, so every instance it returns is encodable, which is what "parsing MUST invert encode" needs to mean. The strict half of the rule is unaffected and is where `#76` puts the rejection: `QueryParamsBuilder.add` throws `UrlConstructionError` for the same input, and `substitutePathParams` throws `OperationAssemblyError`. That asymmetry is not new to the query model — it is exactly the outbound/inbound split `Headers` already draws for `HTTP-18` against `HTTP-19`, applied to the one requirement pair that needs it here. Substitution matches the platform rather than inventing a policy: `new URL('https://x/?a=\uD800').search` is `?a=%EF%BF%BD` (measured 2026-09-05). *Rejected:* letting `parse` throw the builder's error, which breaks a MUST. *Rejected:* dropping the offending parameter, which loses a name the caller may be matching on | audit #67 / #76 | 2026-09-05 | `docs/product-spec/04-core-http-domain-model.md:42` carries `HTTP-31`'s wording (shared with `HTTP-30`). `packages/core/src/http/rfc3986.ts:17-18` are the two patterns, `:31` `hasLoneSurrogate` (strict) and `:44` `toWellFormed` (lenient) — one rule, two entry points, so no caller can pick the wrong one; `packages/core/src/http/query-params.ts:144-150` is `parse`'s repair with the `HTTP-18`/`HTTP-19` comparison stated inline, against `:44-50` and `:240-241` for the strict `add` path; `packages/core/src/seams/operation.ts:139-144` is the path-param half. `/\p{Surrogate}/u` rather than `String.prototype.isWellFormed()` because the latter is ES2024 and `tsconfig.base.json:5-11` pins `lib: ES2023`, though the `engines.node >= 20.3` runtime has it. Pinned by the `lone surrogates are rejected where they are supplied (HTTP-29, HTTP-31)` block in `packages/core/src/http/query-params.test.ts:170` — "parse() stays lenient and substitutes U+FFFD, because HTTP-31 forbids throwing" (`:197`) and the `no anything escapes parse()` property (`:233`) | not yet in §10 | | **`OBS-35`'s "MUST NOT bake in a default config key name" is satisfied by making the key configurable, not by removing the default.** `OBS-35` (SHOULD) asks for a tolerant, layered log-level resolution and adds one MUST: no baked-in default key name. The port ships `CFG_KEY_LOG_LEVEL` (`DEXPACE_LOG_LEVEL`) as `CFG-14`'s well-known key and, until 2026-09-05, read it unconditionally. It is now `LoggingStepSettings.configKey`'s default: a caller names their own key and the resolution is otherwise identical. **Why the default stays.** A required key would mean no caller gets ambient granularity without naming one first, which trades a MUST about *naming* for a worse default experience, and `CFG-14` — which this port also implements — exists precisely to standardise the name. The layered resolution itself is `CFG-1`'s (override → environment → normalised property → default) and is tolerant as the requirement asks. **A second, quieter half:** the process-wide configuration slot starts empty (`CFG-13`), so no key of any name resolves until a host calls `setGlobalConfiguration(defaultConfiguration())`. That is deliberate — defaulting the slot to a configuration that reads `process.env` would make an import-time environment read the SDK's default behaviour — and it is now documented as the required wiring rather than left to be discovered | audit #67 / #80 | 2026-09-05 | `packages/core/src/observability/logging-step.ts:64-79,94-104` (the setting and the resolution); `packages/core/src/config/configuration.ts:311,354-358,368` (`CFG_KEY_LOG_LEVEL`, `defaultConfiguration`, the empty default slot); `docs/sdk-documentation/pipelines.md` "Turning logging on from the environment"; `docs/product-spec/15-instrumentation-and-observability.md:66` (the requirement) | not yet in §10 | -| **`CFG-22`'s SOCKS proxy types are resolved by the configuration layer and supported by neither shipped transport; the refusal is at the transport factory, and `ProxyType` keeps them.** `CFG-22` (MUST) requires the proxy model to carry "the proxy protocol type (HTTP, SOCKS4, SOCKS5)", and the port implements it in full: `ProxyType` is `'http' \| 'socks4' \| 'socks5'`, and `resolveProxyOptions` maps `ALL_PROXY`/`HTTPS_PROXY`'s `socks:`, `socks4:`, `socks4a:`, `socks5:` and `socks5h:` schemes onto it. Nothing can then send over one. `@dexpace/transport-undici` builds undici's `ProxyAgent`, which is an HTTP `CONNECT` tunnel reading its `uri` as a URL, and `@dexpace/transport-fetch` ships no `proxy` option at all because Node's bare global `fetch` exposes no proxy hook outside undici internals. So a configuration that resolves cleanly has no transport that can honour it. **What changed on 2026-09-05 (audit #67 / #81).** Until then the discovery was `new ProxyAgent({uri: 'socks5://…'})` throwing undici's `InvalidArgumentError('Invalid URL protocol: socks5:')` out of a public factory — untyped, undocumented, and outside the SDK's error vocabulary. `undiciTransport()` now refuses `proxy.type !== 'http'` at construction with a `TypeError` naming the type, before any dispatcher is allocated, deliberately outside the `IoError` tree so `retry/classify.ts`'s allow-list makes it non-retryable (RETRY-2). That is `TRANSPORT-30`'s "make the limitation discoverable rather than silently misbehaving" applied at the earliest point that can. **Why `ProxyType` still admits `socks4`/`socks5`.** Narrowing a `@public` union is a breaking change and therefore a release-pass decision, which this run is not taking (the run's release machinery is suspended); and `CFG-22`'s MUST is about the *model*, which would then no longer satisfy it. The honest state is a configuration layer that is complete and a transport layer that is not, which is what this row records. A future transport — a `node:net` SOCKS dialer, or a `ProxyAgent` replacement — closes the gap without a model change. | audit #67 / #81 | 2026-09-05 | `packages/core/src/config/proxy.ts:34` (`ProxyType`), `:372-380` (the scheme map); `packages/transport-undici/src/undici-transport.ts:138,151-158,192` (the supported type, the refusal, and where it runs); `packages/transport-fetch/src/fetch-transport.ts:76-79` (no `proxy` option, and why); `docs/product-spec/16-configuration.md:42` (`CFG-22`); `docs/product-spec/17-transport-adapter-conformance-contract.md:48` (`TRANSPORT-30`) | not yet in §10 | +| **`CFG-22`'s SOCKS proxy types are resolved by the configuration layer and supported by neither shipped transport; the refusal is at the transport factory, and `ProxyType` keeps them.** `CFG-22` (MUST) requires the proxy model to carry "the proxy protocol type (HTTP, SOCKS4, SOCKS5)", and the port implements it in full: `ProxyType` is `'http' \| 'socks4' \| 'socks5'`, and `resolveProxyOptions` maps `ALL_PROXY`/`HTTPS_PROXY`'s `socks:`, `socks4:`, `socks4a:`, `socks5:` and `socks5h:` schemes onto it. Nothing can then send over one. `@dexpace/transport-undici` builds undici's `ProxyAgent`, which is an HTTP `CONNECT` tunnel reading its `uri` as a URL, and `@dexpace/transport-fetch` ships no `proxy` option at all because Node's bare global `fetch` exposes no proxy hook outside undici internals. So a configuration that resolves cleanly has no transport that can honour it. **What changed on 2026-09-05 (audit #67 / #81).** Until then the discovery was `new ProxyAgent({uri: 'socks5://…'})` throwing undici's `InvalidArgumentError('Invalid URL protocol: socks5:')` out of a public factory — untyped, undocumented, and outside the SDK's error vocabulary. `undiciTransport()` now refuses `proxy.type !== 'http'` at construction with a `TypeError` naming the type, before any dispatcher is allocated, deliberately outside the `IoError` tree so `retry/classify.ts`'s allow-list makes it non-retryable (RETRY-2). That is `TRANSPORT-30`'s "make the limitation discoverable rather than silently misbehaving" applied at the earliest point that can. **Why `ProxyType` still admits `socks4`/`socks5`.** Narrowing a `@public` union is a breaking change and therefore a release-pass decision, which this run is not taking (the run's release machinery is suspended); and `CFG-22`'s MUST is about the *model*, which would then no longer satisfy it. The honest state is a configuration layer that is complete and a transport layer that is not, which is what this row records. A future transport — a `node:net` SOCKS dialer, or a `ProxyAgent` replacement — closes the gap without a model change. | audit #67 / #81 | 2026-09-05 | `packages/core/src/config/proxy.ts:34` (`ProxyType`), `:372-380` (the scheme map); `packages/transport-undici/src/undici-transport.ts:147,160-167,201` (the supported type, the refusal, and where it runs); `packages/transport-fetch/src/fetch-transport.ts:79-82` (no `proxy` option, and why); `docs/product-spec/16-configuration.md:42` (`CFG-22`); `docs/product-spec/17-transport-adapter-conformance-contract.md:48` (`TRANSPORT-30`) | not yet in §10 | +| **`TRANSPORT-20`'s "any transport failure that produced no HTTP response" is read as an exchange that failed, not as a request the native client refused to make.** `TRANSPORT-20` (MUST) names four instances — connection refused, DNS/TLS failure, peer reset, connect/read timeout — and requires the retryable `TransportFailureError`. A scheme the client will not speak (`ftp://`), a forbidden method (`CONNECT`), a method that is not a token, a body on a GET: all of them also produce no HTTP response, so the literal reading makes them retryable too. The port refuses that. `retry/classify.ts:90` is an allow-list over `instanceof IoError`, so retryable would mean the caller's entire retry budget spent re-proving a URL that cannot change between attempts, and the requirement's own enumeration is four ways an *exchange* fails, not four ways an argument is rejected. Such a refusal surfaces as a bare `TypeError` carrying the native error as `cause`, outside the `IoError` tree, which is the same class both transports already raise for a misconfiguration caught at construction. **What changed on 2026-09-05 (audit #67 / #82).** The reading is not new — `@dexpace/transport-undici` has applied it to undici's `UND_ERR_INVALID_ARG` / `UND_ERR_NOT_SUPPORTED` since Phase 8a — but it was recorded only in that phase's checklist, and `@dexpace/transport-fetch` did the opposite for the identical condition: every native rejection became `TransportFailureError`. The decision now lives in one table in `@dexpace/transport-shared` that both adapters call, so the two cannot answer differently again, and `docs/sdk-documentation/write-a-transport.md`'s rule 4 tells a third transport to use it. **The MUST is still the default.** The table is an allow-list of three positive recognitions and everything else falls through to retryable; `'bad port'` is excluded by name, because port 1 is on WHATWG's blocked list and so `TRANSPORT-20`'s own dead-port conformance probe arrives with that reason on Node's `fetch`. | audit #67 / #82 | 2026-09-05 | `packages/transport-shared/src/dispatch-classification.ts:17,40,81,123` (the two tables, the predicate and the mapping, with the `'bad port'` exclusion documented at `:35`); `packages/transport-fetch/src/fetch-transport.ts:340` and `packages/transport-undici/src/undici-transport.ts:308,317` (the two call sites); `packages/core/src/retry/classify.ts:89-90` (the allow-list this is answerable to); `docs/product-spec/17-transport-adapter-conformance-contract.md:38` (the requirement); `docs/work/mvp/phase8/phase8a/2026-07-28-phase8a-transport-checklist.md:57` (where the reading was recorded before this row). Pinned by `packages/transport-conformance/src/run-suite.ts:387` on both adapters and by `tests/node-conformance/transport.test.mjs:351`, whose runtimes report the same refusal in entirely different shapes | not yet in §10 | ### Proposed erratum for `PIPE-40` (drafted 2026-09-04, not applied) diff --git a/packages/transport-conformance/src/run-suite.ts b/packages/transport-conformance/src/run-suite.ts index bbe2402..1e28538 100644 --- a/packages/transport-conformance/src/run-suite.ts +++ b/packages/transport-conformance/src/run-suite.ts @@ -16,8 +16,12 @@ // full proxy-challenge flow is transport-undici's challenge-handler.test.ts, and only its // unsupported-type refusal is a row here. TRANSPORT-22 is NOT driven from here -- // forcing an adaptation throw needs a per-transport hook into the native response, so each adapter -// asserts it against its own (transport-fetch's fetch-transport.test.ts:118, transport-undici's -// undici-transport.test.ts:614). +// asserts it against its own (transport-fetch's fetch-transport.test.ts:123, transport-undici's +// undici-transport.test.ts:615). TRANSPORT-9's producer-failure race is not driven from here +// either, and for the same reason: proving that a native call still pending when the producer fails +// is CANCELLED needs the signal the adapter handed it, which only an instrumented native client can +// show -- a `FetchLike` in transport-fetch's suite and a bring-your-own `Dispatcher` in +// transport-undici's, each with a call that resolves after the producer has already lost the race. import {mkdtemp, rm, truncate, writeFile} from 'node:fs/promises'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; diff --git a/packages/transport-fetch/README.md b/packages/transport-fetch/README.md index cf97160..60891f4 100644 --- a/packages/transport-fetch/README.md +++ b/packages/transport-fetch/README.md @@ -69,6 +69,18 @@ records the decision and the four reasons the floor does not move instead. it (`SEAM-16`). Cancellation stays live for the whole in-flight window. - A timeout surfaces as the retryable `TransportFailureError`; a caller abort as the terminal `CancellationError` (`TRANSPORT-3`/`TRANSPORT-4`). A raw `DOMException` is never surfaced. +- A request `fetch` refused to make — an unsupported scheme such as `ftp://`, a forbidden method, an + argument its own validation rejects — is a bare `TypeError` outside the `IoError` tree, so + `retry/classify.ts`'s allow-list makes it non-retryable (`RETRY-2`). A failed *exchange* stays the + retryable `TransportFailureError` (`TRANSPORT-20`). The table that tells them apart is + `@dexpace/transport-shared`'s, shared with `@dexpace/transport-undici`, because the runtimes report + the same refusal in three different shapes (audit #67 / #82). +- A 204, a 304 and every HEAD response carry `body === null`. Node's `fetch` says so itself; Bun + 1.3.14's returns a live `ReadableStream` for all three, which this transport cancels and replaces + with `null` so the shape is the SDK's rather than the runtime's. +- `defaultTimeoutMs` must be an integer number of milliseconds in `1 .. 2**32 - 1` — + `AbortSignal.timeout()`'s range. Anything else is a `TypeError` out of `fetchTransport()`, not a + failure on the first send (`HTTP-35`). ## Conformance diff --git a/packages/transport-shared/README.md b/packages/transport-shared/README.md index dc04786..91242ee 100644 --- a/packages/transport-shared/README.md +++ b/packages/transport-shared/README.md @@ -18,5 +18,8 @@ other, not merely of the rest of the tree. | `header-mapping.ts` | `TRANSPORT-10`/`TRANSPORT-12`'s outbound drop-and-degrade pass and `TRANSPORT-14`'s lenient inbound copy, which preserves obs-text values rather than rejecting them | | `drop-log.ts` | `TRANSPORT-13`'s bounded, case-insensitive, drain-to-cap dedup of already-logged drop names. Names only — never values | | `abort-mapping.ts` | The single mapping from an aborted signal to a canonical SDK error: `TransportFailureError` on timeout, `CancellationError` otherwise. A raw `DOMException` is never surfaced | -| `body-pump.ts` | Turning a `Body` into a request stream the transport owns the closing of, plus `TRANSPORT-19`'s idempotent teardown for an abandoned producer | -| `signal-fork.ts` | `SEAM-16`'s abort-after-delivery rule: both native clients tie a response body's lifetime to the signal they were given, so the transport dispatches over a fork it detaches at delivery | +| `dispatch-classification.ts` | The single mapping from a *native rejection* to one: the retryable `TransportFailureError` `TRANSPORT-20` requires for a failed exchange, and a bare `TypeError` outside the `IoError` tree for a request the client refused to make. An allow-list, so an unrecognised rejection stays retryable | +| `body-less.ts` | Which method/status pairs can carry no response body at all, so `Response.body` is `null` for a 204, a 304 or a HEAD on every runtime rather than on whichever ones agree with the spec | +| `default-timeout.ts` | `HTTP-35`'s range check for a transport-wide default timeout: an integer in `1 .. 2**32 - 1`, which is `AbortSignal.timeout()`'s and therefore every transport's | +| `body-pump.ts` | Turning a `Body` into a request stream the transport owns the closing of, plus `TRANSPORT-19`'s idempotent teardown for an abandoned producer, and the classification of a producer's own failure | +| `signal-fork.ts` | `SEAM-16`'s abort-after-delivery rule: both native clients tie a response body's lifetime to the signal they were given, so the transport dispatches over a fork it detaches at delivery — and `TRANSPORT-9`'s other direction, a handle the transport pulls to cancel a native call it has abandoned | diff --git a/packages/transport-undici/README.md b/packages/transport-undici/README.md index c67c411..54edeee 100644 --- a/packages/transport-undici/README.md +++ b/packages/transport-undici/README.md @@ -110,6 +110,19 @@ silently misbehaving (`TRANSPORT-30`): (`TRANSPORT-12`), never a failed send. - Destroying the dispatcher mid-flight surfaces as the terminal `CancellationError`, while a timeout on the same path stays the retryable `TransportFailureError` (`TRANSPORT-8`). +- An argument undici refuses outright — a non-`http(s)` origin such as `ftp://`, `CONNECT` as a + method, a per-request `Proxy-Authorization` on a bring-your-own `ProxyAgent` — is a bare + `TypeError` outside the `IoError` tree, so `retry/classify.ts`'s allow-list makes it non-retryable + (`RETRY-2`). A failed *exchange* stays the retryable `TransportFailureError` (`TRANSPORT-20`). The + table that tells them apart moved to `@dexpace/transport-shared` on 2026-09-05 so + `@dexpace/transport-fetch` answers identically (audit #67 / #82). +- A 204, a 304 and every HEAD response carry `body === null`. undici's dispatcher always hands back a + `BodyReadable`, so this transport `dump()`s the one it declines to expose — returning the + connection to the pool — rather than wrapping an empty stream the caller would have to read to + discover was empty. +- `defaultTimeoutMs` must be an integer number of milliseconds in `1 .. 2**32 - 1` — + `AbortSignal.timeout()`'s range. Anything else is a `TypeError` out of `undiciTransport()`, raised + before any dispatcher is allocated, not a failure on the first send (`HTTP-35`). - `Response.protocol` is always `HTTP_1_1`: undici's `ResponseData` does not surface the negotiated version. A Deviation Ledger row, not a silent gap. From 262fb8f9d3dda1fda9401f30146382b7b0909814 Mon Sep 17 00:00:00 2001 From: Mohammad Wahbeh Date: Sat, 5 Sep 2026 13:44:37 +0300 Subject: [PATCH 100/100] =?UTF-8?q?docs:=20ledger=20=E2=80=94=20wave=206b?= =?UTF-8?q?=20landed=20(#82),=20run=20complete;=20re-anchor=20item=2013=20?= =?UTF-8?q?after=20#81/#82?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/audit-67-decisions.md | 33 +++++++++++++++++++++++++++++++++ docs/deviations.md | 10 +++++----- 2 files changed, 38 insertions(+), 5 deletions(-) diff --git a/docs/audit-67-decisions.md b/docs/audit-67-decisions.md index c839cd6..94c08f8 100644 --- a/docs/audit-67-decisions.md +++ b/docs/audit-67-decisions.md @@ -548,6 +548,38 @@ Known merge seams: `deviations.md` (item 17 section vs OBS-29 row — disjoint r PR #96 (#81) merged into the umbrella at `808f6b0`. Single branch on the umbrella tip, so its own preflight (all 20 steps passed, at `ca682c5`; round 2 changed a README only) is the merged tree's. Identity checked. +## Wave 6b — landed 2026-09-05; the run is complete +PR #97 (#82) merged into the umbrella at `901b50e`. Single branch on the umbrella tip, so its own preflight (all +20 steps passed) is the merged tree's. Identity checked. All fifteen subtasks of #67 are merged; milestones 1–5 +are done. The seven `file:line` citations in `deviations.md` item 13 and the §10 `sideEffects` item that #81 +and #82 shifted were re-anchored on the umbrella after this merge. + +**D20 outcome (2026-09-05, #82, PR #97).** Three new `transport-shared` modules: `dispatch-classification.ts` +(allow-list of permanent recognitions — terminal argument codes, a cause-less `TypeError`, three known scheme +messages; `'bad port'` deliberately excluded because Node's `fetch` reports TRANSPORT-20's own dead-port probe +that way), `body-less.ts` (`body === null` for HEAD, 101/103/204/205/304, 2xx CONNECT; each adapter releases +the handle it declines — **Bun's `fetch` also returned a live stream for these, so both adapters needed it**), +`default-timeout.ts` (carried from #76; `TypeError` at both factories). `CONTROL_BYTE` now covers LF. +`ForkedSignal` always returns a live signal and gains `abort(reason)`, so the producer-failure branch can +cancel the native call; `producerFailure` classifies its own rejection as `TransportFailureError` so the shared +table cannot read it as permanent. **One `deviations.md` row beyond the task's letter, accepted:** the +classification departs from TRANSPORT-20's "*any* transport failure that produced no HTTP response", and +`write-a-transport.md` now prescribes the reading to third-party transports. The producer race is proven by an +instrumented-transport test, not a shared row (the suite cannot see the signal handed to the native call). +*Correction to D19's outcome:* `ftp://` on undici was already terminal via `UND_ERR_INVALID_ARG`. + +## Closing state (2026-09-05) +- Umbrella `audit/remediation-67`: fifteen task branches merged, PRs #83–#97 closed as merged, #67 checklist + fully ticked. Base `mvp` untouched at `1f48926`. +- Every merged wave preflighted on its exact tree (all 20 steps) before the merge; every task-branch commit + authored as the repository's configured identity. +- Left deliberately for the release pass: the "Deferred — release machinery" table below (fifteen changesets' + worth of notes), `docs/first-release.md`, the `ProxyType` union question, `packages/core/src/io/index.ts`'s + dead barrel, `packages/core/README.md`'s Observability pointer, the `bindAbort` second report in SSE, and the + unknown-length `pipeTo` path forwarding empty chunks. +- Not done by this run, on purpose: opening a PR from the umbrella into `mvp`, and any `gh issue create` (D4's + PIPE-37 tracking issue). Both are the maintainer's. + ## Decisions taken for wave 6 (M5: #81, then #82) — pre-taken 2026-09-05 ### D19 — #81: degrade what undici cannot carry; one short-write detector; a typed refusal for SOCKS @@ -630,3 +662,4 @@ the `fromWeb` leak sits in the producer-race code D20 edits; `ftp://` still reac | #79 / PR #94 | patch changesets for `@dexpace/core` (paginator closes on malformed `PageInfo`; SSE release failure reported once; splice surrogate typed) and `@dexpace/codec-json` (abortable reads/writes; `tristate` rejects present null) | | #80 / PR #95 | minor changeset for `@dexpace/core`: `PipelineOptions` (new public shape), `LoggingStepSettings.configKey`, context restored after `send()`; `.changeset/2026-09-04-per-operation-span.md`'s `createRuntime(...)` example is stale | | #81 / PR #96 | patch changesets for `@dexpace/transport-undici` (header degrade, file body via `writeTo`, SOCKS refusal, file framing change) and `@dexpace/transport-fetch` (three names added to the drop set); `ProxyType` narrowing declined — keep the union | +| #82 / PR #97 | patch changesets for `@dexpace/transport-shared` (`CONTROL_BYTE`, classification table, body-less rule, `ForkedSignal.abort`, `requireValidDefaultTimeoutMs`), `@dexpace/transport-fetch` and `@dexpace/transport-undici` (permanent failures non-retryable; `body === null` for body-less responses; producer failure aborts the native call; `defaultTimeoutMs` validated) | diff --git a/docs/deviations.md b/docs/deviations.md index dc59fb7..0467a45 100644 --- a/docs/deviations.md +++ b/docs/deviations.md @@ -265,8 +265,8 @@ configuration file would be empty by construction. The structurally equivalent J 2026-08-30).** The `[Symbol.asyncDispose]` repair replaced three class members with a **module-scope `Object.defineProperty` statement** — a top-level side effect — in four files across three packages that all declare `"sideEffects": false` (`packages/core/src/sse/stream.ts:209`, -`packages/core/src/pagination/page.ts:114`, `packages/transport-fetch/src/fetch-transport.ts:314`, -`packages/transport-undici/src/undici-transport.ts:566`; the manifest field at `packages/core/package.json:25`, +`packages/core/src/pagination/page.ts:114`, `packages/transport-fetch/src/fetch-transport.ts:366`, +`packages/transport-undici/src/undici-transport.ts:674`; the manifest field at `packages/core/package.json:25`, `packages/transport-fetch/package.json:26`, `packages/transport-undici/package.json:26`). That field entitles a bundler to drop a module whose exports go unused, and nothing forbids a future one from also dropping a top-level statement it judges inert — which would silently un-install disposal in the shipped artifact while @@ -310,9 +310,9 @@ readings; reversing either would widen an attack surface for a caller convenienc ## 13. Transport adapters have platform-shaped gaps the reference does not **Verified.** `Protocol.HTTP_1_1` is hardcoded in both adapters -(`packages/transport-fetch/src/fetch-transport.ts:178`, `packages/transport-undici/src/undici-transport.ts:346`). +(`packages/transport-fetch/src/fetch-transport.ts:201`, `packages/transport-undici/src/undici-transport.ts:429`). `transport-fetch` documents having no `proxy` option at all -(`packages/transport-fetch/src/fetch-transport.ts:62-65`). The proxy `challengeHandler` is surfaced with a +(`packages/transport-fetch/src/fetch-transport.ts:77-83`). The proxy `challengeHandler` is surfaced with a warning rather than dispatched (`packages/transport-undici/src/challenge-handler.ts:27,50`). **Why it cannot be corrected.** Four of the five clauses are closed by the platform, not by choice: @@ -493,7 +493,7 @@ frozen tree and is amended only deliberately, by hand. When §10 is next amended | **`BODY-9`'s mark/reset replay path for a stream-backed body is not built: `StreamBody` is always single-use.** `BODY-9` is a SHOULD, and it is conditional — replayable "when and only when the stream supports mark/reset". Node's `ReadableStream` has no generic mark/reset to support, so the condition is never met and the SHOULD's own fallback ("otherwise it MUST be single-use") is the branch that applies. `StreamBody.replayable` is a hardcoded `false`, which every consumer already reads: `decide.ts` fails a non-303 redirect carrying one (REDIR-6), and the retry pillar refuses to re-send it. A caller who wants replay materializes first or uses `byteArrayBody`. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/stream-body.ts:24` (`readonly replayable = false`); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:441` | not yet in §10 | | **`BODY-34`'s one shared preview cap covers the two logging tees only, not `toHttpError`'s error-body capture.** Read literally as "every in-memory capture in the package", `BODY-34` would put the request-side tee, the response-side drain and `toHttpError`'s error buffer behind one configurable value. They cannot share one: `HTTP-52` **fixes** the error-body cap at 1 MiB, and a spec-fixed constant cannot also be the configurable shared setting. The two capture sites `BODY-34` actually names — the request-side tee-capture and the response-side drain-on-first-access — do share one cap, so the requirement's own enumeration is satisfied; only the wider reading is not. `http-status-error.ts` states the split at the constant rather than leaving it to be inferred. **Recorded in the Phase 3b design's ledger since 2026-07-25 and nowhere else** | audit #67 / #69 | 2026-09-04 | `packages/core/src/body/http-status-error.ts:17-19` (`ERROR_BODY_CAP_BYTES`, with the "Deliberately NOT BODY-34's shared preview cap" note); `packages/core/src/body/response-body-logging.ts:47` and `packages/core/src/body/response-body-logging.ts:105-115` (the shared `cap` and the bounded drain that honours it); `docs/work/mvp/phase3/phase3b/2026-07-25-phase3b-body-lifecycle-design.md:447` | not yet in §10 | | **`IO-38`'s cross-thread close visibility has no subject on this platform and is recorded as not applicable, not as satisfied.** The requirement presupposes that a source or buffer instance can reach a second thread, so that closing it there invalidates a slice being read here. None can. Class instances are not structured-cloneable at all — `postMessage`/`structuredClone` preserve neither prototypes nor `#private` fields, so a `ByteQueue` or `BufferedSource` sent to a worker arrives as a plain object with no methods and no close state to observe. `BufferedSource` is doubly excluded: it holds a `ReadableStreamDefaultReader`, which is neither cloneable nor transferable. A raw `ArrayBuffer` *can* be transferred, but it carries no close state and derives no slices, so the hazard has no subject there either. This row exists because "not applicable" and "done" look identical in a checklist and are not the same claim. **Recorded in the Phase 3a design's ledger since 2026-07-24 and nowhere else** | audit #67 / #69 | 2026-09-04 | `docs/work/mvp/phase3/phase3a/2026-07-24-phase3a-io-contracts-design.md:401`; `packages/core/src/io/buffered-source.ts:47` (the reader it holds); no `worker_threads`, `postMessage` or `structuredClone` call exists anywhere in `packages/core/src/` | not yet in §10 | -| **The two shipped transports disagree on `HTTP-6`'s optional reason phrase: `transport-fetch` sets it, `transport-undici` leaves it `undefined`.** `HTTP-6` (MUST) has a response carry "an optional reason phrase", and the domain model provides the slot. `fetch-transport` fills it from the WHATWG `Response.statusText`, normalizing the empty string to `undefined`. `undici-transport` does not call `.reasonPhrase()` at all, because undici's `ResponseData` carries `statusCode` and no phrase — HTTP/2 has no reason phrase and undici's parser does not surface HTTP/1.1's. **Platform-shaped, and it sits beside §10 item 13's `Protocol.HTTP_1_1` gap rather than under it.** Item 13 already ledgers the negotiated-version gap in both adapters for the same reason — the value is not readable — but it does not name the reason phrase, so this row does. The field is optional in the requirement, and no shipped step or conformance row reads it, so the disagreement is observable to a caller and to nothing else. Recorded rather than papered over: synthesizing a phrase from the status code in the undici adapter would report a value the server never sent | audit #67 / #69 | 2026-09-04 | `packages/transport-fetch/src/fetch-transport.ts:180` (`.reasonPhrase(...)` fed from the WHATWG `statusText`, with the empty string normalized to `undefined`) against `packages/transport-undici/src/undici-transport.ts:342-350` (the builder chain, with no `.reasonPhrase` call); `docs/product-spec/04-core-http-domain-model.md:13`; the neighbouring gap is item 13 of the audit above | not yet in §10 | +| **The two shipped transports disagree on `HTTP-6`'s optional reason phrase: `transport-fetch` sets it, `transport-undici` leaves it `undefined`.** `HTTP-6` (MUST) has a response carry "an optional reason phrase", and the domain model provides the slot. `fetch-transport` fills it from the WHATWG `Response.statusText`, normalizing the empty string to `undefined`. `undici-transport` does not call `.reasonPhrase()` at all, because undici's `ResponseData` carries `statusCode` and no phrase — HTTP/2 has no reason phrase and undici's parser does not surface HTTP/1.1's. **Platform-shaped, and it sits beside §10 item 13's `Protocol.HTTP_1_1` gap rather than under it.** Item 13 already ledgers the negotiated-version gap in both adapters for the same reason — the value is not readable — but it does not name the reason phrase, so this row does. The field is optional in the requirement, and no shipped step or conformance row reads it, so the disagreement is observable to a caller and to nothing else. Recorded rather than papered over: synthesizing a phrase from the status code in the undici adapter would report a value the server never sent | audit #67 / #69 | 2026-09-04 | `packages/transport-fetch/src/fetch-transport.ts:203` (`.reasonPhrase(...)` fed from the WHATWG `statusText`, with the empty string normalized to `undefined`) against `packages/transport-undici/src/undici-transport.ts:425-441` (the builder chain, with no `.reasonPhrase` call); `docs/product-spec/04-core-http-domain-model.md:13`; the neighbouring gap is item 13 of the audit above | not yet in §10 | | **`AUTH-8`'s redaction clause is read as covering EVERY credential type, not the three it enumerates.** The requirement's own list is "API key, name-key secret, bearer token", and it was implemented to the letter: `ApiKeyCredential`, `NameKeyCredential` and `BearerToken` shipped as classes with a `#private` secret, a redacted `toString()` and the `nodejs.util.inspect.custom` hook, while the Basic and Digest credentials shipped as structural interfaces with a public `readonly password: string`. That is a leak the same requirement's first four words forbid: `util.inspect` of an `AuthCredentialSet` printed `password: 'hunter2'` beside `ApiKeyCredential{key=***}`, and `JSON.stringify` serialized both passwords. **Widened, deliberately.** Both are now classes on the same pattern, with the password reachable only through the in-package `credentialPassword()` friend hook. The cost is a public-shape change — `{username, password}` object literals no longer type-check — taken now because it is free before the first version bump. Validation is deliberately NOT duplicated onto the classes: AUTH-14's non-empty-whitespace-permitted rule and AUTH-16's acceptable-set rule stay single-sourced in `basicHandler()`/`digestHandler()`, which `authStep()` builds at construction, so a blank password still fails synchronously from that factory | audit #67 / #71 | 2026-09-04 | `docs/product-spec/11-authentication.md:12` is AUTH-8's three-type enumeration. `packages/core/src/auth/credential.ts:342` (`BasicCredential`) and `:393` (`DigestCredential`), with the friend hooks at `:299-300` and `credentialPassword()` at `:314`; the sole reader is `buildHandlers` in `packages/core/src/auth/auth-step.ts:131-152`. Pinned by "a whole AuthCredentialSet is diagnostic-safe (AUTH-8)" in `packages/core/src/auth/credential.test.ts`, which drives the real `util.inspect` rather than the hook it calls | not yet in §10 | | **`XCUT-16`'s replay guard is keyed on whether the hop was guarded, not on whether the replacement looks credentialed.** `XCUT-16` and `AUTH-28` say the guard applies "on any path where a credential will be attached", and carve out "a deliberately credential-free re-issue MAY proceed over any scheme". Deciding which of the two a challenge replacement is cannot be done by reading header names: the step's own `ApiKeyCredentialConfig.headerName` stamps whatever header the caller names, and a `challengeHook` may invent a carrier this step has never been told about. The port therefore reads "a credential will be attached" as a property of the HOP — if the outbound pass ran the HTTPS guard, so does the replay, whatever URL and headers the hook chose. **Strictly wider than the requirement's letter**, and knowingly so: it refuses a downgraded replacement that carries no credential at all, on a hop that is credentialed. The carve-out is preserved where it is observable — a `NO_AUTH` hop is never guarded outbound, and its replay is guarded only when the replacement carries `Authorization` or `Proxy-Authorization`, which is the previous rule kept as a second arm. *Rejected:* deriving the credential-carrying header names from configuration, which misses the hook-invented carrier and is the shape that let the reported leak through | audit #67 / #71 | 2026-09-04 | `docs/product-spec/19-cross-cutting-invariants-and-policies.md:44` is the requirement and its carve-out. `packages/core/src/auth/auth-step.ts:389` sets `OutboundPlan.guarded`; `:564-575` is `guardReplayScheme` and its two arms. Pinned by "a replacement carrying a NON-standard credential header over plaintext is refused" and "a header-free replacement over plaintext is refused too" in `packages/core/src/auth/auth-step.test.ts`, and by the "XCUT-16: a guarded hop stays guarded across a challenge replay" block in `tests/conformance/xcut/security-by-default.conformance.test.ts` | not yet in §10 | | **`ASYNC-21`'s "MUST NOT close the caller-owned source on any termination" is not honoured: the RxJS SSE adapter takes ownership and closes.** `sseEvents$` and `typedSse$` pass `() => stream.close()` as `fromAsyncIterable`'s `release`, and RxJS runs a subscriber's finalizer on *every* termination — unsubscription, end-of-source and a source error alike, which is the complete list the clause names. **Kept, deliberately, on two grounds.** (1) **The clause has no subject on this platform.** It presumes a source whose iterator return leaves the source open; this port's `SseStream` is deliberately not that one. `#iterate`'s `finally` calls `#releaseQuietly()`, so the resource is released whenever the runtime drives `return()` — which `fromAsyncIterable` must do exactly once (`ASYNC-6`), and which a plain `for await` with `break` does too. Removing the callback would change which channel reports a release failure and when the release runs, not whether the caller-owned source ends up closed. (2) **The ordering is load-bearing.** The release runs *ahead of* `iterator.return()` because an async generator's `return()` queues behind a suspended `next()`, and an SSE stream idling between events is parked in exactly that pull — so without the callback an `unsubscribe()` stays pending until the server next sends a byte, holding the socket open indefinitely. Measured: deleting the two `release` arguments turns four cases red — the two suspended-pull ones, as "the teardown did not settle within 500ms", and the two pre-existing idle-unsubscribe assertions — while every exactly-once release count stays green, which is the shape of the claim. Pagination attaches no release for the complementary reason: a `Paginator`'s pulls are bounded HTTP exchanges, never a wait on a server that may never answer. *Rejected:* dropping the callback to match the letter (reintroduces the hang for no change in what closes). *Rejected:* a caller-facing `{ownership}` option (two behaviours to document for a case with one correct answer). The public TSDoc and `packages/rx/README.md` now state the transfer outright — subscribing hands the stream over, do not close it yourself and do not iterate it afterwards — rather than leaving the `ASYNC-21` citation on the doc comment's first line to read as satisfied | audit #67 / #75 | 2026-09-05 | `packages/rx/src/sse.ts:46` and `:73-75` are the two `release` arguments; `packages/rx/src/from-async-iterable.ts:103-108` is the teardown that runs one on every termination; `docs/product-spec/18-asynchronous-runtime-adapter-contract.md:42` is the requirement. Ground 1: `packages/core/src/sse/stream.ts:136-139` (`#iterate`'s `finally` → `#releaseQuietly()`) with `:117-121` (`close()` memoized, `SSE-28`). Ground 2: `packages/rx/src/from-async-iterable.ts:44-48` states the ordering and why. Pinned by the two `resource ownership` blocks in `packages/rx/src/sse.test.ts`, which count the release the OWNED resource sees rather than `SseStream.close()` calls — the facade memoizes, so a facade-level count reads "once" however many paths call it — and by "SSE ownership transfer releases once on Node" in `tests/node-conformance/rx-bridge.test.mjs`. Phase 8b marked `ASYNC-21` ✅ with this clause dropped from its gist (`docs/work/mvp/phase8/phase8b/2026-07-28-phase8b-async-runtime-checklist.md:67`); that is a dated record and is left as written. The other half is `SSE-41`'s own "documented source ownership" clause, which the same checklist marked ✅ (`:74`) on the strength of documentation that named unsubscription only — completed by the TSDoc and README rewrite this row accompanies | not yet in §10 |