"identity_documents" section in ramp.json - #38
Open
EugeneDymo wants to merge 28 commits into
Open
Conversation
…nts (RAMP-243) An agent's manifest is a bare role marker today, and a verifier finds its keys by appending a fixed path to the host in the covered Signature-Agent header. That works only while every participant serves its WBA directory at exactly that path, and it has no way at all to name a Signature Agent Card, which the protocol did not model until now. WellKnownManifest.identity_documents is an optional block with two optional members, wba_directory and signature_agent_card. It is discovery metadata and nothing else: verification is unchanged and still resolves the directory from the covered header, so a participant that publishes nothing behaves exactly as before. The block relocates a document's path; it does not add a lookup step, which is what separates it from the keys_uri pointer pattern this protocol rejected. Absence means different things on the two members. A missing wba_directory falls back to the WBA-canonical path on the serving host, which is today's behaviour. A missing signature_agent_card means NOT ADVERTISED — RAMP defines no canonical path for that document, so inventing a fallback would send consumers fetching a URL nobody agreed to serve. An empty block is the same as no block. Both members are RFC 3986 URI references resolved against the URL the manifest was fetched FROM, never the self-asserted domain member, which a hostile manifest could set to validate itself — the argument endpoint already carries. After resolution the URL must be on the exact same origin, and a subdomain is refused. That is stricter than the endpoint rule on purpose: whoever takes over the host named by endpoint misdirects signed calls they still cannot sign for, while whoever takes over the host named by wba_directory publishes their own keys and becomes the participant. A dangling DNS record on one unused subdomain is enough for that, without the participant ever losing the host that serves ramp.json. The allowance would buy nothing anyway: an Exchange's API is a live service that often runs elsewhere, while identity documents are static JSON served beside a ramp.json already on that exact host. Field 32 rather than the free 29: 5, 6 and 29 are all unassigned but all carry history, and 29 held registration_schema before account_registration replaced it. Pre-v1 this repo frees numbers rather than reserving them, so recycling is legal, but appending avoids giving one number two meanings in one file's history. No max_len, because RFC 3986 sets no length limit, the 1 MiB well-known cap already bounds parser input, and no regex engine runs on this field — a cap here would be an invented constant with no derivation. No CEL rule, so a publisher or Exchange, which serves a WBA directory and has no agent card, can still use the block; which documents a REGISTERED AGENT must publish is the identity service's rule to enforce, not the generic wire schema's. Two comments that still described the pre-WBA-split world are corrected in the same change: an agent's keys being inline in ramp.json, and Requester.domain being "used for public key lookup". Both now say the same three things and stop there — keys live in the WBA directory, verification resolves that directory from the covered Signature-Agent header, and identity_documents merely advertises where it is served. The corpus seed, the generated artifacts and the reference-page directive land here rather than separately because the drift gates admit no intermediate state. The descriptor embeds source comments, so even a comment-only proto edit changes its bytes, and doccoverage walks every message and field with no exemption map, so the contract cannot be green until the reference page names the new message. Seeding the block into WellKnownManifest/valid is what puts it through the proto-JSON round-trip; standalone vectors only prove it validates in isolation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t's origin identity_documents names URI references, and reading one is the part every consumer has to get identically right: a reference one SDK accepts and another refuses is a participant whose keys some consumers can find and others cannot. ResolveIdentityDocument is that one answer, pure string work in the IO-free tier, so it runs before anything is fetched. The rule is exact same origin — https, equal hostname, equal effective port, no userinfo — and it deliberately does not reuse HostAnchored. That predicate implements the endpoint rule, host or subdomain, and this field refuses a subdomain, because the two fail differently: a hijacked endpoint host misdirects calls the attacker still cannot sign for, while a hijacked identity-document host lets them publish their own keys and become the participant. The base is vetted before anything resolves against it, and that is a refusal rather than a courtesy. A base of http://a.example/ramp.json resolving to https://a.example/doc passes every later check — the hostnames match and both sides fold to no port — so accepting it means trusting a manifest that arrived unauthenticated, which lets an on-path attacker who rewrote that plaintext document name any path on the host and have it served back over TLS from the legitimate origin. Neither the base nor the reference may carry userinfo, and neither refusal echoes the value it is refusing, since that is where a leaked credential would end up. The host shape is IsBareDomain rather than a new predicate. It is the repo's one answer to "is this the host shape the wire admits", it is already ported and vector-tested in all three languages, and it refuses everything the rebuilt authority would otherwise have to special-case — a non-ASCII label, a bracketed IP literal, an empty host, a trailing root dot. It runs BEFORE case folding, the ordering the audience check already learned: EqualFold reports a host spelled with U+212A KELVIN SIGN and a plain ASCII host as the same name, so folding first would admit a homograph. The returned URL is canonical — host lowercased, a default port folded away — so all three SDKs answer with the same string instead of echoing however the manifest spelled the authority. Without that they could not: WHATWG URL parsing in TypeScript strips :443, lowercases the host and runs IDNA, which maps the Kelvin sign to a plain k. That port therefore reads the authority off the raw string and uses URL only to resolve the path, and the Python port reads netloc rather than .hostname for the same reason — str.lower() maps the Kelvin sign too. The three faces land together because the corpus gate admits no orphan. The 35 cases cover every reference form RFC 3986 defines, both halves of the network-path case, the ports that fold and the padded port that does not, the subdomain that separates this rule from the endpoint rule, both homographs, and four base-vetting cases that nothing downstream would catch. Each carries the author's intended verdict AND the exact resolved URL, and the emitter refuses to write a file where the real face disagrees. One guard is narrowed to allow this: pure modules could not import urllib at all, and the resolution needs urllib.parse. The ban now names the submodule, so urllib.request and everything else under urllib stay refused, with meta-tests pinning both directions. urllib.parse splits and joins strings and opens no socket. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reference page carried only the stub the doc-coverage gate demands, and nothing anywhere told a reader what the block is for or how to read it. The manifest page gains an "Identity documents" section beside "Endpoint host binding", which is where a reader comparing the two rules will look. It states the origin rule, states plainly that it is STRICTER than the endpoint rule, and says why: whoever takes over the host named by endpoint misdirects signed calls they still cannot sign for, while whoever takes over the host named by wba_directory publishes their own keys and becomes the participant. It also spells out the asymmetric absence semantics, since "not advertised" and "fall back to the canonical path" look interchangeable until someone writes them down. The authentication page gets a short note and nothing more. The verification steps are untouched and still resolve the directory from the covered Signature-Agent header, because that is what the code does; the note exists so a reader who meets the block elsewhere does not assume verification follows it. The example page was publisher-only, so an agent manifest — the shape this field was added for — had no worked example anywhere. It has one now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two questions a reader will ask, answered where the reasoning is kept rather than left in a ticket. The first is whether naming a document's location reopens the keys_uri / jwks_uri pointer pattern this protocol rejected. It does not: that decision was about public_keys, and the objection was that verification would have to FOLLOW the pointer — a second fetch on every verification path, and a second thing that can 404 or go stale. Verification is unchanged here and never reads the manifest, so the block relocates a path rather than adding a lookup step, and keys are still never republished. The second is why this field's origin rule is stricter than the endpoint rule, which the history already records as deliberately allowing a subdomain. The two fields fail differently, and the subdomain allowance buys nothing for a static document served beside a ramp.json the participant already serves on that exact host. The changelog entry is mirrored byte-for-byte into both files, which are hand-kept with no gate between them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…twice The reference-parse branch formatted the reference with %q and wrapped the *url.Error, which prints the same input a second time. So the one refusal that fires on unparseable input echoed the userinfo the file promises three times over not to echo. The manifest-URL parse branch already got this right with a static message. Match it. The existing no-echo test could not catch this: both of its cases use references that PARSE, so neither ever reached this branch. Add a third with an invalid percent-escape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The helper promised that every SDK returns the same string for the same
input. It did not. Python and TypeScript guarded the RAW reference with
regexes and then handed resolution to a different URL engine, so the
three answers split wherever those engines preprocess differently — and
the corpus held no input from any diverging class, so every parity suite
passed while the contract was false.
The worst case was a verdict split, not a formatting one. A leading tab
made the Python and TypeScript authority regexes read an ABSOLUTE
reference as a relative path, which skipped the https check, the
userinfo check, the origin check and the port check in one go. Only the
authority rebuild at the end kept the result on the manifest's origin.
That rebuild is now commented as load-bearing in both ports, because it
read like a formatting convenience.
Rather than reproduce three URL parsers byte for byte, untame input is
refused. Both the reference AND the manifest URL must be written in the
coarse RFC 3986 character set, with every percent-escape valid and no
percent-encoded dot segment. One rule covers control characters, spaces,
backslashes, "|", "^" and every non-ASCII byte. The set is deliberately
coarse and not the per-component pchar grammar: pchar would refuse "["
and "]", which all three already agree on, and a vector now pins that.
Four cases are fixed instead of refused, because refusing costs more:
- a port outside 1-65535, which Go accepted and which reached the
WHATWG parser as a raw TypeError outside TypeScript's documented
error family;
- an empty authority ("//", "///x"), which Go read as a plain path;
- a colon in the first segment of a schemeless reference, which is
path-noscheme under RFC 3986;
- the empty path, where RFC 3986 6.2.3 names "/" as the normalized
form, so the WHATWG port was right and this oracle was the deviant.
Python no longer resolves through urljoin. urljoin routes through
urlparse, which splits ";params" off the last segment and drops an empty
one, silently turning "/x;" into "/x"; and it returns an absolute
reference untouched, so this port alone left "/a/../b" unnormalized.
RFC 3986 5.2.2, 5.2.3 and 5.2.4 are written out instead.
A fragment on the manifest URL is now refused outright. A fragment is
never sent to a server, so a fetch URL cannot carry one, and while it
could the three parsers disagreed about whether a reference of "#"
defines an empty fragment or none at all.
Verified by a differential sweep of 21141 base/reference pairs run
through all three implementations: 0 disagreements, 6457 of them
accepted. 29 of those pairs are now generated vectors, one per class.
No previously recorded vector changed its verdict or its string.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The audience comparison and the identity-document rule each wrote out the same split-on-the-last-colon and fold-443 logic, in the same file, about ten lines apart. The 443 fold is half of BOTH origin comparisons, so a change to that rule had to be made twice per port to stay correct. The identity-document versions were the better extraction. Move them above the audience comparison, which now calls them. Behaviour-preserving, and the audience vector corpus is what says so: _normalize_domain and normalizeDomain are replayed against the Go oracle for every audience case, and Go is untouched here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard banned urllib while allowing urllib.parse, but the lookahead was anchored to the import keyword, so it only ever inspected the FIRST module on the line. "import urllib.parse, urllib.request" satisfied it on .parse and never looked further — a pure module could pull in the HTTP client and the guard stayed silent. The guard's own meta-tests missed it because every one of them used a single-module statement. ruff E401 rejects a multi-module import line today, so nothing could reach the hole in practice. It is fixed anyway: this guard is defence in depth, and holding on its own is the reason it was narrowed rather than deleted. Two patterns become one that scans the whole import line. The comma form joins the positive meta-tests, and "import urllib.parse, json" joins the negatives so the fix bans the second module rather than the comma. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The parity fix narrowed what identity_documents may legally carry, and the rule was written down in four places that all became wrong. The proto comments, the spec site, the reference page and both changelogs now say the same thing the three SDKs do. The character rule is stated as a refusal with its reason attached, because "must be written in the RFC 3986 character set" reads like pedantry until you know why: the three SDKs resolve this reference with three different URL parsers, and those parsers disagree about almost everything outside that set. One percent-encodes a literal "|" where another keeps it. One strips a tab and so reads an absolute reference to another host as a relative path, skipping every origin check. One decodes "%2e" before removing dot segments and fetches a different document. The docs say that, so a reader knows the rule is about interoperability rather than taste, and that every value the field is meant to carry is unaffected. design-history.md gets the decision itself, including the two cases fixed rather than refused and the reason the corpus missed all of this: a parity corpus assembled from realistic values covers only the region where independent implementations already agree. The descriptor embeds source comments, so a comment-only proto edit still changes gen/descriptor.binpb and pulls the generated types with it. Both changelog entries are byte-identical; the older drift further down those files is pre-existing and left alone. Nothing here implies that signature verification follows the pointer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine refusals in the Go helper formatted the input with %q. The userinfo checks were treated as the one place a credential could appear, but they only see a parsed authority. Go reads "https:u:pw@a.example/x" as an opaque reference and "u:pw@a.example/x" as one under the scheme "u", and neither has a userinfo component, so both reach a later refusal that then printed the whole string. Errors get logged; that is where the credential ended up. Every refusal is now a fixed string, which is also what the Python and TypeScript ports already returned for the same inputs. The rule is stated once on the function instead of per call site. The no-echo test in all three languages now covers the same five cases, chosen to land on different refusals: userinfo in the reference, userinfo in the manifest URL, a reference that does not parse, and the two opaque forms above. Python and TypeScript were missing the last three. Vectors are unchanged: the corpus records verdicts, not messages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
The parity suite asserted a bare toThrow() on every refused vector, so any exception satisfied it. Tightening that to the documented error family is worth doing, but on its own it catches nothing today: all 45 refused vectors already return a message in that family, and the try/catch around the URL constructor rewrites a TypeError into the same family, so a prefix check passes on that path too. The gap is that the backstop is indistinguishable from a rule. A vector refused only because the parser threw looks exactly like one refused by a rule that fired, and the corpus records verdicts rather than messages, so nothing could tell them apart. A rule ported as a comment would still show green. Each refused vector now checks both halves: the message is in the documented family, and it is not the backstop text. Both pass for every vector today. Checked by reverting each half in turn - a rule path throwing a TypeError fails the suite, and a rule path refusing with the backstop wording fails it too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
Python accepted a manifest URL that Go and TypeScript refuse, ran every
origin check against the wrong hostname, and returned a malformed URL:
base https://a.example:8443:9/ramp.json, ref /x
-> https://a.example:8443:9/x
The split ran on the LAST colon, so the host half was "a.example:8443",
and is_bare_domain accepts that through its optional-port branch. The
host half was never checked for a leftover colon. With a foldable port
the output even looked well-formed: a base of :443:443 returned
https://a.example:443/x.
Not an origin escape - the output authority is rebuilt from the base
host, and the base is the URL the caller fetched from, not attacker
input. It was a verdict split plus a malformed output string.
Both ports now split on the first colon, so anything after it lands in
the port half where the port rule already refuses it. Values that reach
the split through is_bare_domain hold at most one colon, so the shared
audience comparison is unchanged, which its corpus proves.
The four-step check - userinfo, split, host, port - was written out twice
per file, once for the base and once for the reference. Fixing the split
would have meant the same edit in four places, so it is now one private
helper per file. The docstring that claimed is_bare_domain guards this
was wrong and is what hid the bug; it now says what the code does.
TypeScript refused the double-colon base before this only because the URL
constructor threw and the backstop caught it. It now refuses by rule,
which the parity suite asserts.
Three refused vectors added, derived from the real Go face. No existing
vector changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
A query-only or fragment-only reference takes the base path unchanged
under RFC 3986 5.2.2. This port then never ran 5.2.4 over it:
base https://a.example/a/../ramp.json, ref ?q
Go https://a.example/ramp.json?q
TS https://a.example/ramp.json?q
Py https://a.example/a/../ramp.json?q
Same origin, a different document name - an accepted-input output split,
which is what this function exists to prevent.
The call to _remove_dot_segments sat inside three of the four branches
and was missing from the fourth, so the miss was invisible at the call
site. It now runs once, after the branches, on whichever path they
produced. Its docstring claimed it ran on every branch; that is now true
rather than aspirational.
No vector had a dot segment in the MANIFEST URL, which is why three
parity suites passed over this. Three accepted vectors added for a dotted
base, derived from the real Go face. Go and TypeScript already answered
correctly and are unchanged. No existing vector changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
The grammar appeared in two regex literals per file: one matching a scheme at the start of a reference, one matching an authority optionally preceded by that same scheme. The refusal that separates "https:/dir" from a relative path only means what it says while the two agree. Widen one and not the other and a reference is read as having a scheme by one check and no scheme by the other. One constant per file, both patterns built from it. No behaviour change: the scheme-without-authority and uppercase-scheme vectors exercise the pair, and all three corpora replay unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
The resolved PATH was aligned across the three SDKs last round; the query
and the fragment were not. They still came out of each language's URL
serializer, and inside the character set the tame predicate deliberately
admits, those serializers disagree:
ref /x?a='b Go and Python keep the apostrophe, TypeScript said %27
ref /x#f#g Go said #f%23g, Python and TypeScript kept #f#g
Two deviants in two directions. Same manifest member, a different fetch
URL per SDK - the exact split this function exists to prevent. The corpus
could not see it: no accepted vector carried a non-empty query, and the
hash appeared nowhere except in one refused base.
All three ports now read both components as substrings of the strings the
author wrote, the same way the authority is already rebuilt from checked
values rather than taken from the join. RFC 3986 3.4 and 3.5 define them
that way. Only the path still comes from the parser, where a wide
differential sweep found the three agree.
A SECOND HASH IS NOW REFUSED. RFC 3986 3.5 gives a reference one fragment
running to the end of the string, so a second hash is not a URI reference
- a hash inside a fragment has to be written %23. Measured across 29
fragment forms, it is the only character Go re-encoded, so refusing it
removes that divergence instead of picking a winner. A fragment is
otherwise KEPT on the reference, though the manifest URL refuses one: the
base is a URL a document was fetched from and could never have carried
one, while the reference is a value an author wrote, where a fragment
names an entry inside the document.
Query inheritance is unchanged and now pinned. RFC 3986 5.2.2 inherits
the base's query only when the reference has an empty path and defines no
query of its own; a reference carrying any path drops it. Nothing in the
corpus had a query on the manifest URL, so that whole branch was unpinned
in all three languages.
Sixteen vectors added. Re-running the sweep with every ASCII byte in
path, query and fragment position across nine bases - 10296 pairs, 7373
accepted - reports 0 disagreements. No existing vector changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
The design history credited a differential sweep with finding three divergence classes no reviewer had named. The tool was not in the tree and never had been - it was a scratch script, run once and thrown away. So the document promised a safety net that did not exist, and nothing would re-run it after the next Go, CPython or Node upgrade changed a URL parser. The script had also made the same mistake one level up that its own lesson warns corpora make. It dropped every ASCII byte into a PATH and nowhere else: never a query, never a fragment, never a base carrying dot segments or a second colon. Three of the four divergences the next review round found are ones a wider alphabet would have caught. It is now a second emitter beside the existing one, with a curated alphabet: every ASCII byte in path, query and fragment position across four bases, plus fifty-six structural forms across thirteen manifest URLs. 2789 cases, 1580 of them accepted, 513 KB. Both ports replay it, and the corpus-replay completeness gate makes that mandatory with no exemption available. The two corpora answer different questions and the emitter header says so. The hand-written one carries the verdict its author INTENDED and refuses to emit when the real face disagrees, so it catches "the behaviour changed". This one records whatever Go answers, because nobody can hand-declare a verdict for thousands of machine-made inputs, so it catches "the three implementations answer differently". A diff here means something moved, not that something broke. All 2789 cases agree across the three languages today. The file is byte-stable across two consecutive regenerations. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
…hat was too narrow The IdentityDocuments comment listed the character-set, percent-escape, dot-segment, fragment, port and empty-authority refusals but not the path-noscheme rule, which says a reference with no scheme must not carry a colon in its first path segment. Both changelogs stated it, both website pages stated it, and all three implementations enforce it. An implementer working from the proto alone would have accepted "1:x", which every SDK refuses. One sentence was also narrower than the code. It said "no percent-encoded dot segment", but the predicate refuses a percent-encoded dot anywhere in the string, mid-segment included, so "card%2ejson" is refused too. The wording changed rather than the check: refusing it everywhere costs nothing for the values this field carries, and a rule that applied only at segment boundaries would be harder to write identically in three languages, which is the whole point of the rule. Three rules settled by the sibling changes are carried across to every surface: an authority holds at most one colon, a reference may carry a fragment but never a second one while the manifest URL may carry none, and the resolved query and fragment are the substrings the author wrote rather than a re-serialization. The asymmetry on fragments is stated where it appears - the base is a URL a document was fetched from, the reference is a value an author wrote. Descriptor and generated types regenerated; the descriptor embeds source comments, so a comment-only edit changes its bytes. The two changelog entries are byte-identical, checked programmatically. Nothing added implies that signature verification follows the pointer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
… was The TypeScript sweep replay caught bare and mapped every throw to null, which is the expected value for all 1209 refused vectors. A raw TypeError escaping the URL constructor would have left the whole refused half green - and the hostile inputs in that corpus are exactly where such an escape would surface, so the one place the check mattered most was the one place it was not made. This was a regression rather than old code. The TypeScript INTENT suite was tightened two beads earlier to require the documented error family and to exclude the constructor backstop. The sweep replay was then written with the loose form, in a new file, after that fix landed. The Python suites had the weaker form of the same gap in both files: they pinned ValueError but not the message, and urlsplit raises ValueError too, so a stray one from inside the resolver counted as a correct refusal. Every replay now reports a refusal outside the documented family, or one that came from the backstop instead of a rule, as its own string - which makes it a mismatch instead of a pass. Checked by reverting each half: a rule throwing a TypeError fails the TypeScript sweep, a rule refusing with the backstop wording fails it too, and a rule raising outside the family fails both Python suites. No vector changes: the corpus records verdicts, not messages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
When a dot segment popped past the root and the next segment was empty,
Go dropped the empty segment and both ports kept it:
base https://a.example/ramp.json, ref ..//x
Go https://a.example/x
Python, TypeScript https://a.example//x
Two different documents on the same server, from one manifest member.
RFC 3986 5.2.4 gives the ports' answer. Merging ..//x onto /ramp.json
yields /..//x; step 2C matches the prefix, replaces it with a slash and
removes nothing from the empty output buffer - the underflow - and two
applications of step 2E then move the empty segment and then x. The
oracle was the deviant. It is also inconsistent with itself, which is
what pins the trigger to the underflow rather than to empty segments
generally: from a deeper base the same reference pops without underflow,
and there net/url keeps the empty segment and agrees.
Root cause is structural. Both ports hand-write 5.2.2, 5.2.3 and 5.2.4.
The oracle still took its path from ResolveReference. The query and the
fragment were moved off the library serializers for exactly this reason,
and the path was left on them after measuring that the three agreed -
over an alphabet holding no double-dot-then-empty shape.
So 5.2.2, 5.2.3 and 5.2.4 are written out here too, and ResolveReference
is left doing only what it is still trusted for: the scheme and the
authority.
Both corpora were regenerated with no existing case changing its verdict
or its string, in 83 intent vectors and 2789 sweep cases, which is the
evidence that the transcription is faithful rather than merely different.
The sweep alphabet gains the class that hid this: dot atoms crossed with
EMPTY-segment atoms, over bases of four different depths, since whether a
pop underflows depends on how deep the base is. 784 cases added. An
independent dot-segment-dense sweep of 17100 pairs went from 1995 output
splits to 0, and the wide-alphabet sweep of 10296 pairs stays at 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
…ast the root behaves The IdentityDocuments comment scoped its userinfo and plain-host rules to the RESOLVED URL - the sentence begins "After resolution the URL". Nothing in it constrained the manifest URL's own userinfo or host shape. All three SDKs vet the base on those conditions, both changelogs said so, and the exchange-manifest page said so; the surface implementers read first stated the weaker contract. A base of https://u:p@a.example/ramp.json passes every rule the comment stated and every SDK refuses it. That is the third round running of one habit: fix the path the review named, do not grep the siblings. The previous comment fix added the rules that review named and not these. The comment now also states why vetting the base is a refusal rather than a courtesy - a non-https base resolving to an https URL on the same host passes every later check - and states the dot-segment corner the sibling change settled: a ".." that pops past the root removes nothing and leaves what follows in place, so an empty segment there survives. URL libraries disagree about that one, which is why it is written down rather than left to the reader's parser. All four surfaces carry both statements. The two changelog entries stay byte-identical, checked programmatically. Descriptor and generated types regenerated. Nothing added implies that signature verification follows the pointer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
The shared no-echo table put the credential on the reference in five of
its five positions. Two Go refusals are reachable only from a base:
base https:u:s3cr3t@a.example/ramp.json -> "manifest URL names no host"
base u:s3cr3t@a.example/ramp.json -> "manifest URL is not https"
Both are opaque forms, so no parser reads a userinfo component and the
base userinfo check never fires. A base that parses normally is stopped
by that check long before either refusal, which is why nothing else
reaches them. The invariant holds today because every refusal in that
file is a fixed string, but the table could not have caught a regression
on those two paths - the same argument that put the reference-side opaque
forms in it.
Two rows added in all three languages, so the table covers seven
positions everywhere. Checked by interpolating the manifest URL back into
those two refusals: all three suites fail.
No production code and no vector changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
Node's URL parser stops removing dot segments for the rest of a path once
it meets a segment that BEGINS with a dot without being a dot segment:
base https://a.example/ramp.json, ref x/.x/..
Go, Python https://a.example/x/
TypeScript https://a.example/x/.x/..
The bare platform call returns that pathname too, so it is the parser and
not this code. WHATWG is on the other two's side, which means no
judgement call was needed: the spec's path state classifies a segment as
single-dot or double-dot from an explicit list, ".x" is on neither, so it
is appended and the following ".." shortens the path.
A leading-dot sweep of 32985 pairs measured the class before the fix: 0
verdict splits, 0 Go-versus-Python splits, 427 TypeScript-alone output
splits. Every split was accepted against accepted, so only an output
comparison could ever see it. That sweep now reports 0 splits, as do the
other two at 17100 and 10296 pairs.
TypeScript was the last implementation taking any part of its answer from
a URL library. It now hand-writes RFC 3986 5.2.2, 5.2.3 and 5.2.4, ported
from Python, so all three compute path, query, fragment and authority
from the strings the author wrote.
THE URL CONSTRUCTOR IS GONE, not merely bypassed. Its only consumer was
the path. Running both corpora through the real face first: of 3896
inputs, 2397 accepted, 1499 refused by a rule, 0 through the backstop -
and the 273 inputs that do make a bare constructor call throw are all
refused by a rule before it runs. So deleting it changes no verdict. The
backstop message went with it, along with the constant and the guard in
each of the two test files, which would otherwise have been checks
against an error that can no longer be raised.
The sweep alphabet gains a plain segment followed by a dotted one, which
is the shape that breaks; a dotted atom alone would have passed on the
broken code. 356 pairs added. Checked by INPUT PAIR rather than by vector
name, since the sweep names are positional and adding atoms renumbers
them: 0 pairs changed their verdict or resolved string, 0 disappeared.
Two prose claims corrected. A comment here said both ports hand-write
5.2.4; Python did, TypeScript got the earlier case right by accident of
its parser. The design history now records all three fixes as one rule
rather than three accidents.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
Brings in the registration-schema validation work. Seven textual conflicts and one the merge could not see. Both sides added an entry at the top of the Unreleased changelog, a row to the manifest field-reference table, a subsection to the manifest page, and entries to the Python corpus registry. All of those keep both sides. The account_registration table row was rewritten on main and untouched here, so main's version wins outright. The two new manifest subsections are ordered to follow the field-reference table: registration schema rules, then identity documents. Both changelog files carry both entries, and the identity-documents entry is byte-identical across them, checked programmatically. The generated files - the descriptor, the Zod schemas and the parity matrix - were resolved by regenerating from the merged sources rather than by merging their text. Regenerating twice produces identical output, so the tree is self-consistent. THE CONFLICT GIT COULD NOT SEE: both sides added an isHexDigit to package helpers, in different files, so the merge succeeded textually and the package stopped compiling. The two definitions are the same predicate. Main's copy in regschema.go is kept untouched and the duplicate here is dropped, so a later merge from main does not conflict on it again. Main also adds two dependencies the suites need, jsonschema for Python and ajv for TypeScript. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
…usal Both identity-document replay helpers read the caught value as an Error if it was one and stringified it otherwise, then checked the message prefix. So a rule that threw a bare STRING carrying that prefix passed as a correct refusal, even though the function's signature promises an Error. Verified by execution rather than by inspection: replacing one rule's throw with a bare string of the same text left both suites fully green at 91 passed. With this change the same edit fails 3 tests across both files. Each helper now checks the TYPE first and separately. The sweep helper reports a non-Error as its own mismatch string so it lands in the failure list; the intent helper throws, and deliberately does not quote the caught value, because the no-echo probe feeds credentials through that same helper. This is the third turn of the same screw on these two files. The assertion started as a bare throw check, was tightened to require the documented message prefix, then lost its backstop clause when the backstop was deleted - each time getting stronger about the MESSAGE while staying silent about the TYPE. Python needs nothing here: it already pins ValueError, and a raised string is not possible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
_remove_dot_segments reassigned a shorter and shorter suffix of the path on
every step. CPython copies on each slice, so a linear RFC algorithm ran in
quadratic time. Go and TypeScript do the same thing linearly, because a Go
slice and a V8 sliced string are both O(1).
The input is reachable. The reference is a member of a manifest fetched from a
third party, the well-known fetch ceiling is 1 MiB, and the two members carry a
minimum length of 1 with no maximum. The tame-character predicate runs first
but is linear and accepts slashes and letters, so it does not stop this. A
hostile manifest could hold a Python worker for about five seconds per call.
Resolving a reference of "/a" repeated to the size shown, before and after:
bytes before after Go TypeScript
65536 0.041s 0.016s 0.004s 0.012s
131072 0.169s 0.030s 0.008s 0.007s
262144 0.442s 0.061s 0.015s 0.019s
524288 1.388s 0.122s 0.025s 0.027s
1048000 5.094s 0.243s 0.040s 0.062s
Before, each doubling roughly quadrupled the time; after, it roughly doubles.
These numbers are evidence for the algorithm, not a threshold - they are
machine-dependent and are deliberately not asserted anywhere.
The branches are the RFC's, unchanged. Only the way the prefix is dropped is
different. Checked against the old implementation over 111150 generated paths
built from every dot-segment atom: 0 disagreements. Both corpora replay
identically in all three languages, and an independent 21840-pair dot-segment
sweep reports 0 verdict splits and 0 output splits.
No length cap is introduced. A cap would be a new protocol constant with no
derivation, would have to be agreed across three languages and written into the
contract, and would not make the algorithm right.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
…t empty
All three SDKs dropped a trailing "?" and a trailing "#", and the committed
corpora recorded the collapsed answers as expected:
ref /x? all three answered https://a.example/x
ref /x# all three answered https://a.example/x
ref /x?# all three answered https://a.example/x
RFC 3986 6.2.3 says not to. "Normalization should not remove delimiters when
their associated component is empty", and it names "http://example.com/?" as a
URL that cannot be assumed equivalent to "http://example.com/". On the
fragment: two URIs that differ only by a trailing "#" "are considered different
regardless of the scheme". No scheme specification licenses removing either one
here.
It matters because "/x?" and "/x" are different request targets on the wire. A
server may return different bytes for them, and this field names the document a
verifier fetches to learn a participant's keys.
Measured what each native parser does, which shows whose answer got recorded:
reference WHATWG Go Python
/x? preserved preserved dropped
/x# preserved dropped dropped
The collapsed answer was Python's - the most lossy of the three, and the one
agreeing with neither the RFC nor WHATWG. The corpus then locked it in. The
code carried a note calling Go's ForceQuery flag "not RFC 3986 semantics";
ForceQuery is exactly how Go preserves the distinction the RFC calls
significant, so the note is gone with the behaviour.
Presence now decides the delimiter and the contents decide nothing. Go already
had a presence flag on both components and discarded the fragment's; Python and
TypeScript now return fragment presence too, matching Go.
The declared answers in the intent emitter were corrected FIRST, so the face
does not certify itself, and five vectors pin the empty forms on both
components. Corpora regenerated: 34 pairs gained a "?" and 33 gained a "#",
matching the blast radius measured before starting. No pair changed its verdict
and no pair changed in any other way - checked by rekeying both corpora on
(manifest_url, ref) and diffing each pair against its old value.
The same round's port question resolves the other way, and the contract is what
was wrong there. All three SDKs accept https://a.example:/x and normalise it to
https://a.example/x, on the base side and the reference side, while the proto
comment said any port "written out" must be a number in 1-65535. A bare colon
IS written and carries no digits. RFC 3986 6.2.3 lists http://example.com:/ and
http://example.com:80/ as equivalent to http://example.com/, and says an
explicit port that is empty or the scheme default "should be removed by
scheme-based normalization". WHATWG does the same. Refusing it would also
invert this design's own logic: every other refusal in this rule exists because
the three parsers disagreed about an input, and here all three already agree
with each other and with the RFC. The sentence was written to catch a doubled
colon such as a.example:8443:9, which stays refused and stays pinned.
So the four surfaces - the proto comment, proto-ramp.mdx, exchange-manifest.mdx
and the mirrored changelog entry - now state both rules: a NON-EMPTY port must
carry digits in 1-65535 while an empty or default port is elided, and a query
or fragment delimiter is kept whenever the component is defined. Neither rule
was stated either way before, so without this the round would have shipped a
behaviour change no contract describes. Three vectors pin the empty port on the
base, on the reference, and on both.
Descriptor and generated types regenerated. Both changelog entries verified
byte-identical. All three SDK suites, the conformance suite, the website test
and the website build pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
The previous round fixed this loop in Python and left it live in the other two
ports. Both drop a consumed prefix by REBUILDING the remainder with a prepended
slash - `path = "/" + path[3:]` in Go and the template-literal equivalent in
TypeScript. The slice is O(1) in both runtimes; the concatenation is not. It
allocates and copies the whole remainder, and four branches do it: the two
"/./" and "/../" prefixes and their two terminal forms.
Only a DOT SEGMENT reaches any of those branches, which is what hid the cost.
Resolving a reference of the atom shown, repeated to the size shown:
bytes Go /. TS /. Py /. Go /a TS /a Py /a
-- before --
65536 0.192s 0.047s 0.009s 0.003s 0.005s 0.016s
131072 0.687s 0.179s 0.019s 0.005s 0.006s 0.031s
262144 1.939s 2.186s 0.040s 0.010s 0.012s 0.063s
524288 6.600s 9.195s 0.076s 0.019s 0.022s 0.129s
-- after --
65536 0.001s 0.004s 0.009s 0.003s 0.006s 0.016s
131072 0.002s 0.004s 0.018s 0.006s 0.010s 0.031s
262144 0.005s 0.007s 0.037s 0.011s 0.025s 0.062s
524288 0.009s 0.012s 0.074s 0.021s 0.033s 0.124s
1048000 0.017s 0.022s 0.148s 0.042s 0.067s 0.250s
Before, the dot atom roughly quadrupled per doubling in Go and TypeScript while
the plain atom stayed flat. After, both atoms double per doubling in all three.
These numbers are evidence for the algorithm, not a threshold - they are
machine-dependent and are deliberately not asserted anywhere.
The input is reachable the same way it was in Python: the reference is a member
of a manifest fetched from a third party, the well-known fetch ceiling is 1
MiB, and the two members carry a minimum length of 1 with no maximum. It was
worse here than it ever was in Python - 6.6s and 9.2s at HALF a megabyte,
against Python's old 1.4s at that size.
Two method mistakes are worth recording, because neither is about this loop.
The previous round measured with the plain atom, since that is the atom its
ticket named, and a plain segment only ever reaches the default branch - so the
Go and TypeScript columns came back flat and were written up as linear. A cost
measurement has to use the input that reaches the branch under suspicion. And
the Python comment then stated that conclusion as a fact about both other
runtimes; it is corrected here, and now explains that the three are quadratic
for two different reasons.
The corpora cannot catch this class at all. All three languages answer
CORRECTLY here - only the cost differs, and no vector records cost.
The branches are the RFC's, unchanged; only the way the prefix is dropped is
different. Checked three ways: each new implementation agrees with the one it
replaces over 111150 generated paths built from every dot-segment atom, 0
disagreements in each language; both committed corpora replay identically in
all three with no regeneration needed; and an independent 21840-pair
dot-segment sweep reports 0 verdict splits and 0 output splits.
No length cap is introduced, and no max_len is added to the two proto fields.
ResolveIdentityDocument is a public pure-string helper: protovalidate runs on
the MESSAGE, while the helper's own boundary is a plain string. A caller that
validates the manifest first is protected either way, and one that calls the
helper directly is not protected by a field rule at all - so a cap would guard
nothing the linear loop does not already guard, and would miss the path it is
meant to guard. It would also be a new protocol constant with no derivation:
RFC 3986 sets no length limit. The original reasoning - the 1 MiB document
ceiling bounds parser input - failed only because the parser was not linear,
and holds again now that it is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
Main brought the endpoint-rule work: the rule is now enforced in the Python and TypeScript ports as well as Go, credentials are kept out of every refusal the ports raise, and the host predicates move into one shared place - host_of, is_bare_host and host_anchored in each SDK's hosts module, over a new shared host-reference parser. Five conflicts, all in files both sides extended. sdk/python/ramp_sdk/hosts.py was the only substantive one. Both sides appended to the same file and git could not align them, so the whole region came back as one block. Rebuilt from main's version: main's module docstring, its _hostref import and its routing-predicate section are kept as they are; this branch's _split_host_port and _fold_default_port are reinstated with the _normalize_domain that uses them, and the identity-document block sits between the two. Keeping this branch's _normalize_domain over main's inline one is a real choice, not a formatting preference. Main's splits host from port on the LAST colon; this one splits on the FIRST, so a doubled colon lands in the port half where the port rule refuses it, rather than in the host half where is_bare_domain accepts it and every origin check downstream then runs against the wrong hostname. For values is_bare_domain has already accepted the two agree, since those hold at most one colon, so the audience corpus is unaffected either way. Main renamed the helper this branch's comments told readers not to reuse. It is _wba_host_anchored in the Python resolvers now, and in TypeScript it is the shared hostAnchored in the same file as the identity-document code. Both comments now name what exists, since their whole job is to stop the next reader reaching for the host-or-subdomain predicate where a subdomain must be refused. sdk/python/ramp_sdk/__init__.py: both new exports kept, in sorted order. The two changelogs each held a new entry under Unreleased. Both kept, main's above this branch's, which preserves the newest-first order already in the files. The two copies of each entry are hand-mirrored with no gate, so both were checked byte-identical afterwards. docs/sdk-parity-matrix.md is generated. Regenerated rather than hand-merged, then re-run with --check. Verified after resolving: go build, go vet and the full Go suite; 1242 Python tests; 1195 sdk/ts tests plus tsc; 579 gen/ts tests; the website test and build with links valid. Every generator re-run produced no change, so the tree is self-consistent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sJcDZcRwvL7miha9WQMGX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.