feat(xmp): exiv2-parity schema registry and XMP sidecar files - #507
Open
justin13888 wants to merge 10 commits into
Open
justin13888 wants to merge 10 commits into
justin13888 wants to merge 10 commits into
Conversation
`WellKnownNs` grows from 18 to 30 entries - exifEX, aux, plus, mwg-rs, mwg-kw, GPano, lr, MicrosoftPhoto, digiKam, acdsee, crss and dwc - the schemas exiv2 documents (https://exiv2.org/metadata.html) that the registry lacked. Each URI is cited on its variant; the vendored reference for all twelve is exiv2's own registry (third_party/exiv2/src/properties.cpp). The registry stays a registry: it fixes the prefix a schema serializes under and interprets no value. exifEX is registered under http://cipa.jp/exif/1.0/ (CIPA DC-010, the URI the reference engine and deployed writers use); the vendored Exif 3.0 text binds the same prefix to .../exif/2.32/ in its annotation examples, which is documented and not registered. tests/oracle.rs gains one test per schema, reading a documented property back from Adobe XMPCore by its `Xmp.<prefix>.<name>` key - a lookup that fails for a wrong URI or a wrong prefix. One oracle normalization is pinned as such rather than hidden: exiv2 appends `/` to a namespace URI ending in neither `/` nor `#` when registering it with XMPCore (XmpProperties::registerNs), so the engine re-serializes Darwin Core as .../index.htm/; gamut writes the URI exiv2 documents. Refs #421
`XmpSidecar::read(&[u8])` and `XmpSidecar::write(&XmpMeta)` are the bytes of a standalone `.xmp` file, the interchange RAW workflows keep beside a non-extensible raw image (Adobe XMP Part 3, "External storage of metadata"). `read` accepts everything `XmpMeta::from_packet` does - XML declaration, BOM, packet wrapper or bare - but requires the `x:xmpmeta` document element, resolved by namespace rather than prefix: Part 1 s7.3.3 gives that element exactly one purpose, identifying XMP inside general XML text, which a standalone file is; exiv2's sidecar sniffer keys on the same `<?xpacket`/`<x:xmpmeta`. A bare `rdf:RDF` document is rejected with `XmpError::Prohibited` naming the element found. `write` emits the XML declaration Part 3 asks for, then a read-only, unpadded packet in canonical form inside `x:xmpmeta` - byte-stable per graph. No filesystem API; the `photo.xmp`-beside-`photo.dng` convention is documented, not enforced. tests/oracle.rs pins both directions against Adobe XMPCore: the file gamut writes is accepted and its properties read back by key, and the engine's own `<?xpacket?>`/`<x:xmpmeta>` serialization - what exiv2 stores in a .xmp file - is read as a sidecar. Closes #421
… into feat/421-xmp-schemas-sidecars
Review findings on the lane's own diff. The sidecar document-element scan tested `Event::Eof` as a match arm; with that arm removed the loop spins forever, so the mutant would time out instead of being caught. The check is now a guard before the match, as `reader.rs` does, and end of input is terminal regardless of the arms. The MWG oracle helper compares the re-parsed structure under the engine's output URI like its sibling, so it stays correct for a URI exiv2 normalizes; a count comment in the registry test now adds up; README states that exiv2's sidecar sniffer accepts `<?xpacket` or `<x:xmpmeta` and that gamut is stricter. Refs #421
Review finding on the lane's own diff. `XmpSidecar::read` reported a document element other than `x:xmpmeta` as `XmpError::Prohibited`, whose documented meaning is a construct the *specification* prohibits (Part 1 §7.8/§7.9.3: `rdf:_n` items, `rdf:value` carrying `xml:lang`). A bare `rdf:RDF` document is not such a construct - §7.3.3 makes the wrapper optional, `XmpWriter::wrap_xmpmeta(false)` emits exactly that form, and `XmpMeta::from_packet` reads it - so the message asserted a prohibition that does not exist. `XmpError::MissingXmpMeta` now names the element found. The enum is `#[non_exhaustive]`, so this is additive. The docs state precisely where gamut is stricter than its oracle: exiv2's sidecar sniffer (`isXmpType`) accepts a `.xmp` file starting with `<?xpacket` *or* `<x:xmpmeta`, so a file exiv2 reads as a sidecar and gamut rejects is expected, and the caller reads those bytes with `XmpMeta::from_packet`. Also records why the sidecar root check is a second lex: reached through `read` it can only ever see `rdf:RDF`, because the reader admits exactly one document element over a tree with a single root, so the namespace comparison, the `Empty` arm and the end-of-input error are unreachable from that path. The helper stays total over any XML and unit-tested for the cases `read` cannot produce, rather than threading the root element out of `find_rdf` and widening the blast radius in `reader.rs`. Refs #421
Review finding on the lane's own diff. exiv2 appends `/` to any namespace URI ending in neither `/` nor `#` before registering it with XMPCore (`third_party/exiv2/src/properties.cpp`), and Darwin Core is the only registered schema whose URI ends in neither. A packet the reference engine wrote - including a sidecar - therefore declares `http://rs.tdwg.org/dwc/index.htm/`, which `from_uri` did not recognise, so such a graph re-serialized under a synthesized `ns1` prefix instead of `dwc`. Prefixes are non-semantic (Part 1 §6.2), so this was a round-trip fidelity gap rather than a correctness defect - but one the oracle test's own normalization helper hid from the suite. `from_uri` now recognises the slashed form as a read-only alias, `DWC_URI_TRAILING_SLASH`. `uri()` still emits the unslashed URI exiv2 documents, so gamut's bytes are unchanged, and the alias is not an `ALL` entry, so iteration and the registry's URI/prefix uniqueness are untouched. The nested oracle helper is now driven once with Darwin Core, so its expected-graph re-namespacing runs with source and target actually differing; both existing callers pass a URI the engine leaves alone, which left that whole recursion a verified no-op. Also splits the registry-count assertion into its own drift guard named for what it guards, since every future addition to the registry edits it, and records in STATUS that `exifEX`'s URI rests on the oracle rather than on a vendored specification (issue #516). Refs #421
`XmpSidecar::read` requires `x:xmpmeta`, and the docs argued that follows
from Part 1 §7.3.3. It does not: §7.3.3 reads "An optional x:xmpmeta
element may be placed around the rdf:RDF element" and asks a processor to
"tolerate" one, and Part 3's external-storage bullets never mention the
element — `xmpmeta` occurs in Part 3 once, in an SVG example. The
requirement stands, but as gamut's own rule, stricter than the
specification and stated as such.
The same passages claimed parity with exiv2 where there is none: exiv2's
`isXmpType` keys on `<?xpacket` *or* `<x:xmpmeta`, so an xpacket-wrapped
bare `rdf:RDF` is a sidecar to exiv2 and `MissingXmpMeta("RDF")` here.
The Darwin Core read alias was likewise oversold. It maps the slashed URI
XMPCore emits to the `dwc` prefix; it does not rewrite the URI, so a graph
parsed from such a packet keeps the slashed URI and
`get_text(WellKnownNs::DarwinCore.uri(), …)` returns `None` for it. Say
that where a caller meets it — `from_uri`, README and STATUS — and pin the
law it breaks: `from_uri(u).map(uri) == Some(u)` holds for every registry
entry and for no other URI but the one documented alias. Canonicalizing on
read is filed as #547.
Also: move the unbounded-read residual into STATUS's intentional skips so
it survives the merge, read back the per-schema oracle property under
`xmpcore_output_uri` as its sibling helper already does, and drop an
`assert_ne!` implied by the `assert_eq!` above it.
Refs #421
…I differently The variant's own rustdoc is where a caller about to call `.uri()` lands, and it said nothing about the trailing-slash read alias. A caller reading a packet XMPCore wrote would find `get_text(DarwinCore.uri(), ..)` returning None with no clue why, because reading does not canonicalize and the graph keeps the slashed URI the packet declared. State it where they meet it, and point at `from_uri` for the whole rule. Refs #421
`the_trailing_slash_alias_is_the_only_uri_from_uri_does_not_hand_back` looped over ALL asserting `from_uri(u).map(uri) == Some(u)` — which `uri_and_prefix_are_exact_and_round_trip` already asserts more strongly, so the test killed nothing, and its name's claim (that the Darwin Core alias is the *only* exception) went unpinned: a second, undocumented read alias added to `from_uri` passed the whole suite. Pin the exclusivity instead. exiv2's `XmpProperties::registerNs` appends `/` to a URI ending in neither `/` nor `#`, so an alias is a registry URI with a trailing slash added or removed; assert that no schema but Darwin Core answers to either form of its own URI. Verified against both directions of that family: with the test present each fails it and nothing else (106 passed, 1 failed); with the test deleted both pass the suite (106 passed, 0 failed). Refs #421
The Part 3 bullet these docs quote was cut short. It reads "as though it were embedded and then had the XMP packets extracted **and catenated by a postprocessor**" — and the dropped phrase is the one that matters, because it says a conforming producer may hand over a .xmp file holding several <?xpacket?> packets end to end. XmpSidecar::read does not read one. XmpPacket::scan ends the body at the next <?xpacket instruction, which in a catenated file is the first packet's own trailer, so everything after it is discarded with no error: two XmpSidecar::write outputs concatenated read back as the properties of the first alone. Adobe XMPCore rejects those same bytes. Restore the quotation and state the truncation on XmpSidecar::read, in the module docs, in STATUS.md and in README.md. The behaviour predates this branch and is not changed here: rejecting, merging every packet (which needs a conflict rule Part 3 does not supply) and keeping the truncation are three different products, and that choice is #562. Also re-wrap the sidecar prose in README.md and STATUS.md, which an earlier edit left short of the column the rest of both files keep. Refs #421, #562
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #503 (
feat/449-xmp-dcterms-provenance); this PR's base is that branch, and its headd6fd0a0is merged in by merge commit6ffe02b. Issue #421. Commits:1c57b2eschemas,bc1102asidecars,40db95eself-review repair,fd6ba22+fce19d6review-round repairs,b3375afdocumentation round.Summary
Schema breadth (
feat(xmp), commit 1).WellKnownNsgrows from 18 to 30 entries —exifEX,aux,plus,mwg-rs,mwg-kw,GPano,lr,MicrosoftPhoto,digiKam,acdsee,crss,dwc— every schema exiv2 documents (https://exiv2.org/metadata.html) that the registry lacked. Each URI is cited on its variant (schema owner's specification where published; the vendored reference for all twelve is exiv2's own registry,third_party/exiv2/src/properties.cppxmpNsInfo, which is also the oracle's). Additions are minor:WellKnownNsis#[non_exhaustive]from feat(metadata): report remote provenance from dcterms:provenance #503. The registry stays a registry — it fixes the prefix a schema serializes under and interprets no value.Sidecars (
feat(xmp), commit 2). Newgamut_xmp::XmpSidecarwithread(&[u8]) -> Result<XmpMeta>andwrite(&XmpMeta) -> Vec<u8>: the bytes of a standalone.xmpfile (XMP Part 3, "External storage of metadata").readaccepts everythingXmpMeta::from_packetdoes (XML declaration, BOM, packet wrapper or bare) but requires thex:xmpmetadocument element, resolved by namespace, rejecting a document whose element is notx:xmpmetawithXmpError::MissingXmpMetanaming the element found (it wasProhibiteduntilfd6ba22; see decision 14/R1).writeemits the XML declaration Part 3 asks for, then a read-only (end="r"), unpadded canonical packet insidex:xmpmeta— byte-stable per graph. No filesystem API; thephoto.xmp-beside-photo.dngconvention is documented, not enforced.Oracle.
tests/oracle.rsgains one test per added schema, reading a documented property of that schema back from Adobe XMPCore by itsXmp.<prefix>.<name>key (a lookup that fails for a wrong URI or a wrong prefix; the four struct-only schemas use their documented structure/bag shape), and two sidecar tests: the file gamut writes is accepted by XMPCore and read back by key, and XMPCore's own<?xpacket?>/<x:xmpmeta>serialization — what exiv2 stores in a.xmpfile — is read as a sidecar.One oracle normalization pinned, not hidden. exiv2 appends
/to any namespace URI ending in neither/nor#before registering it with XMPCore (XmpProperties::registerNs,properties.cpp:4966), so the engine re-serializes Darwin Core ashttp://rs.tdwg.org/dwc/index.htm/while keying it correctly on read.xmpcore_output_uriintests/oracle.rsstates the rule with the citation; thedwctest additionally asserts gamut's bytes declare the unslashed URI exiv2 documents. Recorded in STATUS.md (decision 9 below).Review round (
fd6ba22,fce19d6). A read-only review verified all twelve URI/prefix pairs character-for-character against exiv2 0.28.8 and returned three Lows, all taken. (1)XmpSidecar::readreported a missing wrapper asXmpError::Prohibited, whose documented meaning is a construct the specification prohibits — but Part 1 §7.3.3 permits the wrapper-less form andXmpWriter::wrap_xmpmeta(false)emits it, so the message asserted a prohibition that does not exist; there is now an additiveXmpError::MissingXmpMeta(String). (2)from_uridid not recognisehttp://rs.tdwg.org/dwc/index.htm/, the form XMPCore emits, so a graph parsed from an exiv2-written packet re-serialized underns1instead ofdwc; the slashed form is now a read-only alias (DWC_URI_TRAILING_SLASH) whileuri()still emits the unslashed URI, leaving written bytes unchanged. (3) The nested oracle helper's re-namespacing was a verified no-op — both callers passed a URI the engine leaves alone — so it is now driven once with Darwin Core, where source and target actually differ. Plus a reachability note on the sidecar root check and a registry-count drift guard named for what it guards.This is an unattended run. No human approved the plan; the decision record below is what a human reads afterwards.
Validation
Run from the lane worktree. Heavy gates inside
systemd-run --user --scope -q --slice=agents.slice -p MemoryMax=16G -p MemorySwapMax=0 -- env CARGO_BUILD_JOBS=2 CMAKE_BUILD_PARALLEL_LEVEL=2 TMPDIR=/var/mnt/scratch/golem/tmp prlimit --as=12000000000 …(same 12 GB address-space limit asulimit -v 12000000; see decision 10).CI is green. All eight checks pass at the head
b3375af(gh pr checks 507: Format & Metadata, Clippy & Doctests, Coverage (test gate), the four Incremental (PR diff) shards; Full workspace isskippingby design), in runs 34436828105 / 34436828094 — as they did atfce19d6in runs 34385470688 / 34385470677. An earlier revision of this body reported that run as red for issue #517 — an apt/Chrome-repository failure in the runner image — which was true when written and is not true now; the run completed successfully. Nothing was weakened, skipped or bypassed. The local gates below are the primary evidence.CARGO_BUILD_JOBS=2 cargo test -p gamut-xmp --all-features— pass on the review-round tree (fce19d6): 106 unit, 4 golden, 22 oracle, 5 roundtrip, 4 doctests. Earlier on40db95e: 104 unit (9 new insidecar.rs, 1 new innamespace.rs), 4 golden, 21 oracle (14 new), 5 roundtrip, 4 doctests (1 new). Also pass on each commit tree alone: schemas-only1c57b2e95/4/19/5/3; sidecarsbc1102a104/4/21/5/4.__CARGO_TEST_ROOT=<worktree> mise run fmtthenmise run fmt-check— pass on every commit tree (the prefix is the documented workaround for cargo walking past a nested worktree to the primary checkout's manifests).mise run check-tests— pass.mise run check-commits— pass, "no errors in 7 commits" (merge commits skipped);convco check origin/feat/449-xmp-dcterms-provenance..HEAD— pass on the two lane commits before the merge.CARGO_BUILD_JOBS=2 cargo clippy -p gamut-xmp --all-targets --all-features -- -D warnings— pass (exit 0) on every commit tree.mise run lint(scope above,timeout 3000) — pass twice: on6ffe02band again on the final tree40db95e(exit 0; the only output beyond clippy is cargo's future-incompat note forproc-macro-error2, a transitive dependency this PR does not touch).mise run test(scope above,timeout 3000) — pass on the final tree: 203 suites, 3,803 tests, 0 failed, exit 0.mise run mutants-crate gamut-xmp --file crates/gamut-xmp/src/namespace.rs --file crates/gamut-xmp/src/sidecar.rs --file crates/gamut-xmp/src/error.rs --shard 0/1(scope above,timeout 3600) on the review-round tree — pass: 21 mutants tested in 3m, 18 caught, 3 unviable, 0 missed, 0 timeouts.CARGO_BUILD_JOBS=2 cargo clippy -p gamut-xmp --all-targets --all-features -- -D warnings,__CARGO_TEST_ROOT=<worktree> mise run fmt-check,mise run check-tests,mise run check-commits("no errors in 9 commits") — all pass on the review-round tree.mise run lint/mise run testwere not re-run for the review round, deliberately: the box is under heavy load from sibling lanes, and the round's only cross-crate-visible change is an added variant on an#[non_exhaustive]enum plus one newpub const, neither of which can change another crate's compilation or behaviour. Both passed on40db95e(lint exit 0; test 203 suites / 3,803 tests / 0 failed).mise run mutants-crate gamut-xmp --file crates/gamut-xmp/src/namespace.rs --file crates/gamut-xmp/src/sidecar.rs --shard 0/1(the two source files that PR state changed —cargo mutants -p gamut-xmp --listcounts 296 mutants for the crate, all of which link the exiv2 oracle, so the full crate is hours for files this PR does not touch, and CI's blocking gate is--in-diff) — pass: 18 mutants tested in 3m, 16 caught, 2 unviable (Default::default()on types withoutDefault), 0 missed, 0 timeouts.b3375af).CARGO_BUILD_JOBS=2 cargo test -p gamut-xmp --all-features— pass: 107 unit (+1), 4 golden, 22 oracle, 5 roundtrip, 4 doctests.CARGO_BUILD_JOBS=2 cargo clippy -p gamut-xmp --all-targets --all-features -- -D warnings— pass (exit 0).RUSTDOCFLAGS="-D warnings" cargo doc -p gamut-xmp --no-deps --all-features— pass (exit 0), for the new intra-doc link.__CARGO_TEST_ROOT=<worktree> mise run fmtthenmise run fmt-check— pass.mise run check-tests— pass.mise run check-commits— pass, "no errors in 10 commits";convco check origin/feat/449-xmp-dcterms-provenance..HEAD— pass, 6 commits.mise run lint(scope above,timeout 3600) — pass, exit 0 (only output beyond clippy is cargo's future-incompat note for the transitiveproc-macro-error2).mise run mutants-diff(scope above,timeout 3600) — pass: 26 mutants tested in 3m, 24 caught, 2 unviable, 0 missed.mise run testwas not re-run for the documentation round, deliberately: the round changes no executable library code — the source edits are doc comments only — and its three code-shaped changes (one added inline test, one oracle-helper assertion, one dropped redundant assertion) are confined togamut-xmp's own suite, which passed in full above. The doc comments are covered bycargo test -p gamut-xmp(4 doctests) and by the rustdoc run. It passed on40db95e(203 suites / 3,803 tests / 0 failed) andmise run lint, which compiles every crate and target in the workspace, passed on this tree.http://rs.tdwg.org/dwc/index.htm/yieldsget_text(WellKnownNs::DarwinCore.uri(), "Record") == Noneandget_text(DWC_URI_TRAILING_SLASH, "Record") == Some("gamut"), and gamut writes both thedwcprefix and the slashed URI back;XmpSidecar::readon an xpacket-wrapped barerdf:RDFreturnsMissingXmpMeta("RDF")while exiv2'sisXmpTypeaccepts it (third_party/exiv2/src/xmpsidecar.cpp:190). The throwaway test used for this was deleted before the commit.check-release-deps/check-ffi-features(noCargo.tomltouched),check-ffi-header(no C-surface type),coverage(the new module is fully reached by its inline tests and the oracle tests; CI's Coverage job is the gate).9909bb3). Every command below completed in this run, under the memory-capped scope.__CARGO_TEST_ROOT=<worktree> mise run fmtthenmise run fmt-check— pass (exit 0).mise run check-tests— pass, "module docs, pinned proptest seeds and oracle filenames all conform".convco check origin/feat/449-xmp-dcterms-provenance..HEAD— pass, "no errors in 9 commits".RUSTDOCFLAGS="-D warnings" cargo doc -p gamut-xmp --no-deps --all-features— pass (exit 0).CARGO_BUILD_JOBS=2 cargo test -p gamut-xmp --all-features— pass: 107 unit, 4 golden, 22 oracle, 5 roundtrip, 4 doctests, 0 failed.CARGO_BUILD_JOBS=2 cargo clippy -p gamut-xmp --all-targets --all-features -- -D warnings— pass (exit 0).GAMUT_MUTANTS_BASE=origin/feat/449-xmp-dcterms-provenance mise run mutants-diff(scope above) — pass: 18 mutants tested in 10m, 17 caught, 1 unviable, 0 missed.from_uritest was verified against the mutation it is named for, both ways round. Withthe_trailing_slash_alias_is_the_only_uri_from_uri_does_not_hand_backpresent, adding a second read alias tofrom_urifails that test and no other — for a slashed alias (http://ns.adobe.com/lightroom/1.0//→Lightroom): 106 passed, 1 failed; for an unslashed one (http://ns.adobe.com/xap/1.0→Xmp): 106 passed, 1 failed. With the test deleted and both aliases present, the lib suite is green: 106 passed, 0 failed. The test therefore kills something no other test in the crate kills, which its previous form did not.XmpSidecar::writeoutputs concatenated (xmp:Rating=5, thenxmp:CreatorTool=gamut) returnOkwith one property:get_text(xmp, "Rating") == Some("5"),get_text(xmp, "CreatorTool") == None. No error is raised. The throwaway test used for this was deleted before the commit.mise run testfor the documentation round cited "the rustdoc run" among the things covering the change, as though a documentation build guards this repository. It does not: there is no such gate anywhere in this repository. Verified in this run:grep -n "cargo doc\|RUSTDOCFLAGS" .github/workflows/*.yml mise.tomlreturns nothing, andgrep -rn "lints.rustdoc\|rustdoc::" --include=Cargo.toml .returns nothing — no workflow step, no mise task, no rustdoc lint table. A broken intra-doc link ships green anywhere in the workspace. That the localcargo docrun above is a real check was confirmed by breaking a link in this diff on purpose:RUSTDOCFLAGS="-D warnings" cargo doc -p gamut-xmpthen reportserror: unresolved link to \WellKnownNs::no_such_method`` and exits 101 — but nothing in CI would run it. The gap is filed as No gate compiles a README code block or fails on a broken rustdoc link #549; it is repository-wide and deliberately not wired here.mise run testis safe stands on its own, and on stronger ground than the sentence above gave it. The workspace test gate is the Coverage job, and the doctests it cannot instrument are added back separately — both stated by the workflow itself, quoted here from.github/workflows/ci.ymland re-read in this run. Line 163:name: Coverage (test gate). Lines 196–198: "This is the workspace's test gate as well as its coverage gate: llvm-cov runs the same suitecargo testwould and fails the job on any test failure." Lines 137–139, on the Clippy & Doctests job: "The full test suite is NOT run here — the coverage job runs it instrumented and fails on any test failure, so it is the green-test gate;test-docadds back only the doctests, whichcargo llvm-covcannot instrument on stable." The union of those two required checks is therefore a superset of the workspace test gate, and both are required on this PR.mise run lintandmise run test— the round changes no executable library code outside one inline test body ingamut-xmp, no public signature, and noCargo.toml; the rest is doc comments and Markdown. Nothing outsidegamut-xmpcan compile or behave differently, andgamut-xmp's own suite (including the exiv2/XMPCore oracle) plus crate-scoped clippy at-D warningspassed in full above; the two workflow-quoted required checks re-run both on the pushed head.check-release-deps/check-ffi-features/check-ffi-header— noCargo.tomland no C-surface type touched.coverage— no new module; the round adds documentation and assertions to an existing test, which cannot lower the floor.Risks and rollout
#[non_exhaustive]enum variants, one new module and type, one newlib.rsre-export; no existing signature, URI, prefix or canonical byte changes.WellKnownNs::ALL.len()changes from 18 to 30 — a consumer that hard-codes the count would notice; none in the workspace does.nsNprefix and now serializes under the documented prefix. The parsed graph is identical (prefixes are non-semantic, Part 1 §6.2); only the bytes of such packets change. None of the existing golden vectors uses these namespaces.XmpSidecar::readrejects a barerdf:RDFdocument thatXmpMeta::from_packetaccepts. That is the intended boundary (decision 3); a caller with such input usesfrom_packet.dwcnormalization is exiv2's, and pinned as such; if a future exiv2 stops appending/,xmpcore_output_uriis the one place to change.Issue
Closes #421
Filed by this lane, linked rather than done here: #547 — gamut-xmp: consider canonicalizing a well-known namespace URI on read (Darwin Core trailing slash). Reading does not rewrite the URI a packet declares, so a graph parsed from an XMPCore-written Darwin Core packet re-serializes under the
dwcprefix but stays keyed byDWC_URI_TRAILING_SLASH; the issue records the option, the asymmetry it would trade away (byte fidelity for the one schema affected), and what either decision would have to pin.Also filed by this lane: #516 — references/exif: vendor CIPA DC-010 and re-derive the exifEX XMP namespace URI from it.
WellKnownNs::ExifExis bound tohttp://cipa.jp/exif/1.0/, which is what exiv2 and XMPCore bind but is not derived from any specification vendored underreferences/; the only vendored text binding the prefix is Exif 3.0 Annex J.2/J.3, wherehttp://cipa.jp/exif/2.32/appears as anexifEX:ExifANannotation example rather than as the schema URI.Also filed by this lane: #562 — gamut-xmp: XmpSidecar::read silently truncates a catenated multi-packet sidecar to its first packet. Part 3 asks that external metadata be written "as though it were embedded and then had the XMP packets extracted and catenated by a postprocessor", so a conforming producer may hand over a
.xmpfile holding several<?xpacket?>packets end to end;readtakes the first and discards the rest with no error, where Adobe XMPCore rejects the same bytes. This PR documents the behaviour and does not change it — rejecting, merging every packet (which needs a conflict rule Part 3 does not supply) and keeping the truncation are three different products, and that is not this PR's scope.Not filed, because it already exists: the missing repository-wide documentation gate is #549 — No gate compiles a README code block or fails on a broken rustdoc link — which names exactly this gap ("
RUSTDOCFLAGS=\"-D warnings\" cargo docis not run by any mise task or workflow"). Nothing was added to it; it is linked here as the home of the fix.Decisions taken
Appended by the lane, in the record's shape:
Unresolved review notes
Decode limits for gamut-xmp are a crate-wide question, not one this PR opens.
XmpSidecar::readtakes an unbounded&[u8]and parses before checking the wrapper, butXmpMeta::from_packethas identical exposure on the same bytes, so the sidecar API adds no new hostile-input class. Whether the crate should grow a decode-limit convention likegamut-png's is for a human to decide, for the crate as a whole (decision 15, Q6). Recorded incrates/gamut-xmp/STATUS.md's "Intentional skips" (decision 16) so it survives this PR's merge rather than living only here.exifEX's URI rests on the oracle, not on a vendored specification — tracked in the filed issue references/exif: vendor CIPA DC-010 and re-derive the exifEX XMP namespace URI from it #516 (decision 15, Q3). The registry entry is what exiv2 and XMPCore bind, and the oracle test pins it; what is missing is CIPA DC-010 underreferences/.The Darwin Core read alias resolves a prefix, not a URI, and this PR deliberately leaves it that way (decision 16). A consumer that needs
WellKnownNs::DarwinCore.uri()to resolve a graph parsed from an XMPCore-written packet needs issue gamut-xmp: consider canonicalizing a well-known namespace URI on read (Darwin Core trailing slash) #547 decided first; the trap is documented onfrom_uri, in README and in STATUS, and the law and its one exception are pinned by a test.A read alias outside the slash family would still be unpinned. The strengthened test closes
from_uriover the family exiv2'sregisterNsgenerates — a registry URI with a trailing slash added or removed — because that is the family the one real alias comes from and the only one a mutation of this code plausibly produces. An alias of some entirely different shape (a different host, a version bump in the path) would pass it. Pinning that would needfrom_urito be a closed lookup rather than a chain ofor_else, which is a refactor of shipped public behaviour and not a closing-round repair.The catenated-sidecar truncation is documented, not fixed (gamut-xmp: XmpSidecar::read silently truncates a catenated multi-packet sidecar to its first packet #562). A caller merging a sidecar into an image can still lose properties with no signal; the documentation now says so at
XmpSidecar::read, but until gamut-xmp: XmpSidecar::read silently truncates a catenated multi-packet sidecar to its first packet #562 is decided the behaviour is what it is.XmpMeta::from_packethas the same single-packet behaviour for embedded payloads, where a catenation is not something a container hands over — gamut-xmp: XmpSidecar::read silently truncates a catenated multi-packet sidecar to its first packet #562 notes that any fix has to decide about both.A broken intra-doc link anywhere in this workspace still ships green (No gate compiles a README code block or fails on a broken rustdoc link #549). The
RUSTDOCFLAGS="-D warnings" cargo docrun recorded in Validation is this lane's own, not a gate: it protects this diff and nothing else, and it will not protect the next one.