From 1c57b2ebf54220016a3011b082e0cd91384c1666 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Sun, 6 Sep 2026 04:04:08 -0400 Subject: [PATCH 1/9] feat(xmp): register the twelve schemas exiv2 documents `WellKnownNs` grows from 18 to 30 entries - exifEX, aux, plus, mwg-rs, mwg-kw, GPano, lr, MicrosoftPhoto, digiKam, acdsee, crss and dwc - the schemas exiv2 documents (https://exiv2.org/metadata.html) that the registry lacked. Each URI is cited on its variant; the vendored reference for all twelve is exiv2's own registry (third_party/exiv2/src/properties.cpp). The registry stays a registry: it fixes the prefix a schema serializes under and interprets no value. exifEX is registered under http://cipa.jp/exif/1.0/ (CIPA DC-010, the URI the reference engine and deployed writers use); the vendored Exif 3.0 text binds the same prefix to .../exif/2.32/ in its annotation examples, which is documented and not registered. tests/oracle.rs gains one test per schema, reading a documented property back from Adobe XMPCore by its `Xmp..` key - a lookup that fails for a wrong URI or a wrong prefix. One oracle normalization is pinned as such rather than hidden: exiv2 appends `/` to a namespace URI ending in neither `/` nor `#` when registering it with XMPCore (XmpProperties::registerNs), so the engine re-serializes Darwin Core as .../index.htm/; gamut writes the URI exiv2 documents. Refs #421 --- crates/gamut-xmp/README.md | 28 +++- crates/gamut-xmp/STATUS.md | 18 +- crates/gamut-xmp/src/namespace.rs | 204 ++++++++++++++++++++++- crates/gamut-xmp/tests/oracle.rs | 268 +++++++++++++++++++++++++++++- 4 files changed, 504 insertions(+), 14 deletions(-) diff --git a/crates/gamut-xmp/README.md b/crates/gamut-xmp/README.md index c4831afa..9b9db382 100644 --- a/crates/gamut-xmp/README.md +++ b/crates/gamut-xmp/README.md @@ -72,10 +72,17 @@ padding) and `XmpPacket::parse` the graph — `from_packet` is exactly that comp with the reference engine. Per-property and per-item `xml:lang` are fully supported. - **Part 2 (standard schemas) is a namespace registry** (`WellKnownNs`), not per-property validation: values are uninterpreted text in the model, as the wire format allows. The registry - also carries the external schemas image-metadata standards layer on XMP — `dcterms` (DCMI - Metadata Terms), which C2PA uses for `dcterms:provenance`, the URL of an *external* manifest - store (C2PA 2.4 §11.5). gamut-xmp registers the namespace; reading that property as a - provenance signal is [`gamut-metadata`](../gamut-metadata)'s job. + covers every schema exiv2 documents () — the Adobe Parts 1–2 + schemas plus the external ones image-metadata standards and deployed tools layer on XMP: + `dcterms` (DCMI Metadata Terms, which C2PA uses for `dcterms:provenance`, the URL of an + *external* manifest store — C2PA 2.4 §11.5; reading that property as a provenance signal is + [`gamut-metadata`](../gamut-metadata)'s job), `exifEX`, `aux`, `plus`, `mwg-rs`, `mwg-kw`, + `GPano`, `lr`, `MicrosoftPhoto`, `digiKam`, `acdsee`, `crss` and `dwc`. Registering a schema + fixes the prefix its properties serialize under — the one Adobe XMPCore keys them by — and + nothing more; each non-Adobe URI is cited on its variant, and every one is vouched for by the + reference engine in `tests/oracle.rs`. `exifEX` is registered under `http://cipa.jp/exif/1.0/` + (CIPA DC-010, the URI the reference engine and deployed writers use); the Exif 3.0 text's + annotation examples bind the same prefix to `…/exif/2.32/`, which is not registered. - **Part 3 (storage in files) belongs to the format crates by design.** This crate supplies what they need — wrapper-optional parse, bare-body serialization (`to_rdf` / `serialize_body`), and the writability/padding envelope for in-place editing. Locating packets inside JPEG/TIFF/PNG @@ -87,8 +94,9 @@ padding) and `XmpPacket::parse` the graph — `from_packet` is exactly that comp ## Status **Production-ready v1** (issue #189). Implemented: parser + canonical serializer for the full XMP -data model (simple / URI / structured / `Bag`·`Seq`·`Alt`, qualifiers, language alternatives) and -the `` wrapper. See [STATUS.md](STATUS.md). +data model (simple / URI / structured / `Bag`·`Seq`·`Alt`, qualifiers, language alternatives), the +`` wrapper, and a 30-schema registry at parity with exiv2's documented set (issue +#421). See [STATUS.md](STATUS.md). ## Migrating from 1.x @@ -106,8 +114,12 @@ changed — every existing variant, URI, prefix and method keeps its meaning. non-semantic (`tests/roundtrip.rs`). - **Differential oracle** against exiv2's bundled **Adobe XMPCore**: gamut's packets validate and round-trip through the reference engine, every `WellKnownNs` URI is vouched for by its schema - registry, and the default-`xml:lang` posture is pinned to parity (`tests/oracle.rs`; needs the - `third_party/exiv2` + `third_party/expat` submodules and a C++ toolchain). + registry (one test per exiv2-parity schema reads a documented property back by its + `Xmp..` key), and the default-`xml:lang` posture is pinned to parity + (`tests/oracle.rs`; needs the `third_party/exiv2` + `third_party/expat` submodules and a C++ + toolchain). One oracle normalization is pinned as such: exiv2 appends `/` to a namespace URI + ending in neither `/` nor `#` when it registers it with XMPCore, so the engine re-serializes + Darwin Core as `http://rs.tdwg.org/dwc/index.htm/`; gamut writes the URI exiv2 documents. - **Mutation-clean** — `cargo mutants` passes with zero gamut-xmp exclusions in `.cargo/mutants.toml`. - **No benches, intentionally** — the crate has no performance contract; packets are a few KB. diff --git a/crates/gamut-xmp/STATUS.md b/crates/gamut-xmp/STATUS.md index dd0a5a34..27bdebbb 100644 --- a/crates/gamut-xmp/STATUS.md +++ b/crates/gamut-xmp/STATUS.md @@ -30,6 +30,21 @@ placement, and array/struct nesting so output is stable, diffable, and round-tri the namespace only — what the property *means* is read by `gamut-metadata`, consistent with the registry-not-validator posture below — and `tests/oracle.rs` pins that XMPCore reads the property back under the `Xmp.dcterms.provenance` key its own registry defines. +- **Schema breadth = exiv2's documented set (issue #421).** `WellKnownNs` holds 30 entries: the + Adobe Parts 1–2 schemas and structure types, `dcterms`, and the twelve further schemas exiv2 + documents (): `exifEX`, `aux`, `plus`, `mwg-rs`, `mwg-kw`, + `GPano`, `lr`, `MicrosoftPhoto`, `digiKam`, `acdsee`, `crss`, `dwc`. Each URI is taken from the + schema owner's specification where one is published and cited on the variant; the vendored + reference for all twelve is exiv2's own registry (`third_party/exiv2/src/properties.cpp`, + `xmpNsInfo`), and `tests/oracle.rs` reads a documented property of each back from XMPCore by its + `Xmp..` key, which fails for a wrong URI *or* a wrong prefix. The registry stays a + registry: nothing about a value is interpreted. Two divergences are recorded rather than hidden: + `exifEX` is `http://cipa.jp/exif/1.0/` (CIPA DC-010; the reference engine's URI) although the + vendored Exif 3.0 text's annotation examples (Annex J) bind the prefix to `…/exif/2.32/`; and + exiv2 appends `/` to a URI ending in neither `/` nor `#` when registering it with XMPCore + (`XmpProperties::registerNs`), so the engine re-serializes `dwc` as `…/index.htm/` while gamut + writes the `…/index.htm` exiv2 documents — pinned in `tests/oracle.rs` as an oracle + normalization. ## Phases @@ -42,6 +57,7 @@ placement, and array/struct nesting so output is stable, diffable, and round-tri | P5 | Part 1 §7 | **Keystone** — canonical RDF/XML serialization + packet emit (writable padding) | ✅ done | | P6 | — | exiv2 differential conformance gate | ✅ done | | P7 | Parts 1–3 | **v1 stabilization** (issue #189) — API finalization (`XmpPacket::parse` composition, `XmpWriter::with_namespace` prefix registration, model conveniences), conformance audit (control-character escaping fix, trailer `end=` matching, edge-case pins), gamut-iptc dogfood migration, docs | ✅ done | +| P8 | Part 2 | **Breadth** (issue #421) — registry at parity with exiv2's documented schemas (17 → 30 entries), per-schema oracle tests | ✅ done | ## Intentional skips (audited for v1) @@ -61,6 +77,6 @@ deliberately: - **Part 3 per-container embedding and JPEG ExtendedXMP:** owned by the format crates; this crate supplies wrapper-optional parse, bare-body serialization, and the writable/padding envelope. - **Per-schema value validation (Part 2):** values are uninterpreted text; `WellKnownNs` is a - namespace registry, not a validator. + namespace registry, not a validator — still true after the exiv2-parity additions (issue #421). - **Deferred additive API** (post-1.0, no consumer today): an opt-in `XmpMeta::validate()`, and nested-structure field lookup. diff --git a/crates/gamut-xmp/src/namespace.rs b/crates/gamut-xmp/src/namespace.rs index ad4f0e3e..8dbef629 100644 --- a/crates/gamut-xmp/src/namespace.rs +++ b/crates/gamut-xmp/src/namespace.rs @@ -41,9 +41,15 @@ impl From for Namespace { } /// The standard XMP schemas (Adobe XMP Parts 1–2), plus the external schemas image metadata -/// standards layer on XMP. Each maps to a fixed namespace URI and a conventional prefix via -/// [`WellKnownNs::uri`] / [`WellKnownNs::prefix`]; [`WellKnownNs::from_uri`] recovers the schema -/// from a URI. +/// standards and widely deployed tools layer on XMP. Each maps to a fixed namespace URI and a +/// conventional prefix via [`WellKnownNs::uri`] / [`WellKnownNs::prefix`]; +/// [`WellKnownNs::from_uri`] recovers the schema from a URI. +/// +/// This is a **namespace registry, not a validator**: registering a schema fixes the prefix its +/// properties serialize under (the one the reference engine, exiv2's Adobe XMPCore, keys them by), +/// and nothing more — property values stay uninterpreted text. The set covers every schema exiv2 +/// documents (); the non-Adobe URIs are each cited on their +/// variant, and all thirty are cross-checked against XMPCore's own registry in `tests/oracle.rs`. /// /// Marked `#[non_exhaustive]`: the registry grows as gamut's format and metadata crates need /// further schemas, and each addition must not be a breaking change. Match with a wildcard arm, @@ -89,6 +95,82 @@ pub enum WellKnownNs { /// key C2PA 2.4 §11.5 / §15.5.3.1 uses to point at an **external** manifest store; gamut only /// registers the namespace — the C2PA reading of the property lives in `gamut-metadata`. DcTerms, + /// `exifEX` — Exif 2.3+ properties in XMP (`LensModel`, `PhotographicSensitivity`, …), the + /// XMP counterpart of the tags `gamut-exif` reads. + /// + /// URI `http://cipa.jp/exif/1.0/`, from CIPA DC-010-2012 "Exif metadata for XMP" (referenced, + /// not reproduced, by Adobe XMP Part 2 §3.4) and registered under that URI by exiv2 + /// (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). The vendored Exif 3.0 text + /// (`references/exif/exif-3.0-dc-008-translation-2023.pdf`, Annex J.2–J.3) binds the same + /// `exifEX` prefix to `http://cipa.jp/exif/2.32/` in its *annotation* (`exifEX:ExifAN`) + /// examples; that URI is not registered here — the reference engine and deployed writers use + /// `1.0`, and a second variant can be added without a breaking change if a consumer needs it. + ExifEx, + /// `aux` — Exif auxiliary camera/lens properties (`aux:Lens`, `aux:SerialNumber`, …), + /// ubiquitous in Lightroom and Camera Raw output. + /// + /// URI `http://ns.adobe.com/exif/1.0/aux/`, Adobe's "Exif Schema for Additional Exif + /// Properties" (present in the 2008/2010 editions of XMP Part 2, dropped from the vendored + /// 2016 text); registered by exiv2 (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + Aux, + /// `plus` — the PLUS (Picture Licensing Universal System) License Data Format. + /// + /// URI `http://ns.useplus.org/ldf/xmp/1.0/`, from the PLUS LDF XMP specification + /// (); registered by exiv2 + /// (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + Plus, + /// `mwg-rs` — Metadata Working Group image regions (`mwg-rs:Regions`). + /// + /// URI `http://www.metadataworkinggroup.com/schemas/regions/`, from the MWG *Guidelines for + /// Handling Image Metadata* 2.0 (2010), "Regions" schema; registered by exiv2 + /// (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + MwgRegions, + /// `mwg-kw` — Metadata Working Group hierarchical keywords (`mwg-kw:Keywords`). + /// + /// URI `http://www.metadataworkinggroup.com/schemas/keywords/`, from the MWG *Guidelines for + /// Handling Image Metadata* 2.0 (2010), "Keywords" schema; registered by exiv2 + /// (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + MwgKeywords, + /// `GPano` — Google Photo Sphere (panorama) metadata. + /// + /// URI `http://ns.google.com/photos/1.0/panorama/`, from Google's *Photo Sphere XMP Metadata* + /// specification; registered by exiv2 (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + GPano, + /// `lr` — Adobe Lightroom (`lr:hierarchicalSubject`, `lr:privateRTKInfo`). + /// + /// URI `http://ns.adobe.com/lightroom/1.0/`, Adobe's Lightroom schema (not part of the + /// published XMP Parts 1–3); registered by exiv2 (`third_party/exiv2/src/properties.cpp`, + /// `xmpNsInfo`). + Lightroom, + /// `MicrosoftPhoto` — Microsoft Photo 1.0 (Windows Photo Gallery / Explorer rating and camera + /// fields). + /// + /// URI `http://ns.microsoft.com/photo/1.0/`, Microsoft's Photo schema; registered by exiv2 + /// (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + MicrosoftPhoto, + /// `digiKam` — digiKam photo-management properties (`digiKam:TagsList`, + /// `digiKam:ColorLabel`, …). + /// + /// URI `http://www.digikam.org/ns/1.0/`, digiKam's own schema; registered by exiv2 + /// (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + DigiKam, + /// `acdsee` — ACDSee photo-management properties. + /// + /// URI `http://ns.acdsee.com/iptc/1.0/`, ACDSee's own schema; registered by exiv2 + /// (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + Acdsee, + /// `crss` — Camera Raw Saved Settings (snapshots), the companion of `crs` (Camera Raw + /// settings, Part 2 §3.3). + /// + /// URI `http://ns.adobe.com/camera-raw-saved-settings/1.0/`, Adobe's Camera Raw Saved Settings + /// schema (not part of the published XMP Parts 1–3); registered by exiv2 + /// (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + CameraRawSavedSettings, + /// `dwc` — Darwin Core biodiversity terms in XMP (`dwc:Record`, `dwc:Event`, …). + /// + /// URI `http://rs.tdwg.org/dwc/index.htm`, the TDWG Darwin Core namespace as it is used in + /// XMP; registered by exiv2 (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + DarwinCore, } impl WellKnownNs { @@ -112,6 +194,18 @@ impl WellKnownNs { WellKnownNs::Dimensions, WellKnownNs::ResourceRef, WellKnownNs::DcTerms, + WellKnownNs::ExifEx, + WellKnownNs::Aux, + WellKnownNs::Plus, + WellKnownNs::MwgRegions, + WellKnownNs::MwgKeywords, + WellKnownNs::GPano, + WellKnownNs::Lightroom, + WellKnownNs::MicrosoftPhoto, + WellKnownNs::DigiKam, + WellKnownNs::Acdsee, + WellKnownNs::CameraRawSavedSettings, + WellKnownNs::DarwinCore, ]; /// The schema's namespace URI — its canonical identity. @@ -119,7 +213,8 @@ impl WellKnownNs { /// `dc` is Dublin Core (Part 1 §8.3); the `xmp*` schemas are Part 1 §8.4–8.6; `photoshop`, /// `crs` are Part 2 §3.2–3.3. `exif`/`tiff` mirror the EXIF tags into XMP (Part 2 §3.4, defined /// by CIPA DC-010); `Iptc4xmpCore`/`Iptc4xmpExt` are the IPTC Photo Metadata schemas; - /// `dcterms` is the DCMI Metadata Terms namespace (`http://purl.org/dc/terms/`). + /// `dcterms` is the DCMI Metadata Terms namespace (`http://purl.org/dc/terms/`). The URIs of + /// the schemas outside the published XMP Parts are cited on each variant. #[must_use] pub const fn uri(self) -> &'static str { match self { @@ -141,6 +236,20 @@ impl WellKnownNs { WellKnownNs::Dimensions => "http://ns.adobe.com/xap/1.0/sType/Dimensions#", WellKnownNs::ResourceRef => "http://ns.adobe.com/xap/1.0/sType/ResourceRef#", WellKnownNs::DcTerms => "http://purl.org/dc/terms/", + WellKnownNs::ExifEx => "http://cipa.jp/exif/1.0/", + WellKnownNs::Aux => "http://ns.adobe.com/exif/1.0/aux/", + WellKnownNs::Plus => "http://ns.useplus.org/ldf/xmp/1.0/", + WellKnownNs::MwgRegions => "http://www.metadataworkinggroup.com/schemas/regions/", + WellKnownNs::MwgKeywords => "http://www.metadataworkinggroup.com/schemas/keywords/", + WellKnownNs::GPano => "http://ns.google.com/photos/1.0/panorama/", + WellKnownNs::Lightroom => "http://ns.adobe.com/lightroom/1.0/", + WellKnownNs::MicrosoftPhoto => "http://ns.microsoft.com/photo/1.0/", + WellKnownNs::DigiKam => "http://www.digikam.org/ns/1.0/", + WellKnownNs::Acdsee => "http://ns.acdsee.com/iptc/1.0/", + WellKnownNs::CameraRawSavedSettings => { + "http://ns.adobe.com/camera-raw-saved-settings/1.0/" + } + WellKnownNs::DarwinCore => "http://rs.tdwg.org/dwc/index.htm", } } @@ -167,6 +276,18 @@ impl WellKnownNs { WellKnownNs::Dimensions => "stDim", WellKnownNs::ResourceRef => "stRef", WellKnownNs::DcTerms => "dcterms", + WellKnownNs::ExifEx => "exifEX", + WellKnownNs::Aux => "aux", + WellKnownNs::Plus => "plus", + WellKnownNs::MwgRegions => "mwg-rs", + WellKnownNs::MwgKeywords => "mwg-kw", + WellKnownNs::GPano => "GPano", + WellKnownNs::Lightroom => "lr", + WellKnownNs::MicrosoftPhoto => "MicrosoftPhoto", + WellKnownNs::DigiKam => "digiKam", + WellKnownNs::Acdsee => "acdsee", + WellKnownNs::CameraRawSavedSettings => "crss", + WellKnownNs::DarwinCore => "dwc", } } @@ -208,6 +329,81 @@ mod tests { } } + #[test] + fn exiv2_documented_schemas_have_exact_uris_and_prefixes() { + // The twelve schemas added for exiv2 parity (issue #421). Each pair is the exact string + // exiv2's registry binds (`third_party/exiv2/src/properties.cpp`), so the prefix gamut + // serializes under is the key the reference engine reads back by; the differential check + // is `tests/oracle.rs`. Near-misses are the likely defects: `exifEX` vs `exifEx`, `aux/` + // under `exif/1.0/` (not a sibling of it), `crss` vs the `crs` it complements, and + // `photo/1.0/` (not exiv2's separate `MP` = `photo/1.2/`). + let expected = [ + (WellKnownNs::ExifEx, "http://cipa.jp/exif/1.0/", "exifEX"), + (WellKnownNs::Aux, "http://ns.adobe.com/exif/1.0/aux/", "aux"), + ( + WellKnownNs::Plus, + "http://ns.useplus.org/ldf/xmp/1.0/", + "plus", + ), + ( + WellKnownNs::MwgRegions, + "http://www.metadataworkinggroup.com/schemas/regions/", + "mwg-rs", + ), + ( + WellKnownNs::MwgKeywords, + "http://www.metadataworkinggroup.com/schemas/keywords/", + "mwg-kw", + ), + ( + WellKnownNs::GPano, + "http://ns.google.com/photos/1.0/panorama/", + "GPano", + ), + ( + WellKnownNs::Lightroom, + "http://ns.adobe.com/lightroom/1.0/", + "lr", + ), + ( + WellKnownNs::MicrosoftPhoto, + "http://ns.microsoft.com/photo/1.0/", + "MicrosoftPhoto", + ), + ( + WellKnownNs::DigiKam, + "http://www.digikam.org/ns/1.0/", + "digiKam", + ), + ( + WellKnownNs::Acdsee, + "http://ns.acdsee.com/iptc/1.0/", + "acdsee", + ), + ( + WellKnownNs::CameraRawSavedSettings, + "http://ns.adobe.com/camera-raw-saved-settings/1.0/", + "crss", + ), + ( + WellKnownNs::DarwinCore, + "http://rs.tdwg.org/dwc/index.htm", + "dwc", + ), + ]; + for (ns, uri, prefix) in expected { + assert_eq!(ns.uri(), uri, "{ns:?}"); + assert_eq!(ns.prefix(), prefix, "{ns:?}"); + assert!( + WellKnownNs::ALL.contains(&ns), + "{ns:?} must be in ALL so from_uri and the writer's prefix table see it" + ); + } + // exiv2 documents 29 schemas; with `dcterms` counted once and the two structure types + // (`stDim`, `stRef`) the registry holds 30 entries. + assert_eq!(WellKnownNs::ALL.len(), 30); + } + #[test] fn from_uri_rejects_unknown() { assert_eq!(WellKnownNs::from_uri("http://example.com/ns/"), None); diff --git a/crates/gamut-xmp/tests/oracle.rs b/crates/gamut-xmp/tests/oracle.rs index cb066b89..7c2b40fb 100644 --- a/crates/gamut-xmp/tests/oracle.rs +++ b/crates/gamut-xmp/tests/oracle.rs @@ -80,6 +80,25 @@ fn empty_packet_round_trips_through_exiv2() { exiv2_oracle::validate(&packet).expect("exiv2 must accept an empty packet"); } +/// The URI under which the reference engine re-serializes a schema. +/// +/// An oracle limitation, pinned rather than hidden: exiv2 normalizes every namespace URI it +/// registers with XMPCore by appending `/` when the URI ends in neither `/` nor `#` +/// (`third_party/exiv2/src/properties.cpp`, `XmpProperties::registerNs` and +/// `XmpProperties::prefix`, "`if (ns2.back() != '/' && ns2.back() != '#') ns2 += '/'`"). Reading +/// is unaffected — the key lookup resolves through the same normalization, so a `dwc` property +/// reads back by its `Xmp.dwc.*` key — but XMPCore's output then declares the normalized URI. +/// Darwin Core (`http://rs.tdwg.org/dwc/index.htm`) is the only registered schema this touches; +/// gamut writes the URI exactly as exiv2 documents and registers it (`xmp.cpp:519`). +fn xmpcore_output_uri(ns: WellKnownNs) -> String { + let uri = ns.uri(); + if uri.ends_with('/') || uri.ends_with('#') { + uri.to_owned() + } else { + format!("{uri}/") + } +} + #[test] fn every_well_known_namespace_survives_xmpcore() { // One property in each WellKnownNs, its value naming the expected prefix. XMPCore's own @@ -100,8 +119,9 @@ fn every_well_known_namespace_survives_xmpcore() { let out = exiv2_oracle::roundtrip(&packet).expect("exiv2 round-trip"); let parsed = XmpMeta::from_packet(&out).expect("gamut parses exiv2's output"); for ns in WellKnownNs::ALL { + // `xmpcore_output_uri` is the identity for every schema but Darwin Core (see its doc). assert_eq!( - parsed.get_text(ns.uri(), "GamutCheck"), + parsed.get_text(&xmpcore_output_uri(*ns), "GamutCheck"), Some(ns.prefix()), "property in {ns:?} must survive the reference engine" ); @@ -202,3 +222,249 @@ fn default_xml_lang_on_description_matches_reference() { Some("de") ); } + +// --------------------------------------------------------------------------------------------------- +// Schema breadth (issue #421): one test per namespace added for exiv2 parity. +// +// Each writes one *documented* property of the schema (a name from exiv2's own property table for +// that namespace) and reads it back from XMPCore by its exiv2 key, `Xmp..`. The key +// is what makes this differential rather than self-consistent: exiv2 resolves the key's prefix +// through its registry, so a URI XMPCore does not know, or a prefix that is not the one exiv2 +// binds to that URI, fails the lookup — the unit test in `namespace.rs` can only compare the +// strings with themselves. +// --------------------------------------------------------------------------------------------------- + +/// Serializes `meta`, asserts XMPCore accepts it, and returns the packet. +fn packet_xmpcore_accepts(meta: &XmpMeta) -> Vec { + let packet = meta.to_packet(); + exiv2_oracle::validate(&packet).expect("exiv2 (Adobe XMPCore) must accept gamut's packet"); + packet +} + +/// A graph holding one simple text property in `ns`. +fn one_text(ns: WellKnownNs, name: &str, value: &str) -> XmpMeta { + let mut meta = XmpMeta::new(); + meta.set_text(ns.uri(), name, value); + meta +} + +/// Asserts a simple text property in `ns` is keyed `Xmp..` by XMPCore and survives +/// its re-serialization back into gamut. +fn text_property_survives_xmpcore(ns: WellKnownNs, name: &str, value: &str) { + let packet = packet_xmpcore_accepts(&one_text(ns, name, value)); + let key = format!("Xmp.{}.{name}", ns.prefix()); + assert_eq!( + exiv2_oracle::get_property(&packet, &key).unwrap(), + value, + "{key} must read back from the reference engine" + ); + let out = exiv2_oracle::roundtrip(&packet).expect("exiv2 round-trip"); + let parsed = XmpMeta::from_packet(&out).expect("gamut parses exiv2's output"); + assert_eq!(parsed.get_text(ns.uri(), name), Some(value)); +} + +#[test] +fn exif_ex_lens_model_reads_back_under_the_exif_ex_key() { + text_property_survives_xmpcore(WellKnownNs::ExifEx, "LensModel", "GAMUT 50mm F1.4"); +} + +#[test] +fn aux_lens_reads_back_under_the_aux_key() { + text_property_survives_xmpcore(WellKnownNs::Aux, "Lens", "50.0 mm f/1.4"); +} + +#[test] +fn plus_version_reads_back_under_the_plus_key() { + text_property_survives_xmpcore(WellKnownNs::Plus, "Version", "1.2.0"); +} + +#[test] +fn gpano_projection_type_reads_back_under_the_gpano_key() { + text_property_survives_xmpcore(WellKnownNs::GPano, "ProjectionType", "equirectangular"); +} + +#[test] +fn microsoft_photo_rating_reads_back_under_the_microsoft_photo_key() { + text_property_survives_xmpcore(WellKnownNs::MicrosoftPhoto, "Rating", "75"); +} + +#[test] +fn digikam_color_label_reads_back_under_the_digikam_key() { + text_property_survives_xmpcore(WellKnownNs::DigiKam, "ColorLabel", "3"); +} + +#[test] +fn acdsee_caption_reads_back_under_the_acdsee_key() { + text_property_survives_xmpcore(WellKnownNs::Acdsee, "caption", "Harbour at dusk"); +} + +#[test] +fn lightroom_hierarchical_subject_reads_back_under_the_lr_key() { + // `lr:hierarchicalSubject` is the property the schema exists for: a Bag of `|`-separated + // keyword paths. + let lr = WellKnownNs::Lightroom; + let mut meta = XmpMeta::new(); + meta.set(XmpProperty::new( + lr.uri(), + "hierarchicalSubject", + XmpValue::Array(XmpArray::Bag(vec![XmpItem::new(XmpValue::Simple( + "Places|Seoul".into(), + ))])), + )); + let packet = packet_xmpcore_accepts(&meta); + assert_eq!( + exiv2_oracle::get_property(&packet, "Xmp.lr.hierarchicalSubject").unwrap(), + "Places|Seoul" + ); + let out = exiv2_oracle::roundtrip(&packet).expect("exiv2 round-trip"); + let parsed = XmpMeta::from_packet(&out).expect("gamut parses exiv2's output"); + let XmpValue::Array(XmpArray::Bag(items)) = + &parsed.get(lr.uri(), "hierarchicalSubject").unwrap().value + else { + panic!("lr:hierarchicalSubject must round-trip as a Bag"); + }; + let values: Vec<&str> = items.iter().filter_map(XmpItem::text).collect(); + assert_eq!(values, ["Places|Seoul"]); +} + +/// A graph whose one property in `ns` is the structure `name` holding the text fields `fields` +/// (same namespace) — the shape of the four schemas whose top-level properties are all structures. +fn one_struct(ns: WellKnownNs, name: &str, fields: &[(&str, &str)]) -> XmpMeta { + let mut meta = XmpMeta::new(); + meta.set(XmpProperty::new( + ns.uri(), + name, + XmpValue::Structured( + fields + .iter() + .map(|(field, value)| { + XmpProperty::new(ns.uri(), *field, XmpValue::Simple((*value).into())) + }) + .collect(), + ), + )); + meta +} + +/// Asserts the structure field `Xmp../:` reads back from XMPCore and +/// survives its re-serialization. +fn struct_field_survives_xmpcore(ns: WellKnownNs, name: &str, field: &str, value: &str) { + let packet = packet_xmpcore_accepts(&one_struct(ns, name, &[(field, value)])); + let prefix = ns.prefix(); + let key = format!("Xmp.{prefix}.{name}/{prefix}:{field}"); + assert_eq!( + exiv2_oracle::get_property(&packet, &key).unwrap(), + value, + "{key} must read back from the reference engine" + ); + let out = exiv2_oracle::roundtrip(&packet).expect("exiv2 round-trip"); + let parsed = XmpMeta::from_packet(&out).expect("gamut parses exiv2's output"); + // The reference engine's output URI (differs from `ns.uri()` only for Darwin Core). + let out_uri = xmpcore_output_uri(ns); + let XmpValue::Structured(fields) = &parsed.get(&out_uri, name).unwrap().value else { + panic!("{prefix}:{name} must round-trip as a structure"); + }; + let got = fields + .iter() + .find(|f| f.namespace == out_uri && f.name == field) + .and_then(XmpProperty::text); + assert_eq!(got, Some(value)); +} + +/// Asserts the MWG shape — a top-level structure `name` whose field `list` is a Bag of structures +/// each carrying the text `field` — reads back from XMPCore as +/// `Xmp../:[1]/:` and survives its re-serialization. +fn struct_bag_item_field_survives_xmpcore( + ns: WellKnownNs, + name: &str, + list: &str, + field: &str, + value: &str, +) { + let uri = ns.uri(); + let item = XmpItem::new(XmpValue::Structured(vec![XmpProperty::new( + uri, + field, + XmpValue::Simple(value.into()), + )])); + let mut meta = XmpMeta::new(); + meta.set(XmpProperty::new( + uri, + name, + XmpValue::Structured(vec![XmpProperty::new( + uri, + list, + XmpValue::Array(XmpArray::Bag(vec![item])), + )]), + )); + let packet = packet_xmpcore_accepts(&meta); + let prefix = ns.prefix(); + let key = format!("Xmp.{prefix}.{name}/{prefix}:{list}[1]/{prefix}:{field}"); + assert_eq!( + exiv2_oracle::get_property(&packet, &key).unwrap(), + value, + "{key} must read back from the reference engine" + ); + let out = exiv2_oracle::roundtrip(&packet).expect("exiv2 round-trip"); + let parsed = XmpMeta::from_packet(&out).expect("gamut parses exiv2's output"); + assert_eq!( + parsed.get(uri, name), + meta.get(uri, name), + "{prefix}:{name} must round-trip through XMPCore unchanged" + ); +} + +#[test] +fn mwg_regions_list_item_name_reads_back_under_the_mwg_rs_key() { + // MWG Guidelines 2.0, Regions: `mwg-rs:Regions` is a RegionInfo structure whose `RegionList` + // is a Bag of RegionStruct, each with a text `Name`. A hyphenated prefix is also the one + // shape the writer's prefix table had not exercised before. + struct_bag_item_field_survives_xmpcore( + WellKnownNs::MwgRegions, + "Regions", + "RegionList", + "Name", + "Face 1", + ); +} + +#[test] +fn mwg_keywords_hierarchy_item_keyword_reads_back_under_the_mwg_kw_key() { + // MWG Guidelines 2.0, Keywords: `mwg-kw:Keywords` is a KeywordInfo structure whose + // `Hierarchy` is a Bag of KeywordStruct, each with a text `Keyword`. + struct_bag_item_field_survives_xmpcore( + WellKnownNs::MwgKeywords, + "Keywords", + "Hierarchy", + "Keyword", + "Seoul", + ); +} + +#[test] +fn camera_raw_saved_settings_name_reads_back_under_the_crss_key() { + struct_field_survives_xmpcore( + WellKnownNs::CameraRawSavedSettings, + "SavedSettings", + "Name", + "Import", + ); +} + +#[test] +fn darwin_core_record_field_reads_back_under_the_dwc_key() { + // The one schema whose URI ends in neither `/` nor `#`. gamut writes it exactly as exiv2 + // documents it; the reference engine keys it correctly (`Xmp.dwc.*` below) and re-serializes + // it with a `/` appended (`xmpcore_output_uri`) — an oracle normalization, not a defect in + // either direction, so both halves are pinned here. + let dwc = WellKnownNs::DarwinCore; + let packet = one_struct(dwc, "Record", &[("institutionID", "GAMUT")]).to_packet(); + assert!( + std::str::from_utf8(&packet) + .unwrap() + .contains("xmlns:dwc=\"http://rs.tdwg.org/dwc/index.htm\""), + "gamut must declare the URI exiv2 documents, unslashed" + ); + assert_eq!(xmpcore_output_uri(dwc), "http://rs.tdwg.org/dwc/index.htm/"); + struct_field_survives_xmpcore(dwc, "Record", "institutionID", "GAMUT"); +} From bc1102afcfac8e99267a5e1d17a1e17ae9665561 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Sun, 6 Sep 2026 04:04:45 -0400 Subject: [PATCH 2/9] feat(xmp): read and write XMP sidecar files `XmpSidecar::read(&[u8])` and `XmpSidecar::write(&XmpMeta)` are the bytes of a standalone `.xmp` file, the interchange RAW workflows keep beside a non-extensible raw image (Adobe XMP Part 3, "External storage of metadata"). `read` accepts everything `XmpMeta::from_packet` does - XML declaration, BOM, packet wrapper or bare - but requires the `x:xmpmeta` document element, resolved by namespace rather than prefix: Part 1 s7.3.3 gives that element exactly one purpose, identifying XMP inside general XML text, which a standalone file is; exiv2's sidecar sniffer keys on the same ``/`` serialization - what exiv2 stores in a .xmp file - is read as a sidecar. Closes #421 --- crates/gamut-xmp/README.md | 18 ++- crates/gamut-xmp/STATUS.md | 16 +- crates/gamut-xmp/src/lib.rs | 5 + crates/gamut-xmp/src/sidecar.rs | 265 +++++++++++++++++++++++++++++++ crates/gamut-xmp/tests/oracle.rs | 39 ++++- 5 files changed, 338 insertions(+), 5 deletions(-) create mode 100644 crates/gamut-xmp/src/sidecar.rs diff --git a/crates/gamut-xmp/README.md b/crates/gamut-xmp/README.md index 9b9db382..e546bb7c 100644 --- a/crates/gamut-xmp/README.md +++ b/crates/gamut-xmp/README.md @@ -56,6 +56,17 @@ a custom schema. `WellKnownNs` supplies the standard schema URIs and prefixes so hand-write them. For in-place editing, `XmpPacket::scan` exposes the envelope (writability, padding) and `XmpPacket::parse` the graph — `from_packet` is exactly that composition. +**Sidecars.** `XmpSidecar::write` / `XmpSidecar::read` are the bytes of a standalone `.xmp` file — +the interchange RAW workflows use, since a camera's raw format is not extensible (Part 3, "External +storage of metadata"). A sidecar is the packet as embedded, so `read` accepts everything +`from_packet` does but **requires the `x:xmpmeta` document element**: Part 1 §7.3.3 makes it the +marker that identifies XMP inside general XML text, which is what a standalone file is (exiv2's +sidecar sniffer keys on the same `` / ``). `write` emits the XML declaration +Part 3 asks for, then a read-only, unpadded packet in canonical form, so two sidecars of the same +graph are byte-identical. The crate has no filesystem API; the naming convention — `photo.xmp` +beside `photo.dng`, looked up in the image's directory — is documented, not enforced, and the caller +owns the path. + ## Scope - **Part 1 (data model + serialization): both directions.** The permissive reader accepts the @@ -95,8 +106,8 @@ padding) and `XmpPacket::parse` the graph — `from_packet` is exactly that comp **Production-ready v1** (issue #189). Implemented: parser + canonical serializer for the full XMP data model (simple / URI / structured / `Bag`·`Seq`·`Alt`, qualifiers, language alternatives), the -`` wrapper, and a 30-schema registry at parity with exiv2's documented set (issue -#421). See [STATUS.md](STATUS.md). +`` wrapper, a 30-schema registry at parity with exiv2's documented set, and `.xmp` +sidecar read/write (issue #421). See [STATUS.md](STATUS.md). ## Migrating from 1.x @@ -115,7 +126,8 @@ changed — every existing variant, URI, prefix and method keeps its meaning. - **Differential oracle** against exiv2's bundled **Adobe XMPCore**: gamut's packets validate and round-trip through the reference engine, every `WellKnownNs` URI is vouched for by its schema registry (one test per exiv2-parity schema reads a documented property back by its - `Xmp..` key), and the default-`xml:lang` posture is pinned to parity + `Xmp..` key), a sidecar gamut writes is read by the engine and the engine's own + serialization is read as a sidecar, and the default-`xml:lang` posture is pinned to parity (`tests/oracle.rs`; needs the `third_party/exiv2` + `third_party/expat` submodules and a C++ toolchain). One oracle normalization is pinned as such: exiv2 appends `/` to a namespace URI ending in neither `/` nor `#` when it registers it with XMPCore, so the engine re-serializes diff --git a/crates/gamut-xmp/STATUS.md b/crates/gamut-xmp/STATUS.md index 27bdebbb..b9f75496 100644 --- a/crates/gamut-xmp/STATUS.md +++ b/crates/gamut-xmp/STATUS.md @@ -45,6 +45,18 @@ placement, and array/struct nesting so output is stable, diffable, and round-tri (`XmpProperties::registerNs`), so the engine re-serializes `dwc` as `…/index.htm/` while gamut writes the `…/index.htm` exiv2 documents — pinned in `tests/oracle.rs` as an oracle normalization. +- **Sidecars require `x:xmpmeta` (issue #421).** `XmpSidecar::read` accepts everything + `XmpMeta::from_packet` does — XML declaration, BOM, wrapper or bare — but rejects a document + whose element is not `x:xmpmeta` with `XmpError::Prohibited` naming the element found. Part 3 + ("External storage of metadata", in the vendored 2020 edition's Introduction) defines a sidecar as + the packet "as though it were embedded and then … extracted"; Part 1 §7.3.3 gives `x:xmpmeta` + exactly one purpose, identifying XMP inside general XML text, which a standalone `.xmp` file is; + exiv2's sidecar sniffer keys on `.xmp` next to it. Part 3 asks that such a file be +//! "a complete, well-formed XML document, including the leading XML declaration", written "as though +//! it were embedded and then had the XMP packets extracted", with the `.xmp` extension and the same +//! base name as the image; applications find it by looking in the image's directory. +//! +//! [`XmpSidecar::read`] and [`XmpSidecar::write`] are the bytes-in / bytes-out pair. This crate has +//! no filesystem API and the naming convention is documented, not enforced: the caller owns the +//! path, exactly as the format crates own the embedded packet's location in their containers. + +use quick_xml::NsReader; +use quick_xml::events::Event; +use quick_xml::name::ResolveResult; + +use crate::error::{Result, XmpError}; +use crate::model::XmpMeta; +use crate::namespace::XMPMETA_NAMESPACE; +use crate::packet::XmpPacket; +use crate::writer::XmpWriter; + +/// The XMP sidecar file format: an XMP packet as a standalone `.xmp` file. +/// +/// The one thing a sidecar requires beyond an embedded packet is the `x:xmpmeta` element. Part 1 +/// §7.3.3 makes it optional inside a container — its purpose is "to identify XMP metadata within +/// general XML text that might contain other non-XMP uses of RDF" — and a standalone `.xmp` file is +/// exactly that general XML text, so here it is the marker that makes the file XMP at all (exiv2's +/// sidecar sniffer likewise keys on the `` header or the `` element). A +/// bare `rdf:RDF` document is rejected by [`XmpSidecar::read`] and always emitted by +/// [`XmpSidecar::write`] wrapped. +/// +/// ``` +/// use gamut_xmp::{WellKnownNs, XmpMeta, XmpSidecar}; +/// +/// let mut meta = XmpMeta::new(); +/// meta.set_text(WellKnownNs::Xmp.uri(), "Rating", "5"); +/// +/// // The bytes of `photo.xmp`, ready to be written beside `photo.dng`... +/// let file = XmpSidecar::write(&meta); +/// // ...and read back from it. +/// let parsed = XmpSidecar::read(&file)?; +/// assert_eq!(parsed.get_text(WellKnownNs::Xmp.uri(), "Rating"), Some("5")); +/// # Ok::<(), gamut_xmp::XmpError>(()) +/// ``` +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct XmpSidecar; + +impl XmpSidecar { + /// Parses the bytes of a `.xmp` sidecar file into a property graph. + /// + /// Accepts a UTF-8 file with or without the leading XML declaration, with or without the + /// `` wrapper, tolerating a leading byte-order mark — everything + /// [`XmpMeta::from_packet`] accepts — **provided** the document element is `x:xmpmeta` + /// (namespace `adobe:ns:meta/`, Part 1 §7.3.3). Trailing padding inside the packet wrapper is + /// ignored. + /// + /// # Errors + /// + /// Returns [`XmpError::Prohibited`] naming the document element when it is not `x:xmpmeta` + /// (a bare `rdf:RDF` document is a packet body, not a sidecar), and otherwise the same errors + /// as [`XmpMeta::from_packet`]: [`XmpError::Encoding`] for non-UTF-8 input, [`XmpError::Xml`] + /// for malformed XML, [`XmpError::MissingRdf`] when no `rdf:RDF` is found, and the + /// RDF/XML-for-XMP errors for constructs XMP does not permit. + pub fn read(bytes: &[u8]) -> Result { + let packet = XmpPacket::scan(bytes)?; + // Parse first so a malformed file reports its malformation, not a missing wrapper. + let meta = packet.parse()?; + if let Some(root) = document_element_unless_xmpmeta(&packet.body)? { + return Err(XmpError::Prohibited(format!( + "sidecar document element is <{root}>, not x:xmpmeta (a .xmp file is the packet \ + as embedded, wrapped in )" + ))); + } + Ok(meta) + } + + /// Serializes a property graph as the bytes of a `.xmp` sidecar file. + /// + /// The file is the XML declaration Part 3 asks for, then a read-only (`end="r"`, no padding) + /// `` packet whose body is the canonical RDF/XML (Part 1 §7) inside `x:xmpmeta`. + /// UTF-8, no byte-order mark. Being canonical, it is byte-stable for a given graph, so two + /// sidecars of the same metadata diff clean. + #[must_use] + pub fn write(meta: &XmpMeta) -> Vec { + let mut out = XML_DECLARATION.as_bytes().to_vec(); + out.extend_from_slice( + &XmpWriter::new() + .wrap_xmpmeta(true) + .writable(false) + .serialize(meta), + ); + out + } +} + +/// The XML declaration a sidecar leads with (Part 3, "External storage of metadata"). +const XML_DECLARATION: &str = "\n"; + +/// The name of the document element of `xml` when it is **not** `x:xmpmeta`, or `None` when it is. +/// +/// Walks past the prolog (declaration, comments, processing instructions, whitespace) to the first +/// start tag and resolves its namespace, so the check is on the element's identity rather than on +/// the literal `x:` prefix a writer happened to choose. +/// +/// # Errors +/// +/// Returns [`XmpError::Xml`] if the prolog cannot be lexed or the document has no element at all. +/// [`XmpSidecar::read`] parses the body before calling this, so on that path neither happens. +fn document_element_unless_xmpmeta(xml: &str) -> Result> { + let mut reader = NsReader::from_str(xml); + loop { + match reader + .read_event() + .map_err(|err| XmpError::Xml(err.to_string()))? + { + Event::Start(start) | Event::Empty(start) => { + let (resolved, local) = reader.resolver().resolve_element(start.name()); + let local = String::from_utf8_lossy(local.as_ref()).into_owned(); + let is_xmpmeta = matches!(resolved, ResolveResult::Bound(ns) + if ns.as_ref() == XMPMETA_NAMESPACE.as_bytes()) + && local == "xmpmeta"; + return Ok((!is_xmpmeta).then_some(local)); + } + Event::Eof => { + return Err(XmpError::Xml("sidecar has no document element".into())); + } + _ => {} + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::namespace::WellKnownNs; + + const RDF_ONLY: &str = "\ + "; + + /// `RDF_ONLY` inside the wrapper Part 1 §7.3.3 defines. + fn wrapped(body: &str) -> String { + format!("{body}") + } + + #[test] + fn write_emits_the_exact_sidecar_for_an_empty_graph() { + // Byte-exact: the XML declaration, the read-only packet wrapper with no padding, the + // x:xmpmeta wrapper, and the canonical body — the file is meant to diff clean. + let file = XmpSidecar::write(&XmpMeta::new()); + assert_eq!( + core::str::from_utf8(&file).unwrap(), + "\n\ + \n\ + \n \ + \n \ + \n \ + \n\ + \n\ + " + ); + } + + #[test] + fn read_accepts_what_write_produces() { + // The two halves agree on the wrapper requirement: a written sidecar reads back. + let mut meta = XmpMeta::new(); + meta.set_text(WellKnownNs::Xmp.uri(), "CreatorTool", "gamut"); + let parsed = XmpSidecar::read(&XmpSidecar::write(&meta)).unwrap(); + assert_eq!(parsed, meta); + } + + #[test] + fn read_rejects_a_bare_rdf_document() { + // A packet body without x:xmpmeta is valid embedded XMP but not a sidecar; the error names + // the element found so the caller knows which wrapper is missing. + let err = XmpSidecar::read(RDF_ONLY.as_bytes()).unwrap_err(); + assert!( + matches!(&err, XmpError::Prohibited(m) if m.contains("") && m.contains("x:xmpmeta")), + "got {err:?}" + ); + } + + #[test] + fn read_rejects_a_bare_rdf_document_even_inside_the_packet_wrapper() { + // The xpacket wrapper alone does not make a sidecar: the document element must still be + // x:xmpmeta. + let file = format!( + "{RDF_ONLY}" + ); + let err = XmpSidecar::read(file.as_bytes()).unwrap_err(); + assert!( + matches!(&err, XmpError::Prohibited(m) if m.contains("")), + "got {err:?}" + ); + } + + #[test] + fn read_accepts_a_wrapped_document_without_the_packet_wrapper() { + // Part 1 §7.3.3: x:xmpmeta may stand without the xpacket instructions. + let meta = XmpSidecar::read(wrapped(RDF_ONLY).as_bytes()).unwrap(); + assert!(meta.properties.is_empty()); + } + + #[test] + fn read_skips_bom_declaration_comment_and_whitespace_before_the_document_element() { + // Part 3 asks for the leading XML declaration; a BOM and a comment are legal XML prolog + // too. None of them is the document element. + let file = format!( + "\u{FEFF}\n\n {}", + wrapped(RDF_ONLY) + ); + let meta = XmpSidecar::read(file.as_bytes()).unwrap(); + assert!(meta.properties.is_empty()); + } + + #[test] + fn read_reports_malformed_xml_before_the_missing_wrapper() { + // Parse errors take precedence: an ill-formed bare document (mismatched end tag) is + // "malformed XML", not "not a sidecar", so the diagnosis points at the real defect. + let err = XmpSidecar::read( + b"", + ) + .unwrap_err(); + assert!(matches!(err, XmpError::Xml(_)), "got {err:?}"); + } + + #[test] + fn document_element_check_resolves_the_namespace_not_the_prefix() { + // An `xmpmeta` element in the wrong namespace is not the wrapper; the wrapper under + // another prefix is. Identity is the URI (Part 1 §6.2). + assert_eq!( + document_element_unless_xmpmeta( + "" + ) + .unwrap(), + Some("xmpmeta".to_owned()) + ); + assert_eq!( + document_element_unless_xmpmeta( + "" + ) + .unwrap(), + None + ); + // An empty-element wrapper is still the wrapper (quick-xml reports it as `Empty`). + assert_eq!( + document_element_unless_xmpmeta("").unwrap(), + None + ); + } + + #[test] + fn document_element_check_reports_an_element_less_document() { + // Only reachable outside `read` (which has already parsed an rdf:RDF); pinned so the + // loop's EOF arm is a typed error, not a hang or a panic. + let err = document_element_unless_xmpmeta("\n") + .unwrap_err(); + assert!( + matches!(&err, XmpError::Xml(m) if m.contains("no document element")), + "got {err:?}" + ); + } +} diff --git a/crates/gamut-xmp/tests/oracle.rs b/crates/gamut-xmp/tests/oracle.rs index 7c2b40fb..f6aa65ec 100644 --- a/crates/gamut-xmp/tests/oracle.rs +++ b/crates/gamut-xmp/tests/oracle.rs @@ -8,7 +8,8 @@ //! Requires the `third_party/exiv2` + `third_party/expat` submodules and a C++ toolchain. use gamut_xmp::{ - Namespace, WellKnownNs, XmpArray, XmpItem, XmpMeta, XmpProperty, XmpValue, XmpWriter, + Namespace, WellKnownNs, XmpArray, XmpItem, XmpMeta, XmpProperty, XmpSidecar, XmpValue, + XmpWriter, }; const RDF: &str = "http://www.w3.org/1999/02/22-rdf-syntax-ns#"; @@ -468,3 +469,39 @@ fn darwin_core_record_field_reads_back_under_the_dwc_key() { assert_eq!(xmpcore_output_uri(dwc), "http://rs.tdwg.org/dwc/index.htm/"); struct_field_survives_xmpcore(dwc, "Record", "institutionID", "GAMUT"); } + +// --------------------------------------------------------------------------------------------------- +// Sidecars (issue #421): the file gamut writes is XMP to the reference engine, and the file the +// reference engine writes is a sidecar to gamut. +// --------------------------------------------------------------------------------------------------- + +#[test] +fn sidecar_written_by_gamut_is_read_by_xmpcore() { + // The whole file — XML declaration, read-only packet wrapper, x:xmpmeta — is handed to XMPCore + // as-is, the way exiv2's sidecar reader passes a .xmp file's bytes to the parser. + let file = XmpSidecar::write(&sample()); + exiv2_oracle::validate(&file).expect("exiv2 (Adobe XMPCore) must accept gamut's sidecar"); + assert_eq!( + exiv2_oracle::get_property(&file, "Xmp.dc.format").unwrap(), + "text/plain" + ); + assert_eq!( + exiv2_oracle::get_property(&file, "Xmp.xmp.BaseURL").unwrap(), + "http://example.com/" + ); +} + +#[test] +fn gamut_reads_the_sidecar_xmpcore_writes() { + // XMPCore's default serialization is the `` + `` form exiv2 stores in a + // .xmp file (its sidecar writer only prepends the packet header when the packet lacks one), so + // the reference engine's output is exactly what a sidecar on disk looks like. + let reference = exiv2_oracle::roundtrip(&sample().to_packet()).expect("exiv2 round-trip"); + assert!( + String::from_utf8_lossy(&reference).contains(" Date: Sun, 6 Sep 2026 04:13:51 -0400 Subject: [PATCH 3/9] refactor(xmp): check end of input before matching sidecar prolog events Review findings on the lane's own diff. The sidecar document-element scan tested `Event::Eof` as a match arm; with that arm removed the loop spins forever, so the mutant would time out instead of being caught. The check is now a guard before the match, as `reader.rs` does, and end of input is terminal regardless of the arms. The MWG oracle helper compares the re-parsed structure under the engine's output URI like its sibling, so it stays correct for a URI exiv2 normalizes; a count comment in the registry test now adds up; README states that exiv2's sidecar sniffer accepts `` / ``). `write` emits the XML declaration +marker that identifies XMP inside general XML text, which is what a standalone file is. (exiv2's +sidecar sniffer accepts a file starting with `\n"; fn document_element_unless_xmpmeta(xml: &str) -> Result> { let mut reader = NsReader::from_str(xml); loop { - match reader + let event = reader .read_event() - .map_err(|err| XmpError::Xml(err.to_string()))? - { - Event::Start(start) | Event::Empty(start) => { - let (resolved, local) = reader.resolver().resolve_element(start.name()); - let local = String::from_utf8_lossy(local.as_ref()).into_owned(); - let is_xmpmeta = matches!(resolved, ResolveResult::Bound(ns) - if ns.as_ref() == XMPMETA_NAMESPACE.as_bytes()) - && local == "xmpmeta"; - return Ok((!is_xmpmeta).then_some(local)); - } - Event::Eof => { - return Err(XmpError::Xml("sidecar has no document element".into())); - } - _ => {} + .map_err(|err| XmpError::Xml(err.to_string()))?; + // Checked before the match (as `reader.rs` does) so the loop's exit does not depend on a + // match arm: end of input is always terminal. + if matches!(event, Event::Eof) { + return Err(XmpError::Xml("sidecar has no document element".into())); } + if let Event::Start(start) | Event::Empty(start) = event { + let (resolved, local) = reader.resolver().resolve_element(start.name()); + let local = String::from_utf8_lossy(local.as_ref()).into_owned(); + let is_xmpmeta = matches!(resolved, ResolveResult::Bound(ns) + if ns.as_ref() == XMPMETA_NAMESPACE.as_bytes()) + && local == "xmpmeta"; + return Ok((!is_xmpmeta).then_some(local)); + } + // Declaration, comments, processing instructions, DOCTYPE and whitespace: prolog, skipped. } } diff --git a/crates/gamut-xmp/tests/oracle.rs b/crates/gamut-xmp/tests/oracle.rs index f6aa65ec..1f706276 100644 --- a/crates/gamut-xmp/tests/oracle.rs +++ b/crates/gamut-xmp/tests/oracle.rs @@ -408,13 +408,50 @@ fn struct_bag_item_field_survives_xmpcore( ); let out = exiv2_oracle::roundtrip(&packet).expect("exiv2 round-trip"); let parsed = XmpMeta::from_packet(&out).expect("gamut parses exiv2's output"); + // Compare under the engine's output URI (the identity for both MWG schemas; see + // `xmpcore_output_uri`), so the helper stays correct for a `dwc`-style URI too. + let out_uri = xmpcore_output_uri(ns); + let mut expected = meta.get(uri, name).cloned().expect("the property just set"); + rename_namespace(&mut expected, uri, &out_uri); assert_eq!( - parsed.get(uri, name), - meta.get(uri, name), + parsed.get(&out_uri, name), + Some(&expected), "{prefix}:{name} must round-trip through XMPCore unchanged" ); } +/// Rewrites every occurrence of namespace `from` in `property` (itself, its fields, items and +/// qualifiers) to `to`. +fn rename_namespace(property: &mut XmpProperty, from: &str, to: &str) { + if property.namespace == from { + property.namespace = to.to_owned(); + } + for qualifier in &mut property.qualifiers { + rename_namespace(qualifier, from, to); + } + rename_namespace_in_value(&mut property.value, from, to); +} + +/// The value half of [`rename_namespace`]. +fn rename_namespace_in_value(value: &mut XmpValue, from: &str, to: &str) { + match value { + XmpValue::Simple(_) | XmpValue::Uri(_) => {} + XmpValue::Structured(fields) => { + for field in fields { + rename_namespace(field, from, to); + } + } + XmpValue::Array(array) => { + for item in array.items_mut() { + rename_namespace_in_value(&mut item.value, from, to); + for qualifier in &mut item.qualifiers { + rename_namespace(qualifier, from, to); + } + } + } + } +} + #[test] fn mwg_regions_list_item_name_reads_back_under_the_mwg_rs_key() { // MWG Guidelines 2.0, Regions: `mwg-rs:Regions` is a RegionInfo structure whose `RegionList` From fd6ba221402080027c4151124b3230536d171c06 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Wed, 9 Sep 2026 13:50:47 -0400 Subject: [PATCH 4/9] feat(xmp): give a sidecar's missing wrapper its own error MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review finding on the lane's own diff. `XmpSidecar::read` reported a document element other than `x:xmpmeta` as `XmpError::Prohibited`, whose documented meaning is a construct the *specification* prohibits (Part 1 §7.8/§7.9.3: `rdf:_n` items, `rdf:value` carrying `xml:lang`). A bare `rdf:RDF` document is not such a construct - §7.3.3 makes the wrapper optional, `XmpWriter::wrap_xmpmeta(false)` emits exactly that form, and `XmpMeta::from_packet` reads it - so the message asserted a prohibition that does not exist. `XmpError::MissingXmpMeta` now names the element found. The enum is `#[non_exhaustive]`, so this is additive. The docs state precisely where gamut is stricter than its oracle: exiv2's sidecar sniffer (`isXmpType`) accepts a `.xmp` file starting with `, not x:xmpmeta")] + MissingXmpMeta(String), } /// A specialized [`Result`](core::result::Result) for the XMP read path. @@ -100,5 +110,11 @@ mod tests { gamut_core::Error::from(XmpError::DuplicateLang("en".into())), gamut_core::Error::Context(ref diagnostic) if diagnostic.source_error().kind() == gamut_core::ErrorKind::InvalidInput )); + // A sidecar without its wrapper is bad input, not an unimplemented feature: the caller + // fixes it by handing the bytes to `XmpMeta::from_packet` or wrapping them. + assert!(matches!( + gamut_core::Error::from(XmpError::MissingXmpMeta("RDF".into())), + gamut_core::Error::Context(ref diagnostic) if diagnostic.source_error().kind() == gamut_core::ErrorKind::InvalidInput + )); } } diff --git a/crates/gamut-xmp/src/sidecar.rs b/crates/gamut-xmp/src/sidecar.rs index c6111f87..8279cbbb 100644 --- a/crates/gamut-xmp/src/sidecar.rs +++ b/crates/gamut-xmp/src/sidecar.rs @@ -58,20 +58,25 @@ impl XmpSidecar { /// /// # Errors /// - /// Returns [`XmpError::Prohibited`] naming the document element when it is not `x:xmpmeta` - /// (a bare `rdf:RDF` document is a packet body, not a sidecar), and otherwise the same errors - /// as [`XmpMeta::from_packet`]: [`XmpError::Encoding`] for non-UTF-8 input, [`XmpError::Xml`] - /// for malformed XML, [`XmpError::MissingRdf`] when no `rdf:RDF` is found, and the - /// RDF/XML-for-XMP errors for constructs XMP does not permit. + /// Returns [`XmpError::MissingXmpMeta`] naming the document element when it is not `x:xmpmeta` + /// — a bare `rdf:RDF` document is a packet body, valid XMP that [`XmpMeta::from_packet`] + /// accepts, but not a sidecar. Otherwise the same errors as [`XmpMeta::from_packet`]: + /// [`XmpError::Encoding`] for non-UTF-8 input, [`XmpError::Xml`] for malformed XML, + /// [`XmpError::MissingRdf`] when no `rdf:RDF` is found, and the RDF/XML-for-XMP errors for + /// constructs XMP does not permit. + /// + /// Note that exiv2 is more permissive here: its sidecar sniffer (`isXmpType`) accepts a file + /// that starts with ` Result { let packet = XmpPacket::scan(bytes)?; // Parse first so a malformed file reports its malformation, not a missing wrapper. let meta = packet.parse()?; if let Some(root) = document_element_unless_xmpmeta(&packet.body)? { - return Err(XmpError::Prohibited(format!( - "sidecar document element is <{root}>, not x:xmpmeta (a .xmp file is the packet \ - as embedded, wrapped in )" - ))); + return Err(XmpError::MissingXmpMeta(root)); } Ok(meta) } @@ -104,6 +109,15 @@ const XML_DECLARATION: &str = "\n"; /// start tag and resolves its namespace, so the check is on the element's identity rather than on /// the literal `x:` prefix a writer happened to choose. /// +/// **Reachability.** Called from [`XmpSidecar::read`] the answer can only ever be `None` or +/// `Some("RDF")`: `read` parses the body first, and the reader admits exactly one document element, +/// `rdf:RDF` or `x:xmpmeta` (`reader.rs`'s `find_rdf`, over a tree `build_tree` has already +/// rejected multiple roots in). So on that path the namespace comparison, the `Empty` arm and the +/// end-of-input error are all unreachable, and the second lex is a bounded extra pass over bytes +/// already parsed once. It is kept as a total function over any XML — unit-tested directly for the +/// cases `read` cannot produce — rather than threading the root element out of `find_rdf`, which +/// would widen the blast radius in `reader.rs` for a one-off cost. +/// /// # Errors /// /// Returns [`XmpError::Xml`] if the prolog cannot be lexed or the document has no element at all. @@ -174,12 +188,18 @@ mod tests { #[test] fn read_rejects_a_bare_rdf_document() { // A packet body without x:xmpmeta is valid embedded XMP but not a sidecar; the error names - // the element found so the caller knows which wrapper is missing. + // the element found so the caller knows which wrapper is missing. It is deliberately not + // `Prohibited`: Part 1 §7.3.3 permits the wrapper-less form, and `XmpMeta::from_packet` + // reads exactly these bytes. let err = XmpSidecar::read(RDF_ONLY.as_bytes()).unwrap_err(); assert!( - matches!(&err, XmpError::Prohibited(m) if m.contains("") && m.contains("x:xmpmeta")), + matches!(&err, XmpError::MissingXmpMeta(root) if root == "RDF"), "got {err:?}" ); + assert!( + XmpMeta::from_packet(RDF_ONLY.as_bytes()).is_ok(), + "the same bytes must stay valid as an embedded packet" + ); } #[test] @@ -191,7 +211,7 @@ mod tests { ); let err = XmpSidecar::read(file.as_bytes()).unwrap_err(); assert!( - matches!(&err, XmpError::Prohibited(m) if m.contains("")), + matches!(&err, XmpError::MissingXmpMeta(root) if root == "RDF"), "got {err:?}" ); } From fce19d658714752691ace6ef0cdf740b3fb47fde Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Wed, 9 Sep 2026 13:51:03 -0400 Subject: [PATCH 5/9] feat(xmp): accept on read the Darwin Core URI XMPCore emits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review finding on the lane's own diff. exiv2 appends `/` to any namespace URI ending in neither `/` nor `#` before registering it with XMPCore (`third_party/exiv2/src/properties.cpp`), and Darwin Core is the only registered schema whose URI ends in neither. A packet the reference engine wrote - including a sidecar - therefore declares `http://rs.tdwg.org/dwc/index.htm/`, which `from_uri` did not recognise, so such a graph re-serialized under a synthesized `ns1` prefix instead of `dwc`. Prefixes are non-semantic (Part 1 §6.2), so this was a round-trip fidelity gap rather than a correctness defect - but one the oracle test's own normalization helper hid from the suite. `from_uri` now recognises the slashed form as a read-only alias, `DWC_URI_TRAILING_SLASH`. `uri()` still emits the unslashed URI exiv2 documents, so gamut's bytes are unchanged, and the alias is not an `ALL` entry, so iteration and the registry's URI/prefix uniqueness are untouched. The nested oracle helper is now driven once with Darwin Core, so its expected-graph re-namespacing runs with source and target actually differing; both existing callers pass a URI the engine leaves alone, which left that whole recursion a verified no-op. Also splits the registry-count assertion into its own drift guard named for what it guards, since every future addition to the registry edits it, and records in STATUS that `exifEX`'s URI rests on the oracle rather than on a vendored specification (issue #516). Refs #421 --- crates/gamut-xmp/README.md | 4 ++- crates/gamut-xmp/STATUS.md | 12 +++++-- crates/gamut-xmp/src/lib.rs | 4 ++- crates/gamut-xmp/src/namespace.rs | 55 +++++++++++++++++++++++++++++-- crates/gamut-xmp/tests/oracle.rs | 17 ++++++++++ 5 files changed, 84 insertions(+), 8 deletions(-) diff --git a/crates/gamut-xmp/README.md b/crates/gamut-xmp/README.md index 124d9a71..6725194e 100644 --- a/crates/gamut-xmp/README.md +++ b/crates/gamut-xmp/README.md @@ -134,7 +134,9 @@ changed — every existing variant, URI, prefix and method keeps its meaning. (`tests/oracle.rs`; needs the `third_party/exiv2` + `third_party/expat` submodules and a C++ toolchain). One oracle normalization is pinned as such: exiv2 appends `/` to a namespace URI ending in neither `/` nor `#` when it registers it with XMPCore, so the engine re-serializes - Darwin Core as `http://rs.tdwg.org/dwc/index.htm/`; gamut writes the URI exiv2 documents. + Darwin Core as `http://rs.tdwg.org/dwc/index.htm/`; gamut writes the URI exiv2 documents and + additionally accepts the slashed form on read (`DWC_URI_TRAILING_SLASH`), so a packet the engine + wrote round-trips under the `dwc` prefix. - **Mutation-clean** — `cargo mutants` passes with zero gamut-xmp exclusions in `.cargo/mutants.toml`. - **No benches, intentionally** — the crate has no performance contract; packets are a few KB. diff --git a/crates/gamut-xmp/STATUS.md b/crates/gamut-xmp/STATUS.md index af7cc39d..20ce5578 100644 --- a/crates/gamut-xmp/STATUS.md +++ b/crates/gamut-xmp/STATUS.md @@ -39,12 +39,18 @@ placement, and array/struct nesting so output is stable, diffable, and round-tri `xmpNsInfo`), and `tests/oracle.rs` reads a documented property of each back from XMPCore by its `Xmp..` key, which fails for a wrong URI *or* a wrong prefix. The registry stays a registry: nothing about a value is interpreted. Two divergences are recorded rather than hidden: - `exifEX` is `http://cipa.jp/exif/1.0/` (CIPA DC-010; the reference engine's URI) although the - vendored Exif 3.0 text's annotation examples (Annex J) bind the prefix to `…/exif/2.32/`; and + `exifEX` is `http://cipa.jp/exif/1.0/` (the URI exiv2 and XMPCore bind) although the vendored Exif + 3.0 text's annotation examples (Annex J.2–J.3) bind the prefix to `…/exif/2.32/` — CIPA DC-010 + itself is not vendored, so this one entry rests on the oracle rather than on a specification under + `references/`, tracked in issue #516; and exiv2 appends `/` to a URI ending in neither `/` nor `#` when registering it with XMPCore (`XmpProperties::registerNs`), so the engine re-serializes `dwc` as `…/index.htm/` while gamut writes the `…/index.htm` exiv2 documents — pinned in `tests/oracle.rs` as an oracle - normalization. + normalization. Because that normalization is what XMPCore *emits*, `WellKnownNs::from_uri` also + recognises `http://rs.tdwg.org/dwc/index.htm/` as a read-only alias (`DWC_URI_TRAILING_SLASH`), + so a graph parsed from an exiv2-written packet or sidecar re-serializes under `dwc` rather than a + synthesized prefix; `uri()` still emits the unslashed URI, so gamut's own bytes are unchanged and + the alias is not an `ALL` entry. - **Sidecars require `x:xmpmeta` (issue #421).** `XmpSidecar::read` accepts everything `XmpMeta::from_packet` does — XML declaration, BOM, wrapper or bare — but rejects a document whose element is not `x:xmpmeta` with `XmpError::MissingXmpMeta` naming the element found. Part 3 diff --git a/crates/gamut-xmp/src/lib.rs b/crates/gamut-xmp/src/lib.rs index 8522b664..2011473d 100644 --- a/crates/gamut-xmp/src/lib.rs +++ b/crates/gamut-xmp/src/lib.rs @@ -69,7 +69,9 @@ mod reader; pub use error::{Result, XmpError}; pub use model::{XmpArray, XmpItem, XmpMeta, XmpProperty, XmpValue}; -pub use namespace::{Namespace, RDF_NAMESPACE, WellKnownNs, XML_NAMESPACE, XMPMETA_NAMESPACE}; +pub use namespace::{ + DWC_URI_TRAILING_SLASH, Namespace, RDF_NAMESPACE, WellKnownNs, XML_NAMESPACE, XMPMETA_NAMESPACE, +}; pub use packet::XmpPacket; pub use sidecar::XmpSidecar; pub use writer::XmpWriter; diff --git a/crates/gamut-xmp/src/namespace.rs b/crates/gamut-xmp/src/namespace.rs index 56d3e494..45f1593a 100644 --- a/crates/gamut-xmp/src/namespace.rs +++ b/crates/gamut-xmp/src/namespace.rs @@ -291,13 +291,34 @@ impl WellKnownNs { } } - /// The schema whose URI is exactly `uri`, if any. + /// The schema `uri` identifies, if any. + /// + /// An exact match against [`WellKnownNs::uri`], plus one **read alias**: Darwin Core is also + /// recognised under [`DWC_URI_TRAILING_SLASH`], the form Adobe XMPCore emits (see that + /// constant). The alias is read-only — [`WellKnownNs::uri`] keeps returning the unslashed URI + /// exiv2 documents, so gamut's own bytes are unchanged — and it is not a [`WellKnownNs::ALL`] + /// entry, so iteration and the URI/prefix uniqueness of the registry are unaffected. #[must_use] pub fn from_uri(uri: &str) -> Option { - WellKnownNs::ALL.iter().copied().find(|ns| ns.uri() == uri) + WellKnownNs::ALL + .iter() + .copied() + .find(|ns| ns.uri() == uri) + .or_else(|| (uri == DWC_URI_TRAILING_SLASH).then_some(WellKnownNs::DarwinCore)) } } +/// Darwin Core's namespace URI with the trailing slash Adobe XMPCore emits. +/// +/// exiv2 appends `/` to any namespace URI ending in neither `/` nor `#` before registering it with +/// XMPCore (`third_party/exiv2/src/properties.cpp`, `XmpProperties::registerNs`), and Darwin Core +/// (`http://rs.tdwg.org/dwc/index.htm`) is the only registered schema whose URI ends in neither. +/// A packet written by exiv2 — including a sidecar it wrote — therefore declares the slashed form, +/// and without this alias such a graph would re-serialize under a synthesized `ns1` prefix instead +/// of `dwc`. Prefixes are non-semantic (Part 1 §6.2), so this is round-trip fidelity, not +/// correctness. +pub const DWC_URI_TRAILING_SLASH: &str = "http://rs.tdwg.org/dwc/index.htm/"; + #[cfg(test)] mod tests { use super::*; @@ -399,10 +420,38 @@ mod tests { "{ns:?} must be in ALL so from_uri and the writer's prefix table see it" ); } - // 18 entries before this change (the original 17 plus `dcterms`) and twelve added: 30. + } + + #[test] + fn registry_holds_thirty_schemas() { + // A drift guard, deliberately separate from the exiv2-parity test above: every future + // addition to the registry edits this one line, and it fails for exactly that reason. + // 18 entries before the exiv2-parity additions (the original 17 plus `dcterms`) + 12 = 30. assert_eq!(WellKnownNs::ALL.len(), 30); } + #[test] + fn from_uri_accepts_the_darwin_core_trailing_slash_alias_without_emitting_it() { + // XMPCore emits `.../index.htm/`; reading a packet it wrote must still resolve to the + // registered schema, so the graph re-serializes under `dwc` rather than a synthesized + // prefix. The alias is read-only: `uri()` still emits the unslashed URI exiv2 documents, + // and the alias is not an ALL entry. + assert_eq!( + WellKnownNs::from_uri(DWC_URI_TRAILING_SLASH), + Some(WellKnownNs::DarwinCore) + ); + assert_eq!( + WellKnownNs::DarwinCore.uri(), + "http://rs.tdwg.org/dwc/index.htm" + ); + assert_ne!(WellKnownNs::DarwinCore.uri(), DWC_URI_TRAILING_SLASH); + assert!( + !WellKnownNs::ALL + .iter() + .any(|ns| ns.uri() == DWC_URI_TRAILING_SLASH) + ); + } + #[test] fn from_uri_rejects_unknown() { assert_eq!(WellKnownNs::from_uri("http://example.com/ns/"), None); diff --git a/crates/gamut-xmp/tests/oracle.rs b/crates/gamut-xmp/tests/oracle.rs index 1f706276..86f24216 100644 --- a/crates/gamut-xmp/tests/oracle.rs +++ b/crates/gamut-xmp/tests/oracle.rs @@ -466,6 +466,23 @@ fn mwg_regions_list_item_name_reads_back_under_the_mwg_rs_key() { ); } +#[test] +fn darwin_core_nested_bag_survives_the_uri_xmpcore_normalizes() { + // The only registered schema whose URI XMPCore rewrites (see `xmpcore_output_uri`), driven + // through the nested helper so the expected graph is re-namespaced with `from != to` — the + // MWG callers both pass a URI the engine leaves alone, which would leave that normalization + // unexercised. The shape is the helper's bag-of-structure rather than the `bag Text` exiv2 + // declares for dwc's bags: this crate is a namespace registry, not a validator, and XMPCore + // stores the shape the packet carries; what is under test here is the URI, not the type. + struct_bag_item_field_survives_xmpcore( + WellKnownNs::DarwinCore, + "Record", + "dynamicProperties", + "institutionID", + "GAMUT", + ); +} + #[test] fn mwg_keywords_hierarchy_item_keyword_reads_back_under_the_mwg_kw_key() { // MWG Guidelines 2.0, Keywords: `mwg-kw:Keywords` is a KeywordInfo structure whose From b3375afb0623d45b75ad0c99401bdc3828abc1b0 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 00:19:24 -0400 Subject: [PATCH 6/9] docs(xmp): state where the sidecar rule and the dwc alias diverge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `XmpSidecar::read` requires `x:xmpmeta`, and the docs argued that follows from Part 1 §7.3.3. It does not: §7.3.3 reads "An optional x:xmpmeta element may be placed around the rdf:RDF element" and asks a processor to "tolerate" one, and Part 3's external-storage bullets never mention the element — `xmpmeta` occurs in Part 3 once, in an SVG example. The requirement stands, but as gamut's own rule, stricter than the specification and stated as such. The same passages claimed parity with exiv2 where there is none: exiv2's `isXmpType` keys on ` Option { WellKnownNs::ALL @@ -444,7 +453,6 @@ mod tests { WellKnownNs::DarwinCore.uri(), "http://rs.tdwg.org/dwc/index.htm" ); - assert_ne!(WellKnownNs::DarwinCore.uri(), DWC_URI_TRAILING_SLASH); assert!( !WellKnownNs::ALL .iter() @@ -452,6 +460,26 @@ mod tests { ); } + #[test] + fn the_trailing_slash_alias_is_the_only_uri_from_uri_does_not_hand_back() { + // `from_uri(u).map(uri) == Some(u)` — resolving a URI hands the same URI back, because + // reading never canonicalizes — holds for every registered URI. The Darwin Core read alias + // is the sole exception, and pinning it here is what keeps it a documented exception rather + // than a silent hole: a caller that resolved by the slashed URI must keep using it. + for &ns in WellKnownNs::ALL { + assert_eq!( + WellKnownNs::from_uri(ns.uri()).map(WellKnownNs::uri), + Some(ns.uri()), + "{ns:?} must hand back the URI it was resolved by" + ); + } + assert_eq!( + WellKnownNs::from_uri(DWC_URI_TRAILING_SLASH).map(WellKnownNs::uri), + Some("http://rs.tdwg.org/dwc/index.htm"), + "the alias resolves to the unslashed URI, so it is not an identity" + ); + } + #[test] fn from_uri_rejects_unknown() { assert_eq!(WellKnownNs::from_uri("http://example.com/ns/"), None); diff --git a/crates/gamut-xmp/src/sidecar.rs b/crates/gamut-xmp/src/sidecar.rs index 8279cbbb..8d4650bc 100644 --- a/crates/gamut-xmp/src/sidecar.rs +++ b/crates/gamut-xmp/src/sidecar.rs @@ -23,13 +23,19 @@ use crate::writer::XmpWriter; /// The XMP sidecar file format: an XMP packet as a standalone `.xmp` file. /// -/// The one thing a sidecar requires beyond an embedded packet is the `x:xmpmeta` element. Part 1 -/// §7.3.3 makes it optional inside a container — its purpose is "to identify XMP metadata within -/// general XML text that might contain other non-XMP uses of RDF" — and a standalone `.xmp` file is -/// exactly that general XML text, so here it is the marker that makes the file XMP at all (exiv2's -/// sidecar sniffer likewise keys on the `` header or the `` element). A -/// bare `rdf:RDF` document is rejected by [`XmpSidecar::read`] and always emitted by -/// [`XmpSidecar::write`] wrapped. +/// The one thing this crate requires of a sidecar beyond an embedded packet is the `x:xmpmeta` +/// element — **a gamut rule, stricter than the specification rather than derived from it**. Part 1 +/// §7.3.3 says an "optional" `x:xmpmeta` element "may be placed around the rdf:RDF element" and that +/// a processor "should tolerate" one: permission, never a requirement. Part 3's "External storage +/// of metadata" bullets never mention the element at all. A wrapper-less `.xmp` file is therefore +/// spec-conformant, and [`XmpSidecar::read`] rejects it anyway. +/// +/// What §7.3.3 does supply is the element's purpose — "to identify XMP metadata within general XML +/// text that might contain other non-XMP uses of RDF" — and a standalone `.xmp` file is exactly +/// that general XML text: without the element nothing in the bytes says the RDF is XMP. exiv2 draws +/// the line elsewhere again, accepting ` Result { let packet = XmpPacket::scan(bytes)?; // Parse first so a malformed file reports its malformation, not a missing wrapper. diff --git a/crates/gamut-xmp/tests/oracle.rs b/crates/gamut-xmp/tests/oracle.rs index 86f24216..808eadd3 100644 --- a/crates/gamut-xmp/tests/oracle.rs +++ b/crates/gamut-xmp/tests/oracle.rs @@ -261,7 +261,10 @@ fn text_property_survives_xmpcore(ns: WellKnownNs, name: &str, value: &str) { ); let out = exiv2_oracle::roundtrip(&packet).expect("exiv2 round-trip"); let parsed = XmpMeta::from_packet(&out).expect("gamut parses exiv2's output"); - assert_eq!(parsed.get_text(ns.uri(), name), Some(value)); + // The reference engine's output URI, as in `struct_field_survives_xmpcore`: the identity for + // every schema but Darwin Core, so reading back under `ns.uri()` here would fail for a future + // schema whose URI ends in neither `/` nor `#`, for a reason unrelated to that schema. + assert_eq!(parsed.get_text(&xmpcore_output_uri(ns), name), Some(value)); } #[test] From 825353d1051bb0b9c539d1492bdbdc440112e1fe Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 01:05:03 -0400 Subject: [PATCH 7/9] docs(xmp): warn on the Darwin Core variant that XMPCore spells its URI differently The variant's own rustdoc is where a caller about to call `.uri()` lands, and it said nothing about the trailing-slash read alias. A caller reading a packet XMPCore wrote would find `get_text(DarwinCore.uri(), ..)` returning None with no clue why, because reading does not canonicalize and the graph keeps the slashed URI the packet declared. State it where they meet it, and point at `from_uri` for the whole rule. Refs #421 --- crates/gamut-xmp/src/namespace.rs | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/crates/gamut-xmp/src/namespace.rs b/crates/gamut-xmp/src/namespace.rs index 0035e3e0..536fc8d5 100644 --- a/crates/gamut-xmp/src/namespace.rs +++ b/crates/gamut-xmp/src/namespace.rs @@ -170,6 +170,13 @@ pub enum WellKnownNs { /// /// URI `http://rs.tdwg.org/dwc/index.htm`, the TDWG Darwin Core namespace as it is used in /// XMP; registered by exiv2 (`third_party/exiv2/src/properties.cpp`, `xmpNsInfo`). + /// + /// **The only variant with two spellings.** Adobe XMPCore writes this URI with a trailing + /// slash ([`DWC_URI_TRAILING_SLASH`]), which [`WellKnownNs::from_uri`] accepts as a read alias + /// while [`WellKnownNs::uri`] keeps returning the unslashed form above. Reading does not + /// canonicalize, so in a graph parsed from an XMPCore-written packet the properties are keyed + /// by the *slashed* URI and `uri()` will not find them; resolve them by the URI the packet + /// carries. See [`WellKnownNs::from_uri`] for the whole of it. DarwinCore, } From 490dc579d2daa8889bb83db5bf1e7b8fdbeea6e0 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 01:05:12 -0400 Subject: [PATCH 8/9] test(xmp): close from_uri over the alias family exiv2 generates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `the_trailing_slash_alias_is_the_only_uri_from_uri_does_not_hand_back` looped over ALL asserting `from_uri(u).map(uri) == Some(u)` — which `uri_and_prefix_are_exact_and_round_trip` already asserts more strongly, so the test killed nothing, and its name's claim (that the Darwin Core alias is the *only* exception) went unpinned: a second, undocumented read alias added to `from_uri` passed the whole suite. Pin the exclusivity instead. exiv2's `XmpProperties::registerNs` appends `/` to a URI ending in neither `/` nor `#`, so an alias is a registry URI with a trailing slash added or removed; assert that no schema but Darwin Core answers to either form of its own URI. Verified against both directions of that family: with the test present each fails it and nothing else (106 passed, 1 failed); with the test deleted both pass the suite (106 passed, 0 failed). Refs #421 --- crates/gamut-xmp/src/namespace.rs | 33 ++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/crates/gamut-xmp/src/namespace.rs b/crates/gamut-xmp/src/namespace.rs index 536fc8d5..cebf7a33 100644 --- a/crates/gamut-xmp/src/namespace.rs +++ b/crates/gamut-xmp/src/namespace.rs @@ -469,16 +469,31 @@ mod tests { #[test] fn the_trailing_slash_alias_is_the_only_uri_from_uri_does_not_hand_back() { - // `from_uri(u).map(uri) == Some(u)` — resolving a URI hands the same URI back, because - // reading never canonicalizes — holds for every registered URI. The Darwin Core read alias - // is the sole exception, and pinning it here is what keeps it a documented exception rather - // than a silent hole: a caller that resolved by the slashed URI must keep using it. + // That `from_uri(u).map(uri) == Some(u)` for every *registered* URI is already pinned, more + // strongly, by `uri_and_prefix_are_exact_and_round_trip`. What only this test can see is + // that the Darwin Core alias is the *sole* exception, so it closes `from_uri` over the + // alias family exiv2 generates — `XmpProperties::registerNs` appends `/` to a URI ending in + // neither `/` nor `#`, so an alias is a registry URI with a trailing slash added or + // removed. A second, undocumented read alias drawn from that family fails here and passes + // every other test in the crate. for &ns in WellKnownNs::ALL { - assert_eq!( - WellKnownNs::from_uri(ns.uri()).map(WellKnownNs::uri), - Some(ns.uri()), - "{ns:?} must hand back the URI it was resolved by" - ); + let slashed = format!("{}/", ns.uri()); + if ns == WellKnownNs::DarwinCore { + assert_eq!(slashed, DWC_URI_TRAILING_SLASH, "the one documented alias"); + } else { + assert_eq!( + WellKnownNs::from_uri(&slashed), + None, + "{ns:?}: no schema but Darwin Core answers to its slashed URI" + ); + } + if let Some(unslashed) = ns.uri().strip_suffix(['/', '#']) { + assert_eq!( + WellKnownNs::from_uri(unslashed), + None, + "{ns:?}: the alias runs one way only — a URI is not recognised unterminated" + ); + } } assert_eq!( WellKnownNs::from_uri(DWC_URI_TRAILING_SLASH).map(WellKnownNs::uri), From 9909bb39eedf1951b7e07fb19847c0f99908ec75 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 01:05:21 -0400 Subject: [PATCH 9/9] docs(xmp): record that a catenated sidecar keeps only its first packet MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Part 3 bullet these docs quote was cut short. It reads "as though it were embedded and then had the XMP packets extracted **and catenated by a postprocessor**" — and the dropped phrase is the one that matters, because it says a conforming producer may hand over a .xmp file holding several packets end to end. XmpSidecar::read does not read one. XmpPacket::scan ends the body at the next ` packets end to end. `write` emits exactly one; `read` takes the first and silently +discards the rest, so two `write` outputs concatenated read back as the first alone, with no error +(Adobe XMPCore rejects those bytes). Whether to reject, merge or keep that is open as issue +[#562](https://github.com/visualcommons/gamut/issues/562). ## Scope diff --git a/crates/gamut-xmp/STATUS.md b/crates/gamut-xmp/STATUS.md index 6797128a..30c120b8 100644 --- a/crates/gamut-xmp/STATUS.md +++ b/crates/gamut-xmp/STATUS.md @@ -64,22 +64,33 @@ placement, and array/struct nesting so output is stable, diffable, and round-tri **should tolerate** an x:xmpmeta element in any input" — permission and tolerance, never a requirement; the vendored Part 3's "External storage of metadata" bullets (a complete, well-formed XML document with the leading XML declaration, the `.xmp` extension, `application/rdf+xml`, - written "as though it were embedded and then had the XMP packets extracted") never mention the - element, and the string `xmpmeta` occurs in Part 3 exactly once, inside an SVG example. So `read` - rejects a sidecar the specification permits. What §7.3.3 supplies is the element's *purpose* — - identifying XMP inside general XML text — and a standalone `.xmp` file is that text, so gamut takes - the element as the marker rather than reading any RDF document beside an image as XMP; exiv2 draws - its own, different line (`isXmpType` keys on `` packets end to end. `XmpSidecar::read` reads the first and silently + discards the rest — `XmpPacket::scan` ends the body at the *next* `.xmp` next to it. Part 3 asks that such a file be //! "a complete, well-formed XML document, including the leading XML declaration", written "as though -//! it were embedded and then had the XMP packets extracted", with the `.xmp` extension and the same -//! base name as the image; applications find it by looking in the image's directory. +//! it were embedded and then had the XMP packets extracted and catenated by a postprocessor", with +//! the `.xmp` extension and the same base name as the image; applications find it by looking in the +//! image's directory. //! //! [`XmpSidecar::read`] and [`XmpSidecar::write`] are the bytes-in / bytes-out pair. This crate has //! no filesystem API and the naming convention is documented, not enforced: the caller owns the //! path, exactly as the format crates own the embedded packet's location in their containers. +//! +//! **One packet, not a catenation.** That "catenated by a postprocessor" phrase describes a file +//! holding *several* `` packets end to end — a shape [`XmpSidecar::write`] never +//! produces (it emits exactly one) and [`XmpSidecar::read`] does not read: it takes the first +//! packet and silently discards the rest. See [`XmpSidecar::read`] for what that costs the caller, +//! and issue [#562](https://github.com/visualcommons/gamut/issues/562) for the open decision. use quick_xml::NsReader; use quick_xml::events::Event; @@ -62,6 +69,20 @@ impl XmpSidecar { /// (namespace `adobe:ns:meta/`, Part 1 §7.3.3). Trailing padding inside the packet wrapper is /// ignored. /// + /// # A catenated file loses everything after its first packet + /// + /// Part 3 asks that external metadata be written "as though it were embedded and then had the + /// XMP packets extracted **and catenated by a postprocessor**", so a conforming producer may + /// hand over a `.xmp` file holding several `` packets end to end. This reads the + /// first one and **silently discards the rest**: no error, and no way to tell the result from + /// a genuine single-packet file. Two [`XmpSidecar::write`] outputs concatenated read back as + /// the properties of the first alone. Adobe XMPCore rejects those same bytes. + /// + /// Read such a file only if you know it holds one packet, or split it on `