feat(xmp): exiv2-parity schema registry and XMP sidecar files

crates/gamut-xmp/README.md

@@ -56,6 +56,28 @@ a custom schema. `WellKnownNs` supplies the standard schema URIs and prefixes so
 hand-write them. For in-place editing, `XmpPacket::scan` exposes the envelope (writability,
 padding) and `XmpPacket::parse` the graph — `from_packet` is exactly that composition.
 
+**Sidecars.** `XmpSidecar::write` / `XmpSidecar::read` are the bytes of a standalone `.xmp` file —
+the interchange RAW workflows use, since a camera's raw format is not extensible (Part 3, "External
+storage of metadata"). A sidecar is the packet as embedded, so `read` accepts everything
+`from_packet` does but **requires the `x:xmpmeta` document element** — a gamut rule that is stricter
+than the specification, not derived from it. Part 1 §7.3.3 calls the element optional and asks only
+that a processor tolerate one, and Part 3's external-storage bullets never mention it, so a
+wrapper-less `.xmp` file is spec-conformant and `read` rejects it anyway. What §7.3.3 supplies is
+the element's purpose — identifying XMP inside general XML text, which is what a standalone file is
+— so gamut takes it as the marker. (exiv2 draws its own line, accepting a file starting with
+`<?xpacket` *or* `<x:xmpmeta`. A caller who wants that latitude uses `XmpMeta::from_packet`, and the
+rejection is its own `XmpError::MissingXmpMeta` rather than a claim that the wrapper-less form is
+prohibited.) `write` emits the XML declaration Part 3 asks for, then a read-only, unpadded packet in
+canonical form, so two sidecars of the same graph are byte-identical. The crate has no filesystem
+API; the naming convention — `photo.xmp` beside `photo.dng`, looked up in the image's directory — is
+documented, not enforced, and the caller owns the path. One packet, not a catenation: Part 3's
+bullet asks for the file to be written "as though it were embedded and then had the XMP packets
+extracted **and catenated by a postprocessor**", so a conforming producer may emit several
+`<?xpacket?>` packets end to end. `write` emits exactly one; `read` takes the first and silently
+discards the rest, so two `write` outputs concatenated read back as the first alone, with no error
+(Adobe XMPCore rejects those bytes). Whether to reject, merge or keep that is open as issue
+[#562](https://github.com/visualcommons/gamut/issues/562).
+
 ## Scope
 
 - **Part 1 (data model + serialization): both directions.** The permissive reader accepts the
@@ -72,10 +94,17 @@ padding) and `XmpPacket::parse` the graph — `from_packet` is exactly that comp
   with the reference engine. Per-property and per-item `xml:lang` are fully supported.
 - **Part 2 (standard schemas) is a namespace registry** (`WellKnownNs`), not per-property
   validation: values are uninterpreted text in the model, as the wire format allows. The registry
-  also carries the external schemas image-metadata standards layer on XMP — `dcterms` (DCMI
-  Metadata Terms), which C2PA uses for `dcterms:provenance`, the URL of an *external* manifest
-  store (C2PA 2.4 §11.5). gamut-xmp registers the namespace; reading that property as a
-  provenance signal is [`gamut-metadata`](../gamut-metadata)'s job.
+  covers every schema exiv2 documents (<https://exiv2.org/metadata.html>) — the Adobe Parts 1–2
+  schemas plus the external ones image-metadata standards and deployed tools layer on XMP:
+  `dcterms` (DCMI Metadata Terms, which C2PA uses for `dcterms:provenance`, the URL of an
+  *external* manifest store — C2PA 2.4 §11.5; reading that property as a provenance signal is
+  [`gamut-metadata`](../gamut-metadata)'s job), `exifEX`, `aux`, `plus`, `mwg-rs`, `mwg-kw`,
+  `GPano`, `lr`, `MicrosoftPhoto`, `digiKam`, `acdsee`, `crss` and `dwc`. Registering a schema
+  fixes the prefix its properties serialize under — the one Adobe XMPCore keys them by — and
+  nothing more; each non-Adobe URI is cited on its variant, and every one is vouched for by the
+  reference engine in `tests/oracle.rs`. `exifEX` is registered under `http://cipa.jp/exif/1.0/`
+  (CIPA DC-010, the URI the reference engine and deployed writers use); the Exif 3.0 text's
+  annotation examples bind the same prefix to `…/exif/2.32/`, which is not registered.
 - **Part 3 (storage in files) belongs to the format crates by design.** This crate supplies what
   they need — wrapper-optional parse, bare-body serialization (`to_rdf` / `serialize_body`), and
   the writability/padding envelope for in-place editing. Locating packets inside JPEG/TIFF/PNG
@@ -87,8 +116,9 @@ padding) and `XmpPacket::parse` the graph — `from_packet` is exactly that comp
 ## Status
 
 **Production-ready v1** (issue #189). Implemented: parser + canonical serializer for the full XMP
-data model (simple / URI / structured / `Bag`·`Seq`·`Alt`, qualifiers, language alternatives) and
-the `<?xpacket?>` wrapper. See [STATUS.md](STATUS.md).
+data model (simple / URI / structured / `Bag`·`Seq`·`Alt`, qualifiers, language alternatives), the
+`<?xpacket?>` wrapper, a 30-schema registry at parity with exiv2's documented set, and `.xmp`
+sidecar read/write (issue #421). See [STATUS.md](STATUS.md).
 
 ## Migrating from 1.x
 
@@ -106,8 +136,18 @@ changed — every existing variant, URI, prefix and method keeps its meaning.
   non-semantic (`tests/roundtrip.rs`).
 - **Differential oracle** against exiv2's bundled **Adobe XMPCore**: gamut's packets validate and
   round-trip through the reference engine, every `WellKnownNs` URI is vouched for by its schema
-  registry, and the default-`xml:lang` posture is pinned to parity (`tests/oracle.rs`; needs the
-  `third_party/exiv2` + `third_party/expat` submodules and a C++ toolchain).
+  registry (one test per exiv2-parity schema reads a documented property back by its
+  `Xmp.<prefix>.<name>` key), a sidecar gamut writes is read by the engine and the engine's own
+  serialization is read as a sidecar, and the default-`xml:lang` posture is pinned to parity
+  (`tests/oracle.rs`; needs the `third_party/exiv2` + `third_party/expat` submodules and a C++
+  toolchain). One oracle normalization is pinned as such: exiv2 appends `/` to a namespace URI
+  ending in neither `/` nor `#` when it registers it with XMPCore, so the engine re-serializes
+  Darwin Core as `http://rs.tdwg.org/dwc/index.htm/`; gamut writes the URI exiv2 documents and
+  additionally maps the slashed form to the `dwc` **prefix** on read (`DWC_URI_TRAILING_SLASH`), so
+  a packet the engine wrote re-serializes under `dwc` rather than a synthesized prefix. It does not
+  rewrite the URI — the graph keeps the slashed URI its packet declared — so a caller resolving such
+  a property by URI passes `DWC_URI_TRAILING_SLASH`, not `WellKnownNs::DarwinCore.uri()`
+  (issue #547).
 - **Mutation-clean** — `cargo mutants` passes with zero gamut-xmp exclusions in
   `.cargo/mutants.toml`.
 - **No benches, intentionally** — the crate has no performance contract; packets are a few KB.

crates/gamut-xmp/STATUS.md

@@ -30,6 +30,67 @@ placement, and array/struct nesting so output is stable, diffable, and round-tri
   the namespace only — what the property *means* is read by `gamut-metadata`, consistent with the
   registry-not-validator posture below — and `tests/oracle.rs` pins that XMPCore reads the property
   back under the `Xmp.dcterms.provenance` key its own registry defines.
+- **Schema breadth = exiv2's documented set (issue #421).** `WellKnownNs` holds 30 entries: the
+  Adobe Parts 1–2 schemas and structure types, `dcterms`, and the twelve further schemas exiv2
+  documents (<https://exiv2.org/metadata.html>): `exifEX`, `aux`, `plus`, `mwg-rs`, `mwg-kw`,
+  `GPano`, `lr`, `MicrosoftPhoto`, `digiKam`, `acdsee`, `crss`, `dwc`. Each URI is taken from the
+  schema owner's specification where one is published and cited on the variant; the vendored
+  reference for all twelve is exiv2's own registry (`third_party/exiv2/src/properties.cpp`,
+  `xmpNsInfo`), and `tests/oracle.rs` reads a documented property of each back from XMPCore by its
+  `Xmp.<prefix>.<name>` key, which fails for a wrong URI *or* a wrong prefix. The registry stays a
+  registry: nothing about a value is interpreted. Two divergences are recorded rather than hidden:
+  `exifEX` is `http://cipa.jp/exif/1.0/` (the URI exiv2 and XMPCore bind) although the vendored Exif
+  3.0 text's annotation examples (Annex J.2–J.3) bind the prefix to `…/exif/2.32/` — CIPA DC-010
+  itself is not vendored, so this one entry rests on the oracle rather than on a specification under
+  `references/`, tracked in issue #516; and
+  exiv2 appends `/` to a URI ending in neither `/` nor `#` when registering it with XMPCore
+  (`XmpProperties::registerNs`), so the engine re-serializes `dwc` as `…/index.htm/` while gamut
+  writes the `…/index.htm` exiv2 documents — pinned in `tests/oracle.rs` as an oracle
+  normalization. Because that normalization is what XMPCore *emits*, `WellKnownNs::from_uri` also
+  recognises `http://rs.tdwg.org/dwc/index.htm/` as a read-only alias (`DWC_URI_TRAILING_SLASH`),
+  so a graph parsed from an exiv2-written packet or sidecar re-serializes under `dwc` rather than a
+  synthesized prefix; `uri()` still emits the unslashed URI, so gamut's own bytes are unchanged and
+  the alias is not an `ALL` entry. The alias buys a **prefix**, not a URI: reading does not
+  canonicalize, so such a graph keeps the slashed URI its packet declared and a caller must look the
+  property up under `DWC_URI_TRAILING_SLASH` — `get_text(WellKnownNs::DarwinCore.uri(), …)` returns
+  `None` for it. `from_uri(u).map(uri) == Some(u)` therefore holds for every registry entry and not
+  for the alias, pinned as such in `namespace.rs`. Canonicalizing on read — which would trade byte
+  fidelity for that symmetry — is filed as issue #547 for when a consumer needs it.
+- **Sidecars require `x:xmpmeta` (issue #421).** `XmpSidecar::read` accepts everything
+  `XmpMeta::from_packet` does — XML declaration, BOM, wrapper or bare — but rejects a document
+  whose element is not `x:xmpmeta` with `XmpError::MissingXmpMeta` naming the element found. **This
+  is stricter than the specification, not derived from it.** The vendored Part 1 §7.3.3 reads "An
+  optional x:xmpmeta element **may** be placed around the rdf:RDF element" and "An XMP processor
+  **should tolerate** an x:xmpmeta element in any input" — permission and tolerance, never a
+  requirement; the vendored Part 3's "External storage of metadata" bullets (a complete, well-formed
+  XML document with the leading XML declaration, the `.xmp` extension, `application/rdf+xml`,
+  written "as though it were embedded and then had the XMP packets extracted and catenated by a
+  postprocessor") never mention the element, and the string `xmpmeta` occurs in Part 3 exactly
+  once, inside an SVG example. So `read` rejects a sidecar the specification permits. What §7.3.3
+  supplies is the element's *purpose* — identifying XMP inside general XML text — and a standalone
+  `.xmp` file is that text, so gamut takes the element as the marker rather than reading any RDF
+  document beside an image as XMP; exiv2 draws its own, different line (`isXmpType` keys on
+  `<?xpacket` **or** `<x:xmpmeta`). `XmpSidecar::write` emits the XML declaration Part 3 asks for,
+  then a read-only (`end="r"`), unpadded packet wrapping the canonical body in `x:xmpmeta` —
+  byte-stable per graph. No filesystem API and no enforced file name: the `.xmp`-beside-the-image
+  convention is documented for the caller. The missing wrapper has its own error,
+  `XmpError::MissingXmpMeta`, naming the element found: the wrapper-less form is *permitted* by
+  §7.3.3 and is what `XmpWriter::wrap_xmpmeta(false)` emits, so reporting it as a prohibited
+  construct would assert a prohibition the spec does not make. exiv2 is more permissive still — its
+  sniffer (`isXmpType`) accepts a `.xmp` file starting with `<?xpacket` **or** `<x:xmpmeta` — so a
+  file exiv2 reads as a sidecar and gamut rejects is expected; the caller reads those bytes with
+  `XmpMeta::from_packet`.
+- **A catenated sidecar is truncated to its first packet (issue #562).** Part 3's bullet asks that
+  external metadata be written "as though it were embedded and then had the XMP packets extracted
+  and catenated by a postprocessor", so a conforming producer may emit a `.xmp` file holding
+  several `<?xpacket?>` packets end to end. `XmpSidecar::read` reads the first and silently
+  discards the rest — `XmpPacket::scan` ends the body at the *next* `<?xpacket` instruction, which
+  in a catenated file is the first packet's own trailer. Two `XmpSidecar::write` outputs
+  concatenated read back as the properties of the first alone, with no error; Adobe XMPCore rejects
+  those same bytes. `XmpSidecar::write` emits exactly one packet, so gamut never produces the
+  shape. The behaviour predates issue #421 and is documented rather than changed: rejecting,
+  merging every packet, or keeping the truncation are three different products (a merge needs a
+  conflict rule Part 3 does not supply), and #562 holds that decision.
 
 ## Phases
 
@@ -42,6 +103,7 @@ placement, and array/struct nesting so output is stable, diffable, and round-tri
 | P5 | Part 1 §7 | **Keystone** — canonical RDF/XML serialization + packet emit (writable padding) | ✅ done |
 | P6 | — | exiv2 differential conformance gate | ✅ done |
 | P7 | Parts 1–3 | **v1 stabilization** (issue #189) — API finalization (`XmpPacket::parse` composition, `XmpWriter::with_namespace` prefix registration, model conveniences), conformance audit (control-character escaping fix, trailer `end=` matching, edge-case pins), gamut-iptc dogfood migration, docs | ✅ done |
+| P8 | Part 2; Part 3 "External storage" | **Breadth** (issue #421) — registry at parity with exiv2's documented schemas (17 → 30 entries), `.xmp` sidecar read/write, per-schema and sidecar oracle tests | ✅ done |
 
 ## Intentional skips (audited for v1)
 
@@ -61,6 +123,13 @@ deliberately:
 - **Part 3 per-container embedding and JPEG ExtendedXMP:** owned by the format crates; this crate
   supplies wrapper-optional parse, bare-body serialization, and the writable/padding envelope.
 - **Per-schema value validation (Part 2):** values are uninterpreted text; `WellKnownNs` is a
-  namespace registry, not a validator.
+  namespace registry, not a validator — still true after the exiv2-parity additions (issue #421).
+- **Sidecar file naming and I/O (Part 3):** `XmpSidecar` is bytes-in / bytes-out; locating
+  `photo.xmp` beside `photo.dng` is the caller's, as embedding is the format crates'.
+- **Decode limits (crate-wide, not sidecar-specific):** `XmpSidecar::read` takes an unbounded
+  `&[u8]` and parses the document before checking for the wrapper. `XmpMeta::from_packet` has
+  identical exposure on the same bytes, so the sidecar API (issue #421) adds no hostile-input class
+  the crate did not already have; whether gamut-xmp should grow a decode-limit convention like
+  `gamut-png`'s is a question for the crate as a whole and is deliberately left open.
 - **Deferred additive API** (post-1.0, no consumer today): an opt-in `XmpMeta::validate()`, and
   nested-structure field lookup.

crates/gamut-xmp/src/error.rs

@@ -44,6 +44,16 @@ pub enum XmpError {
     /// requires the language tags to be unique. The string is the duplicated tag.
     #[error("XMP: duplicate xml:lang '{0}' in an alternative array")]
     DuplicateLang(String),
+
+    /// A sidecar file's document element was not `x:xmpmeta`; the string names the element found.
+    ///
+    /// Raised only by [`crate::XmpSidecar::read`]. The wrapper is **optional** in an embedded
+    /// packet (Part 1 §7.3.3, and [`crate::XmpWriter::wrap_xmpmeta`] can omit it), so this is not
+    /// a prohibited construct — it is the one thing a standalone `.xmp` file needs beyond a packet
+    /// body, because §7.3.3 gives `x:xmpmeta` exactly the job of identifying XMP inside general
+    /// XML text. A bare `rdf:RDF` document is valid XMP for [`crate::XmpMeta::from_packet`].
+    #[error("XMP: sidecar document element is <{0}>, not x:xmpmeta")]
+    MissingXmpMeta(String),
 }
 
 /// A specialized [`Result`](core::result::Result) for the XMP read path.
@@ -100,5 +110,11 @@ mod tests {
             gamut_core::Error::from(XmpError::DuplicateLang("en".into())),
             gamut_core::Error::Context(ref diagnostic) if diagnostic.source_error().kind() == gamut_core::ErrorKind::InvalidInput
         ));
+        // A sidecar without its wrapper is bad input, not an unimplemented feature: the caller
+        // fixes it by handing the bytes to `XmpMeta::from_packet` or wrapping them.
+        assert!(matches!(
+            gamut_core::Error::from(XmpError::MissingXmpMeta("RDF".into())),
+            gamut_core::Error::Context(ref diagnostic) if diagnostic.source_error().kind() == gamut_core::ErrorKind::InvalidInput
+        ));
     }
 }

crates/gamut-xmp/src/lib.rs

@@ -52,19 +52,26 @@
 //!   the public API (errors are surfaced via [`XmpError`]), so it can be changed without a breaking
 //!   change.
 //! - **Memory-safe on hostile input.** `#![forbid(unsafe_code)]` — XMP is XML from untrusted files.
+//! - **Sidecars are bytes too.** [`XmpSidecar`] reads and writes the standalone `.xmp` file RAW
+//!   workflows keep beside an image (Part 3, "External storage of metadata"); the crate has no
+//!   filesystem API, so the caller owns the path.
 #![forbid(unsafe_code)]
 
 pub mod error;
 pub mod model;
 pub mod namespace;
 pub mod packet;
+pub mod sidecar;
 pub mod writer;
 // The reader has no configuration — `XmpMeta::from_packet` is the entry point and auto-detects the
 // wrapper, encoding, and input form — so the module carries only that `impl` and stays private.
 mod reader;
 
 pub use error::{Result, XmpError};
 pub use model::{XmpArray, XmpItem, XmpMeta, XmpProperty, XmpValue};
-pub use namespace::{Namespace, RDF_NAMESPACE, WellKnownNs, XML_NAMESPACE, XMPMETA_NAMESPACE};
+pub use namespace::{
+    DWC_URI_TRAILING_SLASH, Namespace, RDF_NAMESPACE, WellKnownNs, XML_NAMESPACE, XMPMETA_NAMESPACE,
+};
 pub use packet::XmpPacket;
+pub use sidecar::XmpSidecar;
 pub use writer::XmpWriter;