feat(tiff): build a metadata seam, then carry the C2PA manifest store - #520
Open
justin13888 wants to merge 45 commits into
Open
justin13888 wants to merge 45 commits into
justin13888 wants to merge 45 commits into
Conversation
`gamut-tiff` had no metadata read or write surface at all: `tags.rs` named XMP (700), IPTC/NAA (33723), ICC (34675) and the Exif/GPS/Interop pointers only so `deconstruct` would not flag them unknown, and a caller wanting any of them had to drop to the re-exported `gamut_ifd` read/write spine. Add the seam: `TiffMetadata` is a `#[non_exhaustive]` struct of optional payloads that `TiffEncoder::with_metadata` writes into IFD 0 — on the strip, tile and multi-page paths alike — and `TiffDecoder::metadata` reads back. XMP, IPTC-IIM and ICC are opaque bytes carried verbatim in both directions, the raw blocks the workspace's metadata facade consumes; the `ExifIFD` is handed over as `gamut_ifd::Ifd`, because it *is* a directory this crate has already parsed and a byte blob would force every caller to re-parse it. Nothing is validated or completed, so what the caller supplies is what the file gets. The blocks are out-of-line values, so they displace the pixel data; libtiff reads a gamut TIFF carrying all four back pixel-exact.
…ules The manifest store joins the seam as a fifth carrier, but it is the one with a placement rule of its own: C2PA 2.4 §A.3.6 puts its entry in the last IFD of the main chain and its bytes at the end of the file, and §18.5.5 makes a signer exclude two disjoint ranges — the store, and the `count` field of its entry — from the `c2pa.hash.data` binding that §18.7.3.3 leaves as a TIFF asset's only hard binding. None of that is restated here. `gamut_ifd::c2pa` owns the tag, the placement, the reserve-then-append relocation and the locator, and `gamut-dng` already calls it; this crate wires the same helper to its three encode paths, so the two formats cannot drift. `TiffEncoder::with_c2pa_reserved` writes a zero-filled reservation for an external signer to overwrite in place, `encode_with_report` returns the exclusion ranges, and `c2pa_exclusions` recovers them from any TIFF's bytes — including one written through `encode_palette8` or `encode_pages_rgb8`, which the object-safe `EncodeImage` seam cannot report through. The store is opaque and never byte-swapped: the header's `ByteOrder` does not govern it (§A.3.6), which the tests pin with an asymmetric payload in a big-endian file. Tag 52545 joins `is_known_tag`, so the strict deconstruct claims the store as its entry's value span rather than flagging a private tag and an unaccounted trailer, and libtiff still decodes such a file pixel-exact.
STATUS.md gains the semver-minor ledger entry for both halves — what the seam carries, why every payload but the Exif sub-IFD stays opaque bytes, and which clauses of C2PA 2.4 are `gamut_ifd::c2pa`'s rather than this crate's — plus the new public items under the v1 surface's freeze list and a deferred row for the typed `gamut-metadata` wiring this crate deliberately does not do. README.md gains the matching Status bullet.
The tuple pattern `if let (Some((last, _)), true) = (images.last_mut(), store.is_some())` said the right thing awkwardly; a let-chain puts the condition first and the binding second, in reading order. No behaviour change.
`TiffDecoder::metadata` follows the `ExifIFD` pointer into a second directory and `c2pa_exclusions` walks the IFD chain to its end to read an offset/count pair — offset-driven reads of untrusted bytes on paths `decode_page` never takes, so the existing byte-flip corpus could not reach them. `byte_flip_fuzz` now takes the entry point under test as a closure, and both new entry points get the truncation sweep and the 5000-mutation corpus. Separately, BigTIFF is the one place the container variant changes what §18.5.5 names: the entry's count field widens from 4 bytes to 8. Pinned, together with the store still landing verbatim at the end of the file.
…' into feat/446-tiff-metadata-seam
…' into feat/446-tiff-metadata-seam
Three defects found in review, each one the seam contradicting something it already claimed. A decoded Exif sub-IFD carried a dangling pointer. `read_metadata` asked `read_tree` for `ExifIFD` alone, and `read_tree` re-parses only the tags it is given, so an `InteroperabilityIFD` (40965) *inside* the Exif directory — near-universal in camera EXIF — came back as a raw `Long` holding the source file's absolute offset, which `apply` then wrote verbatim into a file laid out at different offsets. Decoding and re-encoding produced a file the crate's own judge rejects: not fully classified, with a `severity: Error` "sub-IFD could not be parsed", breaking gamut-tiff's v1 zero-tolerance byte accounting on a file gamut-tiff itself wrote. Read with `gamut_ifd::tags::STANDARD_POINTER_TAGS` instead — the list the repository already provides and `gamut-dng`'s rewrite path already uses — so a nested pointer directory becomes a child the writer re-points. `metadata()` and `c2pa_exclusions()` disagreed about what a manifest store is, while the docstring claimed they agreed. The reader accepted any `UNDEFINED` value under 52545; `locate` additionally rejects one below `MIN_STORE_LEN` or a duplicated entry. A four-byte entry therefore decoded to `Some(...)` that the encoder then refused — exactly the decode → encode trap `locate` reports absence to avoid. The reader now gates on the locator, as `gamut-dng` does. `with_c2pa_reserved`'s documented minimum was wrong for BigTIFF. `append_store` refuses a store that fits inline, so the true minimum is the variant's inline threshold plus one: 8 in classic TIFF but 9 in BigTIFF. A BigTIFF reservation of 8 compressed the whole image before failing from `gamut-ifd`, despite the `# Errors` promise to catch length "before any pixel work". `c2pa_store` now gates on `min_store_len()` and both doc claims are corrected. Also states three contracts that were decided but unwritten: the Exif directory's entries are carried unchanged while its *ordering* is normalised (ascending tag, duplicates collapsed, a child's next-IFD pointer ignored) rather than being byte-identical; the blocks live in IFD 0 only, so a reader decoding one page of a multi-page document must look there for them; and `metadata()` can fail on a file `decode_image` decodes happily, because reporting "no EXIF" for a directory the file declares is silent loss a caller cannot detect.
STATUS, README and the encoder docs all say a palette file carries a store and reports it through `c2pa_exclusions`. Nothing called `encode_palette8` with `with_metadata`, so that claim held by luck: it works today, and nothing would have failed if it stopped. `encode_palette8` is an inherent method rather than an `EncodeImage` impl — it needs a separate colour table — so `encode_with_report` cannot reach it, which is precisely why the locator is the documented route and why it needed its own test.
STATUS.md and README.md said the metadata was carried "verbatim in both directions". That is exact for the byte payloads and approximate for the Exif sub-IFD, which is a directory model rather than a byte range: `gamut_ifd::Ifd` sorts fields by ascending tag, collapses a duplicated tag to its last occurrence, and ignores a child directory's next-IFD pointer. Say "entries carried unchanged, ordering normalised" and name the three, so nobody uses a re-encode to argue a file is unmodified. Also record what the reader now resolves and where the blocks live: every standard pointer tag rather than `ExifIFD` alone, and IFD 0 only — with the cost of that choice stated, since a reader decoding page 3 of a multi-page document alone sees none of the blocks.
… read Resolving every standard pointer tag fixed a dangling `InteroperabilityIFD` but over-reached: `read_tree` takes one flat list and applies it at every node, so `SubIFDs` (330) and `GPSInfo` (34853) were followed at IFD 0 too. Neither feeds any field of `TiffMetadata` and neither is re-encoded by `apply`, which writes into a directory the encoder builds fresh — so following them could only add failure modes, and did. On a well-formed single-strip RGB file carrying XMP, a dangling `SubIFDs` offset (and likewise a dangling `GPSInfo`) made `metadata()` return `Err(TIFF: read out of bounds)` while `decode_image` succeeded, putting XMP, ICC, IPTC and C2PA all out of reach because of a thumbnail pointer nobody asked for. From the same code, two pages of a multi-page document whose `SubIFDs` share one thumbnail directory failed with "sub-IFD pointer loop", because `visited` spans the whole chain. Scope the list to what the struct actually returns: `ExifIFD`, because that directory is handed to the caller and may be written back, and `InteroperabilityIFD`, the one standard pointer that occurs inside it. A broken Exif pointer is still an error — its content *is* returned, so silence there would be data loss — and both regressed cases are pinned.
`with_c2pa_reserved` promises the refusal is raised "before any pixel work", and `c2pa_store()?` sits in `encode_packed`, the chokepoint every layout funnels through. Two paths reach it late: `encode_16bit` first allocates and fills a byte-order-corrected copy of the samples, and `EncodeImage<Bilevel>` first runs a whole bit-packing pass. Both are O(width x height), so the documented claim was false wherever it mattered most. Add `check_c2pa` at the head of those two paths. The claim is now true on every path, and a test asserts it per path shape — a mutant deleting either call leaves every output byte-identical, so nothing else would have caught it.
Two overclaims, both found by reading the docs against the code. The C2PA reader's rationale said agreeing with `locate` means a store read can be written back. It does not, and a reader cannot promise that: it does not know which container the caller will write. `locate` accepts a store of exactly `MIN_STORE_LEN`, while writing BigTIFF needs 9 bytes, since 8 would pack into the entry's value word instead of being placed out of line. Name the case, and say the input is degenerate — 8 bytes is a JUMBF header with no content — so the break's real scope is visible. `TiffMetadata::exif` said pointer tags inside the directory come back parsed "never as raw offsets". Only the standard pointer tags are recognised as pointers, so a vendor private tag holding a `LONG` offset is carried through and re-encoded verbatim — the same defect as a stale Interop pointer, for a narrower input class, and one `deconstruct` cannot grade because it does not know the tag is a pointer either. STATUS.md was already precise; make the API doc a caller actually reads equally precise, and say plainly that a round trip through this field does not prove a rewrite pointer-safe.
The pre-pixel C2PA check was a call whose value was thrown away, so removing its body changed no output on any input: encode_packed resolved the store again and refused with the same message. Nothing could tell whether it ran. encode_packed now takes the already-resolved store as a parameter, the shape encode_pages_rgb8 and encode_tiled already use. Every entry point must therefore resolve it — and take its refusal — before the pass that produces the packed bytes, so the ordering is held by the signature rather than by a check that could be deleted unnoticed, and a reservation is no longer built twice. The path test now reads the refusal's message under a tile size the layout stage also rejects, which is what separates 'resolved before the layout stage' from 'resolved inside it'; asserting is_err cannot.
`read_metadata` handed one flat `POINTER_TAGS` list to `gamut_ifd::read_tree`, which resolves it at every node of every page, while the blocks and the Exif sub-IFD come from IFD 0 alone. So a pointer on a page the reader throws away could still fail the whole call — the very thing `POINTER_TAGS` exists to prevent, moved from a tag this reader ignores to a page it ignores. Two files showed it, both of which `decode_image` reads happily. A two-page classic TIFF whose page 1 carried `ExifIFD = Long(0xFFFF_FF00)` answered `metadata()` with "TIFF: read out of bounds"; two pages whose `ExifIFD` entries named one directory answered "TIFF: sub-IFD pointer loop", because `read_tree` walks a whole file with a single `visited` set and a second page naming a directory the first named looks exactly like a cycle. Both are now read, and both are pinned. The chain is read with `gamut_ifd::read`, which follows no pointer at all, and the list is then resolved over IFD 0's subtree by hand through `read_ifd_at` — the per-pointer control `read_tree`'s own documentation points at. It stays one flat list applied at every node of that subtree, so IFD 0's behaviour is unchanged, including the one over-reach `POINTER_TAGS` already documented: `InteroperabilityIFD` is followed at IFD 0 too, where a conformant file never puts it. Narrowing that further would take a per-node list, which is a `gamut-ifd` surface rather than a scoping decision this crate makes. The C2PA manifest store is unaffected and is asserted so: §A.3.6 puts its entry in the last IFD of the main chain, and that entry carries the store's bytes rather than an offset, so the store is reached on a page whose pointers are never resolved. The new two-page test reads its store from the same page that carries the dangling pointer. Also asserts the message rather than `is_err` where a dangling `ExifIFD` must still be an error, so a refusal that named some other pointer could not pass for it.
`encode_packed` takes the resolved C2PA store as a parameter, so no entry point can forget to resolve one — but that is an obligation to pass *a* store, not the one `c2pa_store()` returned. Replacing the argument with `None` at the bilevel call site left the suite at 196 passed: `tests/c2pa.rs` covered the strip, tile, palette, multi-page and BigTIFF paths and never encoded 16-bit, RGBA or bilevel, and a dropped store is silent — the file is well formed and only `c2pa_exclusions` disagrees. Those three are pinned now, and they are the three that do pixel work of their own before the call: a byte-order-corrected copy, an extra sample, a whole bit-packing pass. Dropping the store at any of the three call sites now fails. The ordering test grows the two entry points it was missing — RGBA and the palette path — so every entry point that resolves a store of its own is named there, each asserting the refusal is the C2PA one rather than the tiling one. Only `encode_pages_rgb8` stays out, and not by choice: it builds strip images directly, so there is no second refusal for the C2PA one to be told apart from.
`with_c2pa_reserved` returns `Self`, so it cannot refuse anything itself, and the length it stores went straight into `vec![0; len]` at the top of every encode. Past `isize::MAX` that panics with a capacity overflow rather than returning, and the length is a caller's number: a panic out of a library path is this crate's defect. `usize::MAX` was the shortest way to reach it. The bound is now checked with the existing minimum, before the reservation is materialised, so an unusable length costs neither the allocation nor the panic. It is the smaller of what a buffer holds and what the container describes: classic TIFF counts an `UNDEFINED` value with a 32-bit `LONG` and addresses it with a 32-bit offset, so nothing beyond `u32::MAX` could be pointed at, while BigTIFF's words are 64-bit and leave only the buffer. A supplied store is held to the same bound as a reservation, as it already was to the minimum. What stays outside this crate's reach is the allocator's: a reservation the machine has no memory for aborts, as any oversized allocation does. The same decision was taken for `AvifEncoder::with_c2pa_reserved`, so the two container crates of the C2PA epic now agree about it.
Five survivors, four of them the new code's own and each a real gap. The reservation's upper bound was a numeric comparison whose accepting side no test could assert — `>` and `>=` differ only at exactly `u32::MAX` or `isize::MAX` bytes, which is not a reservation a test can allocate. The bound is dropped for the thing it was standing in for: the reservation is now taken with `try_reserve_exact`, so a length no `Vec<u8>` can hold is a typed error at the same point, with no comparison to get wrong and no container arm to keep in step. It also retires the second bound, on the container's 32-bit count and offset words, which refused only lengths the writer already cannot lay out. `resolve_pointers` kept `read_tree`'s depth bound of sixteen, which neither side of could be asserted: a generic reader needs sixteen because it is handed arbitrary tags, and this walk follows two. The deepest tree `ExifIFD` and `InteroperabilityIFD` can legitimately reach is two levels (EXIF 2.3 §4.6.3), so the bound is two and both sides of it are pinned — the Exif → Interop round trip already in tests/metadata.rs, and an Interop directory inside an Interop directory, refused here. That also kills `depth + 1` becoming `depth * 1`, which had no observable effect while the bound was unreachable. `pointer_offsets`' 64-bit arm had no BigTIFF reader test behind it: a resolver that knew only `LONG` would leave every BigTIFF's `ExifIFD` in place as an integer and report `exif: None`. Pinned on the type as well as on the result, so the fixture cannot stop exercising the arm without saying so.
This was referenced Sep 10, 2026
`with_metadata` took any nesting a caller built while `metadata()` refuses a directory more than two levels under IFD 0, so this crate wrote a well-formed four-level Exif tree and then rejected its own file as too deep — the same shape as the round-1 finding on this branch, and a direct contradiction of the seam's contract that what the file holds is what the caller gets. The bound now applies to both sides. `TiffMetadata::check` refuses an Exif sub-IFD nesting below the `ExifIFD` -> `InteroperabilityIFD` pair EXIF 2.3 §4.6.3 allows, as a typed error taken at one chokepoint with the C2PA store's refusals, before any pixel work on every entry point. It is measured against `MAX_POINTER_DEPTH` itself rather than against a second constant, so the two cannot drift. The reader's bound was narrowed from sixteen to two in the previous commit for a reason that is not sufficient on its own: that neither side of sixteen could be asserted, so a mutant lived. What justifies two is the spec — `ExifIFD` and `InteroperabilityIFD` reach two levels and a third is out of spec — and that is what the narrowing rests on now. A mutation-assertability argument never licenses narrowing a contract; it can only ask whether the contract was stated at the right width. The error documentation said "deeper than 16 levels" a round after the bound became two, and now states the shipped one.
`resolve_pointers` remembered the offsets it had followed in a `Vec` and tested membership by scanning it. Nothing bounds how many offsets one pointer array holds — the count is a 32-bit field the file chooses — so the walk was quadratic in a number hostile input picks, on a surface the README calls hardened against hostile input. Measured in release on hand-built classic TIFFs whose single `ExifIFD` array names N distinct all-zero directories, every call returning `Ok`: 0.24 s at 0.59 MB, 0.98 s at 1.17 MB and 4.23 s at 2.34 MB — clean quadratic growth. With a `BTreeSet` the same three files answer in 5.2 ms, 13.6 ms and 26.9 ms. `insert` returning `false` *is* the loop guard, so the guard loses a line rather than gaining one. The shared walk this one restates has the same defect and is fixed separately (#578); what still has no bound at all is the breadth of a single pointer array (#579).
The previous commit dropped `max_store_len` because its comparison had an accepting side no test could allocate. One of the two bounds it carried was standing in for something real: classic TIFF counts an `UNDEFINED` value with a 32-bit `LONG`, so a store past 4 GiB cannot be described whatever else holds it. Without the check a reservation that size is allocated, zero-filled and the whole image compressed before `gamut_ifd::c2pa::append_store` refuses it — with a message about the file's 4 GiB offset limit rather than about the length the caller passed. The documentation of a refusal that still happens went with it. The bound is back, expressed as `u32::try_from` over the variant rather than as a comparison, so there is no boundary whose accepting side needs 4 GiB to assert. It is deliberately *necessary*, not sufficient: whether the store's own offset fits depends on the size of the file it lands after, which only `append_store` knows, and that refusal stays there. The `isize::MAX` reservation test now asks for BigTIFF, since classic TIFF refuses that length one check earlier for a different reason.
`a_broken_pointer_on_a_page_the_metadata_discards_does_not_fail_the_read` named one rule and asserted four things: that the read survives, that IFD 0's XMP comes back, that the last page's C2PA store comes back, and that the pixels decode. Three modules could fail it, and the store assertion killed nothing the inline `read_metadata_takes_the_store_from_the_last_ifd_of_the_chain` does not already kill. It now asserts the rule it is named for and nothing else: the call succeeds and still returns IFD 0's blocks. The fixture loses the store it no longer reads, which drops the file's last reach into the C2PA locator and the strip decoder.
`pointer_offsets` restates `gamut_ifd`'s rule for what a sub-IFD pointer value is, and the source guards its 64-bit arm with `#[cfg(feature = "bigtiff")]` while this copy does not. That reads as a dropped guard; it is not one, and it cannot be restored. `bigtiff` is `gamut-ifd`'s feature, enabled unconditionally by this crate's dependency on it and not re-exported, so the attribute here names a feature `gamut-tiff` does not have: `unexpected_cfgs` rejects it under the workspace's `-D warnings`, and were it accepted the arm would vanish and every BigTIFF's `ExifIFD` would read back as a plain integer. Verified by adding the attribute and building: one `unexpected cfg condition value: bigtiff` warning, which the lint gate takes as an error.
CI's incremental mutation shard timed out on `delete !` in `c2pa_store` — the guard added one commit earlier, inverted into its own opposite. Deleting the `!` lets the test's oversized reservation reach `zeroed`, whose `try_reserve_exact` the runner's allocator granted, and the 4 GiB zero-fill that followed ran past the 60 s test timeout. The same mutant was CAUGHT locally, on a machine that zero-fills 4 GiB inside a minute: the mutant's fate depended on the hardware, which is not a gate. Two changes, neither of them a weakened check. The condition is spelled as the refusing one, `uncountable`, so there is no `!` to delete. And the test asks for `usize::MAX` rather than `u32::MAX + 1`, so a length that gets past any future guard meets a reservation no allocator can satisfy and fails instantly instead of allocating. Which of the two bounds answers first is now what separates this test from the BigTIFF one, since both ask for the same length.
Round 1 followed `STANDARD_POINTER_TAGS` everywhere and round 3 narrowed the list to `ExifIFD` and `InteroperabilityIFD` so that a dangling `SubIFDs` offset on a page could not hide the blocks. The narrowing was right about the page and wrong about the directory: it also stopped resolving `SubIFDs` and `GPSInfo` *inside* the Exif directory, which is the one directory the seam hands back. Executed at the previous head, a `GPSInfo` group under `ExifIFD` encodes with no anomalies, comes back from `metadata()` as `Long([212])` — a raw absolute offset into the source file — and re-encoding that value makes this crate's own `deconstruct` report a `Severity::Error` structural anomaly on a file that is no longer fully classified. The two levels answer opposite questions, so the tag list is now per-level rather than one flat list. At IFD 0 a followed pointer can only add a failure mode, so only `ExifIFD` is followed there and round 3's finding stays closed. Below it an *un*followed pointer becomes a stale offset in a directory the caller is handed, so all four standard pointer tags are followed. The cost is stated where the list is: inside the subtree an unreadable target under any of the four now fails the read, which is the trade `ExifIFD` itself already made. The writer's bound follows the reader's, and splits into the two refusals it always conflated. A group under a tag outside the standard set is refused by its own message instead of one citing the Interop depth clause, which was the wrong clause for that mistake; a group nested past the reader's depth keeps the depth message. Neither counts a childless group, so the depth bound still counts children rather than groups.
`exif_ifd` filtered on `fields()` alone, so a directory holding nothing but a sub-IFD group counted as empty and was dropped: it encoded to a file with no Exif directory at all, and read back as absent, with no error to say so. That shape is not hypothetical — it is exactly what `read_metadata` returns for an Exif directory whose only entry is its `InteroperabilityIFD` pointer, so a decode/re-encode round trip lost the directory silently. A group is one on-disk entry, so a directory holding one is not an empty directory. The filter now accepts either, and `is_empty` says so.
This PR restructured the encoder so that every entry point resolves the C2PA store and the metadata check at one chokepoint, before any pixel work. Nothing in the suite held that: reverting `encode_palette8` alone to the unchecked form and running the whole crate suite failed zero tests, because the per-path tests added in round 4 pin which store is resolved rather than that it was checked at all — a path that drops the check still refuses a bad C2PA store. The claim is about entry points, so the table is over entry points: all twelve public encode surfaces, given an Exif tree the decoder could not read back, each asserted on the refusal's message rather than on `is_err` so that a surface refusing for an unrelated reason is not mistaken for one honouring the bound. It replaces the single-surface test it subsumes. Verified against the regression it exists for: with `encode_palette8` reverted, it is the only test in the crate that fails.
The guard refusing a store no classic TIFF `count` word could describe was two lines inline, and its *magnitude* was unpinned: narrowing the conversion to sixteen bits fails zero tests, because every fixture is far below either bound. The mutation gate was satisfied only because substituting true and false for the whole condition is caught by other assertions — neither of which can see a bound that is merely wrong. No test going through `c2pa_store` can reach the boundary either: a length large enough to be refused there is large enough that `zeroed`'s reservation answers first, which is what the previous commit series deliberately arranged so the mutant could not time out. So the bound moves into a named predicate and is asserted directly, at `u32::MAX` and one past it, plus BigTIFF's freedom from it — three assertions, no allocation. Spelling it as the refusing condition keeps the call site free of a `!` for a mutant to delete.
Three places asserted a round trip cannot emit a stale offset — the README's "what the encoder writes the decoder reads back", the status file's "the writer is bounded by what the reader accepts", and the module docs. Two of the four standard pointer tags did not hold it, so all three said more than was true. They now state the rule that is actually implemented: which pointers are resolved depends on the level, `ExifIFD` alone at IFD 0 and all four standard tags inside the returned Exif directory, with the cost of the second named rather than left implicit — an unreadable target under any of the four fails the read. The vendor-private caveat the crate already carried is extended to say plainly that such a field is not round-trip safe and that a round trip through it proves nothing. The writer's bound is restated as the two refusals it now is, tag and depth, each with its own message. Also rewraps the paragraphs this branch introduced to the ~100 columns the surrounding files use, instead of the 110-125 they had drifted to.
…s count Self-review of the previous commit: the new test cloned the Interop directory it built and then compared only `sub_ifds().len()`, so the clone was needless and the assertion could not tell the written group from any other group of one. Comparing against the directory itself is both stronger and shorter.
`check_exif_subtree` inspected a directory's sub-IFD *groups*; `resolve_pointers` inspects its *fields*. That asymmetry was the defect: the one shape the reader misreads was the one shape the writer never looked at. A standard pointer tag carried as a plain `LONG` — `SubIFDs` (330), `ExifIFD` (34665), `GPSInfo` (34853) or `InteroperabilityIFD` (40965) — passed the check, encoded cleanly, and then failed this crate's own `metadata()` with `read out of bounds` for a large integer or `sub-IFD pointer loop` for a small one. That is precisely the file `TiffMetadata::check` documents it exists to prevent: one "read back as something other than what was written". The refusal now inspects what the resolver inspects, at every level of the subtree, and is shaped by the value's *type* rather than by its tag: `pointer_offsets` accepts only LONG/IFD/LONG8/IFD8, so a `SHORT` under `SubIFDs` is a pointer to neither side and is written and read back unchanged. Both refusal messages become named constants so the tag set the public contract enumerates in prose has one place that fails when it stops being true — the set itself is a sibling crate's constant, and a fifth member arriving upstream would otherwise widen the contract silently.
The set of sites was derived by sweeping the crate for the contract's own terms — every `///`/`//!` line naming a pointer tag by name or number, every one naming the follow/resolve contract, the crate's markdown, and the error-message strings — rather than by listing them from memory. That sweep found five restatements of this contract and two that had gone false: `TiffDecoder::metadata` still said only `ExifIFD` and `InteroperabilityIFD` are followed, which was false in both directions: a dangling non-listed tag *inside* the Exif directory now fails the call, which the doc implied cannot happen, and a dangling `InteroperabilityIFD` at IFD 0 now succeeds, which the doc said is followed. It now states the per-level rule and what each level costs. `TiffEncoder::with_metadata` said nesting is "the one thing" that could break the agreement between what it writes and what the decoder reads. There are three, and each has its own message; all three are now named there, in README.md and in STATUS.md.
`a_standard_pointer_at_ifd_0_that_feeds_no_field_is_left_alone` passed under the exact defect its comment describes — `pointer_tags` returning the full set at every level. Its first assertion called `read`, which resolves no pointer under any configuration, and its second asked for `exif`, which stays `None` whether a `SubIFDs` pointer at IFD 0 was resolved or not. The matching mutant was caught, but by the other half of the rule, so that axis was green for the wrong reason. It now drives `resolve_pointers` directly at the depth `read_metadata` calls it with, and asserts the field stays the integer it was read as and becomes no group — which resolution at IFD 0 would break. Verified: the previous body passes under that change and this one fails, as its only failure. Also names `IFD0_POINTER_TAGS`, which is what the round-6 split left of the `POINTER_TAGS` a neighbouring comment still referred to in the present tense.
This was referenced Sep 10, 2026
`check_exif_subtree` asked `pointer_offsets` about a caller's `Value`, so it classified by the in-memory variant. The reader classifies by the type code the entry carries, and `Value::Unknown` is the one shape where the two disagree: its constructor validates only the value word's width, so an `Unknown` built at code 4, 13, 16 or 18 is a plain field to the variant-shaped predicate and a sub-IFD pointer to the reader. `write` emits the code verbatim, and 64 of 384 end-to-end cases encoded cleanly and then failed this crate's own `read_metadata` with `read out of bounds` or `value offset out of bounds`, or -- in BigTIFF at the top level -- read back as a group where a field was written. The writer now asks what the reader asks: `Value::type_code`, total over every variant, against `POINTER_TYPE_CODES`. That membership is pinned against `pointer_offsets` by sweeping the whole `u16` code space rather than by repeating four numbers, and the boundary itself is swept by a matrix derived from the type -- every representable entry type, natural and `Unknown`, across the four pointer tags, classic and BigTIFF, on the Exif directory and one level below it. The sibling half, a `gamut-ifd` constructor that admits a recognised code into `Unknown` at all, is #608 and is not fixable from this crate. Refs #446
`Ifd` keeps fields and sub-IFD groups in two lists, so one tag can sit in both. The writer then emitted two entries under that tag -- not a TIFF directory (TIFF 6.0 §2) -- and the field was silently lost: this crate's model collapses a duplicated tag to its last occurrence, and the group is written last. Verified on disk: an Exif directory given `InteroperabilityIFD` as both a `SHORT` field and a group encoded to entries `[(33434, 5, 1), (40965, 3, 1), (40965, 4, 1)]`, and read back without the field. Three documentation sites promised the opposite unconditionally -- that what the caller supplies is what the file gets. The encode now refuses the pair with its own message, because the alternative is a non-conformant file plus a reader that drops a field the caller set, and normalising it would mean choosing silently which of the two the caller meant. A non-minimal writer is the cheaper cost. Refs #446
`deconstruct` is this crate's own judge, and it graded a file carrying two entries under one tag clean -- which is why a round trip could not see the writer that emitted one. The report walked the parsed tree, and `Ifd` is a directory model: by the time the tree exists a repeated tag has already collapsed to its last occurrence, so the defect is invisible there by construction. The scan now re-reads the raw entry records of every directory the audit walked (`SpanKind::IfdBody`), a second pass over bytes already claimed rather than a second walk of the pointer graph, and reports `Anomaly::DuplicateTag` with the directory's offset, the tag and the count. `Anomaly` is `#[non_exhaustive]`, so the variant is additive. Which entry survives a repeated tag is a property of the reader, not of the file: this crate keeps the last occurrence, libtiff marks every occurrence after the first to be ignored and warns the directory is unsorted. Refs #446
The scoping decision -- IFD 0 resolves `ExifIFD` and nothing else -- was held by a test that iterated a hand-written list of the two tags it expected to be left alone. Adding the fourth standard pointer tag to `IFD0_POINTER_TAGS` therefore left all 25 of this crate's test binaries green, and a `const`'s contents are not a mutable expression, so the mutation gate cannot see it either. The membership is now asserted directly and the sweep derived from the constant by difference, so it can never claim "left alone" about a tag the constant says is followed. Verified: the widening now fails this test and only this test. Refs #446
Nothing in this pull request consulted the crate's oracle about the seam, and a repeated tag is exactly the defect a round trip cannot see: gamut writes and reads by one rule, so a rule that is wrong is wrong symmetrically. libtiff marks every occurrence after the first to be ignored and warns that the directory is unsorted; this crate's directory model keeps the last. A hand-built 2x2 RGB TIFF carrying `PhotometricInterpretation` twice -- `RGB` then `BlackIsZero` -- is therefore decoded as RGB by libtiff and reported as `BlackIsZero` by `TiffDecoder::info`. That disagreement is the whole reason the encode refuses a tag carrying both a field and a group, and the reason `deconstruct` grades one. The fixture is built byte by byte because no directory model can express two entries under one tag, which is the normalisation the test exists to look underneath. Refs #446
…re each is claimed Five sites state this seam's writer/reader bound, derived by sweeping the crate for the contract's own terms rather than listed by hand: `with_metadata`, `TiffDecoder::metadata`, `TiffMetadata::exif`, README.md and STATUS.md. Three of them promised unconditionally that what the caller supplies is what the file gets, which a tag carrying both a field and a group falsified, and all of them named the refused set by the in-memory `Value` variant, which `Value::Unknown` falsified. Each now says what the code does: the discriminator is the on-disk type code, and a tag may carry a field or a group but not both. The STATUS paragraph round 7 left at 152 columns is rewrapped with the rest; no line this branch adds to README.md or STATUS.md now exceeds 100 columns, measured rather than asserted. Refs #446
The fixture repeated `PhotometricInterpretation`, but the oracle harness never queries that tag and for uncompressed chunky data the scanline bytes do not depend on it: with the two entries swapped, libtiff's decode is byte-identical, and with libtiff patched to keep the *last* duplicate the test still passes. Only this crate's half was load-bearing, so the test could not fail for the disagreement it is named for. Repeat `StripOffsets` instead, with the two entries pointing at two different strips that are both in the file. The scanline bytes are then a function of which entry the reader keeps: libtiff returns the first entry's strip, this crate the last, and the assertion fails if either reader changes its rule. Both orderings are built, so "follows the first entry" is separated from "follows the lower offset".
…rries `Anomaly::Structure` reports a `Severity`; `Anomaly::DuplicateTag`, added in this pull request, did not, so a caller triaging a report had to know which variants carry one and infer the rest. A repeated tag is a structural defect of the same kind — TIFF 6.0 §2 gives a directory one entry per tag, and the field a caller set is dropped by at least one reader — so it is graded `Severity::Error`. Added now rather than later because the variant is new and unreleased: the same field arriving afterwards would be a breaking change to a `#[non_exhaustive]` match that binds it. `flags_a_directory_that_repeats_a_tag` matches the severity, so grading it a warning fails that test alone.
`check_duplicate_tags` walks the audit's `IfdBody` spans and `continue`s past anything it cannot re-read, on the recorded ground that such a directory "is already reported by the audit's own finding". That is a contract a caller may rely on — a file is never graded fully accounted on the strength of a directory nobody read — and nothing failed when it stopped being true. State it precisely at the site, separating the two silent skips: a span the audit claimed re-parses by construction, so that arm is unreachable; a directory the audit could not parse is claimed as no span at all and arrives as `AuditFinding::SkippedSubIfd`. Pin the second with a file whose `SubIFDs` target lies past the end, the plain-unreadable reason that neither the cycle nor the depth guard produces, and which no test reached before.
…n the reader `TiffMetadata::check` refuses a sub-IFD group hung off a tag outside the four the reader resolves, while the reader takes such a file without error and hands the tag back as the raw absolute offset it was written as — the child directory is lost silently. That asymmetry is the stated reason refusing is the conservative choice, and it was stated only in prose: nothing failed if the reader began resolving the tag, and nothing failed if it began refusing it, either of which would remove the reason. Drive the reader over a file the writer would refuse, built through `gamut_ifd::write` so `check` is not in the path, and assert both halves — the tag comes back as a bare offset, and no group carries it.
The comment said the sweep is derived "from the constant" beside a paragraph about `IFD0_POINTER_TAGS`, while the code iterates `gamut_ifd::tags::STANDARD_POINTER_TAGS`. Same members — this crate's `EXIF_SUBTREE_POINTER_TAGS` is defined as that constant — but a repair whose point is deriving a set from the constant it names should name the right one. Say which, and why the sibling crate's constant is the stronger choice: the domain the sweep must cover is every standard pointer tag that exists, so a fifth added upstream has to enter it whatever this crate's alias does, and deriving from the alias would let a later narrowing shrink the sweep silently.
Self-review of the previous commit: the two strips are laid out back to back and the second one's offset was written as `+ 12`, the first strip's length as a literal, while `StripByteCounts` derived the same number from the array. The "value layout drifted" assertion only covers the first strip, so widening the pixel blocks would have overlapped them silently. Both now come from one binding.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Issue #446 asks
gamut-tiffto carry the C2PA manifest store, and says up front that buildingthe metadata seam is the bulk of the work — the crate had no metadata read or write surface at
all. This PR does both halves, in that order.
Stacked on #508 (
feat/442-dng-c2pa-manifest-store), which adds the sharedgamut_ifd::c2pamodule. This PR is opened against that branch and must not merge before it. #508's head has moved
three times under review; every move is merged in here by merge commit (never rebase), most
recently
677d38e, at which point #508 is itself fully delivered. This PR has also been throughone review round of its own — see Review round 1 below.
feat(tiff): carry ICC, XMP, IPTC-IIM and an Exif sub-IFD—TiffMetadata(
#[non_exhaustive],new()+with_*builders) is written into IFD 0 byTiffEncoder::with_metadataon the strip, tile and multi-page paths alike, and read back byTiffDecoder::metadata. XMP (700), IPTC-IIM (33723) and ICC (34675) are opaque bytes carriedverbatim — the raw
MetadataBlock-ready payloadsgamut-pngandgamut-webphand over — andthe
ExifIFD(34665) is handed over as agamut_ifd::Ifd, because it is a directory thiscrate has already parsed and a byte blob would force every caller to re-parse it. Nothing is
validated or completed.
feat(tiff): carry the C2PA manifest store over the shared placement rules— the storejoins as a fifth carrier. §A.3.6 and §18.5.5 are not restated:
gamut_ifd::c2pa(fromfeat(dng)!: type the C2PA manifest store and report both exclusion ranges #508) owns the tag, the last-main-chain-IFD placement, the end-of-file store, the
reserve-then-append relocation and the locator, and
gamut-dngcalls the same helper, so thetwo formats cannot drift.
with_c2pa_reserved(len)writes a zero-filled reservation for anexternal signer to overwrite in place;
encode_with_reportreturns the two exclusion ranges;c2pa_exclusions(file)recovers them from any TIFF's bytes — including files written throughencode_palette8orencode_pages_rgb8, which the object-safeEncodeImageseam cannotreport through. Tag 52545 joins
is_known_tag.docs(tiff),style(tiff)andtest(tiff)— STATUS.md/README.md, one readabilityfix, and robustness coverage for the two new byte-consuming entry points plus a BigTIFF store.
Semver: additive only, so minor.
gamut-ifd,gamut-dngand every other crate are untouched bythis branch's own commits; the only non-
gamut-tiffcontent here comes in through the base merges.No human approved this plan. This is an unattended automated run; the decision record below is
what a human reads afterwards.
Validation
Every command below was run in this worktree at head
20268b8, after merging base677d38e, andevery one of them completed.
mise run fmt-checkneeds the__CARGO_TEST_ROOTprefix in a nestednested worktree (cargo otherwise walks past the worktree root to the primary
checkout's
tooling/*manifests and exits 101 on an untouched tree); the prefix is recordedverbatim.
Workspace-wide commands ran inside
systemd-run --user --scope --slice=agents.slice -p MemoryMax=16G -p MemorySwapMax=0withCARGO_BUILD_JOBS=2 CMAKE_BUILD_PARALLEL_LEVEL=2andulimit -v 12000000; the mutation runnerapplies its own equivalent guards from
--budget 16.mise run check-release-deps/check-ffi-features/check-ffi-headerwere not run and arenot applicable: no
Cargo.tomlchanged and no public C-surface type changed.Not run:
mise run coverage(CI's own Coverage gate passed on the previous head),check-cross,check-msrv,test-dng-real(extended CI, master/manual).Review round 1
Four findings, all fixed in
c26c851,98c734aand20268b8.InteroperabilityIFDpointer.read_metadataaskedread_treeforExifIFDalone, andread_treere-parses only the tagsit is given, so an Interop pointer inside the Exif directory came back as a raw
Longholdingthe source file's absolute offset, which
applythen wrote into a file laid out atdifferent offsets. Decode → re-encode produced a file the crate's own judge rejects:
is_fully_classified() == falsewith aseverity: Error"sub-IFD could not be parsed" —breaking gamut-tiff's v1 zero-tolerance byte accounting on a file gamut-tiff itself wrote.
Reproduced as a failing test first, then fixed by reading with
gamut_ifd::tags::STANDARD_POINTER_TAGS, the list the repository already provides andgamut-dng's rewrite path already uses.metadata()andc2pa_exclusions()disagreed, and the docstring said theyagreed. The reader accepted any
UNDEFINEDvalue under 52545;locatealso rejects one belowMIN_STORE_LENor a duplicated entry, so a short entry decoded to aTiffMetadatathe encoderthen refused. The reader now gates on the locator, as
gamut-dngdoes, making the two agree byconstruction rather than by comment.
append_storerefuses a storethat fits inline, so the bound is
max(MIN_STORE_LEN, inline_threshold + 1). A BigTIFFreservation of 8 compressed the whole image before failing from
gamut-ifd, despite the# Errorspromise to catch length "before any pixel work". Gated onmin_store_len(); both docclaims corrected.
with_metadataand astore, and a
deconstructassertion covers a file carrying the Exif sub-IFD — the gap F1escaped through.
Risks and rollout
configured (
TiffMetadata::applywrites nothing for an empty set).the failure this could plausibly cause, and it is pinned against libtiff rather than against
gamut's own reader (
tests/oracle_metadata.rs, three sizes, with and without a store).zero-tolerance byte-accounting guarantee therefore had to keep holding:
tests/c2pa.rsassertsdeconstructreports the store as its entry's typedValuespan, with nothing unclassified, noTrailersegment, and no unknown tag.a big-endian (
MM) file, so a byte-swapping implementation cannot pass; a palindrome wouldhave.
TiffDecoder::metadataandc2pa_exclusionsare new entry points taking untrusted bytes onpaths
decode_pagenever takes (anExifIFDpointer into a second directory; a walk to the endof the chain for an offset/count pair). Both are now in
tests/robustness.rsunder thetruncation sweep and the 5000-mutation byte-flip corpus.
TiffMetadatais now safe to hand straight back towith_metadata: every pointerunder the Exif directory is a parsed child the writer re-points, so a round trip cannot emit a
stale offset. This is the F1 fix and is pinned by a decode → re-encode →
deconstructtest.TiffDecoder::metadatacan now returnErron a file whoseExifIFDpointer is malformed,where
decode_imagestill succeeds. Deliberate, and documented under# Errors: reportingexif: Nonefor a directory the file declares is silent loss a caller cannot detect.feat(tiff): carry the C2PA manifest store …commit to keep the seam alone;drop this branch's own commits to return to the base.
Issue
Refs #446. Not
Closes— the issue's own text namesc2pa-rsas the behavioural oracle, alreadyfiled separately as #447, and this PR carries the store without validating it.
Two existing issues cover what this PR deliberately leaves out, so no new issue was filed:
gamut-metadata: wire the facade into gamut-png, gamut-webp, gamut-avif andgamut-tiff) — the typed metadata wiring. This PR carries raw payloads only; adding the
gamut-metadatadependency edge here would collide with that work.tooling/c2pa-oracle) — the behaviouralc2pa-rsoracle. libtiff is the containeroracle used here; nothing in this workspace validates a manifest store's contents yet.
Decisions taken
Appended by this lane
Unresolved review notes
None.
Round 4 — review repairs F1–F5, and a bound on the reservation
Branch
feat/446-tiff-metadata-seam; four commits on top of01668a81. Still an unattended run:no human approved this round either, and the record below is what a human reads afterwards.
F1 (Medium) — a pointer on a page the reader discards no longer fails the read
read_metadatahanded one flatPOINTER_TAGSlist togamut_ifd::read_tree, which resolves it atevery node of every page, while the blocks and the Exif sub-IFD come from IFD 0 alone. The
regression class the list exists to prevent therefore survived, moved from a tag this reader ignores
to a page it ignores. Both falsifiers from the review were reproduced here before the fix and are
now pinned as tests (
crates/gamut-tiff/tests/metadata.rs):metadata()beforeExifIFD = Long(0xFFFF_FF00)invalid input: TIFF: read out of bounds [origin: gamut-ifd, byte offset: 4294967040]Ok, XMP returnedExifIFDentries name one directoryinvalid input: TIFF: sub-IFD pointer loop [origin: gamut-ifd]Ok, page 0's Exif returnedThe chain is now read with
gamut_ifd::read, which follows no pointer at all, and the list isresolved over IFD 0's subtree only, by hand through
gamut_ifd::read_ifd_at— the per-pointercontrol
read_tree's own documentation points at. Within that subtree it stays one flat list atevery node, exactly as
read_treeapplies one to a whole file, so IFD 0's behaviour is unchangedand the flat-list dependence is now explicit in the code (
resolve_pointers' signature takes thelist) rather than implicit in a call to another crate. The one over-reach that dependence causes —
InteroperabilityIFDfollowed at IFD 0, where a conformant file never puts it — survives and isstill documented as harmless on
POINTER_TAGS, inSTATUS.md§(b), and now onTiffDecoder::metadata's# Errors(F3).The C2PA entry stays reachable, shown rather than assumed. §A.3.6 puts it in the last IFD of the
main chain, and that entry carries the store's bytes rather than an offset, so it is reached without
following a pointer on a page whose pointers are never resolved. Two pieces of evidence: the new
two-page test reads its manifest store from the same page that carries the dangling
ExifIFDand asserts it comes back verbatim; and
tests/c2pa.rs's existing multi-page and last-IFD tests(
a_multipage_document_puts_the_entry_in_its_last_page,read_metadata_takes_the_store_from_the_last_ifd_of_the_chain) still pass unchanged. Revertingread_metadatatoread_treein place makes only the two new tests fail, with the two messagestabulated above — the store assertions pass either way, which is the same result stated as an
experiment.
F2 (Low/Medium) — the compile-time obligation, and the hole beside it
The obligation is to pass a
Option<Cow<[u8]>>, not the onec2pa_store()returned. ReplacingstorewithNoneatencoder.rs:823left the suite green, becausetests/c2pa.rscovered thestrip, tile, palette, multi-page and BigTIFF paths and never encoded 16-bit, RGBA or bilevel. Two
repairs, both executed:
so every entry point that resolves a store of its own is named there, each asserting the refusal
is the C2PA one rather than the tiling one.
encode_16bitandBilevelwere already in it.encode_pages_rgb8stays out and the comment says why: it builds strip images directly, so thereis no second refusal for the C2PA one to be told apart from.
tests/c2pa.rsgainsthe_pixel_paths_that_pack_their_own_buffer_place_the_store_too, coveringthe three paths nothing else there encoded — the three that do pixel work of their own first.
Dropping
storeat each of the three call sites (encoder.rs:361,:744,:823) was appliedby hand and each fails that test now; before it, all three left the suite at 196 passed.
F5 (Nit) — and a correction to a published commit message
tests/metadata.rs's dangling-ExifIFDtest asserts the message (read out of bounds) rather thanis_err(). And the correction, made here because the commit is pushed and is not rewritten:80f40853's message claims "both regressed cases are pinned", and that was not true — only thedangling case was asserted; the shared-directory case was described but not pinned. Both are pinned
as of this round.
F4 — the mutation counts did not reproduce, so they are re-measured
The round-3 figure
142: 130 caught, 12 unviabledoes not reproduce and is withdrawn; theverdict behind it (0 missed, gate green) does. This round measured
83 → 5 missed, repaired, then79 — 76 caught, 3 unviable, 0 missed, with the command and base in the table below. The mutantpopulation of a
--diffrun is a function of the base ref's current tip, which moves while a stackedbranch is open, so a bare count is worth less than the command that produced it; both are given.
A bound on
with_c2pa_reserved(same decision as #506)with_c2pa_reservedreturnsSelf, so it refuses nothing itself, and its length went straight intovec![0; len]at the top of every encode. Pastisize::MAXthat panics with a capacity overflowinstead of returning — reproduced with
usize::MAX, which raisesraw_vec: capacity overflow.The first attempt was a numeric bound (buffer's
isize::MAX, and classic TIFF's 32-bit count andoffset words), and the mutation gate rejected it:
>and>=differ only at exactlyu32::MAXorisize::MAXbytes, which is not a reservation any test can allocate, so the boundary's acceptingside was unassertable and the mutant survived. The bound is therefore taken by the operation instead
of by a comparison —
Vec::try_reserve_exact, so a length noVec<u8>can hold isError::InvalidInputat the same point, before any pixel work, with no comparison to get wrong andno container arm to keep in step. The container bound went with it; it refused only lengths
gamut_ifd::writealready cannot lay out. What stays outside this crate's reach is the allocator's:a request the kernel overcommits succeeds here and aborts only when the bytes are written — the same
residual #506 records.
gamut-dnghas the identical shape atcrates/gamut-dng/src/encoder.rs:237but belongs to #508, soit is filed, not touched: #552. That issue's text suggests the numeric bound #506 took; the
fallible reservation above is the better mechanism and is noted here rather than by editing a filed
issue.
Validation (round 4)
Every command below completed in this round, from the worktree, in the memory-capped scope where the
contract requires one.
cargo test -p gamut-tiff --all-featurescargo clippy -p gamut-tiff --all-targets --all-features -- -D warnings__CARGO_TEST_ROOT=$(git rev-parse --show-toplevel) mise run fmtthen… mise run fmt-checkmise run check-testsconvco check origin/feat/442-dng-c2pa-manifest-store..HEADmise run lint(systemd scope, MemoryMax=16G, MemorySwapMax=0, CARGO_BUILD_JOBS=2,ulimit -v 12000000)mise run test(same scope)GAMUT_MUTANTS_BASE=origin/feat/442-dng-c2pa-manifest-store mise run mutants-diff --budget 1611264899: 83 mutants — 74 caught, 4 unviable, 5 missed (exit 2). All five repaired infix(tiff): kill the diff mutants the round-4 repairs left alive. Second run on175b0ff5: 79 mutants — 76 caught, 3 unviable, 0 missed (exit 0)The five survivors, and what each was:
encoder.rs:234> → >=inc2pa_store(the numeric bound'sunassertable boundary — see appended decision 13);
metadata.rs:281> → ==and> → >=, and:301+ → *, all inresolve_pointers(a depth bound of sixteen neither side of was reachable —appended decision 14);
metadata.rs:251deletingpointer_offsets'Value::Long8 | Value::Ifd8arm(no BigTIFF reader test stood behind the 64-bit pointer form, so every BigTIFF's
ExifIFDcould havebeen dropped silently). None was excluded; each is killed by a test.
Falsifier runs, each applied by hand to the working tree and reverted afterwards:
read_metadatareverted toread_tree(data, POINTER_TAGS)tests/metadata.rscases fail with the two messages tabulated under F1; the other 8 in that file passstore→Noneatencoder.rs:361/:744/:823the_pixel_paths_that_pack_their_own_buffer_place_the_store_toofails in each of the threevec![0; len]restored in place of the fallible reservationa_reservation_no_buffer_could_hold_is_refused_instead_of_panickingpanics inraw_vec(capacity overflow) instead of returningNo
Cargo.tomlwas touched, socheck-release-deps/check-ffi-features/check-ffi-headerarenot triggered; no public C-surface type changed.
Issue (round 4)
Filed: #552 — "gamut-dng: bound
DngEncoder::with_c2pa_reserved's length, as gamut-tiff andgamut-avif do".
Decisions taken (round 4)
The decision record for this round was taken by the orchestrator on review of
01668a81and isreproduced verbatim:
Appended by the lane (forks the round-4 record did not settle), in the record's shape:
Unresolved review notes (round 4)
POINTER_TAGSdocuments is unchanged, by decision.InteroperabilityIFDisstill followed at IFD 0, where a conformant file never puts it, and its resolved group feeds no
field of
TiffMetadata— so by the docstring's own rule ("a pointer whose target this readerthrows away must not be able to fail the whole call") it is a residual instance of the same class,
bounded to one out-of-spec tag on one directory. Removing it takes either a per-node pointer list
in
gamut-ifdor the per-subtree shape decision 1 rejected; both are outside this lane.resolve_pointersrestatesgamut_ifd's resolver, including the depth and cycle guards andthe rule for which value types are pointers, because the gamut-ifd surface is frozen for this
lane (see appended decision 11). The two can now drift. The loop and depth refusals also carry
origin: gamut-tiffrather thanorigin: gamut-ifdfor IFD 0's subtree; the messages areunchanged.
encode_pages_rgb8is not in the ordering test, because it builds strip images directly andoffers no competing refusal to distinguish the C2PA one from. Its store placement is pinned in
tests/c2pa.rs.gamut-dngstill panics on an oversized reservation (crates/gamut-dng/src/encoder.rs:237).Filed as gamut-dng: bound DngEncoder::with_c2pa_reserved's length, as gamut-tiff and gamut-avif do #552 rather than fixed, because that crate belongs to feat(dng)!: type the C2PA manifest store and report both exclusion ranges #508.
Round 5 — review repairs N1–N6
Branch
feat/446-tiff-metadata-seam; five commits on top of175b0ff5. Still an unattended run:no human approved this round either, and the record below is what a human reads afterwards.
Every round-4 finding is repaired on this branch. Each was reproduced by execution before it was
touched, and the repair re-run against the same reproduction.
N1 (Medium) — the encoder wrote what its own reader refused
Reproduced, debug and release: a four-level Exif tree (
ExifIFD→InteroperabilityIFD→InteroperabilityIFD→ leaf) encoded to a well-formed 380-byte TIFF, andTiffDecoder::metadataon those same bytes returnedinvalid input: TIFF: sub-IFD tree too deep [origin: gamut-tiff]. The crate emitted a file itcould not read — the round-1 finding's shape, reintroduced by lowering the reader's bound from 16
to 2 without lowering the writer's.
The bound now applies to both sides.
TiffMetadata::checkrefuses an Exif directory nesting belowthe pair EXIF 2.3 §4.6.3 allows, measured against
MAX_POINTER_DEPTHitself so the two cannotdrift, and it is taken at one chokepoint (
TiffEncoder::checked_store) together with the C2PAstore's refusals — before any pixel work on every entry point. After the repair the same
reproduction fails at the encode:
TIFF: an Exif sub-IFD may nest one further directory (ExifIFD -> InteroperabilityIFD, EXIF 2.3 §4.6.3) and this one nests deeper. The depth the pair does reach still round-trips(
a_decoded_exif_sub_ifd_re_encodes_into_a_fully_classified_file), so the agreement is proved fromboth sides: what the writer accepts, the reader reads back; what the reader refuses, the writer
will not write.
Correction to appended decision 14 (round 4), which is not edited in place. That entry gives
mutation-assertability as the reason the reader's bound became two. That is not on its own a
sufficient reason to narrow a contract. The narrowing is retained, and it now rests on the clause:
ExifIFDandInteroperabilityIFDreach two levels and a third is out of spec (EXIF 2.3 §4.6.3).Assertability is a property of a contract stated at the right width, never an argument for changing
its width.
N2 (Low) — the error documentation
crates/gamut-tiff/src/decoder.rssaid "nesting deeper than 16 levels" a round after the shippedbound became two. It now states the shipped bound and names the clause and the encoder-side twin.
N3 (Medium, robustness) — a wide pointer array made the walk quadratic
Reproduced in release on hand-built classic TIFFs whose single
ExifIFDarray names N distinctall-zero directories (each parses as an empty IFD, so every call returns
Ok):Clean quadratic against near-linear; 157× at 2.34 MB.
visitedis now aBTreeSet, andinsertreturning
falseis the loop guard, so the guard lost a line rather than gaining one. The samedefect in the shared crate this walk restates is filed as #578; the separate question — nothing
bounds the breadth of one pointer array — as #579.
N4 (Low) — the refusal that still happens
Restored, both the check and its documentation. Classic TIFF counts an
UNDEFINEDvalue with a32-bit
LONG, so a store past 4 GiB cannot be described whatever else holds it; without an earlycheck the reservation is allocated, zero-filled and the whole image compressed before
gamut_ifd::c2pa::append_storerefuses it with a message about the file's offset limit rather thanabout the length the caller passed. The bound is expressed as
u32::try_fromover the variantrather than as a comparison, so it has no boundary whose accepting side needs a 4 GiB allocation to
assert — which is what made the deleted
max_store_lenunkillable. It is deliberately necessary,not sufficient: whether the store's own offset fits depends on the size of the file it lands after,
and that refusal stays with
append_store, where the file exists. Theisize::MAXreservation testnow asks for BigTIFF, since classic TIFF refuses that length one check earlier for a different
reason.
Correction to appended decision 13 (round 4), which is not edited in place. Its "Rejected"
clause says the container's 32-bit bound "refuses only lengths
gamut_ifd::writealready cannotlay out". That is true of the outcome and false of the cost: the later refusal arrives after
the reservation and the whole image have been materialised. The fallible reservation is kept; the
container bound is restored beside it.
N5 (Low) — one test, one reason
a_broken_pointer_on_a_page_the_metadata_discards_does_not_fail_the_readasserted four thingsacross three modules, and its store assertion killed nothing the inline
read_metadata_takes_the_store_from_the_last_ifd_of_the_chaindoes not already kill. It nowasserts the rule it is named for and nothing else. The fixture lost the store it no longer reads,
which also drops the file's reach into the C2PA locator and the strip decoder.
N6 (Nit) — the feature guard that cannot be restored
Reported as a dropped guard; it cannot be restored, and the reason is now in the source.
bigtiffis
gamut-ifd's feature, enabled unconditionally by this crate's dependency on it and notre-exported, so
#[cfg(feature = "bigtiff")]ingamut-tiffnames a feature this crate does nothave. Verified by adding the attribute and building: one
warning: unexpected cfg condition value: bigtiff, which the lint gate takes as an error under-D warnings; and were it accepted the arm would vanish and every BigTIFF'sExifIFDwould readback as a plain integer.
Validation (round 5)
Commands verbatim, each run in this branch's worktree; the workspace-wide ones inside a
memory-capped scope (
systemd-run --user --scope --slice=agents.slice -p MemoryMax=16G -p MemorySwapMax=0 -- env CARGO_BUILD_JOBS=2 CMAKE_BUILD_PARALLEL_LEVEL=2 sh -c 'ulimit -v 12000000; exec …').cargo test -p gamut-tiff --all-featurescargo clippy -p gamut-tiff --all-targets --all-features -- -D warnings__CARGO_TEST_ROOT=$(git rev-parse --show-toplevel) mise run fmtthen… mise run fmt-checkmise run check-testsconvco check origin/feat/442-dng-c2pa-manifest-store..HEADmise run check-commitsmise run lintmise run testGAMUT_MUTANTS_BASE=origin/feat/442-dng-c2pa-manifest-store mise run mutants-diffmissed.txtandtimeout.txtboth emptyThe
__CARGO_TEST_ROOTprefix is a nested-worktree artefact of thefmttasks, not a property ofthis change: without it cargo walks past the worktree root when loading the
tooling/*manifests.mise run coveragewas not run: this round adds no module, and the code it adds is reached by thesuite above.
check-release-deps/check-ffi-features/check-ffi-headerwere not run: noCargo.tomland no public C-surface type changed.The N1 and N3 reproductions were run from a scratch integration test that is not part of the
branch — it was written to falsify, executed before and after each repair, and deleted. What ships
is the narrow inline/integration tests named above.
Issues filed (round 5)
Nothing this round declines is left as an appended decision; each is a GitHub issue.
read_treequadratic (the shared copy of the walk N3 fixed locally, with the measurements).question N3 does not answer).
(appended decision 11's "Reverses", which an appended decision cannot carry once this PR merges).
two-page file recovers (the asymmetry the round-3 fix leaves).
InteroperabilityIFDis followed at IFD 0, where no conformant file putsit (the documented over-reach that remains).
Decisions taken (round 5)
The decision record for this round was taken by the orchestrator on review of
175b0ff5and isreproduced verbatim:
Appended by the lane (forks the round-5 record did not settle), in the record's shape:
Unresolved review notes (round 5)
POINTER_TAGSdocuments is still unchanged, and is now filed as gamut-tiff: InteroperabilityIFD is followed at IFD 0, where no conformant file puts it #582rather than carried as prose. Same for the resolver restatement (gamut-tiff: hoist the scoped sub-IFD resolver into gamut-ifd and delete the local copy #580) and the single-page
store asymmetry (gamut-tiff: a dangling IFD-0 pointer costs a single-page file its C2PA store, while a two-page file recovers #581).
checkis stricter than the reader on tags the reader does not follow. An Exif directorycarrying a
SubIFDsorGPSInfosub-IFD group two levels down is refused by the encoder,although
metadata()would have read the file back without error — returning that pointer as araw offset rather than as a directory. Refusing is the conservative direction (the writer stays
inside what the reader delivers) but it is not the reader's exact complement, and it is stated on
TiffMetadata::check.visitedset fixes the quadratic term, not the linear one. A file that is nothing but awide pointer array still costs one directory parse per offset; what the bound on that should be
is gamut-ifd: nothing bounds the breadth of a sub-IFD pointer array #579.
gamut-dngstill panics on an oversized reservation (gamut-dng: bound DngEncoder::with_c2pa_reserved's length, as gamut-tiff and gamut-avif do #552, unchanged from round 4), and theshared resolver still scans a list (gamut-ifd: the sub-IFD visited set is a linear scan, so a wide pointer array makes read_tree quadratic #578).
Round 5 (continued) — a mutation timeout the first push introduced
Correction to the round-5 validation table above, which is left as written: its
mutants-diffrowreports 95 tested, 91 caught, 4 unviable, 0 missed, measured locally on
91d79117. That figurewas reproducible on this machine and not on CI's. The incremental shard failed with exit 3 —
a timeout, not a survivor — on
delete !inTiffEncoder::c2pa_store, the classic-count guardN4 restored. Inverting the guard lets the test's oversized reservation reach
zeroed, whosetry_reserve_exactthe CI runner's allocator granted; the 4 GiB zero-fill that followed ran pastthe 60 s test timeout. Locally the same mutant was caught, because this machine zero-fills
4 GiB inside a minute. A mutant whose verdict depends on the hardware is not a gate.
Repaired in
b6dd0d23without weakening the check: the condition is spelled as the refusing one(
uncountable), so there is no!for the mutation to delete, and the test asks forusize::MAXrather than
u32::MAX + 1, so a length that ever gets past a guard meets a reservation noallocator can satisfy and fails instantly instead of allocating. Which of the two bounds answers
first is now what distinguishes the test from its BigTIFF twin, since both ask for the same length.
Re-measured on
b6dd0d23, same command(
GAMUT_MUTANTS_BASE=origin/feat/442-dng-c2pa-manifest-store mise run mutants-diff):94 tested, 90 caught, 4 unviable, 0 missed, 0 timeouts (exit 0),
missed.txtandtimeout.txtboth empty. One mutant fewer than before, which is the deleted!.Gates re-run on
b6dd0d23after the repair:cargo test -p gamut-tiff --all-featurespass;cargo clippy -p gamut-tiff --all-targets --all-features -- -D warningspass;__CARGO_TEST_ROOT=… mise run fmt-checkpass;mise run check-testspass;convco check origin/feat/442-dng-c2pa-manifest-store..HEADpass (22 commits);mise run lintandmise run testpass (exit 0, 206 test binaries ok).Appended to the round-5 record, in the record's shape:
Round 6 — review repairs F1–F6
No human approved this plan. This is an unattended run; the decision record below is what a human
reads afterwards.
F1 (Medium) — the writer still emitted an Exif directory the reader could not return
Reproduced first, at the round-5 head
b6dd0d23. ASubIFDs(330) orGPSInfo(34853) groupplaced under
ExifIFDencoded with no anomalies;TiffDecoder::metadatahanded the pointer backas
Long([212])— a raw absolute offset into the source file — and re-encoding that decoded valueproduced a file this crate's own judge graded:
Provenance, as the reviewer bisected it. Round 1 (
20268b8) followedSTANDARD_POINTER_TAGSand the round trip was clean. Round 3 (65c48cb4) narrowed the list to[EXIF_IFD, INTEROPERABILITY_IFD]to stop a danglingSubIFDsoffset on a page hiding theblocks — and in doing so re-opened round 1's own finding for the two tags it dropped, inside the
one directory the seam hands back. Rounds 4 and 5 did not notice and asserted the opposite in
three places.
Fix — decision 1, which restores the round-2 decision round 3 implemented only half of. The
tag list is now per level rather than one flat list, because the two levels answer opposite
questions:
ExifIFDonlySTANDARD_POINTER_TAGSThe page scoping round 3 correctly introduced is untouched: the walk still runs over IFD 0's
subtree and no other page's. After the fix, the same reproduction prints
raw field back = None,groups back = [330],re-encode structural errors = [],fully classified = true, forall four tags.
The cost is stated rather than hidden, in the module docs, the README and STATUS: inside the Exif
subtree an unreadable target under any of the four now fails the whole read. That is the trade
ExifIFDitself already made — reportingexif: Nonefor a directory the file declares is silentloss — extended to the pointers that directory contains.
This closes #582 as a side effect. That issue asked for exactly this per-node list
(
ExifIFDat IFD 0, the rest below it) and judged it agamut-ifdsurface. It is not: the list isthis crate's scoping decision and the local resolver already owned it.
InteroperabilityIFDis nolonger followed at IFD 0. This run may not edit or close a filed issue, so it is recorded here.
F2 (Low) — five of six chokepoint sites could regress invisibly
Confirmed: reverting only
encode_palette8to the uncheckedc2pa_store()and running thewhole crate suite failed zero tests. The round-4 per-path tests pin which store is resolved,
not that the configuration was checked — a path that drops the check still refuses a bad C2PA
store, so the C2PA message still comes back.
Decision 3: the invariant is now a test over entry points, not over the internal call sites —
every_public_encode_surface_refuses_an_exif_tree_its_own_decoder_could_not_read_back, table-drivenover all twelve public encode surfaces (the eight
EncodeImageimpls,encode_to_vec,encode_with_report,encode_palette8,encode_pages_rgb8), asserting the refusal's messagerather than
is_err, and that a refused encode appends nothing. It replaces the single-surface testit subsumes.
What it catches that nothing else does: with
encode_palette8reverted, it is the only test inthe crate that fails (
11 passed; 1 failed). Before it, that revert was invisible.F3 (Low) — the count bound's magnitude was unpinned
Confirmed: changing
u32::try_fromtou16::try_fromfailed zero tests, because every fixture isfar below either bound; the mutation gate passed only because substituting true and false for the
whole condition is caught by other assertions, neither of which can see a bound that is merely
wrong. No test going through
c2pa_storecan reach the boundary either — a length large enough tobe refused there is large enough that
zeroed's reservation answers first, which is exactly whatb6dd0d23arranged so the mutant could not time out.Decision 4: the bound is now the predicate
uncountable_store_len(variant, len), unit-tested atu32::MAX(accepted),u32::MAX + 1(refused) andVariant::Bigatusize::MAX(accepted) —three assertions, no allocation, the pattern this repository already uses for arithmetic
unreachable in place. Verified: with the sixteen-bit narrowing applied, it is the only test that
fails.
F4 (Low) — silent loss
Confirmed: an Exif directory whose only content is a sub-IFD group has an empty
fields()list,so
exif_ifd's filter dropped it. It encoded to a file with no Exif directory at all and readback as
exif: None, with no error. Reproduced atb6dd0d23(f4: exif back = None); after thefix the same case returns the directory with its group intact.
That shape is not hypothetical — it is precisely what
read_metadatareturns for an Exif directorywhose only entry is its
InteroperabilityIFDpointer, so a decode/re-encode round trip lost thedirectory silently. A group is one on-disk entry, so a directory holding one is not empty.
F5 (Nit) — the message named the wrong clause
checkcounted every sub-IFD group but reported every refusal with theExifIFD -> InteroperabilityIFDdepth clause, so a tree nested through a non-Interop tag was refused by amessage citing a rule it had not broken. The check is now two refusals with two messages, tag
checked first (a group under an unfollowed tag is unreturnable whatever its depth). The depth arm
still counts children, not groups, so a childless group is not newly refused.
F6 (Nit) — wrapping
The three markdown paragraphs this branch introduced at 110–161 columns are rewrapped to the ~100
the surrounding files use. Verified: no line added by this branch to
README.mdorSTATUS.mdnow exceeds 100 columns; the >100 lines that remain inSTATUS.md(24, 25, 37, 44, 45,52) predate this branch and are left alone.
Corrections to what this PR previously claimed
Decision 2. Three places asserted more than was true and are now corrected:
explicitly, names all four resolved tags, names the cost of resolving them, and states that a
vendor-private tag holding an offset is carried verbatim and is not round-trip safe.
the two axes it is bounded on, tag and depth, each with its own message, on every public encode
surface.
src/metadata.rs— the "a round trip cannot emit a stale offset. This is the F1 fix" bodytext is replaced by the per-level rule and its stated cost, on
TiffMetadata::exif,IFD0_POINTER_TAGS,EXIF_SUBTREE_POINTER_TAGSandpointer_tags.Corrections to the five issues filed in round 5
Decision 8. This run may not edit or comment on a filed issue, so the corrections are recorded
here for whoever picks them up.
gamut-ifd. The issue namesreader::resolve_pointers_with'svisited.contains(&off)and "the tworead_treepaths" —which are the two callers of that one walk (
reader.rs,stream.rs), not two walks.audit::follow_subifds(crates/gamut-ifd/src/audit.rs:333) keeps its ownvisited: &mut Vec<u64>and its ownvisited.contains(&offset)at line 348, and is not mentioned. It isthe worse of the two: it walks the sub-IFD chain (
while offset != 0) as well as the pointerarray, so the list grows on both axes — and its direct consumer is
gamut_tiff::deconstruct,the archival path whose whole claim is surviving hostile input. Fixing only
reader.rsleavesthat path quadratic.
pointer array still costs 5M directory parses" cannot be taken at face value alongside the
issue's own closing note that every offset must be distinct and in bounds, i.e. that N is
bounded by file length: a 20 MB file whose entire content is a 5M-element
LONGarray has noroom left for 5M distinct target directories, which can only be reached by aliasing the array's
own bytes and therefore constrains the offsets. The shape of the concern (breadth is unbounded
where depth is bounded) stands; the arithmetic in the headline does not, and the issue
contradicts it three paragraphs later.
resolver into
gamut-ifdand deletes the local copy; gamut-ifd: the sub-IFD visited set is a linear scan, so a wide pointer array makes read_tree quadratic #578 substitutes a set intogamut-ifd'sexisting walk. Taken gamut-ifd: the sub-IFD visited set is a linear scan, so a wide pointer array makes read_tree quadratic #578-first, the substitution lands twice — once in
resolve_pointers_with, then again carried over by the hoist — in two walks that then have tobe reconciled into one. Taken gamut-tiff: hoist the scoped sub-IFD resolver into gamut-ifd and delete the local copy #580-first there is one walk to change, and gamut-ifd: the sub-IFD visited set is a linear scan, so a wide pointer array makes read_tree quadratic #578 becomes the
one-line change it describes (plus
audit.rs, which is independent of both).tag list, not just subtree scoping.
pointer_tags(depth)is the local shape it would replace.implemented here.
Validation (round 6)
cargo test -p gamut-tiff --all-featurestests/metadata.rs+ the rest__CARGO_TEST_ROOT=$(git rev-parse --show-toplevel) mise run fmt-checkmise run check-testsconvco check origin/master..HEADmise run lint-D warningsmise run testtest result: ok, zero FAILEDmise run mutants-difffmt-checkneeds the__CARGO_TEST_ROOTprefix in a nested worktree (cargo otherwise walks pastthe worktree root when loading the
tooling/*manifests); the prefix changes no manifest. Allworkspace-wide gates ran inside a 16 GiB memory-capped scope with
CARGO_BUILD_JOBS=2.Three targeted falsification runs, each reverting one thing and running the whole crate suite:
encode_palette8→c2pa_store()u32::try_from→u16::try_fromRisks and rollout (round 6)
SubIFDs/GPSInfotarget inside an Exifdirectory now fails
metadata()where it previously returned a stale offset. A real camera EXIFwith a broken GPS pointer therefore now errors instead of silently handing back an unusable
offset. This is the decision record's instruction and the same trade
ExifIFDalready made; itis documented in all three places. The alternative — degrading silently — is what
gamut-dngdoes and what this crate deliberately does not.
is_empty. ATiffMetadatawhose Exif directory holds only a group now reportsnon-empty and is written. That is the point of the fix, but it is an observable change for a
caller that constructed such a directory and relied on it being dropped.
uncountable_store_len,pointer_tags,check_exif_subtree,IFD0_POINTER_TAGSandEXIF_SUBTREE_POINTER_TAGSare all private.a31a47df..1aa39fb5, in reverse.Decisions taken (round 6)
Map of the record — which earlier entries later rounds have corrected. The record is
append-only and nothing above is edited; this is the map a reader needs to read it correctly.
POINTER_TAGSto[EXIF_IFD, INTEROPERABILITY_IFD]checkcounts every sub-IFD group"u32::try_fromunder a matchInteroperabilityIFDis followed at IFD 0")bigtifffeature guard)The decision record for this round was taken by the orchestrator on review of
b6dd0d23and isreproduced verbatim:
Appended by the lane (forks the round-6 record did not settle), in the record's shape:
Unresolved review notes (round 6)
metadata()call, forall four standard tags rather than the two it previously affected. This is decision 1 carried
out and is documented in three places, but it is a real widening: a file whose GPS pointer is
broken and whose XMP/IPTC/ICC are fine now returns an error where it previously returned the
blocks plus a stale offset. Whether the four should share
ExifIFD's fate is the same contractquestion gamut-tiff: a dangling IFD-0 pointer costs a single-page file its C2PA store, while a two-page file recovers #581 already asks one level up, and it is not settled here.
checkis still not the reader's exact complement. It refuses a group under any tag outsidethe standard four; the reader would read such a file back without error, returning that pointer
as a raw offset. Refusing is the conservative direction — the writer stays inside what the
reader can hand back unchanged — and it is stated on
TiffMetadata::check.here: it is indistinguishable from an integer field, and neither this crate nor
deconstructcan grade it. Now stated plainly in the README, STATUS and the
TiffMetadata::exifdocs ratherthan implied.
audit::follow_subifdsstill scans aVec(the third linear scan above), sogamut_tiff::deconstructremains quadratic in a number a hostile file chooses. It isgamut-ifdcode, outside this manifest; recorded against gamut-ifd: the sub-IFD visited set is a linear scan, so a wide pointer array makes read_tree quadratic #578 above rather than filed as asixth issue, since gamut-ifd: the sub-IFD visited set is a linear scan, so a wide pointer array makes read_tree quadratic #578 is the issue for exactly this defect and this run may not edit it.
prose only.
Round 6 (continued) — one self-review finding, and the gates re-run on the final head
Reviewing this round's own diff before it was final surfaced one defect in it, in
an_exif_directory_whose_only_content_is_a_group_is_still_written: the test cloned the Interopdirectory it had just built and then asserted only
sub_ifds().len() == 1, so the clone wasneedless and the assertion could not distinguish the group it wrote from any other group of one.
Fixed in
81cab24fby comparing against the directory itself, which is both stronger and shorter.No library code changed.
All gates re-run on the final head
81cab24f:cargo test -p gamut-tiff --all-features --libcargo clippy -p gamut-tiff --all-targets --all-features -- -D warnings__CARGO_TEST_ROOT=$(git rev-parse --show-toplevel) mise run fmt-checkmise run check-testsconvco check origin/master..HEADmise run lintmise run testtest result: ok, zero FAILEDmise run mutants-diffNo
Cargo.tomlchanged, socheck-release-deps,check-ffi-featuresandcheck-ffi-headerarenot applicable; no new module with thin test reach was added, so
coveragewas left to CI'sCoverage (test gate).
Round 7 — review repairs N1–N6
Issue #446, branch
feat/446-tiff-metadata-seam, stacked on PR #508. No human approved thisround: it is an unattended run, and this record is what a human reads afterwards.
The lesson of the round, carried forward: round 6 closed with 178 mutants tested, 166 caught,
0 missed. That was not evidence this class had stopped. None of the four findings below is
reachable by any mutation operator — N1 is a missing branch, N2 and N3 are prose, and N4 is a
test that passes under the very mutation its comment names.
N1 (Medium) — a regression of the class round 6 was closing
check_exif_subtreeinspected a directory's sub-IFD groups;resolve_pointersinspects itsfields. The asymmetry was the defect: the one shape the reader misreads was the one shape the
writer never looked at.
Reproduced first, at the round-6 head
81cab24f. For each of the four tags, an Exif directorycarrying the tag as a plain
Value::Long:That is exactly the file
TiffMetadata::checkdocuments it exists to prevent — one "read back assomething other than what was written".
The regression is confirmed, and it is narrower than the review stated. With
pointer_tagsreverted to round 5's flat
[EXIF_IFD, INTEROPERABILITY_IFD], tags 330 and 34853 do round-trip(
Some(Long([4294901760])),Some(Long([8]))) — so round 6 newly broke those two. But 34665 and40965 fail identically at round 5: the blind spot itself predates round 6 for the two tags round
5 already followed. Round 6 widened an existing hole from two tags to four rather than opening it.
The fix inspects what the resolver inspects, at every level of the subtree, and is shaped by
the value's type rather than by its tag.
pointer_offsetsaccepts onlyLONG/IFD/LONG8/IFD8, so those four types under those four tags are refused, and every other type is left alone— a
SHORTunderSubIFDsis a pointer to neither side.After, all 20 combinations (4 tags × 4 pointer types + 4 tags ×
SHORT):No case remains in which this encoder emits a file its own decoder rejects. Literal round-tripping
of a pointer-typed field is impossible by the reader's own rule — a
LONGunder a pointer tag isa pointer, there being no other signal in a TIFF — so the shape that round-trips is the refusal,
plus the non-pointer types, both shown above.
Filed as #601:
gamut_ifd::Ifd::set_sub_ifdalready documents that its tag "must not alsobe
setas a regular field", and nothing enforces it.gamut-ifdis frozen base territory forthis lane, so the repair was made in the consumer and the question raised there.
N2, N3 — two false closure claims, and how the set was derived instead of listed
This is the third false closure claim on this pull request, so the sites were derived
mechanically, not listed. Four greps over
crates/gamut-tiff/://////!line naming a pointer tag by name or number//////!line naming the follow/resolve contract (follow*,resolve*,raw offset,stale offset,absolute offset,dangling)Deduplicated, that is eight sites that state this contract:
TiffMetadata::exif,TiffMetadata::check,IFD0_POINTER_TAGS,EXIF_SUBTREE_POINTER_TAGS,pointer_tags,check_exif_subtree(allsrc/metadata.rs),TiffDecoder::metadata(src/decoder.rs),TiffEncoder::with_metadata(src/encoder.rs), plusREADME.mdandSTATUS.md. Two had gonefalse and two were incomplete; the sweep is what found
README.mdandSTATUS.md, which thereview did not name.
src/decoder.rs— claimed onlyExifIFD(34665) andInteroperabilityIFD(40965) arefollowed. False in both directions: a dangling non-listed tag inside the Exif directory now
fails the call, which the doc implied cannot happen, and a dangling
InteroperabilityIFDatIFD 0 now succeeds, which the doc said is followed. It now states the per-level rule and what
each level costs.
src/encoder.rs— said nesting is "the one thing" that could break the agreement. Thereare three, each with its own message; all three are now named.
TiffMetadata::check— said "two bounds", "two distinct refusals". Now three, and it stateswhy only the Exif subtree is checked (the blocks
applywrites into IFD 0 sit under tags nolevel treats as a pointer).
README.mdandSTATUS.md— the writer's bound is restated in both; both now carry thefield clause and the reason the writer must inspect fields rather than groups.
Pinning the enumeration
EXIF_SUBTREE_POINTER_TAGSisgamut_ifd::tags::STANDARD_POINTER_TAGS— a sibling crate'sconstant — while eight sites and two error messages name its four members one by one. A fifth
member added upstream would widen a public contract with nobody deciding it.
Both refusal messages are now named constants (
POINTER_FIELD_REFUSAL,FOREIGN_GROUP_REFUSAL),and
the_exif_subtree_pointer_tags_are_the_four_this_crate_documentsasserts the constant equals[330, 34665, 34853, 40965]and that each message names all four by name. Reading constants only,it exercises no code path, so it is the sole failure under a fifth member and under nothing else.
N4 — the test that passed under the regression it named
a_standard_pointer_at_ifd_0_that_feeds_no_field_is_left_alonepassed withpointer_tagsreturning the full set at every level. Verified at
81cab24f:1 passed; 0 failed. Its firstassertion called
read, which resolves no pointer under any configuration; its second asked forexif, which staysNonewhether aSubIFDspointer at IFD 0 was resolved or not.It now drives
resolve_pointersdirectly at the depthread_metadatacalls it with, because IFD 0is the one directory the seam never hands back, and asserts the field stays the integer it was read
as and becomes no group. Under the regression it now fails, as its only failure.
N5 — the risks statement, corrected against this lane's favour
Round 6's risks section said: "A real camera EXIF with a broken GPS pointer therefore now errors
instead of silently handing back an unusable offset." That is wrong, and the correction shrinks
the stated risk.
GPSInfo(34853) belongs to the 0th IFD, not to the Exif directory — thisworkspace's own reader says so,
crates/gamut-exif/src/exif.rs:15: "GPSInfopointer (0th IFD →GPS sub-IFD)". At IFD 0 the widening does not apply: only
ExifIFDis followed there, so acamera's broken GPS pointer is tolerated exactly as before.
The widening therefore bites only a directory that nests
SubIFDsorGPSInfoinside theExifIFD, which no conformant file does. The round-6 sentence is superseded by this one; the round6 section is left unedited above, as the record requires.
N6 — the stale reference
tests/metadata.rsreferred in the present tense toPOINTER_TAGS, which round 6 split intoIFD0_POINTER_TAGSandEXIF_SUBTREE_POINTER_TAGS. It now namesIFD0_POINTER_TAGS.Each change is the sole failure under its own defect
Verified by reverting one thing at a time and running the whole
gamut-tiffsuite:the_writer_refuses_an_exif_pointer_tag_carried_as_a_pointer_typed_field— onlya_value_no_reader_would_follow_survives_under_a_pointer_tag— onlypointer_tagsreturns the full set at every levela_standard_pointer_at_ifd_0_that_feeds_no_field_is_left_alone— onlythe_exif_subtree_pointer_tags_are_the_four_this_crate_documents— onlyValidation (round 7)
All commands run in this worktree at head
31c08a63, in the mandated memory-capped scope for theworkspace-wide ones.
uptimechecked before each: load average 2.9–3.6, well under 32.cargo test -p gamut-tiff --all-features__CARGO_TEST_ROOT=$(git rev-parse --show-toplevel) mise run fmtthenmise run fmt-checkmise run check-testsconvco check origin/master..HEADmise run lint-D warningsmise run testmise run mutants-diffforover aconstslice guarding an earlyreturn Erryields no operator beyond the function-level one already in the set. Both mutants coveringcheck_exif_subtree—replace check_exif_subtree -> Result<()> with Ok(())anddelete ! in check_exif_subtree— are caught.mutants.out/missed.txtandmutants.out/timeout.txtare both empty. This is exactly the round's lesson: the gate cannot see a missing branch, which is what N1 was.No
Cargo.tomlwas touched, socheck-release-deps/check-ffi-features/check-ffi-headerdonot apply. No new module was added, so
coveragedoes not apply.Risks and rollout (round 7)
as a pointer-typed field is rejected by
with_metadata's encode. Nothing this crate's owndecoder produces has that shape — since round 6 such a file fails
metadata()outright — so theexposure is hand-built directories and directories parsed by third-party code. The alternative is
emitting a file this crate cannot read back, which is the defect.
encodes and reads back verbatim, because the reader leaves it alone. Refusing it would reject a
legal directory for a shape no reader misreads.
widening does not affect a camera file with a broken GPS pointer.
POINTER_FIELD_REFUSAL,FOREIGN_GROUP_REFUSAL,check_exif_subtree,pointer_tags,IFD0_POINTER_TAGSandEXIF_SUBTREE_POINTER_TAGSare allprivate. The only observable change is which inputs
TiffEncoderrefuses.Issues filed (round 7)
gamut-tiff: an ExifIFD pointer array naming two directories silently loses the second. Reproduced: a written field ofLong([26, 44])reads back as one directory andre-encodes as
Long([170]). Documented nowhere; three options set out, the contract decisionleft to whoever owns the seam.
gamut-ifd: Ifd::set_sub_ifd documents an invariant nothing enforces, and a violation is undetectable downstream. The sibling-crate half of N1.Decisions taken (round 7)
Unresolved review notes (round 7)
under any of the four tags inside the Exif directory still fails the whole
metadata()call.N5 establishes that this bites only non-conformant nestings, but it is still a widening and it is
not settled here.
checkis still not the reader's exact complement in one direction. It refuses a groupunder any tag outside the standard four; the reader would read such a file back without error,
returning that pointer as a raw offset. Refusing is the conservative direction and is stated on
TiffMetadata::check.file whose
ExifIFDnames more than one directory loses all but the first, silently.Round 8 — review repairs F1–F5, the enumeration gap, and a correctly-based mutation figure
Issue #446, branch
feat/446-tiff-metadata-seam, stacked on PR #508 (basefeat/442-dng-c2pa-manifest-store). No human approved this round: it is an unattended run, andthis record is what a human reads afterwards.
The lesson of the round. Round 7 shaped the writer's check by the in-memory
Valuevariant.The reader classifies by the on-disk type code.
Value::Unknownis the one value where thosetwo disagree, so the class round 7 was closing stayed open one shape wider. Every repair below is
stated at the write/read boundary, and every enumeration it depends on is now derived from a
constant or a function rather than written out by hand — a hand list short by one is what produced
seven rounds of this.
F1 — the discriminator is now the on-disk type code
gamut_ifd::UnknownValue::newvalidates only that the value word is the file variant's offsetwidth. It accepts a recognised type code,
writeemits that code verbatim, and the readerclassifies the entry by it. So a
Value::Unknownbuilt at code 4, 13, 16 or 18 is a plain field toa variant-shaped predicate and a pointer to the reader. It is reachable through the public API
with no
unsafeand no hand-built bytes.Reproduced at the round-7 head's predicate, over this round's derived sweep (see below), by
reverting the one clause and counting instead of asserting:
TIFF: read out of boundsThe 48 are exactly the
Unknown-at-a-pointer-code cases: 16 in classic TIFF (codes 4 and 13, theonly pointer codes whose type fits a 4-byte value word) and 32 in BigTIFF (codes 4, 13, 16, 18),
each across the four tags and both nesting levels. Post-fix they are refused by
check, which iswhy the exercised count falls by 48.
Reconciliation with the review's figure, which was untrusted evidence and is now checked: the
review reported 64 of 384. The 16-case difference is construction, not disagreement — its sweep
also built an
Unknownat a code whose type is wider than the variant's value word (LONG8inclassic TIFF), which is written out of line, so the word is a raw file offset and the entry is
unreadable under any tag. That is a malformed entry rather than a misclassified pointer, this
crate cannot refuse it without refusing every genuine vendor entry, and
every_representable_valuedocuments the exclusion in place. The review's other two messages (
value offset out of bounds,sub-IFD pointer loop) and its BigTIFF read-back-as-a-group case are properties of the value wordchosen, not of the classification; this sweep's word (
0x08repeated) yields one message. Theclassification defect is the same defect in both accounts.
The fix.
POINTER_TYPE_CODES = [4, 13, 16, 18]andis_pointer_typed, asked ofValue::type_code(), replace thepointer_offsets-shaped clause incheck_exif_subtree. Thecodes are pinned to the resolver by
the_pointer_type_codes_are_exactly_the_codes_the_resolver_follows, so the two sides of theboundary cannot drift apart silently.
The sweep is derived, not listed.
every_type_code()filters0..=u16::MAXthroughgamut_ifd::FieldType::from_code;every_representable_value(variant)yields, per code, thenatural
Valuevariant and theUnknownform carrying that same code, plus threeUnknowns atcodes no field type claims. Crossed with the four pointer tags, classic and BigTIFF, and
top-level and one-level-nested, that is 544 enumerated / 432 exercised cases in
every_value_the_writer_accepts_under_a_pointer_tag_reads_back_as_a_field— measured, not countedby hand. A type code added upstream enters the sweep with no edit here.
The sibling half is filed, not fixed (
gamut-ifdis out of this lane's bounds and other lanesare stacked on it): #608,
gamut-ifd: UnknownValue::new accepts a recognised type code, so an Unknown value lies about what it is on disk. It extends #601's shape — a constructor that admits avalue nothing downstream can distinguish — and is cited from
POINTER_TYPE_CODESand fromevery_representable_valuein the code. #601 was not edited.F2 — a field and a group under one tag is refused, and the judge now grades it
Ifdkeeps fields and groups in two lists, so one tag can sit in both. The writer then emits twoentries under that tag, which TIFF 6.0 §2 does not allow, and the reader's duplicate-collapse
drops one.
deconstruct— this crate's own judge — graded such a file clean and fullyaccounted, which is precisely why no round trip in seven rounds could see it.
FIELD_BESIDE_GROUP_REFUSAL, its own message, its own clause, pinnedby
the_writer_refuses_a_field_and_a_group_under_one_tagacross all four tags with aSHORTvalue, so neither of the other two clauses can be what fires.
Anomaly::DuplicateTag { ifd, tag, entries }(the enum and thevariant are both
#[non_exhaustive], so this is additive).check_duplicate_tagsre-reads theraw entry records of every directory the audit says it walked, because by the time the parsed
tree exists the duplicate has already collapsed. Naming the directory by offset rather than by
page is what lets it report a metadata sub-IFD several levels down.
TiffEncoder::with_metadata,README.mdandSTATUS.md— said unconditionally that what the caller supplies is what the file gets. They nowstate the refusal.
Refusing rather than allowing is decision 3: the alternative is an encoder that emits
non-conformant bytes and a reader that silently drops a field the caller set. A non-minimal writer
is the cheaper cost.
The oracle, on the one question a round trip cannot answer
Nothing in this pull request had consulted libtiff about the seam, and a repeated tag is exactly
the defect a self-consistent round trip is blind to: gamut writes and reads by one rule, so a wrong
rule is wrong symmetrically.
tests/oracle_metadata.rsnow builds a 2×2 RGB TIFF byte by byte— no directory model can express two entries under one tag — carrying
PhotometricInterpretationtwice,
RGBthenBlackIsZero. libtiff decodes it as RGB (it marks every occurrence after thefirst to be ignored,
tif_dirread.c, and warns the directory is unsorted);TiffDecoder::inforeports BlackIsZero, because
gamut-ifd's model keeps the last. Which entry survives is aproperty of the reader, not of the file — which is the whole justification for refusing to write
one.
F3 — the sole-failure table, re-derived by executing every row
Round 7's table was asserted. Executed at this head, one revert at a time, running the whole
gamut-tiffsuite (--no-fail-fast), it is:Valuevariant again (F1 restored)every_value_the_writer_accepts_under_a_pointer_tag_reads_back_as_a_field— onlythe_writer_refuses_a_field_and_a_group_under_one_tag— onlyIFD0_POINTER_TAGSby a fourth standard pointer tagifd_0_resolves_the_exif_directory_and_no_other_standard_pointer— onlypointer_tagsreturns the full set at every levelifd_0_resolves_the_exif_directory_and_no_other_standard_pointeranda_broken_pointer_the_metadata_does_not_use_does_not_hide_the_blocks(2)POINTER_TYPE_CODESshort by one (IFD8dropped)every_value_…_reads_back_as_a_field,the_pointer_type_codes_are_exactly_the_codes_the_resolver_follows,the_writer_refuses_an_exif_pointer_tag_carried_as_a_pointer_typed_fieldgamut_ifd::tags::STANDARD_POINTER_TAGSthe_exif_subtree_pointer_tags_are_the_four_this_crate_documents— onlyentries > 2)flags_a_directory_that_repeats_a_tag— onlyentries > 0)flags_a_directory_that_repeats_a_tagplus the three whole-file accounting testsTwo rows fail more than one test and are published as such rather than trimmed. Both are
defensible: dropping a pointer type code is a wider defect than one test's claim, and a judge that
reports every tag breaks every file that has one. The
pointer_tagsrow is the one that matters forF3 — round 7's table claimed one failure there and there are two.
Correction to round 7's decision 4. Its recorded ground was "no public observation can see
whether IFD 0's pointers were resolved". That is false: the second failure in that row,
a_broken_pointer_the_metadata_does_not_use_does_not_hide_the_blocksintests/metadata.rs, isexactly such an observation, in this crate, today. The rewrite itself stands — driving
resolve_pointersat the depthread_metadatacalls it with is still the narrower reach, and it isthe only test that fails for the IFD-0 scoping rule alone — but it is justified by reach, not by
impossibility. Round 7's entry is left unedited above, as the publishing contract requires.
F4 — the IFD-0 scoping rule is pinned, from the constant
Round 6 changed it and round 7 documented it, and nothing held it: a fifth tag added to
IFD0_POINTER_TAGSleft 25 test binaries green, and aconst's contents are not a mutableexpression, so the mutation gate cannot reach it either.
ifd_0_resolves_the_exif_directory_and_no_other_standard_pointernow derives its iteration setfrom
EXIF_SUBTREE_POINTER_TAGS— it asserts thatExifIFDis resolved at IFD 0 and that everyother member of that constant is left as a field — so it fails under a widening without anyone
having repeated the list. Repeating a list by hand is the thing that let this through.
F5 — the line length is measured, not asserted
Round 6 claimed no added line exceeds 100 columns; round 7 added one of 151. Rewrapped, and the
claim is now a measurement made in this round:
The six lines over 100 columns still in
STATUS.mdare all present unchanged atorigin/master;this branch adds none. (
.rsfiles are excluded from the claim: rustfmt does not rewrap comments,and the workspace carries ~1200 such lines.)
The enumeration — 44 candidate sites, derived by script
The review derived 14 contract-stating sites against round 7's 10, and one of the three missed —
pointer_offsets— is where "a pointer's own type" was defined, and therefore the block F1 had tochange. So this round derived the set with a script rather than a list. Four sweeps over
crates/gamut-tiff/src, plus one repo-wide://////!) naming a pointer tag by name or numberLONG8,IFD8,`LONG`,`IFD`,POINTER_TYPE_CODES, "type code")STANDARD_POINTER_TAGS,pointer_offsets,is_pointer_typed,EXIF_SUBTREE_POINTER_TAGS,IFD0_POINTER_TAGS)Each hit is then mapped to its enclosing declared item by walking forward to the first item
declaration, and deduplicated: 44 distinct sites across 8 files, of which 22 are reached by a
pointer sweep (1 or 4). That superset contains all 14 the review named,
pointer_offsetsincluded, and it is reproducible — the script is what future rounds re-run instead of writing a
list. The five false closure claims are the ones fixed above: the N1 sentence and
check's headline(F1), the
with_metadata/README/STATUSsentence (F2), the round-7 table row (F3) and round 6'sline-length claim (F5).
Decision 10 — the same asymmetry probed in
gamut-dng, and filedNobody had checked whether the defect reaches the crate that shares the reader. It does. Probed
read-only; nothing in
gamut-dngwas changed, and no file outsidecrates/gamut-tiff/is inthis diff.
DngEncoderapplies no shape check toDngMetadata::exif.DngRewrite::opencallsgamut_ifd::read_tree(data, STANDARD_POINTER_TAGS), which resolves the flat list at everynode, so a pointer-typed field inside the Exif directory is followed as a file offset. All nine
combinations tested (three tags ×
Long([8]),Long([26]),Unknown(code 4)) encode and thenfail
DngRewrite::open— the crate's own preserving open→edit→write path.absolute offset as a plain integer, and
gamut_dng::deconstructgrades the file clean and fullyaccounted.
Filed as #610,
gamut-dng: DngEncoder writes an Exif directory DngRewrite cannot open, and deconstruct grades the loss clean, with the reproductions and the note that a check copied fromthis crate's first attempt would miss the
Unknownrow.Validation (round 8)
All commands run in this worktree at head
dc49d235, the workspace-wide ones inside the mandatedmemory-capped scope.
uptimechecked before each: load average 2.4–27.5, under 32.cargo test -p gamut-tiff --all-features__CARGO_TEST_ROOT=$(git rev-parse --show-toplevel) mise run fmt-checktooling/*/Cargo.toml, so it is needed regardless of this crate)mise run check-testsconvco check origin/feat/442-dng-c2pa-manifest-store..HEADmise run lint-D warningsmise run testmise run mutants-diffwithGAMUT_MUTANTS_BASE=origin/feat/442-dng-c2pa-manifest-storemise run mutants-diffwith the default baseorigin/mastergamut-tiff, 46gamut-ifd, 32gamut-dng— so 78 of the 187 (42 %) belong to PR #508 underneath, and the stacked-base run selects exactly the 109 that do notNo
Cargo.tomlwas touched (git diff --name-onlyagainst the base lists twelve files, all undercrates/gamut-tiff/), socheck-release-deps/check-ffi-features/check-ffi-headerdo notapply. No new module was added, so the local
coveragegate does not apply; CI'sCoverage (test gate)is green at this head.Which base the mutation figure is about (decision 8). Round 7 published 178 / 166 caught / 12
unviable / 0 missed against
origin/master; the review reproduced that byte for byte, so it is notwrong — it is a statement about the wrong thing. Both bases are re-run at this head above, and the
split is exact: the
origin/masterselection is 187 mutants of which 78 (42 %) belong to PR #508underneath, while the stacked-base selection is precisely the 109 that are this branch's own. Both
are true; only the second is about this round, and the two agree on the only number that decides the
gate — 0 missed, 0 timeouts, both artefact files empty, so neither is an exit-3 timeout
masquerading as a pass.
What the mutation gate does not hold, stated plainly. F1 was a missing branch, F2 a missing
refusal and a missing judge clause, F4 the contents of a
const. No mutation operator reaches anyof them, so a green
mutants-diffis no evidence about this repair at all. What holds them is theexecuted sole-failure table above — each is the failure of a test that fails for it, and the
POINTER_TYPE_CODESandEXIF_SUBTREE_POINTER_TAGSrows show the two constants are pinned againstdrift in both directions.
Risks and rollout (round 8)
with_metadata. AValue::Unknowncarrying a pointer typecode under one of the four standard pointer tags, and a tag carrying both a field and a group. The
first was a file this encoder could not read back; the second was two entries under one tag, which
is not a TIFF directory. Neither is producible by this crate's own decoder, so the exposure is
hand-built directories and directories parsed by third-party code.
deconstructreports on files it previously graded clean.Anomaly::DuplicateTagis new, soa caller that asserted
anomalies.is_empty()on a file with a repeated tag now sees it. That isthe point; the enum and the variant are
#[non_exhaustive], so no match arm breaks.deliberately stored an opaque
Unknownat code 4 underSubIFDs— expecting it back verbatim —is now refused. There is no reader for which that expectation could have held.
(
POINTER_TYPE_CODES,is_pointer_typed,FIELD_BESIDE_GROUP_REFUSAL,check_duplicate_tags)is private.
Issues filed (round 8)
gamut-ifd: UnknownValue::new accepts a recognised type code, so an Unknown value lies about what it is on disk. The sibling-crate half of F1 (decision 2); extends gamut-ifd: Ifd::set_sub_ifd documents an invariant nothing enforces, and a violation is undetectable downstream #601's shape.gamut-dng: DngEncoder writes an Exif directory DngRewrite cannot open, and deconstruct grades the loss clean. The sibling exposure of both F1 and F2 (decision 10).Neither #601 nor any other existing issue was edited, commented on, labelled or closed.
Decisions taken (round 8)
Corrections to earlier entries in this record, collected here because a frozen entry is never
edited in place: round 7's decision 4 — its stated ground is false; see F3 above for the true
one. Round 6's line-length claim — false at the round-7 head; see F5, now measured.
Unresolved review notes (round 8)
missing branch, a missing clause and the contents of a
const. The sole-failure table is whatholds them, and it is executed rather than asserted — but it is a table this lane wrote, checked
by this lane.
gamut-ifdconsumer. This crate now classifies correctly; theconstructor still admits the value, so
gamut-exif,gamut-dngand any downstream user has toknow to ask
type_code(). Onlygamut-dngwas probed (gamut-dng: DngEncoder writes an Exif directory DngRewrite cannot open, and deconstruct grades the loss clean #610);gamut-exifwas not.ExifIFDloss is still live and silent.
checkis still not the reader's exact complement in one direction — it refuses a groupunder any tag outside the standard four, which the reader would read back without error. Refusing
is the conservative direction and is stated on
TiffMetadata::check.Round 9 — the oracle test that could not fail for its own claim, and three contracts pinned
Issue #446, branch
feat/446-tiff-metadata-seam, stacked on PR #508 (basefeat/442-dng-c2pa-manifest-store). No human approved this round: it is an unattended run, andthis record is what a human reads afterwards.
The lesson of the round. Round 8 put the repeated-tag claim to libtiff, and the test it wrote
asserted a decode the repeated tag could not move. An oracle test that passes without consulting
the oracle is the shape this pull request has now produced twice; the repair is to make the
repeated tag reach the readout, and then to falsify the test in both readers rather than one.
M1 — the fixture now repeats a tag the decoded bytes depend on
The old fixture repeated
PhotometricInterpretation(262) and asserted libtiff's dimensions,sample count and pixels. The oracle harness returns width, height, samples-per-pixel and 8-bit
pixels and no photometric field at all; it reads the tag only to invert 1-bit
MinIsWhitedata,and this fixture is 8-bit. So libtiff's half of the assertion was blind to the repeat, and both
halves of the review's claim reproduce here by execution:
262entries swapped in the fixture(2, 2, 3)and the same twelve pixel bytes, byte-identical; only this crate's answer moves (Rgbinstead ofBlackIsZero)ma->tdir_ignoreforna->tdir_ignore, both sites intif_dirread.c, rebuilt)The replacement repeats
StripOffsets(273). Two 12-byte strips are laid down back to back andthe two entries point at different ones, so for uncompressed chunky data the scanline bytes a
reader hands back are the answer to "which entry did you keep". Both orderings are built —
leading entry at the first strip, then at the second — which separates "follows the first entry"
from "follows the lower offset". Proved in both directions at this head, one falsifier at a time,
each reverted afterwards:
gamut_ifd::Ifd::setkeeps the first occurrence (libtiff's rule)[1..12], wanted[101..112][101..112], wanted[1..12]The submodule and
gamut-ifdwere restored and the suite re-run green after each; neither is inthis branch's diff (
git statusclean,git -C third_party/libtiff statusclean).M1, second half — a correction to round 8's decision 4
Round 8's decision 4 recorded, as its rejected alternative: "asserting libtiff's warning text
(rejected: wording is not a contract — the decoded pixel interpretation is)". That ground is
false as stated, and the round-8 entry is left unedited above as the publishing contract requires.
What the harness actually observes is:
width,height,samples_per_pixel, and the 8-bitpixelsbuffer. It does not surface libtiff's warnings, and it does not surface libtiff'sPhotometricInterpretation— the value is read internally and used only on the 1-bit path. So thechoice was never "warning text versus pixel interpretation": neither was observable, and the test
asserted a decode the repeat could not change. The decision to prefer an observable over a warning
string stands; its ground is that the pixels are observable and both other candidates were not,
which is exactly why the repaired fixture moves the pixels.
L1 — the sole-failure row is true for one of its three instantiations
Round 8's table row "widen
IFD0_POINTER_TAGSby a fourth standard pointer tag →ifd_0_resolves_the_exif_directory_and_no_other_standard_pointer— only" does not hold for thewhole class. Re-executed at this head, one tag at a time, whole
gamut-tiffsuite with--no-fail-fast:SubIFDs(330)ifd_0_resolves_the_exif_directory_and_no_other_standard_pointeranda_broken_pointer_the_metadata_does_not_use_does_not_hide_the_blocksGPSInfo(34853)InteroperabilityIFD(40965)ifd_0_resolves_the_exif_directory_and_no_other_standard_pointeronlyThe row as published is true only of
InteroperabilityIFD. The reason is visible in the secondtest: it hangs a dangling pointer off
SubIFDsand offGPSInfoand asserts the blocks survive,so widening IFD 0 by either of those two makes the dangling pointer fatal; it never uses
InteroperabilityIFD. Two failures is the better outcome — a widening that costs the metadataof a readable file should fail more than a naming test — so the correction is to the table, not to
the code. This is the same imprecision round 7's table was corrected for one row above; the fix
is to name the instantiation rather than the class.
I1 — the record named the wrong constant, and the code's choice is the stronger one
Round 8's F4 and its decision 6 say the pin "derives its iteration set from
EXIF_SUBTREE_POINTER_TAGS". The code iteratesgamut_ifd::tags::STANDARD_POINTER_TAGS. Samemembers —
EXIF_SUBTREE_POINTER_TAGSis defined as that constant — and for a repair whose wholepoint is deriving a set from the constant it names, naming the wrong one is not a detail. The
round-8 entries are left unedited; this is the correction.
Deriving from the sibling crate's constant is deliberate, and it is the stronger of the two. The
domain this sweep has to cover is "every standard pointer tag that exists", because the failure it
guards is a tag IFD 0 silently starts following. Derived from
STANDARD_POINTER_TAGS, a fifthmember added upstream enters the sweep whatever this crate's alias is doing. Derived from the local
alias, a later narrowing of the alias would shrink the sweep silently — the hand-maintained-set
failure this pull request has now hit three times, moved one indirection further out. The comment
at the test now names the constant and says why, rather than the name being swapped in the record
to match.
The enumeration is now a command, not a count
Round 8 published "44 candidate sites across 8 files" from a script that was never committed and
queries stated in prose. That is half a derivation: nobody who was not here can re-run it, and the
review's re-sweeps disagreed on three of the five counts. The queries are therefore published
verbatim. Paste this at the repository root; it needs only
sh,grepandawk:At this head it prints, byte-identically on repeated runs:
These supersede round 8's figures, and are not a correction of them, because they are not a
measurement of the same thing: round 8's sweeps ran over a different head (this round adds ~90
lines of documentation to the same files), and its mapping step was described but not published,
so the two cannot be reconciled — which is the defect. Stated plainly: the mapping from a hit to a
"site" is a heuristic — the nearest item declaration at or after the hit — so 63 is a
reproducible proxy for "distinct contract-stating sites", not a semantic enclosure count. Its value
is that it is the same number for everyone and moves when the crate does. The script is not
committed because
crates/gamut-tiff/**is this round's manifest and a sweep over the crate is notcrate content; publishing it verbatim is the alternative the record allows.
Design question 1 — the new anomaly variant carries a severity
Anomaly::Structurereports aSeverity;Anomaly::DuplicateTag, added by this pull request, didnot, so a caller triaging a report had to know which variants carry one. It now carries
severity: Severity::Error— a repeated tag is a structural defect of exactly that kind, sinceTIFF 6.0 §2 gives a directory one entry per tag and the field a caller set is dropped by at least
one reader. Added now because the variant is new and unreleased: the same field arriving later is a
breaking change to any
#[non_exhaustive]match that binds it.flags_a_directory_that_repeats_a_tagmatchesSeverity::Error, and grading itWarningfailsthat test alone (executed).
Design question 2 — the skip's promise is stated precisely and pinned
check_duplicate_tagswalks the audit'sIfdBodyspans andcontinues past anything it cannotre-read, on the recorded ground that such a directory "is already reported by the audit's own
finding". Nothing failed when that stopped being true. The doc now separates the two silent skips
and states the contract as what a caller may rely on:
IfdBodyspan re-parses here by construction — thespan exists only because the audit already read a directory at that offset, so that
continueisunreachable rather than lenient;
reaches it; it arrives as
AuditFinding::SkippedSubIfd, which becomes anAnomaly::StructureofSeverity::Error. No file is gradedis_fully_accountedon the strength of a directory nobodyread.
an_unparsable_sub_ifd_is_reported_rather_than_silently_skippedpins the second on the plainunreadable target — a
SubIFDsoffset past the end of the file. That isSkipReason::Unparseable,the arm neither the cycle test nor the depth test reaches, and no test reached it before: changing
its detail string fails this test alone (executed).
Design question 3 — the asymmetry is pinned
TiffMetadata::checkrefuses a sub-IFD group under a tag outside the four the reader resolves,while the reader takes such a file without error and hands the tag back as the raw absolute
offset it was written as — the child directory is lost silently. That asymmetry is the stated
reason refusing is the conservative direction, and it was stated only in prose: nothing failed if
the reader began resolving the tag, and nothing failed if it began refusing it either.
the_reader_accepts_the_exif_group_tag_the_writer_refusesdrives the reader over a file builtthrough
gamut_ifd::write, socheckis not in the path, and asserts both halves — the tag comesback as a bare file offset, and no group carries it. Making the reader resolve the tag fails it
alone (executed). The refusal's own half is
the_writer_refuses_an_exif_group_under_a_tag_the_reader_does_not_resolve, unchanged.Validation (round 9)
All commands run at head
8c09576e, in the lane worktree, the workspace-wide ones inside themandated memory-capped scope.
uptimechecked before each: load average 13.6–17.7, under 32.cargo test -p gamut-tiff --all-features__CARGO_TEST_ROOT=$(git rev-parse --show-toplevel) mise run fmt-checktooling/*/Cargo.toml, so it is needed regardless of this crate)mise run check-testsconvco check origin/feat/442-dng-c2pa-manifest-store..HEADmise run lint-D warningsmise run testtest result: oklines, 0 failures across the workspacemise run mutants-diffwithGAMUT_MUTANTS_BASE=origin/feat/442-dng-c2pa-manifest-storeThe base is stated beside the count deliberately: with the default
origin/masterbase theselection folds in PR #508's own mutants underneath and is not evidence about this branch.
No
Cargo.tomlwas touched, socheck-release-deps/check-ffi-features/check-ffi-headerdonot apply. No new module was added, so the local
coveragegate does not apply.What the mutation gate does not hold, again stated plainly. This round's four repairs are: a
test fixture (no mutation operator applies to a test), a struct-literal field (
severity: Severity::Erroris not a mutable expression), a documentation paragraph, and two new tests. Agreen
mutants-diffis therefore no evidence about any of them. What holds them is that each wasfalsified by executing the change it is supposed to catch — the two-way falsification table under
M1, and the three "fails alone" measurements under the design questions. The oracle fixture is the
one that could not be held by anything inside this repository at all: proving its libtiff half
required patching the vendored libtiff, rebuilding it and reverting.
Decisions taken (round 9)
Corrections to earlier entries in this record, collected here because a frozen entry is never
edited in place: round 8's decision 4 — its rejected-alternative ground is false; the harness
observes neither libtiff's warning text nor its photometric interpretation. See M1's second half.
Round 8's F4 and decision 6 — they name
EXIF_SUBTREE_POINTER_TAGS; the code derives fromgamut_ifd::tags::STANDARD_POINTER_TAGS. See I1. Round 8's sole-failure table, row 3 — true ofone of its three instantiations. See L1.
Unresolved review notes (round 9)
field, a documentation paragraph and two tests. Each was falsified by execution instead, and the
falsifications are listed above with their observed failure messages; but they are falsifications
this lane ran and this lane recorded.
by patching the vendored submodule, rebuilding, observing the failure and reverting. A future
libtiff bump that changed the duplicate rule would fail the test — which is the point — but
nothing checks that the rule is still what the comment says between bumps.
mapping takes the nearest item declaration at or after a hit, which is right for a doc comment
and approximate for a hit inside a function body.
gamut-ifdconstructor)is live in every other consumer and only
gamut-dngwas probed; gamut-dng: DngEncoder writes an Exif directory DngRewrite cannot open, and deconstruct grades the loss clean #610 is unfixed livebehaviour in the crate this branch is stacked above; round 6's pointer-resolution widening is
still unsettled (gamut-tiff: InteroperabilityIFD is followed at IFD 0, where no conformant file puts it #582, gamut-tiff: a dangling IFD-0 pointer costs a single-page file its C2PA store, while a two-page file recovers #581) and gamut-tiff: an ExifIFD pointer array naming two directories silently loses the second #599's multi-directory
ExifIFDloss is still liveand silent.
Round 9 (continued) — one self-review finding, and the gates re-run on the final head
Reviewing this round's own diff before finishing: the replacement fixture lays its two strips back
to back and wrote the second one's offset as
+ 12— the first strip's length as a literal — whileStripByteCountsderived the same number from the array. The "value layout drifted" assertioncovers only the first strip, so widening the pixel blocks would have overlapped them silently.
Both now come from one binding (
41178154). Nothing else was changed.The head is therefore
41178154, not8c09576eas the table above records, and every gate wasre-run there rather than the table being edited:
41178154cargo test -p gamut-tiff --all-features__CARGO_TEST_ROOT=$(git rev-parse --show-toplevel) mise run fmt-checkmise run check-testsconvco check origin/feat/442-dng-c2pa-manifest-store..HEADmise run lintmise run testtest result: oklines, 0test result: FAILEDmise run mutants-diffwithGAMUT_MUTANTS_BASE=origin/feat/442-dng-c2pa-manifest-storeThe enumeration counts published above were taken at
8c09576e; the only later change is inside atest function body, so re-running the script at
41178154prints the same seven lines (verified).