From e7621d510daac223828235bbb4a68924c644932d Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 01:07:06 -0400 Subject: [PATCH 01/24] test(fuzz): drive the parser entry points on untrusted bytes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The tier so far drove `invariants` laws over normalised inputs. This adds the other half `docs/testing.md`'s per-crate table asks for: a driver on each crate's untrusted-input surface, handed the engine's bytes unchanged. Six targets — `ifd_read`, `tiff_decode`, `dng_decode`, `isobmff_boxes`, `heic_container`, `heic_hvcc` — one per entry point rather than per crate, since gamut-heic's container walk and its `hvcC`/NAL layer are independent surfaces. Their primary oracle is the engine's own: each crate is `forbid(unsafe_code)` and promises a typed error, so a panic, a hang or an allocation past the malloc limit is the defect. Each target adds one check beyond that — reader agreement and the dual-ledger audit for IFD, the segment tiling for the two container walks, the documented `annex_b` composition for the NAL layer — so a defect that produces no crash is still visible. `corpus/` carries a curated seed set, force-added past the ignore that exists to keep the engine's search state out of the tree: the malformed cases enumerated on the issue for `ifd_read`, and one small well-formed file per decoder target so it starts from something that reaches its pixel path. Refs #264 --- tooling/gamut-fuzz/Cargo.toml | 57 +++++++++++- tooling/gamut-fuzz/README.md | 47 ++++++++++ .../corpus/dng_decode/cfa-12bit.dng | Bin 0 -> 872 bytes .../heic_container/heic-single-item.heic | Bin 0 -> 272 bytes .../heic_hvcc/main-still-vps-sps-pps.bin | Bin 0 -> 57 bytes .../corpus/ifd_read/01-truncated-header.tif | Bin 0 -> 4 bytes .../corpus/ifd_read/03-invalid-byte-order.tif | Bin 0 -> 8 bytes .../corpus/ifd_read/04-invalid-magic.tif | Bin 0 -> 8 bytes .../ifd_read/05-ifd0-offset-past-eof.tif | Bin 0 -> 8 bytes .../ifd_read/06-truncated-entry-count.tif | Bin 0 -> 8 bytes .../corpus/ifd_read/07-truncated-entries.tif | Bin 0 -> 16 bytes .../ifd_read/08-value-offset-past-eof.tif | Bin 0 -> 26 bytes .../corpus/ifd_read/09a-circular-ifd-self.tif | Bin 0 -> 14 bytes .../ifd_read/09b-circular-ifd-two-node.tif | Bin 0 -> 20 bytes .../ifd_read/10a-hostile-entry-count.tif | Bin 0 -> 10 bytes .../10b-hostile-entry-count-bigtiff.tif | Bin 0 -> 24 bytes .../ifd_read/11-hostile-value-count.tif | Bin 0 -> 26 bytes .../ifd_read/12-unknown-tag-preserved.tif | Bin 0 -> 26 bytes .../corpus/ifd_read/13-unknown-field-type.tif | Bin 0 -> 26 bytes .../isobmff_boxes/avif-single-item.avif | Bin 0 -> 229 bytes .../corpus/tiff_decode/rgb8-lzw.tif | Bin 0 -> 264 bytes .../corpus/tiff_decode/rgb8-none.tif | Bin 0 -> 252 bytes tooling/gamut-fuzz/fuzz_targets/dng_decode.rs | 62 +++++++++++++ .../gamut-fuzz/fuzz_targets/heic_container.rs | 87 ++++++++++++++++++ tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs | 78 ++++++++++++++++ tooling/gamut-fuzz/fuzz_targets/ifd_read.rs | 82 +++++++++++++++++ .../gamut-fuzz/fuzz_targets/isobmff_boxes.rs | 71 ++++++++++++++ .../gamut-fuzz/fuzz_targets/tiff_decode.rs | 72 +++++++++++++++ 28 files changed, 553 insertions(+), 3 deletions(-) create mode 100644 tooling/gamut-fuzz/corpus/dng_decode/cfa-12bit.dng create mode 100644 tooling/gamut-fuzz/corpus/heic_container/heic-single-item.heic create mode 100644 tooling/gamut-fuzz/corpus/heic_hvcc/main-still-vps-sps-pps.bin create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/01-truncated-header.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/03-invalid-byte-order.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/04-invalid-magic.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/05-ifd0-offset-past-eof.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/06-truncated-entry-count.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/07-truncated-entries.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/08-value-offset-past-eof.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/09a-circular-ifd-self.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/09b-circular-ifd-two-node.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/10a-hostile-entry-count.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/10b-hostile-entry-count-bigtiff.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/11-hostile-value-count.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/12-unknown-tag-preserved.tif create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/13-unknown-field-type.tif create mode 100644 tooling/gamut-fuzz/corpus/isobmff_boxes/avif-single-item.avif create mode 100644 tooling/gamut-fuzz/corpus/tiff_decode/rgb8-lzw.tif create mode 100644 tooling/gamut-fuzz/corpus/tiff_decode/rgb8-none.tif create mode 100644 tooling/gamut-fuzz/fuzz_targets/dng_decode.rs create mode 100644 tooling/gamut-fuzz/fuzz_targets/heic_container.rs create mode 100644 tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs create mode 100644 tooling/gamut-fuzz/fuzz_targets/ifd_read.rs create mode 100644 tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs create mode 100644 tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs diff --git a/tooling/gamut-fuzz/Cargo.toml b/tooling/gamut-fuzz/Cargo.toml index 3318902a..1b6cf8af 100644 --- a/tooling/gamut-fuzz/Cargo.toml +++ b/tooling/gamut-fuzz/Cargo.toml @@ -30,12 +30,21 @@ cargo-fuzz = true [dependencies] libfuzzer-sys = "0.4" -# Each target's crate is pulled in with `test-support`, the `doc(hidden)` feature that exposes its -# `invariants` module. That feature is never enabled by the `gamut` umbrella, so the shipped +# A *law* target's crate is pulled in with `test-support`, the `doc(hidden)` feature that exposes +# its `invariants` module. That feature is never enabled by the `gamut` umbrella, so the shipped # surface and `mise run check-ffi-features` are unaffected — verified in #434. -gamut-ifd = { path = "../../crates/gamut-ifd", features = ["test-support"] } +# +# A *robustness* target's crate needs no such feature: it drives the crate's ordinary public +# parser entry point on the engine's raw bytes, which is exactly the surface a caller has. +# `bigtiff` is on for gamut-ifd so the 64-bit variant is reachable from the same byte string +# rather than being silently rejected at the magic number. +gamut-ifd = { path = "../../crates/gamut-ifd", features = ["bigtiff", "test-support"] } gamut-core = { path = "../../crates/gamut-core", features = ["test-support"] } gamut-tonemap = { path = "../../crates/gamut-tonemap", features = ["test-support"] } +gamut-tiff = { path = "../../crates/gamut-tiff" } +gamut-dng = { path = "../../crates/gamut-dng" } +gamut-isobmff = { path = "../../crates/gamut-isobmff" } +gamut-heic = { path = "../../crates/gamut-heic" } [[bin]] name = "ifd_read_ledger" @@ -58,5 +67,47 @@ test = false doc = false bench = false +[[bin]] +name = "ifd_read" +path = "fuzz_targets/ifd_read.rs" +test = false +doc = false +bench = false + +[[bin]] +name = "tiff_decode" +path = "fuzz_targets/tiff_decode.rs" +test = false +doc = false +bench = false + +[[bin]] +name = "dng_decode" +path = "fuzz_targets/dng_decode.rs" +test = false +doc = false +bench = false + +[[bin]] +name = "isobmff_boxes" +path = "fuzz_targets/isobmff_boxes.rs" +test = false +doc = false +bench = false + +[[bin]] +name = "heic_container" +path = "fuzz_targets/heic_container.rs" +test = false +doc = false +bench = false + +[[bin]] +name = "heic_hvcc" +path = "fuzz_targets/heic_hvcc.rs" +test = false +doc = false +bench = false + # Prevent this from being picked up as a workspace member of anything above it. [workspace] diff --git a/tooling/gamut-fuzz/README.md b/tooling/gamut-fuzz/README.md index 0e8bc62e..d533e878 100644 --- a/tooling/gamut-fuzz/README.md +++ b/tooling/gamut-fuzz/README.md @@ -70,6 +70,13 @@ same rule `docs/testing.md` applies to a shrunk `proptest` counterexample, and t ## Targets +There are two kinds, and the difference is what the target's oracle is. + +### Law targets + +They drive a crate's `invariants` module — the same functions the pinned-seed properties drive — +over normalised inputs, per the section above. + | target | crate | laws | |---|---|---| | `ifd_read_ledger` | `gamut-ifd` | `ledger_is_canonical`, `subtract_is_set_difference` | @@ -78,6 +85,46 @@ same rule `docs/testing.md` applies to a shrunk `proptest` counterexample, and t One file per crate, deliberately, so adding a crate is an additive change. +### Robustness targets (#264) + +They hand the engine's bytes, unchanged, to the **parser entry point** `docs/testing.md`'s +per-crate table names in its "Fuzz entry point" column — the surface an untrusted file arrives on. +There is no law function to share, because the primary oracle is the engine's own: every one of +these crates is `#![forbid(unsafe_code)]` and promises a *typed error* on hostile input, so a +panic, a hang, or an allocation past libFuzzer's limit is the defect. + +Each target adds at least one check the engine cannot make on its own, so that a defect producing +no crash is still visible: + +| target | crate | entry points | check beyond the crash oracle | +|---|---|---|---| +| `ifd_read` | `gamut-ifd` | `read`, `read_tree`, `read_audited`, `IfdReader` | slice and streaming readers agree; the dual-ledger audit is complete | +| `tiff_decode` | `gamut-tiff` | `TiffDecoder::{page_count,info_page,decode_page}` | the page index is bounded by `page_count`; describing and decoding agree on geometry | +| `dng_decode` | `gamut-dng` | `DngDecoder::{decode,verify_new_raw_image_digest}` | the decoded raw is self-consistent; the digest verdict agrees with the decoded model | +| `isobmff_boxes` | `gamut-isobmff` | `walk_segments`, `walk_meta_children`, `read`, `BoxReader` | the box cursor strictly advances; the segments tile `0..len` exactly | +| `heic_container` | `gamut-heic` | `HeifContainer::parse` | the segments tile `0..len` exactly and every accessor agrees with that tiling | +| `heic_hvcc` | `gamut-heic` | `HevcConfig::parse`, `annex_b*`, `validate_still_payload`, `iter_nal_units` | `annex_b` is its two documented halves, concatenated and appended | + +An **allocation** defect needs the engine's malloc hook to be visible at all: an oversized +`Vec::with_capacity` costs no resident memory on an overcommitting kernel, so measuring RSS finds +nothing and `-malloc_limit_mb` (which libFuzzer defaults to `-rss_limit_mb`, 2048) is the oracle. +That is how `dng_decode` reports a 780-byte file asking for a 34 GB allocation. + +## Seeds + +`corpus//` holds a small **curated seed set**, tracked despite `.gitignore` listing +`tooling/gamut-fuzz/corpus/` — that ignore is there so the engine's *search state* is never +committed, and force-adding the seeds keeps exactly that split: the seeds are tracked, everything +libFuzzer writes beside them stays ignored. `cargo fuzz` uses the directory as its corpus with no +extra wiring, so `mise run fuzz ` picks them up. + +They are seeds, **not** the regression record. `corpus/ifd_read/` carries the malformed-TIFF cases +enumerated on issue #264 (contributed from rawshift's deleted in-repo TIFF parser); the other +directories carry one small well-formed file each, written by this workspace's own encoders, so a +decoder target starts from something that reaches its pixel path instead of spending its budget +rediscovering a header. Real-camera corpora are deliberately not vendored: they run to hundreds of +megabytes and live in `justin13888/rawshift-test-fixtures` releases. + `Drago` is held to monotonicity only where `Drago::is_monotonic` says it claims it (#439); every other operator promises it unconditionally, and all of them are driven through the other three laws. diff --git a/tooling/gamut-fuzz/corpus/dng_decode/cfa-12bit.dng b/tooling/gamut-fuzz/corpus/dng_decode/cfa-12bit.dng new file mode 100644 index 0000000000000000000000000000000000000000..e54e4eb14082e861302501eee24db40c57912df7 GIT binary patch literal 872 zcmZuvO=uHA6n?Y2*`}LpvdLyQYm+AH&!H45>7haof>94vdT1~wK_O77U@Jn~i>J~{ zdx@Y2Z(>h+ZNa0LBCdjh1wDB1Qt+5dK`4qOes6ZxMsSA7_syGc-n{p1=H@0TO+*3p z$shs7i-a!GV5+zdu7NW)G$LKaS#K(iFgZe_9Pk1*1rg``4PG45dx~=|e~6cr{v6_C zA|-3Min|rX=fNk2-kQc+7PM`|udBP~QBW}w=mxdp$4xlmr6{iLHuR?>i$+F3GK`g#AlgGQrs;b|s7TKbOvKoH!H0g2WBJleoco z5;v}a-%vSO)7{jG3(_zjZN{CV0L@3I>?pyNvg(5Mr@&b+FvnD!W#ftu@keY80 zhKhd&?}D?FJx+j;mUF4Ivc49!@2sv}>8udf$L}O4Pp1~|FVVlc_f3l~UCz^wk->oR z%T{0F;`&dC4c(RJjX;;U@GL&xo{#s=i?ob~pGPEe4qnS|qy>m3!d;jy)GOLRWe_xFm^e(CY+`h!Vvun;u&YKIk%x|yBhTI(PvG); zP<$>zHOU5jAhOEHvWkF7>Q!03i`{9SFdIY8#l&lJ%*IstOUff1SiJE0zTZIGKXJJq qs_WTtqHY(*nYweTBU_V}^FT-r>46v>RW<(2JGk3TokaF}yFY$JgD;~1 literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/heic_hvcc/main-still-vps-sps-pps.bin b/tooling/gamut-fuzz/corpus/heic_hvcc/main-still-vps-sps-pps.bin new file mode 100644 index 0000000000000000000000000000000000000000..3fb4f7006abefc0a714d92a2d870f1fb070c9f65 GIT binary patch literal 57 zcmZS3XJk%bU|^U4#0(5k9~l1p{qci=fuDH+10w^o1LLZNAliv>_aYGO!gvO#ibaEQ GJp%x3hYT(N literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/ifd_read/01-truncated-header.tif b/tooling/gamut-fuzz/corpus/ifd_read/01-truncated-header.tif new file mode 100644 index 0000000000000000000000000000000000000000..e5f09c679df4cb86e9612435d50b4c83260801dd GIT binary patch literal 4 LcmebD)M5Yt0#^XN literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/ifd_read/03-invalid-byte-order.tif b/tooling/gamut-fuzz/corpus/ifd_read/03-invalid-byte-order.tif new file mode 100644 index 0000000000000000000000000000000000000000..87bdae045ab8c0ce683de563bf64843dd7270b60 GIT binary patch literal 8 Pcma!u&|=_VU|;|M2223s literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/ifd_read/04-invalid-magic.tif b/tooling/gamut-fuzz/corpus/ifd_read/04-invalid-magic.tif new file mode 100644 index 0000000000000000000000000000000000000000..b27c9d34ec82940634bd3ce5d49bd5d85e9652b3 GIT binary patch literal 8 PcmebDOlIIV!Z literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/ifd_read/09b-circular-ifd-two-node.tif b/tooling/gamut-fuzz/corpus/ifd_read/09b-circular-ifd-two-node.tif new file mode 100644 index 0000000000000000000000000000000000000000..cf4a55df165e14e1a60a50c0d53c06c2e002b9e1 GIT binary patch literal 20 TcmebD)MDUZ00BNQ31R~P4^sf! literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/ifd_read/10a-hostile-entry-count.tif b/tooling/gamut-fuzz/corpus/ifd_read/10a-hostile-entry-count.tif new file mode 100644 index 0000000000000000000000000000000000000000..8bc4b2ad34ecdac211ceeb6b2555d1924b1752b3 GIT binary patch literal 10 RcmebD)MDUZU|{(F9{>sR0>c0R literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/ifd_read/10b-hostile-entry-count-bigtiff.tif b/tooling/gamut-fuzz/corpus/ifd_read/10b-hostile-entry-count-bigtiff.tif new file mode 100644 index 0000000000000000000000000000000000000000..a6dc8bc08ba8d7080938d7f1f313990f0909e16f GIT binary patch literal 24 XcmebD)MnsdU|J0Wk;w05iS<@&Et; literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/ifd_read/13-unknown-field-type.tif b/tooling/gamut-fuzz/corpus/ifd_read/13-unknown-field-type.tif new file mode 100644 index 0000000000000000000000000000000000000000..9fe2729ecd7427d5587b41eb5638b01bb2453844 GIT binary patch literal 26 ZcmebD)MDUZU|?VbqGS-spasMr000^+0aXA1 literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/isobmff_boxes/avif-single-item.avif b/tooling/gamut-fuzz/corpus/isobmff_boxes/avif-single-item.avif new file mode 100644 index 0000000000000000000000000000000000000000..10a7c9b5525ca5a2e6c9be8c1188e6d0b8f0ab90 GIT binary patch literal 229 zcmXv{OA5j;5S`vViq& z0kTD*;vhX@P&UXMaj2RQAX|!&6|8;PI~fCB(=8YtNS literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/tiff_decode/rgb8-none.tif b/tooling/gamut-fuzz/corpus/tiff_decode/rgb8-none.tif new file mode 100644 index 0000000000000000000000000000000000000000..9d3b5b753ea27800ed5e9a7db6ce7d6ee5f9b2ab GIT binary patch literal 252 zcmebD)MDUZU|`^3U|?isU<9(*fS3`=W&yI9fNW+UJr9VPq2ge5P&N}#T#%6ktal5L zEeaI}=@EmnLFR}<)p!8eQjDx%^-F+kX()RQkgb7aCXfqs83=-OfNX@(4BTRhI_6Hk z5y^R#E&a2WZrFY7;+^N8{&EP*X`0x3g(hT|HTFzjw06hgb2p#7|HUdGt!8NB9u%8V UT-P~e{;I79PG5WY<~tKF0R8AFPXGV_ literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs b/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs new file mode 100644 index 00000000..f5d11ae2 --- /dev/null +++ b/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs @@ -0,0 +1,62 @@ +//! fuzz · robustness — `gamut_dng::DngDecoder`, the entry point a camera file from anywhere hits. +//! +//! `docs/testing.md`'s per-crate table names `DngDecoder` as this crate's untrusted-input +//! surface. A DNG is a TIFF/EP sub-IFD tree whose every geometry, level and opcode field is a +//! number the file chose, so the decoder is `#![forbid(unsafe_code)]` and a hostile file must end +//! in a typed error — never a panic, a hang, or an allocation sized from a declared field rather +//! than from the bytes that are actually there. libFuzzer's malloc hook is what makes that last +//! one observable: an over-sized `Vec::with_capacity` costs no resident memory on an +//! overcommitting kernel, so the engine's limit, not the OS, is the oracle. +//! +//! Two checks beyond the crash oracle: +//! +//! - **the raw image is self-consistent**: a decoded `RawImage` holds exactly +//! `width × height × samples_per_pixel` samples. The constructors enforce it; a decode path +//! that builds one another way is what this notices. +//! - **the digest verdict agrees with the decoded model**: the file either carries a +//! `NewRawImageDigest` — in which case `verify_new_raw_image_digest` must reach a verdict — or +//! it does not, in which case the verdict must be `Absent`. Two entry points read the same tag +//! by different routes (`decode` models it, `verify` re-reads it), so a disagreement means one +//! of them found a tag the other did not. +//! +//! A crash found here is **minimised and promoted into a named deterministic case** in +//! `gamut-dng`'s own suite. The corpus is a search aid, not the regression record. + +#![no_main] + +use gamut_dng::{DigestCheck, DngDecoder}; +use libfuzzer_sys::fuzz_target; + +fuzz_target!(|data: &[u8]| { + let decoder = DngDecoder::new(); + + // The digest path reads the container and, for losslessly-stored raws, the image — it is a + // second route through the same tree, and it must be refused rather than crash on its own. + let verdict = decoder.verify_new_raw_image_digest(data); + + let Ok(decoded) = decoder.decode(data) else { + return; + }; + + let dims = decoded.raw.dimensions(); + let expected = (dims.width as usize) + .checked_mul(dims.height as usize) + .and_then(|n| n.checked_mul(usize::from(decoded.raw.samples_per_pixel()))); + assert_eq!( + Some(decoded.raw.samples().len()), + expected, + "decoded raw holds {} samples for {dims:?} × {} planes", + decoded.raw.samples().len(), + decoded.raw.samples_per_pixel() + ); + + // `decode` succeeded, so the container is walkable and the digest path must have reached a + // verdict too — the two routes disagree only if they selected different directories. + let verdict = verdict.expect("digest check on a file that decoded"); + assert_eq!( + decoded.new_raw_image_digest.is_none(), + verdict == DigestCheck::Absent, + "digest tag {:?} but verdict {verdict:?}", + decoded.new_raw_image_digest + ); +}); diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs new file mode 100644 index 00000000..6ae32136 --- /dev/null +++ b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs @@ -0,0 +1,87 @@ +//! fuzz · robustness — `gamut_heic::HeifContainer::parse`, the container half of the crate's +//! untrusted-input surface (`docs/testing.md`'s per-crate table names container `parse`). +//! +//! `gamut-heic` is decode-only and its stated product guarantee is a **full-fidelity byte +//! accounting**: every input byte maps to a box, to an appended motion-photo stream, or to an +//! explicit trailer. That is a claim the engine can be pointed at directly, and it is stronger +//! than "did not crash": a container that silently drops a region still parses. +//! +//! So this target checks, on every successful parse: +//! +//! - the segments tile `0..len` exactly — start at 0, contiguous, non-overlapping, none empty, +//! last ending at end of file; +//! - the accessors are consistent with that tiling: `appended_stream` and `trailer` are `Some` +//! exactly when a segment of that kind exists, and `boxes()` yields one entry per `Box` +//! segment; +//! - every borrowed slice the accessors hand out is a subslice of the input the container was +//! given, which is what `data()` promises. +//! +//! Real files reach here: phones append a whole second MP4 after the HEIC, and camera apps leave +//! trailers, so the accounting path is not an exotic branch. +//! +//! A crash found here is **minimised and promoted into a named deterministic case** in +//! `gamut-heic`'s own suite. The corpus is a search aid, not the regression record. + +#![no_main] + +use gamut_heic::{HeifContainer, SegmentKind}; +use libfuzzer_sys::fuzz_target; + +fuzz_target!(|data: &[u8]| { + let Ok(container) = HeifContainer::parse(data) else { + return; + }; + + // Walking a cursor states the whole tiling claim once — start at 0, contiguous, + // non-overlapping, no empty segment, ending at end of file — and stays correct for a + // zero-length input, where an empty segment list already tiles `0..0`. + let segments = container.segments(); + let mut cursor = 0usize; + for segment in segments { + assert_eq!( + segment.range.start, cursor, + "segment {:?} leaves a gap or overlaps at {cursor}", + segment.range + ); + assert!( + segment.range.end > segment.range.start, + "empty segment {:?}", + segment.range + ); + cursor = segment.range.end; + } + assert_eq!(cursor, data.len(), "coverage does not run to end of file"); + + // The accessors report exactly what the tiling holds. + let boxes = container.boxes().count(); + let mut kinds = (0usize, 0usize, 0usize); + for segment in segments { + match segment.kind { + SegmentKind::Box { .. } => kinds.0 += 1, + SegmentKind::AppendedStream(_) => kinds.1 += 1, + SegmentKind::Trailer(_) => kinds.2 += 1, + _ => {} + } + } + assert_eq!(boxes, kinds.0, "boxes() disagrees with the Box segments"); + assert_eq!( + container.appended_stream().is_some(), + kinds.1 > 0, + "appended_stream() disagrees with the AppendedStream segments" + ); + assert_eq!( + container.trailer().is_some(), + kinds.2 > 0, + "trailer() disagrees with the Trailer segments" + ); + + // `data()` returns the input, and every borrowed region lies inside it. + assert_eq!(container.data().as_ptr(), data.as_ptr()); + assert_eq!(container.data().len(), data.len()); + + // The item model and the unknown-box ledger are built on the same walk; drive them so a + // defect there is reachable too. + let _ = container.image().items().count(); + let _ = container.image().primary_item().id(); + let _ = container.unknown_meta_boxes().len(); +}); diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs b/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs new file mode 100644 index 00000000..00cca66f --- /dev/null +++ b/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs @@ -0,0 +1,78 @@ +//! fuzz · robustness — `gamut_heic`'s typed `hvcC` record and NAL layer, the second half of the +//! crate's untrusted-input surface (`docs/testing.md`'s table names container `parse` **and** NAL +//! `parse`). +//! +//! An `hvcC` record is a length-prefixed array-of-arrays whose counts and lengths all come from +//! the file, and the item payload behind it is a chain of `nal_length_size`-byte length prefixes. +//! Both are `#![forbid(unsafe_code)]` offset arithmetic, so a hostile record must end in a typed +//! error rather than a panic or a spin. +//! +//! The check beyond the crash oracle is the **composition the API documents**: +//! `annex_b` is defined as `annex_b_parameter_sets` followed by `annex_b_payload`, appended to +//! the caller's buffer. Two callers rely on that split — an Annex-B decoder takes the whole +//! stream, an Android MediaCodec-shaped API takes `csd-0` and the samples separately — so the +//! halves drifting from the whole is a real defect that produces no crash at all. The target +//! asserts byte equality *and* that the two forms agree on success, including the documented +//! "bytes already appended are left in place" behaviour on error. +//! +//! `validate_still_payload` is driven for its own sake: it re-walks the payload through +//! `NalHeader::parse`, a different reach from the Annex-B emitters. +//! +//! ## Input framing +//! +//! `u16` big-endian record length, the `hvcC` record, then the item payload. A two-byte length +//! rather than a byte lets the engine reach records past 255 bytes (a real record with several +//! parameter sets is bigger), and taking it from the front means a mutation inside the record +//! does not also reframe the payload. +//! +//! A crash found here is **minimised and promoted into a named deterministic case** in +//! `gamut-heic`'s own suite. The corpus is a search aid, not the regression record. + +#![no_main] + +use gamut_heic::{HevcConfig, NalHeader, iter_nal_units}; +use libfuzzer_sys::fuzz_target; + +fuzz_target!(|data: &[u8]| { + let Some((length, rest)) = data.split_first_chunk::<2>() else { + return; + }; + let split = usize::from(u16::from_be_bytes(*length)).min(rest.len()); + let (record, payload) = rest.split_at(split); + + let Ok(config) = HevcConfig::parse(record) else { + return; + }; + + // The documented composition: `annex_b` is the two halves, in order, appended to `out`. + let mut whole = Vec::new(); + let whole_result = config.annex_b(payload, &mut whole); + let mut halves = Vec::new(); + config.annex_b_parameter_sets(&mut halves); + let halves_result = config.annex_b_payload(payload, &mut halves); + assert_eq!( + whole_result.is_ok(), + halves_result.is_ok(), + "annex_b and its two halves disagree on success" + ); + assert_eq!(whole, halves, "annex_b is not its two halves concatenated"); + + // Appending, not replacing: a caller reusing a scratch buffer keeps what was there. + let mut reused = vec![0xEE; 3]; + let _ = config.annex_b(payload, &mut reused); + assert_eq!(&reused[..3], &[0xEE; 3], "annex_b overwrote the buffer"); + assert_eq!( + &reused[3..], + &whole[..], + "annex_b appended something different to a non-empty buffer" + ); + + // The NAL layer on its own reach: the still-image constraint re-walks the payload through + // `NalHeader::parse`. + let _ = config.validate_still_payload(payload); + for nal in iter_nal_units(payload, config.nal_length_size()) { + let Ok(nal) = nal else { break }; + assert!(!nal.is_empty(), "iter_nal_units yielded an empty NAL unit"); + let _ = NalHeader::parse(nal); + } +}); diff --git a/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs b/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs new file mode 100644 index 00000000..9d8e40d2 --- /dev/null +++ b/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs @@ -0,0 +1,82 @@ +//! fuzz · robustness — `gamut_ifd`'s reader entry points, on wholly untrusted bytes. +//! +//! The sibling `ifd_read_ledger` target drives the crate's `invariants` laws over *normalised* +//! range lists. This one is the other half `docs/testing.md`'s per-crate table asks for — the +//! **driver** over the untrusted-input surface it names (`IfdReader`, `read`) — and it hands the +//! engine's bytes to the parser unchanged, because the thing under test is precisely what the +//! parser does with a byte string nobody normalised. +//! +//! Its oracle is not a law function, and deliberately so: +//! +//! - the engine's own — a panic, a hang, or an allocation past libFuzzer's RSS limit is a crash, +//! and `gamut-ifd` is `#![forbid(unsafe_code)]` precisely so that hostile offsets end in a typed +//! error rather than any of those; +//! - a **differential between two public entry points**: the slice functions are thin wrappers +//! over the streaming engine, so `read` and `IfdReader::read_file` must either both fail or +//! parse to equal files. One parser, two doors. +//! - the **dual-ledger audit** (#263): whenever `read_audited` succeeds, every byte the parser +//! physically read is inside a structural claim and every `Parsed` claim was physically read. +//! +//! `tests/robustness.rs` states the same three checks over a bounded, exhaustive corpus — +//! truncations, single-byte overwrites, a named malformed list. That corpus is the reproducible +//! per-PR gate; this is the unbounded search beside it. A crash found here is **minimised and +//! promoted into a named deterministic case in that file**; the corpus is a search aid, not the +//! regression record. + +#![no_main] + +use gamut_ifd::{IfdReader, TiffFile, read, read_audited, read_tree}; +use libfuzzer_sys::fuzz_target; + +/// Renders a parsed file to its structural `Debug` form, for comparing two parses. +/// +/// `TiffFile` derives `PartialEq`, not `Eq`, because a `FLOAT`/`DOUBLE` field holds `f32`/`f64` — +/// and an arbitrary byte string decodes to `NaN` often enough that the engine finds one within a +/// minute. `NaN != NaN` makes `PartialEq` non-reflexive there, so `a == b` reports a +/// disagreement between two *identical* parses. `Debug` renders `NaN` as a value like any other, +/// which is what makes it the total comparison this differential needs; the field order and the +/// enum variants are all in the rendering, so nothing structural is lost by going through it. +fn structure(file: &TiffFile) -> String { + format!("{file:?}") +} + +/// Sub-IFD pointer tags a DNG/EXIF-shaped consumer would follow. +/// +/// The same three `tests/robustness.rs` walks with: `SubIFDs`, `ExifIFD`, `GPSInfoIFD`. Fixed +/// rather than drawn from the input, so every execution exercises the recursive tree walk instead +/// of spending most of its draws on an empty tag list. +const POINTER_TAGS: &[u16] = &[330, 34665, 34853]; + +fuzz_target!(|data: &[u8]| { + // The flat chain, through both doors. + let slice = read(data); + let stream = IfdReader::open(data).and_then(|mut r| r.read_file()); + match (&slice, &stream) { + (Ok(a), Ok(b)) => assert_eq!(structure(a), structure(b), "flat parse disagreement"), + (Err(_), Err(_)) => {} + _ => panic!("flat readers disagree: slice {slice:?} vs stream {stream:?}"), + } + + // The sub-IFD tree walk, through both doors. A tree parse reaches offsets the flat chain + // never visits, so it is a separate reach rather than a stronger version of the above. + let slice_tree = read_tree(data, POINTER_TAGS); + let stream_tree = IfdReader::open(data).and_then(|mut r| r.read_tree(POINTER_TAGS)); + match (&slice_tree, &stream_tree) { + (Ok(a), Ok(b)) => assert_eq!(structure(a), structure(b), "tree parse disagreement"), + (Err(_), Err(_)) => {} + _ => panic!("tree readers disagree: slice {slice_tree:?} vs stream {stream_tree:?}"), + } + + // The byte audit. A parser that eats bytes it never declares — or declares bytes it never + // touched — is what this catches, and it is only meaningful on a parse that succeeded. + if let Ok((_, report)) = read_audited(data) { + assert!( + report.unclaimed_reads.is_empty(), + "parser read bytes it never claimed: {report:?}" + ); + assert!( + report.unread_claims.is_empty(), + "parser claimed bytes it never read: {report:?}" + ); + } +}); diff --git a/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs b/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs new file mode 100644 index 00000000..a87a3b3b --- /dev/null +++ b/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs @@ -0,0 +1,71 @@ +//! fuzz · robustness — `gamut_isobmff`'s box walk and model reader, on untrusted bytes. +//! +//! `docs/testing.md`'s per-crate table names `read` as this crate's untrusted-input surface. An +//! ISOBMFF file is a tree of length-prefixed boxes whose every length the file chose, so the +//! `#![forbid(unsafe_code)]` reader must end in a typed error rather than a panic, a hang (a box +//! whose declared size does not advance the cursor) or an allocation sized from a declared count. +//! +//! Beyond the crash oracle it checks the crate's own **byte-accounting totality**: when +//! `walk_segments` succeeds, its segments tile `0..len` exactly — starting at 0, contiguous, +//! non-overlapping, none empty, the last ending at end of file. That is the guarantee +//! `tests/accounting.rs` pins over hand-built files; here it is asked of whatever the engine +//! produces, which is where a size-0 or a wrapping box length would show up as a hole or an +//! overlap rather than as a crash. +//! +//! `BoxReader` is driven separately from `walk_segments` because it is the lower layer and a +//! caller may use it directly: the check there is that the cursor advances strictly, so a walk of +//! a hostile file cannot spin. +//! +//! A crash found here is **minimised and promoted into a named deterministic case** in +//! `gamut-isobmff`'s own suite. The corpus is a search aid, not the regression record. + +#![no_main] + +use gamut_isobmff::{BoxReader, read, walk_meta_children, walk_segments}; +use libfuzzer_sys::fuzz_target; + +fuzz_target!(|data: &[u8]| { + // The raw box layer: every successful step must consume at least one byte, or a walk of a + // hostile file never terminates. + let mut reader = BoxReader::new(data); + let mut position = reader.position(); + while let Ok(Some(_)) = reader.next_box() { + let next = reader.position(); + assert!( + next > position, + "BoxReader did not advance past {position} (len {})", + data.len() + ); + assert!(next <= data.len(), "BoxReader ran past the end: {next}"); + position = next; + } + + // The segment walk: byte-accounting totality. Walking a cursor rather than asserting the + // four properties separately states the whole claim once — start at 0, contiguous, + // non-overlapping, no empty segment, ending at end of file — and it is the form that stays + // correct for a zero-length input, where an empty segment list already tiles `0..0`. + if let Ok((segments, meta_body)) = walk_segments(data) { + let mut cursor = 0usize; + for segment in &segments { + assert_eq!( + segment.range.start, cursor, + "segment {:?} leaves a gap or overlaps at {cursor}", + segment.range + ); + assert!( + segment.range.end > segment.range.start, + "empty segment {:?}", + segment.range + ); + cursor = segment.range.end; + } + assert_eq!(cursor, data.len(), "coverage does not run to end of file"); + if let Some(body) = meta_body { + let _ = walk_meta_children(body); + } + } + + // The model reader, the entry point the table names. It validates items and properties on top + // of the walk, so it reaches offsets the walk alone never resolves. + let _ = read(data); +}); diff --git a/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs b/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs new file mode 100644 index 00000000..a9048660 --- /dev/null +++ b/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs @@ -0,0 +1,72 @@ +//! fuzz · robustness — `gamut_tiff::TiffDecoder`, the entry point a TIFF from the network hits. +//! +//! `docs/testing.md`'s per-crate table names `TiffDecoder` as this crate's untrusted-input +//! surface. The decoder is `#![forbid(unsafe_code)]` and carries an explicit `MAX_IMAGE_BYTES` +//! cap, so a hostile file must end in a typed error rather than a panic, a hang or a runaway +//! allocation — the three things libFuzzer itself detects. +//! +//! Two further checks make the target able to fail for something other than a crash: +//! +//! - **the page index is bounded by `page_count`**: `info_page` at the count itself must be +//! refused. A count that over-reports the chain is how an out-of-range page reaches the tag +//! reader at all. +//! - **describing and decoding agree**: a page that decodes must also describe, and the two must +//! report the same dimensions. `info` reads tags only and `decode_page` reads pixels, so a +//! disagreement means the two paths read the geometry differently — exactly the split that +//! turns a size check into a false guarantee. +//! +//! The policy is [`ConvertPolicy::permissive`] so the decode reaches the pixel and conversion +//! paths for pages the default lossless policy would refuse at the layout gate. +//! +//! A crash found here is **minimised and promoted into a named deterministic case** in +//! `gamut-tiff`'s `tests/robustness.rs`. The corpus is a search aid, not the regression record. + +#![no_main] + +use gamut_core::convert::ConvertPolicy; +use gamut_tiff::TiffDecoder; +use libfuzzer_sys::fuzz_target; + +/// How many pages of a multi-page file one execution decodes. +/// +/// A chained TIFF may declare up to `gamut-ifd`'s 65 536 directories, and decoding all of them +/// would make a single execution slow enough to look like a hang; the interesting per-page +/// behaviour is reached in the first few. The page-index bound below is still checked against the +/// *full* count. +const MAX_PAGES: usize = 4; + +fuzz_target!(|data: &[u8]| { + let decoder = TiffDecoder::new().convert_policy(ConvertPolicy::permissive()); + + let Ok(pages) = decoder.page_count(data) else { + // A file whose chain does not parse must still be refused — not crash — by the entry + // points that do not consult `page_count` first. + let _ = decoder.info(data); + let _ = decoder.decode_page(data, 0); + return; + }; + + // One past the last page is out of range, whatever the chain claimed. + assert!( + decoder.info_page(data, pages).is_err(), + "page {pages} described although page_count is {pages}" + ); + + for page in 0..pages.min(MAX_PAGES) { + let info = decoder.info_page(data, page); + let image = decoder.decode_page(data, page); + match (&info, &image) { + (Ok(info), Ok(image)) => { + assert_eq!( + (image.width(), image.height()), + (info.width, info.height), + "page {page}: decoded geometry differs from the described geometry" + ); + } + (Err(error), Ok(_)) => { + panic!("page {page} decoded although it could not be described: {error}") + } + _ => {} + } + } +}); From f17f229daa544031ac7686aa4b0e47e196ee5c3d Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 01:07:14 -0400 Subject: [PATCH 02/24] test(ifd): reject a 65 535-entry directory against the source length The corpus already held a directory whose entry count is truncated mid-word. It did not hold one whose count is whole and well-formed but has no entry bytes at all behind it, which is the case that reaches the point where the directory would be sized from the count: 65 535 entries claimed in a ten-byte file. Asserted on the error text rather than `is_err`, so a guard that stops bounding the count against the source cannot be masked by a later failure. Refs #264 --- crates/gamut-ifd/tests/robustness.rs | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/crates/gamut-ifd/tests/robustness.rs b/crates/gamut-ifd/tests/robustness.rs index 56917a21..bcc8cca6 100644 --- a/crates/gamut-ifd/tests/robustness.rs +++ b/crates/gamut-ifd/tests/robustness.rs @@ -83,7 +83,11 @@ fn specific_malformed_inputs_error_without_panic() { b"MM\x00\x2a\xff\xff\xff\x7f", // first-IFD offset past EOF (big-endian) b"II\x2a\x00\x08\x00\x00\x00", // first IFD at EOF b"II\x2a\x00\x08\x00\x00\x00\xff", // truncated IFD count - b"II\x2a\x00\x00\x00\x00\x00", // first-IFD offset 0 (no IFD) + // A whole entry count of 65 535 with no entry bytes at all behind it — the count is + // present and well-formed, so the reader reaches the point of sizing the directory from + // it. Nothing may be reserved for the 786 KiB it claims in a ten-byte file. + b"II\x2a\x00\x08\x00\x00\x00\xff\xff", + b"II\x2a\x00\x00\x00\x00\x00", // first-IFD offset 0 (no IFD) // A 1-entry IFD whose value count is huge (byte-length overflow path), then truncated. b"II\x2a\x00\x08\x00\x00\x00\x01\x00\x00\x01\x03\x00\xff\xff\xff\xff\x08\x00\x00\x00\x00\x00\x00\x00", // An IFD whose next-IFD pointer loops back to itself. @@ -94,6 +98,13 @@ fn specific_malformed_inputs_error_without_panic() { } // The loop case must be a typed error, not a hang. assert!(read(b"II\x2a\x00\x08\x00\x00\x00\x00\x00\x08\x00\x00\x00").is_err()); + // The hostile entry count must be refused against the source length, not merely survived. + assert!( + read(b"II\x2a\x00\x08\x00\x00\x00\xff\xff") + .expect_err("65 535 entries in a ten-byte file") + .to_string() + .contains("IFD extends past end of file") + ); } #[test] From 6ccd998a4f66e6cff16470a84c543a50f2b2389a Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 01:07:14 -0400 Subject: [PATCH 03/24] ci(fuzz): run the parser-entry-point targets in the extended lane Six new rows in the existing fuzz matrix, so each entry point gets the whole ten-minute budget in parallel rather than the job's wall time growing with every surface added. The lane stays off the per-PR path for the reason docs/testing.md gives: the coverage job is the only gate that runs tests, so anything in it must be bounded and reproducible, and a coverage-guided engine is neither. Refs #264 --- .github/workflows/extended.yml | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/.github/workflows/extended.yml b/.github/workflows/extended.yml index 3597bbc2..f6a204cb 100644 --- a/.github/workflows/extended.yml +++ b/.github/workflows/extended.yml @@ -115,10 +115,22 @@ jobs: # One target's crash says nothing about another's, so a failure must not cancel the rest. fail-fast: false matrix: - # One entry per file in tooling/gamut-fuzz/fuzz_targets. A matrix rather than three steps - # in one job, so each target gets the full time budget below in parallel instead of the - # job's wall time growing with every crate that gains laws. - target: [ifd_read_ledger, core_convert, tonemap_curves] + # One entry per file in tooling/gamut-fuzz/fuzz_targets. A matrix rather than one job with + # a step per target, so each target gets the full time budget below in parallel instead of + # the job's wall time growing with every crate that gains laws or a parser entry point. + target: + - ifd_read_ledger + - core_convert + - tonemap_curves + # The parser entry points (#264): the untrusted-input surface `docs/testing.md`'s + # per-crate table names, one target per entry point rather than per crate, because + # gamut-heic's container walk and its `hvcC`/NAL layer are two independent surfaces. + - ifd_read + - tiff_decode + - dng_decode + - isobmff_boxes + - heic_container + - heic_hvcc env: MISE_TASK_RUN_AUTO_INSTALL: false steps: From 8b9a9db13648ae062c0a44c41133266a2a798c0f Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 01:38:58 -0400 Subject: [PATCH 04/24] docs(fuzz): warn against force-adding the corpus directory a second time MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The seeds are tracked past the ignore rule; the engine writes its own findings beside them and those stay untracked, which is the split the ignore exists for. A second blanket force-add would collapse it — a few minutes of one target adds a couple of hundred files. Refs #264 --- tooling/gamut-fuzz/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tooling/gamut-fuzz/README.md b/tooling/gamut-fuzz/README.md index d533e878..1123ea0a 100644 --- a/tooling/gamut-fuzz/README.md +++ b/tooling/gamut-fuzz/README.md @@ -118,6 +118,11 @@ committed, and force-adding the seeds keeps exactly that split: the seeds are tr libFuzzer writes beside them stays ignored. `cargo fuzz` uses the directory as its corpus with no extra wiring, so `mise run fuzz ` picks them up. +Running a target writes its new findings into the same directory — a few minutes of `heic_hvcc` +adds a couple of hundred files — and those stay untracked, which is the point. **Never +`git add -f` the whole directory a second time**: add the one seed you mean by path, or the +engine's search state goes in with it. + They are seeds, **not** the regression record. `corpus/ifd_read/` carries the malformed-TIFF cases enumerated on issue #264 (contributed from rawshift's deleted in-repo TIFF parser); the other directories carry one small well-formed file each, written by this workspace's own encoders, so a From 7f046c4a82f5ef306aeb0f9c8092d3924efd735c Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 06:01:16 -0400 Subject: [PATCH 05/24] test(fuzz): replace the tautological differentials with structure pins `ifd_read` advertised a differential between two public doors: `read(data)` against `IfdReader::open(data)?.read_file()`. `reader.rs` *defines* `read` as exactly that expression, and `read_tree` likewise, so the two sides were one function call written twice and the comparison could not fail for any input. Falsified before changing anything: making `read_file` drop the last directory of a multi-directory chain and running a hand-built two-directory file produced no report at all, because both sides dropped it. `stream.rs` says as much in its own module docs -- that module is the parser, and the slice functions are thin wrappers over it, so there is exactly one directory-body walk to disagree about. `heic_hvcc` had the same shape twice over: it asserted that `annex_b` equals the concatenation of the two calls its own body makes, and that two `is_ok()` values computed from the same expression are equal. The claim about two bodies agreeing is worth keeping, but it is a structure pin, not a search: one bounded run answers it, and repeating it nine thousand times a second searches an empty space. So - `ifd_read` drops both duplicate parses and promotes the dual-ledger byte audit -- the check the engine cannot make, and the one that can actually fail -- to the headline. The wrapper pin stays where it is bounded and exhaustive, in `crates/gamut-ifd/tests/robustness.rs`. Throughput measured over 120 s went from roughly 12 000 exec/s to 20 596 exec/s. - `heic_hvcc` drops the trivially-true `is_ok()` comparison, promotes the append contract -- which nothing in any of the three emitter bodies makes true by construction -- to the headline, and folds the remaining composition pin into the append check's buffer so it costs no third emitter pass. 59 936 exec/s over 120 s. Refs #264 --- tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs | 56 +++++++++------ tooling/gamut-fuzz/fuzz_targets/ifd_read.rs | 74 ++++++++------------ 2 files changed, 63 insertions(+), 67 deletions(-) diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs b/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs index 00cca66f..fbca2f7e 100644 --- a/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs +++ b/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs @@ -7,13 +7,21 @@ //! Both are `#![forbid(unsafe_code)]` offset arithmetic, so a hostile record must end in a typed //! error rather than a panic or a spin. //! -//! The check beyond the crash oracle is the **composition the API documents**: -//! `annex_b` is defined as `annex_b_parameter_sets` followed by `annex_b_payload`, appended to -//! the caller's buffer. Two callers rely on that split — an Annex-B decoder takes the whole -//! stream, an Android MediaCodec-shaped API takes `csd-0` and the samples separately — so the -//! halves drifting from the whole is a real defect that produces no crash at all. The target -//! asserts byte equality *and* that the two forms agree on success, including the documented -//! "bytes already appended are left in place" behaviour on error. +//! The check beyond the crash oracle is the **append contract**: `annex_b`, +//! `annex_b_parameter_sets` and `annex_b_payload` all document that they *append* to the caller's +//! buffer — bytes already there are left in place, including on the error path — so a caller can +//! reuse one scratch buffer across items. Nothing in any of the three bodies makes that true by +//! construction: each one is free to `clear()` or to write through an index, and doing so breaks +//! every reusing caller while producing no crash at all. That is what this target searches for. +//! +//! Alongside it, and explicitly **not** a differential, is a **structure pin**: `annex_b`'s body +//! *is* `annex_b_parameter_sets` followed by `annex_b_payload`, so asserting the whole equals the +//! two halves cannot fail for any input while that body stands. It is kept because the split is a +//! documented API contract with two distinct callers — an Annex-B decoder takes the whole stream, +//! an Android MediaCodec-shaped API takes `csd-0` and the samples separately — so a future +//! `annex_b` that stops delegating is a real regression. It is folded into the append check's +//! second pass rather than costing a third emitter run: the equality of two `is_ok()` calls on the +//! same expression, which an earlier draft also asserted, is trivially true and is gone. //! //! `validate_still_payload` is driven for its own sake: it re-walks the payload through //! `NalHeader::parse`, a different reach from the Annex-B emitters. @@ -33,6 +41,10 @@ use gamut_heic::{HevcConfig, NalHeader, iter_nal_units}; use libfuzzer_sys::fuzz_target; +/// The bytes a reused scratch buffer is pre-filled with, so an emitter that replaces rather than +/// appends is visible as a missing prefix rather than as a length that happens to match. +const SCRATCH: [u8; 3] = [0xEE; 3]; + fuzz_target!(|data: &[u8]| { let Some((length, rest)) = data.split_first_chunk::<2>() else { return; @@ -44,27 +56,25 @@ fuzz_target!(|data: &[u8]| { return; }; - // The documented composition: `annex_b` is the two halves, in order, appended to `out`. + // Pass one: the whole stream into an empty buffer, as an Annex-B decoder takes it. let mut whole = Vec::new(); - let whole_result = config.annex_b(payload, &mut whole); - let mut halves = Vec::new(); - config.annex_b_parameter_sets(&mut halves); - let halves_result = config.annex_b_payload(payload, &mut halves); + let _ = config.annex_b(payload, &mut whole); + + // Pass two: the same stream through the two halves, into a buffer that is *not* empty. The + // prefix assertion is the live check — it fails if either half replaces instead of appending, + // on the success path or the error path. The tail assertion is the structure pin above. + let mut reused = SCRATCH.to_vec(); + config.annex_b_parameter_sets(&mut reused); + let _ = config.annex_b_payload(payload, &mut reused); assert_eq!( - whole_result.is_ok(), - halves_result.is_ok(), - "annex_b and its two halves disagree on success" + &reused[..SCRATCH.len()], + &SCRATCH[..], + "an annex_b emitter overwrote what was already in the buffer" ); - assert_eq!(whole, halves, "annex_b is not its two halves concatenated"); - - // Appending, not replacing: a caller reusing a scratch buffer keeps what was there. - let mut reused = vec![0xEE; 3]; - let _ = config.annex_b(payload, &mut reused); - assert_eq!(&reused[..3], &[0xEE; 3], "annex_b overwrote the buffer"); assert_eq!( - &reused[3..], + &reused[SCRATCH.len()..], &whole[..], - "annex_b appended something different to a non-empty buffer" + "annex_b is not its two documented halves concatenated" ); // The NAL layer on its own reach: the still-image constraint re-walks the payload through diff --git a/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs b/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs index 9d8e40d2..8f42612e 100644 --- a/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs +++ b/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs @@ -11,35 +11,33 @@ //! - the engine's own — a panic, a hang, or an allocation past libFuzzer's RSS limit is a crash, //! and `gamut-ifd` is `#![forbid(unsafe_code)]` precisely so that hostile offsets end in a typed //! error rather than any of those; -//! - a **differential between two public entry points**: the slice functions are thin wrappers -//! over the streaming engine, so `read` and `IfdReader::read_file` must either both fail or -//! parse to equal files. One parser, two doors. -//! - the **dual-ledger audit** (#263): whenever `read_audited` succeeds, every byte the parser -//! physically read is inside a structural claim and every `Parsed` claim was physically read. +//! - the **dual-ledger audit** (#263) is the one check here that the engine cannot make: whenever +//! `read_audited` succeeds, every byte the parser physically read must be inside a structural +//! claim and every `Parsed` claim must have been physically read. A parser that eats bytes it +//! never declares — or declares bytes it never touched — produces no crash at all, and this is +//! what sees it. //! -//! `tests/robustness.rs` states the same three checks over a bounded, exhaustive corpus — -//! truncations, single-byte overwrites, a named malformed list. That corpus is the reproducible -//! per-PR gate; this is the unbounded search beside it. A crash found here is **minimised and -//! promoted into a named deterministic case in that file**; the corpus is a search aid, not the -//! regression record. +//! ## What this target deliberately does *not* check +//! +//! An earlier draft also compared `read(data)` against `IfdReader::open(data)?.read_file()` and +//! called it a differential between two doors. It is not one: `reader.rs` defines `read` as +//! *exactly* that expression (and `read_tree` likewise), so the two sides are the same function +//! call written twice and the comparison cannot fail for any input. It is a **structure pin** on +//! the wrappers staying thin — worth having, but bounded and deterministic work, not a search. +//! It is kept where it belongs, in `crates/gamut-ifd/tests/robustness.rs`, whose `survives` +//! helper drives both doors over the exhaustive truncation and single-byte-overwrite corpus. +//! Dropping the two duplicate parses here doubles this target's execution rate. +//! +//! `tests/robustness.rs` states the audit check too, over that same bounded corpus. That corpus is +//! the reproducible per-PR gate; this is the unbounded search beside it. A crash found here is +//! **minimised and promoted into a named deterministic case in that file**; the corpus is a search +//! aid, not the regression record. #![no_main] -use gamut_ifd::{IfdReader, TiffFile, read, read_audited, read_tree}; +use gamut_ifd::{read, read_audited, read_tree}; use libfuzzer_sys::fuzz_target; -/// Renders a parsed file to its structural `Debug` form, for comparing two parses. -/// -/// `TiffFile` derives `PartialEq`, not `Eq`, because a `FLOAT`/`DOUBLE` field holds `f32`/`f64` — -/// and an arbitrary byte string decodes to `NaN` often enough that the engine finds one within a -/// minute. `NaN != NaN` makes `PartialEq` non-reflexive there, so `a == b` reports a -/// disagreement between two *identical* parses. `Debug` renders `NaN` as a value like any other, -/// which is what makes it the total comparison this differential needs; the field order and the -/// enum variants are all in the rendering, so nothing structural is lost by going through it. -fn structure(file: &TiffFile) -> String { - format!("{file:?}") -} - /// Sub-IFD pointer tags a DNG/EXIF-shaped consumer would follow. /// /// The same three `tests/robustness.rs` walks with: `SubIFDs`, `ExifIFD`, `GPSInfoIFD`. Fixed @@ -48,27 +46,8 @@ fn structure(file: &TiffFile) -> String { const POINTER_TAGS: &[u16] = &[330, 34665, 34853]; fuzz_target!(|data: &[u8]| { - // The flat chain, through both doors. - let slice = read(data); - let stream = IfdReader::open(data).and_then(|mut r| r.read_file()); - match (&slice, &stream) { - (Ok(a), Ok(b)) => assert_eq!(structure(a), structure(b), "flat parse disagreement"), - (Err(_), Err(_)) => {} - _ => panic!("flat readers disagree: slice {slice:?} vs stream {stream:?}"), - } - - // The sub-IFD tree walk, through both doors. A tree parse reaches offsets the flat chain - // never visits, so it is a separate reach rather than a stronger version of the above. - let slice_tree = read_tree(data, POINTER_TAGS); - let stream_tree = IfdReader::open(data).and_then(|mut r| r.read_tree(POINTER_TAGS)); - match (&slice_tree, &stream_tree) { - (Ok(a), Ok(b)) => assert_eq!(structure(a), structure(b), "tree parse disagreement"), - (Err(_), Err(_)) => {} - _ => panic!("tree readers disagree: slice {slice_tree:?} vs stream {stream_tree:?}"), - } - - // The byte audit. A parser that eats bytes it never declares — or declares bytes it never - // touched — is what this catches, and it is only meaningful on a parse that succeeded. + // The byte audit: the live check. Only meaningful on a parse that succeeded, because an + // abandoned parse has no complete claim set to reconcile against. if let Ok((_, report)) = read_audited(data) { assert!( report.unclaimed_reads.is_empty(), @@ -79,4 +58,11 @@ fuzz_target!(|data: &[u8]| { "parser claimed bytes it never read: {report:?}" ); } + + // The flat chain and the sub-IFD tree walk, for the engine's own oracle. A tree parse reaches + // offsets the flat chain never visits, so it is a separate reach rather than a stronger + // version of the flat one; neither result is compared against anything, because the only + // available comparand is the same function under another name (see the module docs). + let _ = read(data); + let _ = read_tree(data, POINTER_TAGS); }); From 67bb97cb3bbc1c174563491908d6fec310a8e2e9 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 06:01:16 -0400 Subject: [PATCH 06/24] test(fuzz): classify a digest check that cannot run `dng_decode` asserted, with `expect`, that any file which decodes also reaches a digest verdict. `verify_new_raw_image_digest` is not a second call to `decode`: it re-reads the container and then, on the file's own `Compression` code, either re-decodes the raw samples or walks the compressed chunk grid -- a route `decode` never takes. Only the first of those is a subset of decode's work, and neither function's documented contract promises the containment; `verify`'s own docs bound its errors by `decode`'s for *lossless storage only*. An `Err` from that call therefore has to be a classified outcome, not a panic. On a tier that runs unattended a false crash is indistinguishable from a real one until a human minimises it, and this one was the target's own stated most-likely false positive. The `Err` arm now returns: a case with nothing to compare, rather than a report. Nothing that could fail is lost. The self-consistency check on the raw image that arrives is untouched, and the digest comparison survives whole -- relabelled as the structure pin it is, since both sides read `NewRawImageDigest` out of IFD 0 with the same expression. Refs #264 --- tooling/gamut-fuzz/fuzz_targets/dng_decode.rs | 41 +++++++++++++------ 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs b/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs index f5d11ae2..4e930d39 100644 --- a/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs +++ b/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs @@ -8,16 +8,31 @@ //! one observable: an over-sized `Vec::with_capacity` costs no resident memory on an //! overcommitting kernel, so the engine's limit, not the OS, is the oracle. //! -//! Two checks beyond the crash oracle: +//! The check beyond the crash oracle is that **the raw image is self-consistent**: whatever the +//! decode pipeline returns holds exactly `width × height × samples_per_pixel` samples. The +//! constructors enforce that at construction, but a `RawImage` is handed on through linearisation, +//! active-area and crop handling before a caller sees it, and it is the value that *arrives* — +//! after everything that may have rewritten `samples` or `dims` — this asserts on. //! -//! - **the raw image is self-consistent**: a decoded `RawImage` holds exactly -//! `width × height × samples_per_pixel` samples. The constructors enforce it; a decode path -//! that builds one another way is what this notices. -//! - **the digest verdict agrees with the decoded model**: the file either carries a -//! `NewRawImageDigest` — in which case `verify_new_raw_image_digest` must reach a verdict — or -//! it does not, in which case the verdict must be `Absent`. Two entry points read the same tag -//! by different routes (`decode` models it, `verify` re-reads it), so a disagreement means one -//! of them found a tag the other did not. +//! `verify_new_raw_image_digest` is driven for its own reach: on a lossy-compressed raw it walks +//! the chunk grid and digests the compressed chunks, which `decode` never does. Its verdict is +//! compared against the decoded model as a **structure pin, not a differential** — both sides read +//! `NewRawImageDigest` out of IFD 0 with the same expression, so the comparison cannot fail while +//! those two bodies agree. It is kept because the two are genuinely separate readers that a future +//! change could let drift apart (a `verify` that started selecting the raw IFD's digest, say), and +//! it costs nothing: the call is made anyway, for the crash oracle. +//! +//! ## Why a failed digest check is a classified outcome, not a crash +//! +//! `verify_new_raw_image_digest` is not a second call to `decode`. It re-reads the container and +//! then, depending on the file's own `Compression` code, takes one of two routes — and only one of +//! them is a subset of what `decode` did. It is therefore free to return `Err` on a file that +//! decoded, and an earlier draft turned that into `expect(...)`: a **false crash**, +//! indistinguishable from a real one until a human minimises it, on a tier that runs unattended +//! with no human at the other end. Nothing in either function's documented contract promises +//! "everything that decodes also verifies" — `verify`'s own docs bound its errors by `decode`'s +//! only *for lossless storage* — so the `Err` arm is simply a case with nothing to compare, and +//! the target returns instead of panicking. //! //! A crash found here is **minimised and promoted into a named deterministic case** in //! `gamut-dng`'s own suite. The corpus is a search aid, not the regression record. @@ -50,9 +65,11 @@ fuzz_target!(|data: &[u8]| { decoded.raw.samples_per_pixel() ); - // `decode` succeeded, so the container is walkable and the digest path must have reached a - // verdict too — the two routes disagree only if they selected different directories. - let verdict = verdict.expect("digest check on a file that decoded"); + // The digest route may legitimately refuse a file `decode` accepted (see the module docs), so + // an `Err` is a case with nothing to cross-check, not a defect. Only a verdict is comparable. + let Ok(verdict) = verdict else { + return; + }; assert_eq!( decoded.new_raw_image_digest.is_none(), verdict == DigestCheck::Absent, From 078d78544a235f3cef0542188f8b5f7d20c5a86e Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 06:01:32 -0400 Subject: [PATCH 07/24] test(ifd): name the malformed-input case for the assertion it makes The case list gained an assertion on the error *text* -- that a 65 535-entry count in a ten-byte file is refused with "IFD extends past end of file" -- while the test was still named for surviving without a panic. A test's name has to cover what it fails for, so it is now `specific_malformed_inputs_yield_typed_errors_not_panics`. The module docs also called the two-reader comparison a differential. It is not one: `read` is defined as `IfdReader::open(data)?.read_file()`, so both sides are the same parser reached twice. It is a structure pin on the wrappers going on delegating rather than growing a second directory walk with a second set of hostile-input guards to drift, and it is named as one now. Being unfalsifiable by input, this bounded exhaustive corpus is where it belongs, rather than in the unbounded fuzz tier that was also running it. Refs #264 --- crates/gamut-ifd/tests/robustness.rs | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/crates/gamut-ifd/tests/robustness.rs b/crates/gamut-ifd/tests/robustness.rs index bcc8cca6..b2e1768a 100644 --- a/crates/gamut-ifd/tests/robustness.rs +++ b/crates/gamut-ifd/tests/robustness.rs @@ -3,9 +3,14 @@ //! panic, a hang, or unbounded allocation (STATUS P6). //! //! Every input is also driven through the streaming [`IfdReader`] and the two entry points must -//! *agree* — both parse to equal files, or both fail. The slice functions are thin wrappers over -//! the streaming engine (one parser), so this differential layer is now a regression gate on the -//! wrappers themselves staying faithful. +//! *agree* — both parse to equal files, or both fail. This is a **structure pin, not a +//! differential**: `read` is *defined* as `IfdReader::open(data)?.read_file()` (and `read_tree` +//! likewise), so the two sides are one parser reached twice and the comparison cannot fail while +//! those bodies stand. It is kept because a `read` that stopped delegating — growing a second +//! directory walk, and with it a second set of hostile-input guards to drift — is exactly the +//! regression the crate's one-parser design exists to prevent. Being unfalsifiable by input, it +//! belongs here, over this bounded corpus, and not in the unbounded fuzz tier, where it would cost +//! half of every execution and search for a counterexample that does not exist. use gamut_ifd::{ ByteOrder, Ifd, IfdReader, TiffFile, Value, Variant, read, read_audited, read_tree, write, @@ -73,7 +78,7 @@ fn survives(data: &[u8]) { } #[test] -fn specific_malformed_inputs_error_without_panic() { +fn specific_malformed_inputs_yield_typed_errors_not_panics() { let cases: &[&[u8]] = &[ b"", b"II", From 8f170ffec3ffc0cf215f7a78dcf0002ba8ee7b1f Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 06:01:32 -0400 Subject: [PATCH 08/24] ci(fuzz): build the targets and reconcile the lists on the pull-request path Two holes, both silent. `tooling/gamut-fuzz` is workspace-excluded and nothing depends on it, so no gate on the pull-request path compiles its targets at all: an API change in gamut-ifd, gamut-tiff, gamut-dng, gamut-isobmff or gamut-heic breaks them and every check stays green until the next Extended run on master. That is the hole `check-dng-real` already closes for the other excluded tier, in the same job, for the same reason, so this follows it exactly: `cargo check --manifest-path tooling/gamut-fuzz/Cargo.toml --all-targets`. Build-only -- no nightly, no sanitizer, no engine -- so nothing unbounded reaches the per-PR path and the rule in docs/testing.md stands. The driven crates are already built by the Clippy step above it. A target also exists in three hand-maintained places nothing reconciled: its file under `fuzz_targets/`, its `[[bin]]` entry, and its row in `extended.yml`'s matrix. Miss the third and the target is written, reviewed, committed -- and never run, with nothing anywhere reporting it. `check-targets.sh` fails on any of the four mismatches, and on a `[[bin]]` whose `name` disagrees with its own `path`. It is pure text -- no cargo, no toolchain -- so it runs in the cheap Format & Metadata job. Each of the five failure modes was fault-injected and confirmed to fire. Refs #264 --- .github/workflows/ci.yml | 17 +++++++ tooling/gamut-fuzz/check-targets.sh | 74 +++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) create mode 100755 tooling/gamut-fuzz/check-targets.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8dde08e9..d5a14fd6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -69,6 +69,14 @@ jobs: # property depending on OS entropy would make cargo-mutants report CAUGHT or MISSED for the # same mutant on different runs. run: mise run check-tests + - name: Fuzz target lists in step + # Three hand-maintained lists describe the same set of fuzz targets: the files under + # tooling/gamut-fuzz/fuzz_targets, the [[bin]] entries in that crate's Cargo.toml, and the + # matrix of extended.yml's fuzz job. Nothing reconciled them, and the failure mode of the + # third is silent -- a target that is written and committed but never run, with no check + # anywhere reporting it. Pure text, no cargo, sub-second, which is why it is here and not + # in lint. + run: ./tooling/gamut-fuzz/check-targets.sh - name: Check PR commit messages # Only PRs have a base..head range; validate just the PR's own commits so the pre-existing # non-conventional history on master doesn't fail the check. @@ -147,6 +155,15 @@ jobs: # corpus, so this job does not grow the ~178 MiB fetch that `test-dng-real` requires. - name: Real-DNG conformance tier compiles run: mise run check-dng-real + # Same hole, same shape: `tooling/gamut-fuzz` is workspace-excluded and nothing depends on + # it, so no gate on the pull-request path compiles its targets at all -- an API change in + # gamut-ifd, gamut-tiff, gamut-dng, gamut-isobmff or gamut-heic breaks them unnoticed until + # the next Extended run on master. This is build-only: no nightly, no sanitizer, no engine, + # so it stays bounded and reproducible and does not put a coverage-guided run on the PR path + # (docs/testing.md, "Why fuzzing is not in the per-PR gate"). The driven crates are already + # built by the Clippy step above, so the marginal cost is the targets themselves. + - name: Fuzz tier compiles + run: cargo check --manifest-path tooling/gamut-fuzz/Cargo.toml --all-targets - name: gamut-ffi feature sync run: mise run check-ffi-features - name: gamut-ffi header sync diff --git a/tooling/gamut-fuzz/check-targets.sh b/tooling/gamut-fuzz/check-targets.sh new file mode 100755 index 00000000..9876b44a --- /dev/null +++ b/tooling/gamut-fuzz/check-targets.sh @@ -0,0 +1,74 @@ +#!/usr/bin/env bash +# Drift guard for the fuzz tier's three hand-maintained lists (issues #264, #311). +# +# A target exists in three places that nothing reconciles: +# +# 1. tooling/gamut-fuzz/fuzz_targets/.rs -- the code +# 2. tooling/gamut-fuzz/Cargo.toml [[bin]] -- what cargo-fuzz can build and list +# 3. .github/workflows/extended.yml fuzz matrix -- what CI actually runs +# +# Miss (2) and `mise run fuzz ` says the target does not exist. Miss (3) and the target is +# written, reviewed, committed -- and never run, silently, because nothing anywhere fails. This +# script is the thing that fails. It is pure text: no cargo, no toolchain, no network, which is +# why CI runs it in the cheap `Format & Metadata` job rather than in lint. +set -euo pipefail + +cd "$(dirname "$0")/../.." + +MANIFEST="tooling/gamut-fuzz/Cargo.toml" +WORKFLOW=".github/workflows/extended.yml" +TARGET_DIR="tooling/gamut-fuzz/fuzz_targets" + +# (1) The files on disk. +files="$(find "$TARGET_DIR" -maxdepth 1 -name '*.rs' -printf '%f\n' | sed 's/\.rs$//' | sort)" + +# (2) The `[[bin]]` entries. Read the name/path pair per section so a `name` that disagrees with +# its own `path` -- a copy-paste that builds the wrong file under the right label -- is caught +# too, not just a missing section. +bins="$( + awk ' + /^\[\[bin\]\]/ { name = ""; path = ""; next } + /^\[/ { name = ""; path = ""; next } + /^name = "/ { name = $3; gsub(/"/, "", name) } + /^path = "fuzz_targets\// { path = $3; gsub(/"|fuzz_targets\/|\.rs/, "", path) } + name != "" && path != "" { if (name != path) { print "[[bin]] name \"" name "\" builds fuzz_targets/" path ".rs" > "/dev/stderr"; exit 1 } + print name; name = ""; path = "" } + ' "$MANIFEST" | sort +)" + +# (3) The workflow matrix. The list is a plain YAML sequence under the fuzz job's `target:` key; +# comment lines inside it are skipped, and the first line that is neither ends the list. +matrix="$( + awk ' + /^ fuzz:$/ { job = 1 } + job && /^ *target:$/ { list = 1; next } + list { + if ($0 ~ /^ *#/) { next } + if ($0 ~ /^ *- [A-Za-z0-9_]+$/) { print $2; next } + exit + } + ' "$WORKFLOW" | sort +)" + +status=0 +report() { + local what="$1" left="$2" right="$3" left_list="$4" right_list="$5" + local missing + missing="$(comm -23 <(printf '%s\n' "$left_list") <(printf '%s\n' "$right_list"))" + if [ -n "$missing" ]; then + echo "fuzz target(s) in $left but not in $right ($what):" >&2 + printf ' %s\n' $missing >&2 + status=1 + fi +} + +report "the target would never be built" "$TARGET_DIR" "$MANIFEST" "$files" "$bins" +report "a [[bin]] points at a file that does not exist" "$MANIFEST" "$TARGET_DIR" "$bins" "$files" +report "the target would never run in CI" "$MANIFEST" "$WORKFLOW" "$bins" "$matrix" +report "CI names a target that cannot be built" "$WORKFLOW" "$MANIFEST" "$matrix" "$bins" + +if [ "$status" -ne 0 ]; then + exit 1 +fi + +echo "fuzz targets in step: $(printf '%s\n' "$files" | wc -l) in $TARGET_DIR, $MANIFEST and $WORKFLOW" From 7cd2e6d9c818325dfa90729d45ca75fd46835494 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 06:01:49 -0400 Subject: [PATCH 09/24] docs(testing): record the wired fuzz entry points The per-crate selection table's own rule is that a row changes only in a pull request that says why. This is that pull request: six entry points it marked "not yet wired" now have a driver in `tooling/gamut-fuzz`, so gamut-ifd's driver mark, gamut-tiff, gamut-dng, gamut-isobmff and gamut-heic's two both flip, and the legend gains the meaning of the tick it never defined. Two rules the tier needed and the file did not state are added beside them. First, the per-PR path carries the *compile* half of an excluded fuzz tier, plus the drift guard over the three lists that describe the target set -- neither runs an engine, so the "bounded and reproducible" rule above is untouched. Second, a robustness target is not a law, does not route through `invariants`, and any check it adds beyond the engine's own oracle must be able to fail: comparing a wrapper against the expression its own body is is a tautology, not a differential, and belongs in the crate's bounded suite as a structure pin if it is worth keeping at all. Refs #264, #565 --- docs/testing.md | 36 +++++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-) diff --git a/docs/testing.md b/docs/testing.md index c38d701a..5a8dd540 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -169,6 +169,23 @@ lives where the DNG sample corpus lives: an excluded `tooling/` crate, a corpus task, and an `extended.yml` job. **A crash it finds is minimised and promoted into a named deterministic case in that crate's `tests/robustness.rs`**, which is where the regression value is. +What the per-PR path *does* carry is the **compile** half, for the reason the excluded real-DNG +tier already carries it: nothing else builds an excluded crate, so an API change in a driven crate +breaks its targets unnoticed until the next run on master. CI's lint job runs +`cargo check --manifest-path tooling/gamut-fuzz/Cargo.toml --all-targets` — no nightly, no +sanitizer, no engine, nothing unbounded — and its `Format & Metadata` job runs +`tooling/gamut-fuzz/check-targets.sh`, which reconciles the three hand-maintained lists that +describe the target set (the files, the `[[bin]]` entries, the `extended.yml` matrix), because a +target missing from the third is one that never runs and nothing reports it. + +A **robustness** target is not a law and does not route through an `invariants` module: its +primary oracle is the engine's own — a panic, a hang, or an allocation past `-malloc_limit_mb` — +which no function can express. Any check it adds beyond that oracle must be able to *fail*: an +assertion comparing a wrapper against the expression its own body is (`gamut_ifd::read` against +`IfdReader::open(..)?.read_file()`) is a tautology, not a differential, and belongs — if it is +worth pinning at all — in the crate's bounded deterministic suite as a **structure pin**, named as +one. + `#[ignore]` is not used in this workspace and must not be introduced: `coverage` is the only test gate, so an ignored test is not deferred, it is unrun. @@ -177,15 +194,16 @@ gate, so an ignored test is not deferred, it is unrun. The authority and primary technique for each crate are decided **here, once**. A row changes only in a pull request that says why. "Authority" is the **in-crate** authority — several crates are additionally covered by a consuming codec's oracle, which their own `STATUS.md` records. "Fuzz -entry point" names the untrusted-input surface a fuzz target takes; ☐ marks one not yet wired -(#264). The binary, binding and stub crates (`gamut`, `gamut-cli`, `gamut-wasm`, `gamut-ffi`, -`gamut-jxl-sys`, `gamut-av2`, `gamut-vvc`) have no row: they are excluded from the coverage and -mutation gates, and the stubs carry no function bodies. +entry point" names the untrusted-input surface a fuzz target takes; ☑ marks one a target in +`tooling/gamut-fuzz` drives today, ☐ one not yet wired (#264). The binary, binding and stub crates +(`gamut`, `gamut-cli`, `gamut-wasm`, `gamut-ffi`, `gamut-jxl-sys`, `gamut-av2`, `gamut-vvc`) have +no row: they are excluded from the coverage and mutation gates, and the stubs carry no function +bodies. | Crate | Authority | Primary technique | Fuzz entry point | | --- | --- | --- | --- | | gamut-core | *none* — no oracle exists | **property** (`convert`, `image` stride math) | — | -| gamut-ifd | *none in-crate* — libtiff/exiv2 reach it via the consuming codecs (STATUS.md P7) | **property** + exact-byte | `IfdReader`, `read` ☑ laws; ☐ driver | +| gamut-ifd | *none in-crate* — libtiff/exiv2 reach it via the consuming codecs (STATUS.md P7) | **property** + exact-byte | `IfdReader`, `read` ☑ laws; ☑ driver | | gamut-tonemap | *none* | **property** (monotonicity, endpoints, no NaN) | — | | gamut-bitstream | *none* — self-inverse | property + exact-byte | — | | gamut-dsp | AV1 §7.13 / T.81 §A.3 transform definitions | example + in-test reference transform | — | @@ -196,13 +214,13 @@ mutation gates, and the stubs carry no function bodies. | gamut-deflate | zlib | differential | — | | gamut-png | libpng (both directions) | differential + conformance | `PngDecoder` ☐ | | gamut-jpeg | libjpeg-turbo | differential + exact-byte | `JpegDecoder` ☐ | -| gamut-tiff | libtiff | differential | `TiffDecoder` ☐ | -| gamut-dng | Adobe DNG SDK; libtiff (container) | conformance + differential | `DngDecoder` ☐ | -| gamut-isobmff | ISO/IEC 14496-12 + 23008-12; libavif/dav1d via gamut-avif | exact-byte + law | `read` ☐ | +| gamut-tiff | libtiff | differential | `TiffDecoder` ☑ | +| gamut-dng | Adobe DNG SDK; libtiff (container) | conformance + differential | `DngDecoder` ☑ | +| gamut-isobmff | ISO/IEC 14496-12 + 23008-12; libavif/dav1d via gamut-avif | exact-byte + law | `read` ☑ | | gamut-riff | libwebp demux | differential + law | `RiffReader` ☐ | | gamut-webp | libwebp (both directions) | differential + size/effort contract | `WebpDecoder` ☐ | | gamut-avif | libavif; dav1d | differential + law | `decode` ☐ | -| gamut-heic | libheif + libde265 | differential + law | container `parse`, NAL `parse` ☐ | +| gamut-heic | libheif + libde265 | differential + law | container `parse`, NAL `parse` ☑ | | gamut-av1 | libaom (definitive); dav1d | differential | — | | gamut-jxl | libjxl (the `jxl` crate is the decoder under test, not an authority) | differential | `decode` ☐ | | gamut-exif | exiv2 | differential + golden | `parse` ☐ | From 6725ee81650214536e362bf94bd51053d4704999 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 06:01:49 -0400 Subject: [PATCH 10/24] docs(fuzz): reprice the robustness checks and state the expected-red rows The targets table sold three checks that cannot fail. They are repriced rather than merely deleted: a claim about two bodies agreeing is a structure pin, kept where it is free and named as one, and the table now states the rule that put them there -- a check is listed only if it can fail. Two rows fail today, on #563 and #564, and a reader meets that first in Extended's aggregate status, which stays red on every push to the default branch until both close. Narrowing either target to make its row green would be weakening a check to make a report green, so instead the expectation is written where it will be met: in the README, and in a comment on the job that produces it. Restoring the aggregate's meaning is #593; the job's cadence, inherited rather than chosen and now nine parallel ten-minute runners per push, is #594. Also: two paragraphs annotating the law-target table had been orphaned under the seed section by a heading added between them; the seed set is described by what it actually contains, `corpus/tiff_decode/` holding two files and not one; and the note that a feature added for one target is on for every target, because Cargo resolves features once per crate -- `bigtiff` went in for `ifd_read` and the pre-existing `ifd_read_ledger` is built with it too. Refs #264 --- .github/workflows/extended.yml | 6 +++ tooling/gamut-fuzz/README.md | 98 +++++++++++++++++++++++++++------- 2 files changed, 86 insertions(+), 18 deletions(-) diff --git a/.github/workflows/extended.yml b/.github/workflows/extended.yml index f6a204cb..3a16fbf8 100644 --- a/.github/workflows/extended.yml +++ b/.github/workflows/extended.yml @@ -170,6 +170,12 @@ jobs: # A crash found here is minimised and promoted into a NAMED DETERMINISTIC TEST in the # crate's own suite -- the corpus is a search aid, not the regression record -- so a # failure of this job is a signal to go and write that test, not to re-run until green. + # + # EXPECTED RED: `tiff_decode` and `dng_decode` fail today on filed, accepted defects + # (#563, #564). They are deliberately not narrowed to make these rows green; read the + # per-row status rather than this workflow's aggregate until both close. Restoring the + # aggregate's meaning is #593; this job's cadence (nine parallel ten-minute runners on + # every push to the default branch, growing with the matrix) is #594. run: mise run fuzz ${{ matrix.target }} -- -max_total_time=600 - name: Upload any crash artifacts if: failure() diff --git a/tooling/gamut-fuzz/README.md b/tooling/gamut-fuzz/README.md index 1123ea0a..1d68a826 100644 --- a/tooling/gamut-fuzz/README.md +++ b/tooling/gamut-fuzz/README.md @@ -62,6 +62,17 @@ same rule `docs/testing.md` applies to a shrunk `proptest` counterexample, and t `x86_64-unknown-linux-musl`, and the sanitizer cannot link against a static libc. Without the pin the build fails before reaching a target at all. - **This crate is workspace-excluded**, so `cargo test --workspace --all-features` never builds it. + Nothing on the pull-request path would otherwise compile these targets at all, and an API change + in a driven crate would break them unnoticed until the next Extended run. CI's lint job therefore + runs `cargo check --manifest-path tooling/gamut-fuzz/Cargo.toml --all-targets` — build-only, no + nightly, no sanitizer, no engine — exactly as it already does for the excluded real-DNG + conformance tier. +- **The dependency graph is shared across every target.** A feature turned on for one target's + crate is on for all of them, because Cargo resolves features once per crate for the whole + package: `bigtiff` was added to `gamut-ifd` for the `ifd_read` driver, and the pre-existing + `ifd_read_ledger` law target is now built with it too. That is harmless here — `bigtiff` widens + the accepted input rather than changing the laws — but it is not free in general, and a feature + added for one target must be checked against the others before it goes in. - **The `--` is reconstructed, not passed through.** mise swallows a task's `--`, so `mise run fuzz t -- -max_total_time=60` reaches `run.sh` as two bare words and cargo-fuzz would reject the second as one of its own options. The runner re-splits on libFuzzer's own flag syntax @@ -85,6 +96,16 @@ over normalised inputs, per the section above. One file per crate, deliberately, so adding a crate is an additive change. +`Drago` is held to monotonicity only where `Drago::is_monotonic` says it claims it (#439); every +other operator promises it unconditionally, and all of them are driven through the other three +laws. + +The `tonemap_curves` target found a defect **in a law** within a minute of first running: the +monotonicity tolerance derived its scale from the sampled outputs, so a sample set drawn entirely +from `Hable`'s near-zero cancellation region measured the noise against itself. Fixed in the same +change, with the case promoted into a named test — which is the workflow this file prescribes, +exercised once. + ### Robustness targets (#264) They hand the engine's bytes, unchanged, to the **parser entry point** `docs/testing.md`'s @@ -98,18 +119,66 @@ no crash is still visible: | target | crate | entry points | check beyond the crash oracle | |---|---|---|---| -| `ifd_read` | `gamut-ifd` | `read`, `read_tree`, `read_audited`, `IfdReader` | slice and streaming readers agree; the dual-ledger audit is complete | +| `ifd_read` | `gamut-ifd` | `read`, `read_tree`, `read_audited` | the dual-ledger audit is complete: no byte read outside a claim, no claim unread | | `tiff_decode` | `gamut-tiff` | `TiffDecoder::{page_count,info_page,decode_page}` | the page index is bounded by `page_count`; describing and decoding agree on geometry | -| `dng_decode` | `gamut-dng` | `DngDecoder::{decode,verify_new_raw_image_digest}` | the decoded raw is self-consistent; the digest verdict agrees with the decoded model | +| `dng_decode` | `gamut-dng` | `DngDecoder::{decode,verify_new_raw_image_digest}` | the raw image that *arrives* holds exactly `width × height × planes` samples, after every rewriting stage | | `isobmff_boxes` | `gamut-isobmff` | `walk_segments`, `walk_meta_children`, `read`, `BoxReader` | the box cursor strictly advances; the segments tile `0..len` exactly | | `heic_container` | `gamut-heic` | `HeifContainer::parse` | the segments tile `0..len` exactly and every accessor agrees with that tiling | -| `heic_hvcc` | `gamut-heic` | `HevcConfig::parse`, `annex_b*`, `validate_still_payload`, `iter_nal_units` | `annex_b` is its two documented halves, concatenated and appended | +| `heic_hvcc` | `gamut-heic` | `HevcConfig::parse`, `annex_b*`, `validate_still_payload`, `iter_nal_units` | the Annex-B emitters append rather than replace, on the success path and the error path | + +**A check is only listed here if it can fail.** Three earlier entries could not. `ifd_read` +compared `read(data)` against `IfdReader::open(data)?.read_file()` — but `reader.rs` *defines* +`read` as that expression, so the two sides were one function call written twice. `heic_hvcc` +compared `annex_b(..).is_ok()` against `annex_b_payload(..).is_ok()` on the same input, and +asserted `annex_b` equals the two calls its own body makes. `dng_decode` compared a digest verdict +against a decoded field that is read with the *same expression* on both sides. None of them had a +reachable failure, and calling any of them a differential overstated what the tier proves. + +They are not all deleted — they are **relabelled and repriced**. A claim about two bodies agreeing +is a **structure pin**: worth keeping where it is free or where a future change could genuinely +split the bodies apart, worth nothing as a search. So `heic_hvcc` still asserts the two halves, +folded into the append check's existing buffer at no extra emitter pass; `dng_decode` still +compares the verdict, on a call it makes anyway for the crash oracle; and the `gamut-ifd` wrapper +pin lives in `crates/gamut-ifd/tests/robustness.rs`, over a bounded exhaustive corpus, rather than +costing half of every one of this target's twenty thousand executions per second to search for a +counterexample that does not exist. Dropping the two duplicate parses raised `ifd_read` from +roughly 12 000 exec/s to roughly 20 000. An **allocation** defect needs the engine's malloc hook to be visible at all: an oversized `Vec::with_capacity` costs no resident memory on an overcommitting kernel, so measuring RSS finds nothing and `-malloc_limit_mb` (which libFuzzer defaults to `-rss_limit_mb`, 2048) is the oracle. That is how `dng_decode` reports a 780-byte file asking for a 34 GB allocation. +### Two of these rows are red on purpose + +`Fuzz tiff_decode` and `Fuzz dng_decode` **fail today**, on the first defects this tier found: +[#563](https://github.com/visualcommons/gamut/issues/563) (a panic on `SamplesPerPixel = 0`) and +[#564](https://github.com/visualcommons/gamut/issues/564) (a raw buffer sized from declared +geometry). They are filed rather than fixed, because narrowing a target so its row goes green is +weakening a check to make a report green — the opposite of what the tier is for. + +What that costs, stated plainly so nobody has to rediscover it: **Extended runs on every push to +the default branch, so its aggregate status stays red until both are fixed.** The blast radius is +bounded — Extended is post-merge and manual-dispatch only, and the fuzz job is `fail-fast: false`, +so no pull request is blocked and no other row is cancelled — but a human scanning one red tick +per push learns nothing from it. Read the per-row status, not the aggregate, until #563 and #564 +close; both rows go green with no change here. Whether these two rows should instead live in a +separate, expected-to-fail lane so the aggregate keeps its meaning is +[#593](https://github.com/visualcommons/gamut/issues/593) — a workflow-topology question, not a +fuzzing one. The job's cadence, which was inherited rather than chosen and now costs nine parallel +ten-minute runners per push, is [#594](https://github.com/visualcommons/gamut/issues/594). + +## Keeping the three lists in step + +A target exists in three hand-maintained places: its `fuzz_targets/.rs` file, its `[[bin]]` +entry in `Cargo.toml`, and its row in `extended.yml`'s fuzz matrix. Miss the third and the target +is written, committed, and never run — silently, because nothing fails. `check-targets.sh` +reconciles all three and is wired into CI's `Format & Metadata` job; run it directly too: + +```bash +./tooling/gamut-fuzz/check-targets.sh +``` + ## Seeds `corpus//` holds a small **curated seed set**, tracked despite `.gitignore` listing @@ -123,19 +192,12 @@ adds a couple of hundred files — and those stay untracked, which is the point. `git add -f` the whole directory a second time**: add the one seed you mean by path, or the engine's search state goes in with it. -They are seeds, **not** the regression record. `corpus/ifd_read/` carries the malformed-TIFF cases -enumerated on issue #264 (contributed from rawshift's deleted in-repo TIFF parser); the other -directories carry one small well-formed file each, written by this workspace's own encoders, so a -decoder target starts from something that reaches its pixel path instead of spending its budget -rediscovering a header. Real-camera corpora are deliberately not vendored: they run to hundreds of +They are seeds, **not** the regression record. `corpus/ifd_read/` carries the thirteen +malformed-TIFF cases enumerated on issue #264 (contributed from rawshift's deleted in-repo TIFF +parser); each of the other five directories carries one or two small well-formed files, written by +this workspace's own encoders, so a decoder target starts from something that reaches its pixel +path instead of spending its budget rediscovering a header. `corpus/tiff_decode/` carries two — +`rgb8-none.tif` and `rgb8-lzw.tif` — because an uncompressed strip and an LZW strip enter the +decoder through different code, and seeding only one leaves the other to be rediscovered. +Real-camera corpora are deliberately not vendored: they run to hundreds of megabytes and live in `justin13888/rawshift-test-fixtures` releases. - -`Drago` is held to monotonicity only where `Drago::is_monotonic` says it claims it (#439); every -other operator promises it unconditionally, and all of them are driven through the other three -laws. - -The `tonemap_curves` target found a defect **in a law** within a minute of first running: the -monotonicity tolerance derived its scale from the sampled outputs, so a sample set drawn entirely -from `Hable`'s near-zero cancellation region measured the noise against itself. Fixed in the same -change, with the case promoted into a named test — which is the workflow this file prescribes, -exercised once. From 0f8662a1ccf091b2307ab04190accfd49490f966 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 07:18:55 -0400 Subject: [PATCH 11/24] test(fuzz): re-anchor the TIFF decode check on the samples produced Both checks `tiff_decode` advertised were unfalsifiable by input. The page-index bound compared a count against the same expression that produces it: `page_count` is `read(data)?.ifds.len()` and `info_page` is `read(data)?.ifds.get(page)`, so the claim reduced to indexing a vector one past its own length, and the defect it named -- a count that over-reports the chain -- moves both sides together. It is dropped. The geometry equality saw only the lines copying a described number into a decoded one: `decode_page_samples` states outright that "everything the page declares comes from one shared reader", so a transposition inside `info::page_info` hands every caller a transposed image and the comparison stays quiet. It is replaced by a count the geometry reader does not produce -- the samples the strip/tile assembly, the predictor pass and the photometric unpack physically yielded -- against the declared dimensions and the channel count of the layout asked for. Injected to prove it fires, and recorded in the module doc so a reader can re-run it: trim the last row at the point the `DecodedImage` is built and report `height - 1`, a crop stage that describes what it cropped. Both `RawImage::new` and `ImageBuf::new` accept it, nothing crashes, and the committed seeds alone report "page 0: decoded 54 samples for the 6 x 4 x 3 the tags declare". Also recorded: the injection that did *not* report (decoding one row more than declared), because the assembly runs out of bytes and the page is refused -- which bounds what the check reaches. The remaining "a page that decodes must also describe" assertion cannot fail either, since decoding calls the tag reader first; it is kept at zero cost and labelled a structure pin at the site. Refs #264 --- .../gamut-fuzz/fuzz_targets/tiff_decode.rs | 75 ++++++++++++++----- 1 file changed, 56 insertions(+), 19 deletions(-) diff --git a/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs b/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs index a9048660..30382590 100644 --- a/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs +++ b/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs @@ -5,15 +5,47 @@ //! cap, so a hostile file must end in a typed error rather than a panic, a hang or a runaway //! allocation — the three things libFuzzer itself detects. //! -//! Two further checks make the target able to fail for something other than a crash: +//! The check beyond the crash oracle is that **the decoded volume matches the declared geometry**: +//! a page that decodes yields exactly `width × height × Rgb8::CHANNELS` samples, where the width +//! and height are the ones `info_page` read out of the tags. The sample count is not read from the +//! geometry reader at all — it is what the strip/tile assembly, the predictor pass and the +//! photometric unpack physically produced — so the check reaches that whole pipeline rather than +//! the handful of lines that copy a described number into a decoded one. //! -//! - **the page index is bounded by `page_count`**: `info_page` at the count itself must be -//! refused. A count that over-reports the chain is how an out-of-range page reaches the tag -//! reader at all. -//! - **describing and decoding agree**: a page that decodes must also describe, and the two must -//! report the same dimensions. `info` reads tags only and `decode_page` reads pixels, so a -//! disagreement means the two paths read the geometry differently — exactly the split that -//! turns a size check into a false guarantee. +//! Injection that proved it fires (re-runnable): at the point `decode_page_samples` builds its +//! `DecodedImage`, trim the last row from the samples *and* report `height - 1` — a crop stage +//! that describes what it cropped. It is internally consistent, so `RawImage::new` and +//! `ImageBuf::new` both accept it and nothing crashes; only the declared geometry contradicts it. +//! Run as `run.sh tiff_decode -- -runs=0`, the committed seeds alone report it: +//! *"page 0: decoded 54 samples for the 6 × 4 × 3 the tags declare"*. +//! +//! Two things this deliberately does **not** assert, both because they cannot fail by input: +//! +//! - **`info_page` at `page_count` is refused.** `page_count` is `read(data)?.ifds.len()` and +//! `info_page` is `read(data)?.ifds.get(page)`, so the claim reduces to indexing a vector one +//! past its own length. Injecting the defect it advertised — a count that over-reports the +//! chain — produces no report, because the over-report moves both sides together. +//! - **decoded dimensions equal described dimensions.** The two sides are one reader: +//! `decode_page_samples` states outright that "everything the page *declares* comes from one +//! shared reader", so a transposition inside `info::page_info` hands every caller a transposed +//! image and this comparison stays quiet. It fires only for a defect in the few lines between +//! that reader and the returned buffer, which is a reach the sample count already covers. +//! +//! One assertion beside it is kept and **named a structure pin** rather than advertised as a +//! check: "a page that decodes must also describe". `decode_page_samples` calls `info::page_info` +//! before it reads a pixel, so a page `info_page` refuses cannot decode, for any input, while that +//! body stands. It costs nothing — both calls are made anyway for the crash oracle — and it is the +//! shape that would report if `decode` ever grew its own tag reader. +//! +//! What the sample count does not see either, stated so nobody over-reads it: a defect that +//! produces the wrong *volume* while leaving the dimensions alone is turned into a typed error by +//! `RawImage::new`/`ImageBuf::new` before it can reach a caller, so it arrives here as a rejected +//! file rather than as a report. Measured, not assumed: decoding `info.height + 1` rows — the +//! first injection tried — produced **no** report, because the strip assembly runs out of bytes +//! and the page is refused. The live class is a stage that rewrites the geometry it hands on — a +//! crop, an orientation, a tile-grid rounding — which both constructors accept and only the +//! declared geometry contradicts. That is the same class the sibling `dng_decode` target checks, +//! where linearisation and active-area handling are such stages today. //! //! The policy is [`ConvertPolicy::permissive`] so the decode reaches the pixel and conversion //! paths for pages the default lossless policy would refuse at the layout gate. @@ -24,6 +56,7 @@ #![no_main] use gamut_core::convert::ConvertPolicy; +use gamut_core::{Pixel, Rgb8}; use gamut_tiff::TiffDecoder; use libfuzzer_sys::fuzz_target; @@ -31,8 +64,7 @@ use libfuzzer_sys::fuzz_target; /// /// A chained TIFF may declare up to `gamut-ifd`'s 65 536 directories, and decoding all of them /// would make a single execution slow enough to look like a hang; the interesting per-page -/// behaviour is reached in the first few. The page-index bound below is still checked against the -/// *full* count. +/// behaviour is reached in the first few. const MAX_PAGES: usize = 4; fuzz_target!(|data: &[u8]| { @@ -46,23 +78,28 @@ fuzz_target!(|data: &[u8]| { return; }; - // One past the last page is out of range, whatever the chain claimed. - assert!( - decoder.info_page(data, pages).is_err(), - "page {pages} described although page_count is {pages}" - ); - for page in 0..pages.min(MAX_PAGES) { let info = decoder.info_page(data, page); let image = decoder.decode_page(data, page); match (&info, &image) { (Ok(info), Ok(image)) => { + // The declared geometry, times the channel count of the layout that was asked for, + // against the samples the decode actually produced. + let declared = (info.width as usize) + .checked_mul(info.height as usize) + .and_then(|n| n.checked_mul(::CHANNELS)); assert_eq!( - (image.width(), image.height()), - (info.width, info.height), - "page {page}: decoded geometry differs from the described geometry" + Some(image.as_samples().len()), + declared, + "page {page}: decoded {} samples for the {} × {} × {} the tags declare", + image.as_samples().len(), + info.width, + info.height, + ::CHANNELS ); } + // Structure pin, not a check: decoding calls the tag reader first, so this cannot + // fail by input while it does (see the module docs). (Err(error), Ok(_)) => { panic!("page {page} decoded although it could not be described: {error}") } From b101b38fbfd3e1e6b70d60cd21ff694d6957e265 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 07:19:06 -0400 Subject: [PATCH 12/24] docs(fuzz): record the injection that fires each robustness check "This check can fail" was a reading of the code, and twice it was wrong. Every robustness target's module doc now names the defect that was injected to make its check fire, the message the target printed, and the command that reproduces it -- all of them report from the committed seeds alone under `-runs=0`, with no search: ifd_read header claimed as `header_size() - 1` -> "parser read bytes it never claimed", unclaimed_reads [7, len 1] isobmff_boxes a box's segment recorded as `b.offset + 8..end` -> "segment 8..24 leaves a gap or overlaps at 0" heic_container `boxes()` skips the ftyp box -> "boxes() disagrees with the Box segments" heic_hvcc `annex_b_parameter_sets` begins with `out.clear()` -> "an annex_b emitter overwrote what was already in the buffer" dng_decode `new_cfa` pushes a sample past `check_sample_count` -> "decoded raw holds 49 samples for 8 x 6 x 1 planes" Two further notes a reader would otherwise have to derive. The `heic_container` accessor check reaches exactly one three-line function per accessor -- that is a limitation, not a flaw, since the accessors are what callers use and the tiling check beside it is the deep one. And `heic_hvcc`'s "no empty NAL unit" assertion cannot fail by input at all: `NalUnitIter::next` errors on a zero length before it can yield an empty slice, so it is labelled a structure pin at the site rather than counted as a check. Refs #264 --- tooling/gamut-fuzz/fuzz_targets/dng_decode.rs | 6 ++++++ tooling/gamut-fuzz/fuzz_targets/heic_container.rs | 12 ++++++++++++ tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs | 14 +++++++++++++- tooling/gamut-fuzz/fuzz_targets/ifd_read.rs | 6 ++++++ tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs | 6 ++++++ 5 files changed, 43 insertions(+), 1 deletion(-) diff --git a/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs b/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs index 4e930d39..6740768d 100644 --- a/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs +++ b/tooling/gamut-fuzz/fuzz_targets/dng_decode.rs @@ -14,6 +14,12 @@ //! active-area and crop handling before a caller sees it, and it is the value that *arrives* — //! after everything that may have rewritten `samples` or `dims` — this asserts on. //! +//! Injection that proved it fires (re-runnable): have `RawImage::new_cfa` push one extra sample +//! *after* `check_sample_count` has passed — a constructor whose own gate no longer describes what +//! it built. The committed seed alone reports it, with no search: +//! `run.sh dng_decode -- -runs=0` gives *"decoded raw holds 49 samples for +//! Dimensions { width: 8, height: 6 } × 1 planes"*. +//! //! `verify_new_raw_image_digest` is driven for its own reach: on a lossy-compressed raw it walks //! the chunk grid and digests the compressed chunks, which `decode` never does. Its verdict is //! compared against the decoded model as a **structure pin, not a differential** — both sides read diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs index 6ae32136..ad59ae60 100644 --- a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs +++ b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs @@ -19,6 +19,18 @@ //! Real files reach here: phones append a whole second MP4 after the HEIC, and camera apps leave //! trailers, so the accounting path is not an exotic branch. //! +//! Injection that proved the accessor check fires (re-runnable): make `HeifContainer::boxes()` +//! skip the `ftyp` box — an accessor that filters what the segments hold. The committed seed alone +//! reports it, with no search: `run.sh heic_container -- -runs=0` gives *"boxes() +//! disagrees with the Box segments"*. +//! +//! Its **reach is one function per accessor**, and that is a limitation rather than a flaw: +//! `boxes`, `appended_stream` and `trailer` are each a three-line `filter_map`/`find_map` over the +//! segment list, so the check sees a defect in those and nothing deeper. It is kept at that size +//! because the accessors are the API every caller actually uses — the segment list is the +//! evidence, the accessors are the product — and because it costs one pass over a list the target +//! walks anyway. The tiling check above is the one with the deep reach: it sees the whole parse. +//! //! A crash found here is **minimised and promoted into a named deterministic case** in //! `gamut-heic`'s own suite. The corpus is a search aid, not the regression record. diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs b/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs index fbca2f7e..15064a76 100644 --- a/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs +++ b/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs @@ -14,6 +14,12 @@ //! construction: each one is free to `clear()` or to write through an index, and doing so breaks //! every reusing caller while producing no crash at all. That is what this target searches for. //! +//! Injection that proved the append check fires (re-runnable): begin +//! `HevcConfig::annex_b_parameter_sets` with `out.clear()` — an emitter that replaces instead of +//! appending, which breaks every reusing caller and crashes nothing. The committed seed alone +//! reports it, with no search: `run.sh heic_hvcc -- -runs=0` gives *"an annex_b emitter +//! overwrote what was already in the buffer"*. +//! //! Alongside it, and explicitly **not** a differential, is a **structure pin**: `annex_b`'s body //! *is* `annex_b_parameter_sets` followed by `annex_b_payload`, so asserting the whole equals the //! two halves cannot fail for any input while that body stands. It is kept because the split is a @@ -24,7 +30,11 @@ //! same expression, which an earlier draft also asserted, is trivially true and is gone. //! //! `validate_still_payload` is driven for its own sake: it re-walks the payload through -//! `NalHeader::parse`, a different reach from the Annex-B emitters. +//! `NalHeader::parse`, a different reach from the Annex-B emitters. The "no empty NAL unit" +//! assertion beside that walk is a **structure pin** too, and is labelled as one at the site: +//! `NalUnitIter::next` returns `Err("zero-length NAL unit")` for `len == 0` *before* it can yield +//! an empty slice, so no input reaches an `Ok` that fails it. It pins that early return staying +//! where it is, at the cost of one `is_empty` on a slice already in hand. //! //! ## Input framing //! @@ -82,6 +92,8 @@ fuzz_target!(|data: &[u8]| { let _ = config.validate_still_payload(payload); for nal in iter_nal_units(payload, config.nal_length_size()) { let Ok(nal) = nal else { break }; + // Structure pin, not a check: `next` errors on a zero length before it can yield an empty + // slice, so this cannot fail by input while that early return stands (module docs). assert!(!nal.is_empty(), "iter_nal_units yielded an empty NAL unit"); let _ = NalHeader::parse(nal); } diff --git a/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs b/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs index 8f42612e..1f14a155 100644 --- a/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs +++ b/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs @@ -17,6 +17,12 @@ //! never declares — or declares bytes it never touched — produces no crash at all, and this is //! what sees it. //! +//! Injection that proved the audit fires (re-runnable): in `IfdReader::read_chain`, claim the +//! header as `header_size() - 1` bytes — an off-by-one that leaves a byte the parser physically +//! reads outside every structural claim. The committed seeds alone report it, with no search: +//! `run.sh ifd_read -- -runs=0` gives *"parser read bytes it never claimed"* carrying +//! `unclaimed_reads: [Range { start: 7, len: 1 }]`. +//! //! ## What this target deliberately does *not* check //! //! An earlier draft also compared `read(data)` against `IfdReader::open(data)?.read_file()` and diff --git a/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs b/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs index a87a3b3b..6c22386b 100644 --- a/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs +++ b/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs @@ -12,6 +12,12 @@ //! produces, which is where a size-0 or a wrapping box length would show up as a hole or an //! overlap rather than as a crash. //! +//! Injection that proved the tiling check fires (re-runnable): in `walk_segments`, record a box's +//! segment as `b.offset + 8..end` — byte accounting that counts box bodies and forgets their +//! headers. The committed seed alone reports it, with no search: +//! `run.sh isobmff_boxes -- -runs=0` gives *"segment 8..24 leaves a gap or overlaps at +//! 0"*. +//! //! `BoxReader` is driven separately from `walk_segments` because it is the lower layer and a //! caller may use it directly: the check there is that the cursor advances strictly, so a walk of //! a hostile file cannot spin. From 6a6f619a3f270d351dca733b6c6faae45ccdbdcb Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 07:19:15 -0400 Subject: [PATCH 13/24] ci(fuzz): name a duplicated target entry as a duplicate A hand-maintained list can name the same target twice, and `comm -23` reports the second copy as a line present on the left and absent on the right. The guard then printed "a [[bin]] points at a file that does not exist" for a file that does exist, and "CI names a target that cannot be built" for a target that builds -- twice sending a reader after the wrong thing. Duplicates are now diagnosed first, by name, and both lists are de-duplicated before the set comparisons so those keep saying what they mean. `find` cannot produce a duplicate filename, so the files list needs no such check. Verified by injecting a repeated `[[bin]]` and a repeated matrix row: both new messages fire, and the restored tree passes. Refs #264 --- tooling/gamut-fuzz/check-targets.sh | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tooling/gamut-fuzz/check-targets.sh b/tooling/gamut-fuzz/check-targets.sh index 9876b44a..512ed6bc 100755 --- a/tooling/gamut-fuzz/check-targets.sh +++ b/tooling/gamut-fuzz/check-targets.sh @@ -51,6 +51,28 @@ matrix="$( )" status=0 + +# A hand-maintained list can name the same target twice, and `comm -23` reports the second copy as +# a line present on the left and absent on the right -- i.e. as "a [[bin]] points at a file that +# does not exist" or "CI names a target that cannot be built", neither of which is what happened. +# Duplicates are therefore diagnosed first, by name, and the lists are de-duplicated before the +# set comparisons below so those keep saying what they mean. (`find` cannot produce a duplicate +# filename, so (1) needs no such check.) +duplicates() { + local what="$1" where="$2" list="$3" dupes + dupes="$(printf '%s\n' "$list" | uniq -d)" + if [ -n "$dupes" ]; then + echo "$where names the same fuzz target more than once ($what):" >&2 + printf ' %s\n' $dupes >&2 + status=1 + fi +} + +duplicates "cargo would build it twice" "$MANIFEST" "$bins" +duplicates "CI would run it twice" "$WORKFLOW" "$matrix" +bins="$(printf '%s\n' "$bins" | uniq)" +matrix="$(printf '%s\n' "$matrix" | uniq)" + report() { local what="$1" left="$2" right="$3" left_list="$4" right_list="$5" local missing From 8c1ce503e02abf162b3038e4f118b76572c8972c Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 07:19:15 -0400 Subject: [PATCH 14/24] ci(fuzz): run the compile and drift checks through mise tasks Both steps this change added to the pull-request path carried their command inline in the workflow, so a contributor could not run what CI runs without reading YAML -- and the two copies drift. Every comparable gate in this repository is a mise task for that reason. `check-fuzz` compiles the excluded fuzz tier (build-only: no nightly, no sanitizer, no engine), mirroring `check-dng-real` for the other excluded tooling crate. `check-fuzz-matrix` runs the drift guard that reconciles the three hand-maintained lists describing the target set. The workflow now calls both by name. Refs #264 --- .github/workflows/ci.yml | 4 ++-- mise.toml | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d5a14fd6..79b0977b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -76,7 +76,7 @@ jobs: # third is silent -- a target that is written and committed but never run, with no check # anywhere reporting it. Pure text, no cargo, sub-second, which is why it is here and not # in lint. - run: ./tooling/gamut-fuzz/check-targets.sh + run: mise run check-fuzz-matrix - name: Check PR commit messages # Only PRs have a base..head range; validate just the PR's own commits so the pre-existing # non-conventional history on master doesn't fail the check. @@ -163,7 +163,7 @@ jobs: # (docs/testing.md, "Why fuzzing is not in the per-PR gate"). The driven crates are already # built by the Clippy step above, so the marginal cost is the targets themselves. - name: Fuzz tier compiles - run: cargo check --manifest-path tooling/gamut-fuzz/Cargo.toml --all-targets + run: mise run check-fuzz - name: gamut-ffi feature sync run: mise run check-ffi-features - name: gamut-ffi header sync diff --git a/mise.toml b/mise.toml index 50ccb405..3a184636 100644 --- a/mise.toml +++ b/mise.toml @@ -195,6 +195,25 @@ run = "git submodule update --init --checkout third_party/gamut-dng-samples" description = "Coverage-guided fuzzing of the invariants laws; `mise run fuzz `" run = "./tooling/gamut-fuzz/run.sh" +# The compile half of the tier above -- the only part of it the per-PR lane can afford, and the +# same argument `check-dng-real` below makes for the other excluded `tooling/` crate: `gamut-fuzz` +# is workspace-excluded *and* nothing depends on it, so `clippy --workspace --all-targets` and +# `test --workspace` never build it, and an API change in gamut-ifd, gamut-tiff, gamut-dng, +# gamut-isobmff or gamut-heic can break every target while all four required checks stay green. +# Build-only: no nightly, no sanitizer, no engine, so it stays bounded and reproducible and does +# not put a coverage-guided run on the pull-request path (docs/testing.md). +[tasks.check-fuzz] +description = "Compile the fuzz tier's targets (no nightly, no engine)" +run = "cargo check --manifest-path tooling/gamut-fuzz/Cargo.toml --all-targets" + +# The drift guard for the three hand-maintained lists that describe the same target set: the files +# under `fuzz_targets/`, the `[[bin]]` entries, and `extended.yml`'s fuzz matrix. Missing from the +# third means the target is written, reviewed, committed -- and never run, silently. Pure text: no +# cargo, no toolchain, no network, which is why CI runs it in the cheap `Format & Metadata` job. +[tasks.check-fuzz-matrix] +description = "Reconcile the fuzz targets across fuzz_targets/, Cargo.toml and extended.yml" +run = "./tooling/gamut-fuzz/check-targets.sh" + [tasks.test-dng-real] description = "Validate gamut-dng against real camera DNGs (issue #174; needs fetch-dng-samples)" run = "cargo test --manifest-path tooling/gamut-dng-real-conformance/Cargo.toml" From 9cd010d99ba222bd06f0be476c044fc948b5728d Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 07:19:26 -0400 Subject: [PATCH 15/24] docs(fuzz): reprice the listed checks and answer the job's cadence The README claimed "a check is only listed here if it can fail" while listing two that could not, and the register carried the same rule. Both now say what holds: five earlier entries were unfalsifiable, the two `tiff_decode` ones are dropped rather than repriced, and the rule gains the half that makes it checkable -- each target's module doc names the injection that made its check fire, with the message and the command to reproduce it. The two assertions kept for their pinning value are listed as pins, not as checks. The register also learns the shape that failed here: a comparison whose two sides come from one reader is a tautology, so anchor a decode check on what the decode physically produced rather than on the geometry the probe read. Cadence, answered rather than left open (#594): the job stays on the workflow's trigger. Actions minutes are free for public repositories and the matrix grows parallel runners rather than wall time, so the cost is queue time on a post-merge lane that blocks no pull request; and frequency is the wrong dial while nothing accumulates between runs -- each starts from the committed seeds and discards what the engine finds, so running less often simply searches less. Persisting the corpus is what would change the tier's yield, filed as #603, and the cadence and time budget are worth re-opening after that lands. Changing the trigger would also move the premise of #593, which is a question about the workflow's shape rather than about this tier. Whether "a check can fail" can be mechanically guarded at all is filed as gate without being one, which is worse than prose. Refs #264 --- .github/workflows/extended.yml | 9 +++- docs/testing.md | 24 +++++---- tooling/gamut-fuzz/README.md | 90 ++++++++++++++++++++++++---------- 3 files changed, 86 insertions(+), 37 deletions(-) diff --git a/.github/workflows/extended.yml b/.github/workflows/extended.yml index 3a16fbf8..c25cb557 100644 --- a/.github/workflows/extended.yml +++ b/.github/workflows/extended.yml @@ -174,8 +174,13 @@ jobs: # EXPECTED RED: `tiff_decode` and `dng_decode` fail today on filed, accepted defects # (#563, #564). They are deliberately not narrowed to make these rows green; read the # per-row status rather than this workflow's aggregate until both close. Restoring the - # aggregate's meaning is #593; this job's cadence (nine parallel ten-minute runners on - # every push to the default branch, growing with the matrix) is #594. + # aggregate's meaning is #593. + # + # CADENCE (#594), decided: this job stays on the workflow's own trigger. Nothing + # accumulates between runs -- each starts from the committed seeds and discards what the + # engine finds -- so a run is ten minutes of cold search whatever the cadence, and running + # less often just searches less. Persisting the corpus (#603) is the change that makes + # frequency and this time budget worth tuning; the README states the full argument. run: mise run fuzz ${{ matrix.target }} -- -max_total_time=600 - name: Upload any crash artifacts if: failure() diff --git a/docs/testing.md b/docs/testing.md index 5a8dd540..56340f65 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -171,20 +171,24 @@ deterministic case in that crate's `tests/robustness.rs`**, which is where the r What the per-PR path *does* carry is the **compile** half, for the reason the excluded real-DNG tier already carries it: nothing else builds an excluded crate, so an API change in a driven crate -breaks its targets unnoticed until the next run on master. CI's lint job runs -`cargo check --manifest-path tooling/gamut-fuzz/Cargo.toml --all-targets` — no nightly, no -sanitizer, no engine, nothing unbounded — and its `Format & Metadata` job runs -`tooling/gamut-fuzz/check-targets.sh`, which reconciles the three hand-maintained lists that +breaks its targets unnoticed until the next run on master. CI's lint job runs `mise run +check-fuzz` — no nightly, no sanitizer, no engine, nothing unbounded — and its `Format & Metadata` +job runs `mise run check-fuzz-matrix`, which reconciles the three hand-maintained lists that describe the target set (the files, the `[[bin]]` entries, the `extended.yml` matrix), because a -target missing from the third is one that never runs and nothing reports it. +target missing from the third is one that never runs and nothing reports it. Both are mise tasks +rather than commands written into the workflow, so a contributor runs exactly what CI runs. A **robustness** target is not a law and does not route through an `invariants` module: its primary oracle is the engine's own — a panic, a hang, or an allocation past `-malloc_limit_mb` — -which no function can express. Any check it adds beyond that oracle must be able to *fail*: an -assertion comparing a wrapper against the expression its own body is (`gamut_ifd::read` against -`IfdReader::open(..)?.read_file()`) is a tautology, not a differential, and belongs — if it is -worth pinning at all — in the crate's bounded deterministic suite as a **structure pin**, named as -one. +which no function can express. Any check it adds beyond that oracle must be able to *fail*, and +**its module doc records the injected defect that made it fail** — the patch, the message the +target printed, and the command that reproduces it. An assertion comparing a wrapper against the +expression its own body is (`gamut_ifd::read` against `IfdReader::open(..)?.read_file()`) is a +tautology, not a differential; so is a comparison whose two sides come from one reader, which is +what a decoded-versus-described geometry check reduces to when the decoder and the probe share a +tag reader. Anchor the check on something the compared reader does not produce — the count of +samples the decode physically yielded, against the geometry the file declares — and keep the +tautology, if it is worth keeping at all, as a **structure pin**, named as one at the site. `#[ignore]` is not used in this workspace and must not be introduced: `coverage` is the only test gate, so an ignored test is not deferred, it is unrun. diff --git a/tooling/gamut-fuzz/README.md b/tooling/gamut-fuzz/README.md index 1d68a826..fdf67299 100644 --- a/tooling/gamut-fuzz/README.md +++ b/tooling/gamut-fuzz/README.md @@ -64,9 +64,8 @@ same rule `docs/testing.md` applies to a shrunk `proptest` counterexample, and t - **This crate is workspace-excluded**, so `cargo test --workspace --all-features` never builds it. Nothing on the pull-request path would otherwise compile these targets at all, and an API change in a driven crate would break them unnoticed until the next Extended run. CI's lint job therefore - runs `cargo check --manifest-path tooling/gamut-fuzz/Cargo.toml --all-targets` — build-only, no - nightly, no sanitizer, no engine — exactly as it already does for the excluded real-DNG - conformance tier. + runs `mise run check-fuzz` — build-only, no nightly, no sanitizer, no engine — exactly as it + already does for the excluded real-DNG conformance tier with `mise run check-dng-real`. - **The dependency graph is shared across every target.** A feature turned on for one target's crate is on for all of them, because Cargo resolves features once per crate for the whole package: `bigtiff` was added to `gamut-ifd` for the `ifd_read` driver, and the pre-existing @@ -120,29 +119,46 @@ no crash is still visible: | target | crate | entry points | check beyond the crash oracle | |---|---|---|---| | `ifd_read` | `gamut-ifd` | `read`, `read_tree`, `read_audited` | the dual-ledger audit is complete: no byte read outside a claim, no claim unread | -| `tiff_decode` | `gamut-tiff` | `TiffDecoder::{page_count,info_page,decode_page}` | the page index is bounded by `page_count`; describing and decoding agree on geometry | +| `tiff_decode` | `gamut-tiff` | `TiffDecoder::{page_count,info_page,decode_page}` | a page that decodes yields exactly `width × height × Rgb8::CHANNELS` samples for the geometry the tags declare | | `dng_decode` | `gamut-dng` | `DngDecoder::{decode,verify_new_raw_image_digest}` | the raw image that *arrives* holds exactly `width × height × planes` samples, after every rewriting stage | | `isobmff_boxes` | `gamut-isobmff` | `walk_segments`, `walk_meta_children`, `read`, `BoxReader` | the box cursor strictly advances; the segments tile `0..len` exactly | | `heic_container` | `gamut-heic` | `HeifContainer::parse` | the segments tile `0..len` exactly and every accessor agrees with that tiling | | `heic_hvcc` | `gamut-heic` | `HevcConfig::parse`, `annex_b*`, `validate_still_payload`, `iter_nal_units` | the Annex-B emitters append rather than replace, on the success path and the error path | -**A check is only listed here if it can fail.** Three earlier entries could not. `ifd_read` -compared `read(data)` against `IfdReader::open(data)?.read_file()` — but `reader.rs` *defines* -`read` as that expression, so the two sides were one function call written twice. `heic_hvcc` -compared `annex_b(..).is_ok()` against `annex_b_payload(..).is_ok()` on the same input, and -asserted `annex_b` equals the two calls its own body makes. `dng_decode` compared a digest verdict -against a decoded field that is read with the *same expression* on both sides. None of them had a -reachable failure, and calling any of them a differential overstated what the tier proves. - -They are not all deleted — they are **relabelled and repriced**. A claim about two bodies agreeing -is a **structure pin**: worth keeping where it is free or where a future change could genuinely -split the bodies apart, worth nothing as a search. So `heic_hvcc` still asserts the two halves, -folded into the append check's existing buffer at no extra emitter pass; `dng_decode` still -compares the verdict, on a call it makes anyway for the crash oracle; and the `gamut-ifd` wrapper -pin lives in `crates/gamut-ifd/tests/robustness.rs`, over a bounded exhaustive corpus, rather than -costing half of every one of this target's twenty thousand executions per second to search for a -counterexample that does not exist. Dropping the two duplicate parses raised `ifd_read` from -roughly 12 000 exec/s to roughly 20 000. +**A check is only listed here if it can fail — and each target's module doc names the injected +defect that made it fail**, with the message it produced and the command that reproduces it. That +second half is the part a reader can re-run; without it "this check is live" is a reading of the +code, which is exactly what put the rows below wrong twice. + +Five earlier entries could not fail. `ifd_read` compared `read(data)` against +`IfdReader::open(data)?.read_file()` — but `reader.rs` *defines* `read` as that expression, so the +two sides were one function call written twice. `heic_hvcc` compared `annex_b(..).is_ok()` against +`annex_b_payload(..).is_ok()` on the same input, and asserted `annex_b` equals the two calls its +own body makes. `dng_decode` compared a digest verdict against a decoded field that is read with +the *same expression* on both sides. `tiff_decode` claimed two: that `info_page` refuses the index +`page_count` returns — which reduces to indexing a vector one past its own length, both sides +being `read(data)?.ifds` — and that the described and decoded geometry agree, which sees only the +few lines copying one into the other, because `decode_page_samples` says outright that +"everything the page *declares* comes from one shared reader". None of them had a reachable +failure, and calling any of them a differential overstated what the tier proves. + +The two `tiff_decode` claims are **dropped**, and the target is re-anchored on something the +geometry reader does not produce: the number of samples the decode physically yielded, against the +declared dimensions and the channel count of the layout asked for. The rest are **relabelled and +repriced**. A claim about two bodies agreeing is a **structure pin**: worth keeping where it is +free or where a future change could genuinely split the bodies apart, worth nothing as a search. +So `heic_hvcc` still asserts the two halves, folded into the append check's existing buffer at no +extra emitter pass; `dng_decode` still compares the verdict, on a call it makes anyway for the +crash oracle; and the `gamut-ifd` wrapper pin lives in `crates/gamut-ifd/tests/robustness.rs`, +over a bounded exhaustive corpus, rather than costing half of every one of this target's twenty +thousand executions per second to search for a counterexample that does not exist. Dropping the +two duplicate parses raised `ifd_read` from roughly 12 000 exec/s to roughly 20 000. + +Two smaller assertions are pins for the same reason and are labelled as such at the site, so +nobody reads them as checks: `tiff_decode`'s "a page that decodes must also describe" (decoding +calls the tag reader before it reads a pixel) and `heic_hvcc`'s "no empty NAL unit" +(`NalUnitIter::next` errors on a zero length before it can yield one). Both cost one comparison on +a value already in hand. An **allocation** defect needs the engine's malloc hook to be visible at all: an oversized `Vec::with_capacity` costs no resident memory on an overcommitting kernel, so measuring RSS finds @@ -165,20 +181,44 @@ per push learns nothing from it. Read the per-row status, not the aggregate, unt close; both rows go green with no change here. Whether these two rows should instead live in a separate, expected-to-fail lane so the aggregate keeps its meaning is [#593](https://github.com/visualcommons/gamut/issues/593) — a workflow-topology question, not a -fuzzing one. The job's cadence, which was inherited rather than chosen and now costs nine parallel -ten-minute runners per push, is [#594](https://github.com/visualcommons/gamut/issues/594). +fuzzing one. + +### The cadence, answered + +Nine parallel ten-minute runners on every push to the default branch, growing by one runner per +target, was inherited from the workflow's trigger rather than chosen ([#594](https://github.com/visualcommons/gamut/issues/594)). +It is **kept**, and here is why, so the next target added does not reopen it: + +- The cost is queue time, not budget — Actions minutes are free for public repositories — and the + job is post-merge with `fail-fast: false`, so it blocks no pull request. The matrix grows the + number of *parallel* runners, not the job's wall time. +- Frequency is the wrong dial, because **nothing accumulates between runs**. Each run starts from + the committed seeds and discards what the engine finds, so a run's yield is ten minutes of cold + search whatever the cadence: running less often searches strictly less, and running more often + re-derives the same shallow space. What would change the tier's yield is persisting the corpus, + filed as [#603](https://github.com/visualcommons/gamut/issues/603) — and the cadence and the + `-max_total_time` budget are both worth re-opening *after* that lands, not before. +- Changing the trigger would also move #593's premise (whether a per-push aggregate is red), which + is a decision about the workflow's shape rather than about this tier. ## Keeping the three lists in step A target exists in three hand-maintained places: its `fuzz_targets/.rs` file, its `[[bin]]` entry in `Cargo.toml`, and its row in `extended.yml`'s fuzz matrix. Miss the third and the target is written, committed, and never run — silently, because nothing fails. `check-targets.sh` -reconciles all three and is wired into CI's `Format & Metadata` job; run it directly too: +reconciles all three and is wired into CI's `Format & Metadata` job as `mise run +check-fuzz-matrix`; run the same thing locally: ```bash -./tooling/gamut-fuzz/check-targets.sh +mise run check-fuzz-matrix # the three lists describe the same target set +mise run check-fuzz # every target still compiles against the crates it drives ``` +Both are tasks rather than bare commands so a contributor runs *what CI runs*: a step whose command +lives only in a workflow is a step nobody can reproduce without reading YAML, and the two copies +drift. `check-targets.sh` also fails a `[[bin]]` whose `name` disagrees with its own `path`, and +names a duplicated entry as a duplicate rather than mis-reporting it as a missing file. + ## Seeds `corpus//` holds a small **curated seed set**, tracked despite `.gitignore` listing From 1f7a9f23e7f23966637e27f7b0f9297c56e23518 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 08:32:18 -0400 Subject: [PATCH 16/24] test(fuzz): check the geometry that arrives, not the count derived from it `convert_from_raw` allocates its output as `ImageBuf::::zeroed(src.dims)`, so the returned sample count is the dimensions' own product for every input. Asserting it against the declared geometry is that comparison times a constant on both sides: a stage that transposes the geometry it hands on passes it, and the target's module doc claimed the opposite. Restore the dimension pair beside it as the live check, name the sample count as the structure pin on `ImageBuf`'s constructor that it is, and record the transposition injection that separates them. Also drop the early-return branch driving `info`/`decode_page` after `page_count` failed: all three begin with the same `read`, so the two extra calls fail at the byte it already failed at. --- .../corpus/ifd_read/padding-unread-claim.tif | Bin 0 -> 22 bytes .../gamut-fuzz/fuzz_targets/tiff_decode.rs | 107 +++++++++++------- 2 files changed, 67 insertions(+), 40 deletions(-) create mode 100644 tooling/gamut-fuzz/corpus/ifd_read/padding-unread-claim.tif diff --git a/tooling/gamut-fuzz/corpus/ifd_read/padding-unread-claim.tif b/tooling/gamut-fuzz/corpus/ifd_read/padding-unread-claim.tif new file mode 100644 index 0000000000000000000000000000000000000000..3284a126da86754ca135006f100a0342002735b0 GIT binary patch literal 22 OcmebD)M5}|KnDO2!2rzw literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs b/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs index 30382590..7ddb06dc 100644 --- a/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs +++ b/tooling/gamut-fuzz/fuzz_targets/tiff_decode.rs @@ -5,46 +5,61 @@ //! cap, so a hostile file must end in a typed error rather than a panic, a hang or a runaway //! allocation — the three things libFuzzer itself detects. //! -//! The check beyond the crash oracle is that **the decoded volume matches the declared geometry**: -//! a page that decodes yields exactly `width × height × Rgb8::CHANNELS` samples, where the width -//! and height are the ones `info_page` read out of the tags. The sample count is not read from the -//! geometry reader at all — it is what the strip/tile assembly, the predictor pass and the -//! photometric unpack physically produced — so the check reaches that whole pipeline rather than -//! the handful of lines that copy a described number into a decoded one. +//! ## The check beyond the crash oracle +//! +//! **The geometry that arrives equals the geometry the tags declare.** `decode_page_samples` reads +//! the page's dimensions once, runs the strip/tile assembly, the predictor pass and the photometric +//! unpack, and only then builds the `DecodedImage` those dimensions travel out in; the buffer a +//! caller receives carries whatever that stage, `RawImage::new` and `convert_from_raw` between them +//! made of it. A stage that rewrites the geometry it hands on — a crop, an orientation, a tile-grid +//! rounding — is accepted by every constructor on the way, produces no crash, and is contradicted +//! only by the tags. //! //! Injection that proved it fires (re-runnable): at the point `decode_page_samples` builds its -//! `DecodedImage`, trim the last row from the samples *and* report `height - 1` — a crop stage -//! that describes what it cropped. It is internally consistent, so `RawImage::new` and -//! `ImageBuf::new` both accept it and nothing crashes; only the declared geometry contradicts it. -//! Run as `run.sh tiff_decode -- -runs=0`, the committed seeds alone report it: -//! *"page 0: decoded 54 samples for the 6 × 4 × 3 the tags declare"*. +//! `DecodedImage`, swap `width` and `height` — an orientation stage, volume-preserving, so nothing +//! downstream refuses it. The committed seeds alone report it, with no search: +//! `run.sh tiff_decode -- -runs=0` gives +//! *"page 0: decoded 4 × 6 for the 6 × 4 the tags declare"*. +//! +//! ## The sample count beside it is a structure pin, not a second check +//! +//! An earlier revision anchored this target on the number of samples the decode physically yielded +//! and claimed that count "is not read from the geometry reader at all". **That is false.** +//! `convert_from_raw` allocates its output as `ImageBuf::::zeroed(src.dims)`, and +//! `ImageBuf::zeroed` sizes that allocation with `expected_len::

(dims)` — so the returned +//! `as_samples().len()` is `width × height × CHANNELS` of the *dimensions*, by construction, for +//! every input. Asserting it against `info.width × info.height × CHANNELS` is therefore the +//! geometry comparison above multiplied by a constant on both sides: it can separate the two sides +//! only if `ImageBuf`'s own length-versus-dimensions invariant breaks, never if the decode pipeline +//! miscounts. +//! +//! Measured, not reasoned: the transposition above gives **exit 0 and no report** under the sample +//! count alone, because `w·h·3 == w·h·3` after a transposition for every input, while the geometry +//! comparison fires on the committed seeds immediately. //! -//! Two things this deliberately does **not** assert, both because they cannot fail by input: +//! The sample count is kept — one comparison on values already in hand — and **named a pin at the +//! site**: it pins `ImageBuf`'s constructor continuing to derive its length from its dimensions. +//! (`ImageBuf::new`, which does validate a caller-supplied buffer against dimensions, is never +//! called on this path; `RawImage::new` is the only gate the decoded samples pass through, and it +//! runs before the output buffer exists.) //! -//! - **`info_page` at `page_count` is refused.** `page_count` is `read(data)?.ifds.len()` and -//! `info_page` is `read(data)?.ifds.get(page)`, so the claim reduces to indexing a vector one -//! past its own length. Injecting the defect it advertised — a count that over-reports the -//! chain — produces no report, because the over-report moves both sides together. -//! - **decoded dimensions equal described dimensions.** The two sides are one reader: -//! `decode_page_samples` states outright that "everything the page *declares* comes from one -//! shared reader", so a transposition inside `info::page_info` hands every caller a transposed -//! image and this comparison stays quiet. It fires only for a defect in the few lines between -//! that reader and the returned buffer, which is a reach the sample count already covers. +//! A second assertion is a pin for the same reason and labelled as one at the site: **"a page that +//! decodes must also describe"**. `decode_page_samples` calls `info::page_info` before it reads a +//! pixel, so a page `info_page` refuses cannot decode, for any input, while that body stands. It is +//! the shape that would report if `decode` ever grew its own tag reader. //! -//! One assertion beside it is kept and **named a structure pin** rather than advertised as a -//! check: "a page that decodes must also describe". `decode_page_samples` calls `info::page_info` -//! before it reads a pixel, so a page `info_page` refuses cannot decode, for any input, while that -//! body stands. It costs nothing — both calls are made anyway for the crash oracle — and it is the -//! shape that would report if `decode` ever grew its own tag reader. +//! ## What is deliberately not asserted //! -//! What the sample count does not see either, stated so nobody over-reads it: a defect that -//! produces the wrong *volume* while leaving the dimensions alone is turned into a typed error by -//! `RawImage::new`/`ImageBuf::new` before it can reach a caller, so it arrives here as a rejected -//! file rather than as a report. Measured, not assumed: decoding `info.height + 1` rows — the -//! first injection tried — produced **no** report, because the strip assembly runs out of bytes -//! and the page is refused. The live class is a stage that rewrites the geometry it hands on — a -//! crop, an orientation, a tile-grid rounding — which both constructors accept and only the -//! declared geometry contradicts. That is the same class the sibling `dng_decode` target checks, +//! **`info_page` at `page_count` is refused.** `page_count` is `read(data)?.ifds.len()` and +//! `info_page` is `read(data)?.ifds.get(page)`, so the claim reduces to indexing a vector one past +//! its own length. Injecting the defect it advertised — a count that over-reports the chain — +//! produces no report, because the over-report moves both sides together. +//! +//! A defect that produces the wrong *volume* while leaving the dimensions alone does not arrive +//! here as a report either: `RawImage::new` turns it into a typed error before a caller sees it, so +//! the file is simply rejected. Measured — decoding `info.height + 1` rows produced **no** report, +//! because the strip assembly runs out of bytes and the page is refused. The live class is the +//! geometry-rewriting stage above, which is also the class the sibling `dng_decode` target checks, //! where linearisation and active-area handling are such stages today. //! //! The policy is [`ConvertPolicy::permissive`] so the decode reaches the pixel and conversion @@ -70,11 +85,10 @@ const MAX_PAGES: usize = 4; fuzz_target!(|data: &[u8]| { let decoder = TiffDecoder::new().convert_policy(ConvertPolicy::permissive()); + // A file whose chain does not parse is simply refused. The other entry points are not driven + // for it: `page_count` is `read(data)?.ifds.len()`, and `info`/`decode_page` both begin with + // that same `read`, so they fail at the byte it already failed at and reach nothing new. let Ok(pages) = decoder.page_count(data) else { - // A file whose chain does not parse must still be refused — not crash — by the entry - // points that do not consult `page_count` first. - let _ = decoder.info(data); - let _ = decoder.decode_page(data, 0); return; }; @@ -83,8 +97,21 @@ fuzz_target!(|data: &[u8]| { let image = decoder.decode_page(data, page); match (&info, &image) { (Ok(info), Ok(image)) => { - // The declared geometry, times the channel count of the layout that was asked for, - // against the samples the decode actually produced. + // The live check: the geometry that arrives, against the geometry the tags + // declare. Every stage between the tag reader and this buffer could rewrite it. + let decoded = image.dimensions(); + assert_eq!( + (decoded.width, decoded.height), + (info.width, info.height), + "page {page}: decoded {} × {} for the {} × {} the tags declare", + decoded.width, + decoded.height, + info.width, + info.height + ); + // Structure pin, not a check: `ImageBuf` sizes its storage from its own + // dimensions, so for every input this is the assertion above times + // `Rgb8::CHANNELS` on both sides (see the module docs). let declared = (info.width as usize) .checked_mul(info.height as usize) .and_then(|n| n.checked_mul(::CHANNELS)); From b2d25165ade0f3a808b625e20616714d4338b528 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 08:32:18 -0400 Subject: [PATCH 17/24] test(fuzz): name the box-cursor and empty-segment assertions as pins `next_box` reads the 4-byte size and 4-byte type through `take` before any success return, so `position()` has grown by 8 before a `RawBox` exists and no hostile box size can fail "the cursor strictly advances". Every segment shape `walk_segments` pushes is non-empty for the same reason. Both are structure pins, not checks beyond the crash oracle, and are labelled as such. Record an injection for the tiling's end-of-file half, which had none. --- .../gamut-fuzz/fuzz_targets/isobmff_boxes.rs | 68 +++++++++++++------ 1 file changed, 47 insertions(+), 21 deletions(-) diff --git a/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs b/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs index 6c22386b..f3234a61 100644 --- a/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs +++ b/tooling/gamut-fuzz/fuzz_targets/isobmff_boxes.rs @@ -2,25 +2,49 @@ //! //! `docs/testing.md`'s per-crate table names `read` as this crate's untrusted-input surface. An //! ISOBMFF file is a tree of length-prefixed boxes whose every length the file chose, so the -//! `#![forbid(unsafe_code)]` reader must end in a typed error rather than a panic, a hang (a box -//! whose declared size does not advance the cursor) or an allocation sized from a declared count. +//! `#![forbid(unsafe_code)]` reader must end in a typed error rather than a panic, a hang or an +//! allocation sized from a declared count. //! -//! Beyond the crash oracle it checks the crate's own **byte-accounting totality**: when -//! `walk_segments` succeeds, its segments tile `0..len` exactly — starting at 0, contiguous, -//! non-overlapping, none empty, the last ending at end of file. That is the guarantee -//! `tests/accounting.rs` pins over hand-built files; here it is asked of whatever the engine -//! produces, which is where a size-0 or a wrapping box length would show up as a hole or an +//! ## The check beyond the crash oracle +//! +//! The crate's own **byte-accounting totality**: when `walk_segments` succeeds, its segments tile +//! `0..len` exactly — starting at 0, contiguous, non-overlapping, the last ending at end of file. +//! That is the guarantee `tests/accounting.rs` pins over hand-built files; here it is asked of +//! whatever the engine produces, which is where a wrapping box length would show up as a hole or an //! overlap rather than as a crash. //! -//! Injection that proved the tiling check fires (re-runnable): in `walk_segments`, record a box's -//! segment as `b.offset + 8..end` — byte accounting that counts box bodies and forgets their -//! headers. The committed seed alone reports it, with no search: -//! `run.sh isobmff_boxes -- -runs=0` gives *"segment 8..24 leaves a gap or overlaps at -//! 0"*. +//! Both halves of the tiling are checked separately, and each has its own injection. Both are +//! re-runnable as `run.sh isobmff_boxes -- -runs=0` and both are reported by the committed +//! seed alone, with no search: +//! +//! - **contiguity.** In `walk_segments`, record a box's segment as `b.offset + 8..end` — byte +//! accounting that counts box bodies and forgets their headers. Gives *"segment 8..24 leaves a +//! gap or overlaps at 0"*. +//! - **coverage to end of file.** In `walk_segments`, drop the last segment (`segments.pop()` +//! before the return) — accounting that stops one box short. Gives *"coverage does not run to +//! end of file"*. +//! +//! ## Two assertions here are structure pins, not checks +//! +//! Both are labelled at the site and neither is listed in the README's "check beyond the crash +//! oracle" column, because **no file can fail them**: +//! +//! - **the box cursor strictly advances.** The defect this names is a box whose declared size does +//! not move the cursor — and `next_box` cannot express it: it reads the 4-byte size and the +//! 4-byte type through `take` *before* any success return, so `position()` has already grown by +//! 8 by the time a `RawBox` exists. Measured, not reasoned: with the `size < header_size` guard +//! removed and the body length taken as `size.saturating_sub(header_size)` — the exact defect +//! the assertion advertises — the committed seed reports nothing, and neither does a bounded +//! search. The control fires immediately: rewinding `self.pos` to the box offset before the +//! `Ok(Some(..))` return reports *"BoxReader did not advance past 0"*. So what the assertion +//! really pins is that unconditional 8-byte header read staying where it is. +//! - **no segment is empty.** `walk_segments` pushes exactly three shapes, and each is non-empty +//! for the same reason: a `Box` segment is `b.offset..reader.position()`, which the 8-byte header +//! read above has already widened; an `AppendedStream` is `b.offset..data.len()` for a box that +//! was read, so `data.len() > b.offset`; and a `Trailer` is `box_start..data.len()` on an `Err`, +//! which `next_box` only returns when at least one byte remained. //! -//! `BoxReader` is driven separately from `walk_segments` because it is the lower layer and a -//! caller may use it directly: the check there is that the cursor advances strictly, so a walk of -//! a hostile file cannot spin. +//! Both cost one comparison inside a loop the target walks anyway. //! //! A crash found here is **minimised and promoted into a named deterministic case** in //! `gamut-isobmff`'s own suite. The corpus is a search aid, not the regression record. @@ -31,8 +55,9 @@ use gamut_isobmff::{BoxReader, read, walk_meta_children, walk_segments}; use libfuzzer_sys::fuzz_target; fuzz_target!(|data: &[u8]| { - // The raw box layer: every successful step must consume at least one byte, or a walk of a - // hostile file never terminates. + // The raw box layer is driven separately from `walk_segments` because it is the lower layer + // and a caller may use it directly. The two assertions in this loop are structure pins on + // `next_box`'s unconditional 8-byte header read, not checks (see the module docs). let mut reader = BoxReader::new(data); let mut position = reader.position(); while let Ok(Some(_)) = reader.next_box() { @@ -46,10 +71,9 @@ fuzz_target!(|data: &[u8]| { position = next; } - // The segment walk: byte-accounting totality. Walking a cursor rather than asserting the - // four properties separately states the whole claim once — start at 0, contiguous, - // non-overlapping, no empty segment, ending at end of file — and it is the form that stays - // correct for a zero-length input, where an empty segment list already tiles `0..0`. + // The segment walk: byte-accounting totality. Walking a cursor states the whole claim once — + // start at 0, contiguous, non-overlapping, ending at end of file — and it is the form that + // stays correct for a zero-length input, where an empty segment list already tiles `0..0`. if let Ok((segments, meta_body)) = walk_segments(data) { let mut cursor = 0usize; for segment in &segments { @@ -58,6 +82,8 @@ fuzz_target!(|data: &[u8]| { "segment {:?} leaves a gap or overlaps at {cursor}", segment.range ); + // Structure pin, not a check: every segment `walk_segments` can push is non-empty by + // construction (see the module docs). assert!( segment.range.end > segment.range.start, "empty segment {:?}", From 52be3fe039a0bdc9c108145464a47f1c931b2586 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 08:32:18 -0400 Subject: [PATCH 18/24] test(fuzz): replace heic_container's duplicated tiling with the promised containment `HeifContainer::parse` stores `gamut_isobmff::walk_segments` verbatim and `segments()` returns it unchanged, so the tiling check here searched the same function the sibling `isobmff_boxes` target searches, over a narrower input set: the same injection produced the identical message in both. Drop it, and make the check the module doc already promised but the code never asserted -- every borrowed slice the accessors hand out lies inside `data()`, which is what makes the crate zero-copy. --- .../gamut-fuzz/fuzz_targets/heic_container.rs | 124 +++++++++++------- 1 file changed, 75 insertions(+), 49 deletions(-) diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs index ad59ae60..b19187ca 100644 --- a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs +++ b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs @@ -3,33 +3,49 @@ //! //! `gamut-heic` is decode-only and its stated product guarantee is a **full-fidelity byte //! accounting**: every input byte maps to a box, to an appended motion-photo stream, or to an -//! explicit trailer. That is a claim the engine can be pointed at directly, and it is stronger -//! than "did not crash": a container that silently drops a region still parses. +//! explicit trailer. Real files reach that path — phones append a whole second MP4 after the HEIC, +//! and camera apps leave trailers — so it is not an exotic branch. //! -//! So this target checks, on every successful parse: +//! ## The checks beyond the crash oracle //! -//! - the segments tile `0..len` exactly — start at 0, contiguous, non-overlapping, none empty, -//! last ending at end of file; -//! - the accessors are consistent with that tiling: `appended_stream` and `trailer` are `Some` -//! exactly when a segment of that kind exists, and `boxes()` yields one entry per `Box` -//! segment; -//! - every borrowed slice the accessors hand out is a subslice of the input the container was -//! given, which is what `data()` promises. +//! - **the accessors report exactly what the segment list holds**: `appended_stream` and `trailer` +//! are `Some` exactly when a segment of that kind exists, and `boxes()` yields one entry per +//! `Box` segment; +//! - **`data()` is the caller's own buffer, and every borrowed slice the accessors hand out lies +//! inside it** — checked as a pointer-range containment over `boxes()`, `appended_stream()`, +//! `trailer()` and `unknown_meta_boxes()`. This is the promise that makes the crate zero-copy: +//! an accessor that normalised or re-allocated on the way out would satisfy every count above +//! and still break it. //! -//! Real files reach here: phones append a whole second MP4 after the HEIC, and camera apps leave -//! trailers, so the accounting path is not an exotic branch. +//! Injections that proved each assertion fires (re-runnable), all reported by the committed seed +//! alone with no search, as `run.sh heic_container -- -runs=0`: //! -//! Injection that proved the accessor check fires (re-runnable): make `HeifContainer::boxes()` -//! skip the `ftyp` box — an accessor that filters what the segments hold. The committed seed alone -//! reports it, with no search: `run.sh heic_container -- -runs=0` gives *"boxes() -//! disagrees with the Box segments"*. +//! | injection in `gamut-heic` | message | +//! |---|---| +//! | `boxes()` skips the `ftyp` box | *"boxes() disagrees with the Box segments"* | +//! | `appended_stream()` returns `None` unconditionally | *"appended\_stream() disagrees with the AppendedStream segments"* | +//! | `trailer()` returns `Some(self.data)` unconditionally | *"trailer() disagrees with the Trailer segments"* | +//! | `data()` returns a leaked copy of the input rather than the input | *"left == right" on the `data()` pointer* | +//! | `boxes()` yields a leaked copy of each body rather than the borrowed body | *"a borrowed slice is not inside data()"* | //! -//! Its **reach is one function per accessor**, and that is a limitation rather than a flaw: +//! The accessors' **reach is one function each**, and that is a limitation rather than a flaw: //! `boxes`, `appended_stream` and `trailer` are each a three-line `filter_map`/`find_map` over the -//! segment list, so the check sees a defect in those and nothing deeper. It is kept at that size -//! because the accessors are the API every caller actually uses — the segment list is the -//! evidence, the accessors are the product — and because it costs one pass over a list the target -//! walks anyway. The tiling check above is the one with the deep reach: it sees the whole parse. +//! segment list. They are checked because they are the API every caller actually uses — the segment +//! list is the evidence, the accessors are the product — and because they cost one pass over a list +//! the target walks anyway. +//! +//! ## The segment tiling is deliberately not checked here +//! +//! `HeifContainer::parse` stores `gamut_isobmff::walk_segments(data)?` verbatim and `segments()` +//! returns it unchanged, so a tiling check here searches **the same function** the sibling +//! `isobmff_boxes` target already searches — and over a strictly narrower input set, since it is +//! reached only for files `gamut_isobmff::read` also accepted. Measured, not reasoned: the +//! `b.offset + 8..end` injection recorded in `isobmff_boxes` produced the *identical* message here, +//! *"segment 8..24 leaves a gap or overlaps at 0"*, because it is the identical assertion over the +//! identical values. Two ten-minute runners searching one function is a real cost at a +//! proven-zero marginal yield, so this target keeps only what is genuinely its own. The tiling +//! guarantee itself is unaffected: `isobmff_boxes` searches it, and `gamut-heic`'s own +//! `tests/accounting.rs` pins it over hand-built files. //! //! A crash found here is **minimised and promoted into a named deterministic case** in //! `gamut-heic`'s own suite. The corpus is a search aid, not the regression record. @@ -39,35 +55,24 @@ use gamut_heic::{HeifContainer, SegmentKind}; use libfuzzer_sys::fuzz_target; +/// Whether `part` is a subslice of `whole`, by pointer range. +/// +/// Comparing raw pointers needs no `unsafe`, and a zero-length borrow at end of input still +/// satisfies `start >= whole.start && end <= whole.end`. +fn is_inside(part: &[u8], whole: &[u8]) -> bool { + let (part, whole) = (part.as_ptr_range(), whole.as_ptr_range()); + part.start >= whole.start && part.end <= whole.end +} + fuzz_target!(|data: &[u8]| { let Ok(container) = HeifContainer::parse(data) else { return; }; - // Walking a cursor states the whole tiling claim once — start at 0, contiguous, - // non-overlapping, no empty segment, ending at end of file — and stays correct for a - // zero-length input, where an empty segment list already tiles `0..0`. - let segments = container.segments(); - let mut cursor = 0usize; - for segment in segments { - assert_eq!( - segment.range.start, cursor, - "segment {:?} leaves a gap or overlaps at {cursor}", - segment.range - ); - assert!( - segment.range.end > segment.range.start, - "empty segment {:?}", - segment.range - ); - cursor = segment.range.end; - } - assert_eq!(cursor, data.len(), "coverage does not run to end of file"); - - // The accessors report exactly what the tiling holds. - let boxes = container.boxes().count(); + // The accessors report exactly what the segment list holds. + let boxes: Vec<_> = container.boxes().collect(); let mut kinds = (0usize, 0usize, 0usize); - for segment in segments { + for segment in container.segments() { match segment.kind { SegmentKind::Box { .. } => kinds.0 += 1, SegmentKind::AppendedStream(_) => kinds.1 += 1, @@ -75,7 +80,11 @@ fuzz_target!(|data: &[u8]| { _ => {} } } - assert_eq!(boxes, kinds.0, "boxes() disagrees with the Box segments"); + assert_eq!( + boxes.len(), + kinds.0, + "boxes() disagrees with the Box segments" + ); assert_eq!( container.appended_stream().is_some(), kinds.1 > 0, @@ -87,13 +96,30 @@ fuzz_target!(|data: &[u8]| { "trailer() disagrees with the Trailer segments" ); - // `data()` returns the input, and every borrowed region lies inside it. - assert_eq!(container.data().as_ptr(), data.as_ptr()); - assert_eq!(container.data().len(), data.len()); + // `data()` is the caller's buffer, and every borrowed region the accessors hand out lies + // inside it. + let whole = container.data(); + assert_eq!(whole.as_ptr(), data.as_ptr(), "data() is not the input"); + assert_eq!(whole.len(), data.len(), "data() is not the whole input"); + let borrowed = boxes + .iter() + .map(|(_, body)| *body) + .chain(container.appended_stream()) + .chain(container.trailer()) + .chain(container.unknown_meta_boxes().iter().map(|b| b.body)); + for part in borrowed { + assert!( + is_inside(part, whole), + "a borrowed slice is not inside data(): {} bytes at {:p}, data() is {} bytes at {:p}", + part.len(), + part.as_ptr(), + whole.len(), + whole.as_ptr() + ); + } // The item model and the unknown-box ledger are built on the same walk; drive them so a // defect there is reachable too. let _ = container.image().items().count(); let _ = container.image().primary_item().id(); - let _ = container.unknown_meta_boxes().len(); }); From 1ddd498c589fa603516a65af107813278a91d371 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 08:32:28 -0400 Subject: [PATCH 19/24] test(fuzz): seed the unread-claim half of the dual-ledger audit Every #264 seed puts IFD0 at offset 8, so byte 8 is read as the entry count and an over-claimed header lands on a byte the ledger already holds: the "no claim unread" half of the audit had no witness in the committed corpus and was left to the engine to synthesise. A 22-byte TIFF pointing IFD0 at offset 16 leaves 8..16 as internal padding nothing reads, and turns the over-claim into a report. --- tooling/gamut-fuzz/fuzz_targets/ifd_read.rs | 30 ++++++++++++++++----- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs b/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs index 1f14a155..15f033f1 100644 --- a/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs +++ b/tooling/gamut-fuzz/fuzz_targets/ifd_read.rs @@ -17,11 +17,26 @@ //! never declares — or declares bytes it never touched — produces no crash at all, and this is //! what sees it. //! -//! Injection that proved the audit fires (re-runnable): in `IfdReader::read_chain`, claim the -//! header as `header_size() - 1` bytes — an off-by-one that leaves a byte the parser physically -//! reads outside every structural claim. The committed seeds alone report it, with no search: -//! `run.sh ifd_read -- -runs=0` gives *"parser read bytes it never claimed"* carrying -//! `unclaimed_reads: [Range { start: 7, len: 1 }]`. +//! The audit has **two halves and each is checked separately**, because a single injection +//! satisfies only one of them — which is how the second half went three rounds without a falsifier. +//! Both injections are in `IfdReader::read_chain`'s header claim and both are re-runnable as +//! `run.sh ifd_read -- -runs=0`, reported by the committed seeds alone with no search: +//! +//! - **no byte read outside a claim.** Claim the header as `header_size() - 1` bytes — an +//! off-by-one that leaves a byte the parser physically reads outside every structural claim. +//! Reports *"parser read bytes it never claimed"* carrying +//! `unclaimed_reads: [Range { start: 7, len: 1 }]`. +//! - **no claim unread.** Claim the header as `header_size() + 1` bytes — an over-claim that +//! declares a byte the parser never touches. Reports *"parser claimed bytes it never read"* +//! carrying `unread_claims: [Segment { range: Range { start: 0, len: 9 }, kind: Header }]`. +//! +//! That second one is only reachable because of one seed. In every file whose IFD0 sits at the +//! usual offset 8, byte 8 *is* read — it is the entry count — so an over-claim of one byte lands +//! on a byte the ledger already holds and the check stays quiet; measured, the #264 cases alone +//! give exit 0 under it. `corpus/ifd_read/padding-unread-claim.tif` is a 22-byte TIFF whose header +//! points IFD0 at offset 16, leaving `8..16` as internal padding that nothing reads, and it is what +//! turns the over-claim into a report. A check whose only witness must be synthesised by the engine +//! is a check the tier is asking luck for. //! //! ## What this target deliberately does *not* check //! @@ -52,8 +67,9 @@ use libfuzzer_sys::fuzz_target; const POINTER_TAGS: &[u16] = &[330, 34665, 34853]; fuzz_target!(|data: &[u8]| { - // The byte audit: the live check. Only meaningful on a parse that succeeded, because an - // abandoned parse has no complete claim set to reconcile against. + // The byte audit: the live check, in two halves that fail for different defects. Only + // meaningful on a parse that succeeded, because an abandoned parse has no complete claim set + // to reconcile against. if let Ok((_, report)) = read_audited(data) { assert!( report.unclaimed_reads.is_empty(), From 07c4f20c85bb379fa54075d17196af329be4777d Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 08:42:40 -0400 Subject: [PATCH 20/24] test(fuzz): seed the appended-stream and trailer sides of the accessor check `heic-single-item.heic` carries neither, so an accessor returning `None` unconditionally agreed with the segment list and reported nothing: one of the two directions of each equality had no witness in the corpus. A HEIC with a second top-level `ftyp` and one with a truncated trailing box header close both. --- .../heic_container/appended-stream.heic | Bin 0 -> 304 bytes .../corpus/heic_container/trailer.heic | Bin 0 -> 276 bytes .../heic_hvcc/truncated-payload-nal.bin | Bin 0 -> 57 bytes .../gamut-fuzz/fuzz_targets/heic_container.rs | 20 +++++++++++++----- 4 files changed, 15 insertions(+), 5 deletions(-) create mode 100644 tooling/gamut-fuzz/corpus/heic_container/appended-stream.heic create mode 100644 tooling/gamut-fuzz/corpus/heic_container/trailer.heic create mode 100644 tooling/gamut-fuzz/corpus/heic_hvcc/truncated-payload-nal.bin diff --git a/tooling/gamut-fuzz/corpus/heic_container/appended-stream.heic b/tooling/gamut-fuzz/corpus/heic_container/appended-stream.heic new file mode 100644 index 0000000000000000000000000000000000000000..f08c19a653095673ee3c83eb57c55572649ea862 GIT binary patch literal 304 zcmZQzV30^FsVvAy%}izh0uY^>nPv!NzR683Nd$=jfnr8VP7#F3z)+BxTmoamXug8X zl3Xx{5lG5q=H!Eob75d$1VRwWz{mrnS%6qMGczv@NP|TgM1h=CCMYkXEE!}q(29bh z0w5inS&*C$q|Jd6&Wy|nK;{G>W?+c=!0_kqj~@&S{LBl0hBG@bu38A9ofvm70?{sv zXMn0iGK&jR!Ri?p9DuX|5Q}COvG); zP<$>zHOU5jAhOEHvWkF7>Q!03i`{9SFdIY8#l&lJ%*IstOUff1SiJE0zTZIGKXJJq ss_WTtqHY(*nYweTBU_V}^FT-r>46v>RW<(2JGk3TokaF}yFaM?9|IvTrT_o{ literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/corpus/heic_hvcc/truncated-payload-nal.bin b/tooling/gamut-fuzz/corpus/heic_hvcc/truncated-payload-nal.bin new file mode 100644 index 0000000000000000000000000000000000000000..37ec02e98de3f8505f6aaef6cb25aaa3eaf4e2a7 GIT binary patch literal 57 zcmZS3XJk%bU|^U4#0(5k9~l1p{qci=fuDH+10w^o1LLZNAliv>_aYGO!gvO#idBPg GJp%x3j0`UT literal 0 HcmV?d00001 diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs index b19187ca..79c04aae 100644 --- a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs +++ b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs @@ -17,16 +17,26 @@ //! an accessor that normalised or re-allocated on the way out would satisfy every count above //! and still break it. //! -//! Injections that proved each assertion fires (re-runnable), all reported by the committed seed -//! alone with no search, as `run.sh heic_container -- -runs=0`: +//! Injections that proved each assertion fires (re-runnable), all reported by the committed seeds +//! alone with no search, as `run.sh heic_container -- -runs=0`. Each `is_some()` equality +//! is injected in **both** directions, because one direction is silent on a file that has no +//! segment of that kind: //! //! | injection in `gamut-heic` | message | //! |---|---| //! | `boxes()` skips the `ftyp` box | *"boxes() disagrees with the Box segments"* | //! | `appended_stream()` returns `None` unconditionally | *"appended\_stream() disagrees with the AppendedStream segments"* | -//! | `trailer()` returns `Some(self.data)` unconditionally | *"trailer() disagrees with the Trailer segments"* | -//! | `data()` returns a leaked copy of the input rather than the input | *"left == right" on the `data()` pointer* | -//! | `boxes()` yields a leaked copy of each body rather than the borrowed body | *"a borrowed slice is not inside data()"* | +//! | `appended_stream()` returns `Some(self.data)` unconditionally | the same | +//! | `trailer()` returns `None` unconditionally | *"trailer() disagrees with the Trailer segments"* | +//! | `trailer()` returns `Some(self.data)` unconditionally | the same | +//! | `data()` returns a leaked copy of the input rather than the input | *"data() is not the input"* | +//! | `boxes()` yields a leaked copy of each body rather than the borrowed body | *"a borrowed slice is not inside data(): 16 bytes at 0x…, data() is 272 bytes at 0x…"* | +//! +//! Two of those rows need a file the corpus did not have. `heic-single-item.heic` carries neither +//! an appended stream nor a trailer, so `appended_stream()`/`trailer()` returning `None` +//! unconditionally agreed with it and reported nothing. `appended-stream.heic` (a second top-level +//! `ftyp`, as a motion-photo phone writes) and `trailer.heic` (a truncated trailing box header, +//! retained once `ftyp` and `meta` are seen) are what make those two directions reachable. //! //! The accessors' **reach is one function each**, and that is a limitation rather than a flaw: //! `boxes`, `appended_stream` and `trailer` are each a three-line `filter_map`/`find_map` over the From 39176adeeb0bc31551eda7bae35c529a19e7e787 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 08:42:40 -0400 Subject: [PATCH 21/24] test(fuzz): seed the error path of the annex_b append contract The append contract is claimed "on the success path and the error path", but the well-formed seed's payload splits cleanly, so `annex_b_payload` never returns `Err` for it and an emitter that unwinds the buffer when it gives up went unreported. The same record with one NAL length prefix raised past the end of the payload closes that half. Take the prefix and tail slices with `get` rather than by indexing, so a truncating emitter reports the assertion's own message instead of a bare out-of-range panic raised inside the target. --- tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs | 33 ++++++++++++++------ 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs b/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs index 15064a76..9bda4e23 100644 --- a/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs +++ b/tooling/gamut-fuzz/fuzz_targets/heic_hvcc.rs @@ -14,11 +14,26 @@ //! construction: each one is free to `clear()` or to write through an index, and doing so breaks //! every reusing caller while producing no crash at all. That is what this target searches for. //! -//! Injection that proved the append check fires (re-runnable): begin -//! `HevcConfig::annex_b_parameter_sets` with `out.clear()` — an emitter that replaces instead of -//! appending, which breaks every reusing caller and crashes nothing. The committed seed alone -//! reports it, with no search: `run.sh heic_hvcc -- -runs=0` gives *"an annex_b emitter -//! overwrote what was already in the buffer"*. +//! The contract has **two halves — the success path and the error path — and one injection covers +//! only one of them**, so each has its own, both re-runnable as `run.sh heic_hvcc -- +//! -runs=0` and both reported by the committed seeds alone with no search: +//! +//! - **success path.** Begin `HevcConfig::annex_b_parameter_sets` with `out.clear()` — an emitter +//! that replaces instead of appending, which breaks every reusing caller and crashes nothing. +//! Reports *"an annex_b emitter overwrote what was already in the buffer"*. +//! - **error path.** Have `annex_b_payload` `out.clear()` before returning the error a malformed +//! NAL length prefix produces — an emitter that unwinds the caller's buffer when it gives up. +//! Reports the same message, on `corpus/heic_hvcc/truncated-payload-nal.bin`. +//! +//! That second seed is what makes the error half reachable: the well-formed record's payload +//! splits cleanly, so `annex_b_payload` never returns `Err` for it and the error-path injection +//! goes unreported. `truncated-payload-nal.bin` is the same record with its one NAL length prefix +//! raised by one, past the end of the payload. +//! +//! The prefix and tail comparisons take their slices with `get`, not by indexing: an emitter that +//! truncates the buffer would otherwise report a bare "range end index out of range" from inside +//! this target rather than the assertion's own message, which is a worse thing to be handed by an +//! unattended run. //! //! Alongside it, and explicitly **not** a differential, is a **structure pin**: `annex_b`'s body //! *is* `annex_b_parameter_sets` followed by `annex_b_payload`, so asserting the whole equals the @@ -77,13 +92,13 @@ fuzz_target!(|data: &[u8]| { config.annex_b_parameter_sets(&mut reused); let _ = config.annex_b_payload(payload, &mut reused); assert_eq!( - &reused[..SCRATCH.len()], - &SCRATCH[..], + reused.get(..SCRATCH.len()), + Some(&SCRATCH[..]), "an annex_b emitter overwrote what was already in the buffer" ); assert_eq!( - &reused[SCRATCH.len()..], - &whole[..], + reused.get(SCRATCH.len()..), + Some(&whole[..]), "annex_b is not its two documented halves concatenated" ); From eea00c339540bb74c62ab15addec7f03877cf189 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 08:46:29 -0400 Subject: [PATCH 22/24] docs(fuzz): reprice the listed checks against a per-check injection audit The robustness table listed the sample count as `tiff_decode`'s check and the box cursor as one of `isobmff_boxes`'s; neither can fail for any input, and `heic_container`'s tiling row searched the same function as `isobmff_boxes`'s. Move them to a structure-pin table, name each row's checks separately so a row with two gets two injections, derive the count of dead entries from the list rather than restating it, and document the #264 seed numbering, the four witness seeds and why the matrix keeps its nine rows. --- tooling/gamut-fuzz/README.md | 153 ++++++++++++++++++++++++----------- 1 file changed, 106 insertions(+), 47 deletions(-) diff --git a/tooling/gamut-fuzz/README.md b/tooling/gamut-fuzz/README.md index fdf67299..8ddbaaba 100644 --- a/tooling/gamut-fuzz/README.md +++ b/tooling/gamut-fuzz/README.md @@ -118,47 +118,81 @@ no crash is still visible: | target | crate | entry points | check beyond the crash oracle | |---|---|---|---| -| `ifd_read` | `gamut-ifd` | `read`, `read_tree`, `read_audited` | the dual-ledger audit is complete: no byte read outside a claim, no claim unread | -| `tiff_decode` | `gamut-tiff` | `TiffDecoder::{page_count,info_page,decode_page}` | a page that decodes yields exactly `width × height × Rgb8::CHANNELS` samples for the geometry the tags declare | +| `ifd_read` | `gamut-ifd` | `read`, `read_tree`, `read_audited` | the dual-ledger audit is complete — no byte read outside a claim, **and** no claim unread | +| `tiff_decode` | `gamut-tiff` | `TiffDecoder::{page_count,info_page,decode_page}` | the geometry the decode hands back equals the geometry the tags declare, after every stage that could rewrite it | | `dng_decode` | `gamut-dng` | `DngDecoder::{decode,verify_new_raw_image_digest}` | the raw image that *arrives* holds exactly `width × height × planes` samples, after every rewriting stage | -| `isobmff_boxes` | `gamut-isobmff` | `walk_segments`, `walk_meta_children`, `read`, `BoxReader` | the box cursor strictly advances; the segments tile `0..len` exactly | -| `heic_container` | `gamut-heic` | `HeifContainer::parse` | the segments tile `0..len` exactly and every accessor agrees with that tiling | -| `heic_hvcc` | `gamut-heic` | `HevcConfig::parse`, `annex_b*`, `validate_still_payload`, `iter_nal_units` | the Annex-B emitters append rather than replace, on the success path and the error path | +| `isobmff_boxes` | `gamut-isobmff` | `walk_segments`, `walk_meta_children`, `read`, `BoxReader` | the segments tile `0..len` exactly — contiguous, **and** covering to end of file | +| `heic_container` | `gamut-heic` | `HeifContainer::parse` | every accessor agrees with the segment list, **and** every borrowed slice lies inside `data()` | +| `heic_hvcc` | `gamut-heic` | `HevcConfig::parse`, `annex_b*`, `validate_still_payload`, `iter_nal_units` | the Annex-B emitters append rather than replace — on the success path **and** on the error path | **A check is only listed here if it can fail — and each target's module doc names the injected -defect that made it fail**, with the message it produced and the command that reproduces it. That -second half is the part a reader can re-run; without it "this check is live" is a reading of the -code, which is exactly what put the rows below wrong twice. - -Five earlier entries could not fail. `ifd_read` compared `read(data)` against -`IfdReader::open(data)?.read_file()` — but `reader.rs` *defines* `read` as that expression, so the -two sides were one function call written twice. `heic_hvcc` compared `annex_b(..).is_ok()` against -`annex_b_payload(..).is_ok()` on the same input, and asserted `annex_b` equals the two calls its -own body makes. `dng_decode` compared a digest verdict against a decoded field that is read with -the *same expression* on both sides. `tiff_decode` claimed two: that `info_page` refuses the index -`page_count` returns — which reduces to indexing a vector one past its own length, both sides -being `read(data)?.ifds` — and that the described and decoded geometry agree, which sees only the -few lines copying one into the other, because `decode_page_samples` says outright that -"everything the page *declares* comes from one shared reader". None of them had a reachable -failure, and calling any of them a differential overstated what the tier proves. - -The two `tiff_decode` claims are **dropped**, and the target is re-anchored on something the -geometry reader does not produce: the number of samples the decode physically yielded, against the -declared dimensions and the channel count of the layout asked for. The rest are **relabelled and -repriced**. A claim about two bodies agreeing is a **structure pin**: worth keeping where it is -free or where a future change could genuinely split the bodies apart, worth nothing as a search. -So `heic_hvcc` still asserts the two halves, folded into the append check's existing buffer at no -extra emitter pass; `dng_decode` still compares the verdict, on a call it makes anyway for the -crash oracle; and the `gamut-ifd` wrapper pin lives in `crates/gamut-ifd/tests/robustness.rs`, -over a bounded exhaustive corpus, rather than costing half of every one of this target's twenty -thousand executions per second to search for a counterexample that does not exist. Dropping the -two duplicate parses raised `ifd_read` from roughly 12 000 exec/s to roughly 20 000. - -Two smaller assertions are pins for the same reason and are labelled as such at the site, so -nobody reads them as checks: `tiff_decode`'s "a page that decodes must also describe" (decoding -calls the tag reader before it reads a pixel) and `heic_hvcc`'s "no empty NAL unit" -(`NalUnitIter::next` errors on a zero length before it can yield one). Both cost one comparison on -a value already in hand. +defect that made *each listed check* fail**, with the message it produced and the command that +reproduces it. That second half is the part a reader can re-run; without it "this check is live" is +a reading of the code, which is exactly what put the rows above wrong twice. + +**One injection per listed check, not per target row.** Every "and" in the column above is a +separate assertion that fails for a separate defect, and a row that records one injection has +evidence for one of them. Two checks survived three rounds of review that way — `ifd_read`'s "no +claim unread" and `heic_hvcc`'s error path — because their row's single injection satisfied only +the other half. Where a check is an equality between an accessor and a count, inject in **both** +directions: one direction is silent on a file that holds no instance of the thing. + +### Entries that could not fail + +Each of these was listed as a check and each was removed or relabelled after an injection into the +defect it advertised produced no report: + +1. `ifd_read` compared `read(data)` against `IfdReader::open(data)?.read_file()` — but `reader.rs` + *defines* `read` as that expression, so the two sides were one function call written twice. +2. `heic_hvcc` compared `annex_b(..).is_ok()` against `annex_b_payload(..).is_ok()` on the same + input. +3. `heic_hvcc` asserted `annex_b` equals the two calls its own body makes. +4. `dng_decode` compared a digest verdict against a decoded field that is read with the *same + expression* on both sides. +5. `tiff_decode` claimed that `info_page` refuses the index `page_count` returns — which reduces to + indexing a vector one past its own length, both sides being `read(data)?.ifds`. +6. `tiff_decode` claimed that the described and decoded geometry agree, which sees only the few + lines copying one into the other, because `decode_page_samples` says outright that "everything + the page *declares* comes from one shared reader". +7. `tiff_decode`'s replacement for 5 and 6 — the number of samples the decode physically yielded, + against the declared geometry — was **also** one of these, and was listed as the row's check for + a round. `convert_from_raw` allocates its output as `ImageBuf::::zeroed(src.dims)`, so the + returned count is the *dimensions'* product by construction: the assertion is the geometry + comparison times a constant on both sides, and a transposition injected where the decode builds + its `DecodedImage` passes it with exit 0. The geometry pair — dropped as entry 6 — is what fires + on that transposition, so it is back, and the sample count is a pin (below). +8. `isobmff_boxes` claimed the box cursor strictly advances. `BoxReader::next_box` reads its + 4-byte size and 4-byte type through `take` before any success return, so no declared box size + can stall the cursor: with the `size < header_size` guard removed, 851 173 executions over + 61 seconds reported nothing, while a `self.pos` rewind reports on the first seed. +9. `heic_container` checked that the segments tile `0..len`. `HeifContainer::parse` stores + `gamut_isobmff::walk_segments(data)?` verbatim, so that is the same assertion over the same + values as `isobmff_boxes`'s — the same injection produced the *identical* message in both — on a + narrower input set. Two ten-minute runners searching one function, at a measured-zero marginal + yield. + +Calling any of them a differential overstated what the tier proves. + +### Structure pins + +A claim about two bodies agreeing is a **structure pin**: worth keeping where it is free or where a +future change could genuinely split the bodies apart, worth nothing as a search. Each is labelled +as one at the site and none is listed in the table above. + +| pin | target | why no input can fail it | +|---|---|---| +| `annex_b` equals its two documented halves | `heic_hvcc` | `annex_b`'s body *is* those two calls; folded into the append check's existing buffer at no extra emitter pass | +| no empty NAL unit | `heic_hvcc` | `NalUnitIter::next` errors on a zero length before it can yield one | +| the digest verdict matches the decoded field | `dng_decode` | both sides read `NewRawImageDigest` out of IFD 0 with the same expression; the call is made anyway for the crash oracle | +| a page that decodes must also describe | `tiff_decode` | `decode_page_samples` calls the tag reader before it reads a pixel | +| the sample count matches the declared geometry's product | `tiff_decode` | `ImageBuf` sizes its storage from its own dimensions, so this is the row's live check times `Rgb8::CHANNELS` | +| the box cursor strictly advances, and never runs past the end | `isobmff_boxes` | `next_box` consumes its 8-byte header through `take` before any success return | +| no segment is empty | `isobmff_boxes` | every shape `walk_segments` pushes is widened by that same header read, or runs to end of file | + +The `gamut-ifd` wrapper pin (entry 1) lives in `crates/gamut-ifd/tests/robustness.rs` instead, over +a bounded exhaustive corpus, rather than costing half of every one of this target's twenty thousand +executions per second to search for a counterexample that does not exist. Dropping the two +duplicate parses raised `ifd_read` from roughly 12 000 exec/s to roughly 20 000. An **allocation** defect needs the engine's malloc hook to be visible at all: an oversized `Vec::with_capacity` costs no resident memory on an overcommitting kernel, so measuring RSS finds @@ -201,6 +235,12 @@ It is **kept**, and here is why, so the next target added does not reopen it: - Changing the trigger would also move #593's premise (whether a per-push aggregate is red), which is a decision about the workflow's shape rather than about this tier. +The **size** of the matrix is a separate question with the same answer. Nine rows is nine targets, +and dropping one to buy back queue time would trade coverage for a cost that #603 already owns — +persisting the corpus is what changes this tier's yield per minute, and until it lands, a shorter +matrix is simply less search. A target is removed when its checks stop having reach, which is what +the injection audit decides, and never to make a job finish sooner. + ## Keeping the three lists in step A target exists in three hand-maintained places: its `fuzz_targets/.rs` file, its `[[bin]]` @@ -232,12 +272,31 @@ adds a couple of hundred files — and those stay untracked, which is the point. `git add -f` the whole directory a second time**: add the one seed you mean by path, or the engine's search state goes in with it. -They are seeds, **not** the regression record. `corpus/ifd_read/` carries the thirteen -malformed-TIFF cases enumerated on issue #264 (contributed from rawshift's deleted in-repo TIFF -parser); each of the other five directories carries one or two small well-formed files, written by -this workspace's own encoders, so a decoder target starts from something that reaches its pixel -path instead of spending its budget rediscovering a header. `corpus/tiff_decode/` carries two — -`rgb8-none.tif` and `rgb8-lzw.tif` — because an uncompressed strip and an LZW strip enter the -decoder through different code, and seeding only one leaves the other to be rediscovered. -Real-camera corpora are deliberately not vendored: they run to hundreds of -megabytes and live in `justin13888/rawshift-test-fixtures` releases. +They are seeds, **not** the regression record. `corpus/ifd_read/` carries the thirteen cases +enumerated on issue #264 (contributed from rawshift's deleted in-repo TIFF parser) — eleven that +must be refused and two that must parse — under that issue's own numbering, so a file maps back to +a row of its table. Two things about the numbering, since neither is guessable from `ls`: **case 2 +carries no file**, because `"II" LE16(42)` *is* `"II" 2A 00` byte for byte and it would be the same +four bytes as case 1; and cases 9 and 10 carry two files each, `a` and `b`, for the two variants +their rows name. + +Each of the other five directories starts from one or two small well-formed files, written by this +workspace's own encoders, so a decoder target begins from something that reaches its pixel path +instead of spending its budget rediscovering a header. + +Beyond those, **a seed is added whenever a listed check turns out to have no witness in the +corpus** — that is what the injection audit above is for, and a check whose only witness has to be +synthesised by the engine is a check the tier is asking luck for. Four exist for that reason and +each names the check it feeds: + +| seed | the check it makes reachable | +|---|---| +| `ifd_read/padding-unread-claim.tif` | "no claim unread". Every #264 case puts IFD0 at offset 8, so byte 8 *is* read as the entry count and an over-claimed header lands on a byte the ledger already holds. This one points IFD0 at offset 16, leaving `8..16` as padding nothing reads. | +| `heic_container/appended-stream.heic` | `appended_stream()` returning `None` when a segment of that kind exists — silent on a file that has no appended stream, which the original seed did not. A second top-level `ftyp`, as a motion-photo phone writes. | +| `heic_container/trailer.heic` | `trailer()` returning `None` when a trailer exists. A truncated trailing box header, retained as a trailer once `ftyp` and `meta` are seen. | +| `heic_hvcc/truncated-payload-nal.bin` | the append contract **on the error path**. The well-formed record's payload splits cleanly, so `annex_b_payload` never returns `Err` for it. Same record, one NAL length prefix raised past the end of the payload. | + +`corpus/tiff_decode/` carries two — `rgb8-none.tif` and `rgb8-lzw.tif` — because an uncompressed +strip and an LZW strip enter the decoder through different code, and seeding only one leaves the +other to be rediscovered. Real-camera corpora are deliberately not vendored: they run to hundreds +of megabytes and live in `justin13888/rawshift-test-fixtures` releases. From 1d4bdde156af9c36bb2a629e470bdde6ac61d945 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 08:46:50 -0400 Subject: [PATCH 23/24] docs(testing): correct the rule for anchoring a robustness check The paragraph told every future author in this workspace to anchor a fuzz target's check on "the count of samples the decode physically yielded". That count is not produced by the decode: `convert_from_raw` allocates its output as `ImageBuf::::zeroed(src.dims)`, so it is the dimensions' own product and the comparison is a tautology a transposition walks through. State the boundary the rule actually needs -- a check can fail only if some defect makes some input fail it, which two sides computed from one another can never do -- give the three shapes that violate it, and require one injection per listed check rather than per target. --- docs/testing.md | 39 ++++++++++++++++++++++++++++++++------- 1 file changed, 32 insertions(+), 7 deletions(-) diff --git a/docs/testing.md b/docs/testing.md index 56340f65..129d5d8a 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -182,13 +182,38 @@ A **robustness** target is not a law and does not route through an `invariants` primary oracle is the engine's own — a panic, a hang, or an allocation past `-malloc_limit_mb` — which no function can express. Any check it adds beyond that oracle must be able to *fail*, and **its module doc records the injected defect that made it fail** — the patch, the message the -target printed, and the command that reproduces it. An assertion comparing a wrapper against the -expression its own body is (`gamut_ifd::read` against `IfdReader::open(..)?.read_file()`) is a -tautology, not a differential; so is a comparison whose two sides come from one reader, which is -what a decoded-versus-described geometry check reduces to when the decoder and the probe share a -tag reader. Anchor the check on something the compared reader does not produce — the count of -samples the decode physically yielded, against the geometry the file declares — and keep the -tautology, if it is worth keeping at all, as a **structure pin**, named as one at the site. +target printed, and the command that reproduces it. + +**"Can fail" means some defect in the code the check names makes some *input* fail it**, and both +halves bite. A check whose two sides are computed from one another cannot be separated by any +input, however hostile: it can only report a defect in that shared computation, never one in the +subject it advertises. Three shapes recur, and every one of them was found here by injecting the +defect the check named and getting nothing back: + +- **a wrapper against the expression its own body is.** `gamut_ifd::read` against + `IfdReader::open(..)?.read_file()` is one function call written twice. +- **two sides that come from one reader.** A decoded-versus-described geometry check sees nothing + when the decoder and the probe share a tag reader. +- **a value silently derived from the value it is compared with.** A decode's sample count looks + like the pixel pipeline's own output, but `gamut_core::convert::convert_from_raw` allocates its + result as `ImageBuf::::zeroed(src.dims)` — so the count *is* the dimensions' product, and + comparing it against the declared geometry's product is the geometry comparison times a + constant. A transposition passes it. In the same shape, "the box cursor strictly advances" + cannot fail for any declared box size, because `BoxReader::next_box` consumes its 8-byte header + before any success return. + +Anchor a check on a value the compared side does not produce, and keep the tautology — if it is +worth keeping at all — as a **structure pin**: named as one at the site, and kept out of the +target's list of checks. A pin earns its one comparison where a future change could genuinely +split the two bodies apart; it is worth nothing as a search. + +**Inject once per listed check, not once per target.** A target that lists two checks and records +one injection has evidence for one of them, and the other can sit dead for rounds — two of this +workspace's did. Where a check is an equality between an accessor and a count, inject in both +directions, because one direction is silent on a file that holds no instance of the thing. And +where an injection reports nothing because no *committed seed* reaches the check, add the seed: a +check whose only witness has to be synthesised by the engine is a check the tier is asking luck +for. `#[ignore]` is not used in this workspace and must not be introduced: `coverage` is the only test gate, so an ignored test is not deferred, it is unrun. From d349d92599e1fe2470bb8bf75c0d4694a97e9285 Mon Sep 17 00:00:00 2001 From: Justin Chung Date: Thu, 10 Sep 2026 09:35:01 -0400 Subject: [PATCH 24/24] docs(fuzz): publish the per-check injection audit The rule "one injection per listed check" was stated but its result was not written down anywhere, so a check with no injection stayed invisible until someone read six module docs. Derive the check set from the robustness table mechanically -- split each row's last cell on its own bold `and` -- and publish the ten checks it yields against the injections that stand behind them. Deriving it that way exposed one hole the rule already covers: `boxes()` is an accessor-versus-count equality with only its under-reporting direction injected, so record the over-reporting one beside it. --- tooling/gamut-fuzz/README.md | 29 +++++++++++++++++++ .../gamut-fuzz/fuzz_targets/heic_container.rs | 8 +++-- 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/tooling/gamut-fuzz/README.md b/tooling/gamut-fuzz/README.md index 8ddbaaba..2b0c95b8 100644 --- a/tooling/gamut-fuzz/README.md +++ b/tooling/gamut-fuzz/README.md @@ -137,6 +137,35 @@ claim unread" and `heic_hvcc`'s error path — because their row's single inject the other half. Where a check is an equality between an accessor and a count, inject in **both** directions: one direction is silent on a file that holds no instance of the thing. +#### The audit that rule produces + +The check set is derived from the table above rather than read off it: split each row's last cell +on its own bold `and`, and every conjunct is one listed check owed one injection. Six rows yield +**ten** listed checks, and sixteen injections stand behind them — more than one apiece wherever a +check is an accessor-versus-count equality, which is injected in both directions. + +| listed check | target | injections recorded | +|---|---|---| +| no byte read outside a claim | `ifd_read` | 1 — header claimed as `header_size() - 1` | +| no claim unread | `ifd_read` | 1 — header claimed as `header_size() + 1` | +| the geometry that arrives equals the geometry the tags declare | `tiff_decode` | 1 — transpose the `DecodedImage` dimensions | +| the raw image holds `width × height × planes` samples | `dng_decode` | 1 — push a sample past `check_sample_count` | +| the segments are contiguous | `isobmff_boxes` | 1 — record a box as `b.offset + 8..end` | +| the segments cover to end of file | `isobmff_boxes` | 1 — `segments.pop()` before the return | +| every accessor agrees with the segment list | `heic_container` | 6 — `boxes`/`appended_stream`/`trailer`, each under- and over-reporting | +| every borrowed slice lies inside `data()` | `heic_container` | 2 — `data()` returns a copy; `boxes()` yields copies | +| the emitters append on the success path | `heic_hvcc` | 1 — `annex_b_parameter_sets` begins `out.clear()` | +| the emitters append on the error path | `heic_hvcc` | 1 — `annex_b_payload` clears before returning `Err` | + +Each injection's message and the command that reproduces it are in the target's own module docs; +this table only accounts for *coverage*, so a row with no injection is visible without reading six +files. The law targets are outside it by construction: their oracle is the `invariants` function +itself, which the property tier already drives. + +Four of these injections report only because of a seed added for them, and each of the four was +first observed to report **nothing** on the seed set that preceded it — the "Seeds" section names +which seed feeds which check. + ### Entries that could not fail Each of these was listed as a check and each was removed or relabelled after an injection into the diff --git a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs index 79c04aae..05fc0d6e 100644 --- a/tooling/gamut-fuzz/fuzz_targets/heic_container.rs +++ b/tooling/gamut-fuzz/fuzz_targets/heic_container.rs @@ -18,13 +18,15 @@ //! and still break it. //! //! Injections that proved each assertion fires (re-runnable), all reported by the committed seeds -//! alone with no search, as `run.sh heic_container -- -runs=0`. Each `is_some()` equality -//! is injected in **both** directions, because one direction is silent on a file that has no -//! segment of that kind: +//! alone with no search, as `run.sh heic_container -- -runs=0`. Every accessor-versus-count +//! equality is injected in **both** directions — an accessor that under-reports and one that +//! over-reports are different defects, and for the two `is_some()` accessors the under-reporting +//! direction is additionally silent on a file that has no segment of that kind: //! //! | injection in `gamut-heic` | message | //! |---|---| //! | `boxes()` skips the `ftyp` box | *"boxes() disagrees with the Box segments"* | +//! | `boxes()` yields every `Box` segment twice | the same | //! | `appended_stream()` returns `None` unconditionally | *"appended\_stream() disagrees with the AppendedStream segments"* | //! | `appended_stream()` returns `Some(self.data)` unconditionally | the same | //! | `trailer()` returns `None` unconditionally | *"trailer() disagrees with the Trailer segments"* |