Skip to content

test: every law listed in the fuzz tier needs its own failure-path test #611

Description

@justin13888

docs/testing.md requires that a law's own failure path be covered by an ordinary
#[cfg(test)] unit test on deliberately-broken input, because .cargo/mutants.toml
excludes crates/*/src/invariants.rs from mutation and the pinned-seed properties only
ever exercise the Ok arm against unmutated code. A law silently weakened to Ok(())
would therefore make every property driving it vacuous while still passing, and nothing
in the gate would notice.

Auditing the three law rows of tooling/gamut-fuzz's target table against that rule
(one falsifier per listed law, mechanically, rather than per module) leaves a gap:

crate law named falsifier today
gamut-tonemap output_is_non_negative_and_never_nan yes — a_nan_producing_curve_violates_the_non_nan_law, a_negative_curve_violates_the_non_negative_law
gamut-tonemap monotonic_non_decreasing yes — a_decreasing_curve_violates_the_monotonicity_law
gamut-tonemap map_slice_is_elementwise_map yes — a_map_slice_that_skips_an_element_violates_the_elementwise_law
gamut-tonemap map_slice_is_order_independent yes — a_position_dependent_map_slice_violates_the_order_law
gamut-ifd ledger_is_canonical yes, via canonical_form_rejects_{an_empty_range,adjacent_ranges,descending_ranges}
gamut-ifd subtract_is_set_difference only the shared canonical_form helper. Its own set-difference comparison arm has no test that reaches it
gamut-core palette_and_cmyk_convert_only_to_themselves yes — the_closure_law_{does_not_govern_an_open_pair,governs_a_closed_pair_and_the_engine_refuses_it}
gamut-core acceptance_is_independent_of_the_samples none
gamut-core output_shape_matches_the_target_layout none
gamut-core the_in_place_door_matches_the_allocating_door none
gamut-core converting_a_layout_to_itself_changes_nothing none

gamut-core's five properties all assert outcome.is_ok(), and the module's three guard
tests cover the closure law, sample_count, normalise_dims and the format tables — not
the other four laws' Err arms.

Wanted: one #[cfg(test)] test per law above marked none, on a deliberately-broken
input, asserting the law reports a violation (and, where cheap, that the violation's
detail() names the right thing) — the shape gamut-tonemap's a_decreasing_curve_…
already has. Plus one for gamut-ifd's subtract_is_set_difference that reaches its own
comparison rather than the shared canonical-form check.

Found while re-auditing the fuzz tier's target table in #568, whose manifest was bounded to
tooling/gamut-fuzz/** and docs/testing.md, so the crates/ change is filed rather than
made there.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions