Context
extended.yml's fuzz job now runs nine targets, each for -max_total_time=600, on every push
to the default branch and on manual dispatch (PR #568 added six of the nine). The cadence was never
chosen — it is inherited from the job's first three targets, when the matrix was small and the
targets were law targets over normalised inputs.
Why the cadence is now a question
- Cost scales with the matrix. Nine parallel ten-minute runners per push, and the design of the
job is explicitly that adding a crate adds a row. The next few entry points in
docs/testing.md's table (PngDecoder, JpegDecoder, WebpDecoder, IccProfile::parse,
RiffReader, decode, parse, read_irb) would roughly double it.
- Ten minutes from a cold, empty corpus is not the same search twice. Each run starts from the
committed seeds only: the engine's accumulated corpus is deliberately not persisted (it is in
.gitignore, and the README explains why — the corpus is a search aid, not the regression
record). So sixty ten-minute runs are sixty shallow searches, not one ten-hour one. A nightly run
with a cached corpus would search deeper for less total compute; a per-push run catches a
regression sooner.
- Per-push is a regression signal; scheduled is an exploration signal. They are different jobs
wearing one name, and the tier currently only does the first badly and the second not at all.
The question
Pick, and record, a cadence for the fuzz job. Candidates:
- Keep per-push at 600 s. Simplest; a defect introduced by a push is caught by that push's
own run. Cost grows linearly with the matrix.
- Nightly
schedule: at a longer budget, per-push dropped. Deeper search, cheaper overall,
regression detection delayed by up to a day.
- Both, split: a short per-push smoke run (60–120 s) plus a nightly deep run. Two rows per
target, more YAML, and the drift guard
(tooling/gamut-fuzz/check-targets.sh) has to reconcile two matrices instead of one.
- Per-push, but only for targets whose crates the push touched. Best cost/signal ratio, most
machinery, and it needs a path filter that cannot go stale — which is the same class of problem
the drift guard exists to solve.
Whether to cache and persist the engine corpus between runs (actions/cache keyed per target) is
part of this decision: it is what makes options 2 and 3 worth more than option 1, and it is in
tension with the "the corpus is not the regression record" rule, which is about the repository,
not about a cache.
Not urgent — nine targets at ten minutes is affordable today. It should be decided before the
matrix doubles.
Refs #264, #311, #568.
Context
extended.yml's fuzz job now runs nine targets, each for-max_total_time=600, on every pushto the default branch and on manual dispatch (PR #568 added six of the nine). The cadence was never
chosen — it is inherited from the job's first three targets, when the matrix was small and the
targets were law targets over normalised inputs.
Why the cadence is now a question
job is explicitly that adding a crate adds a row. The next few entry points in
docs/testing.md's table (PngDecoder,JpegDecoder,WebpDecoder,IccProfile::parse,RiffReader,decode,parse,read_irb) would roughly double it.committed seeds only: the engine's accumulated corpus is deliberately not persisted (it is in
.gitignore, and the README explains why — the corpus is a search aid, not the regressionrecord). So sixty ten-minute runs are sixty shallow searches, not one ten-hour one. A nightly run
with a cached corpus would search deeper for less total compute; a per-push run catches a
regression sooner.
wearing one name, and the tier currently only does the first badly and the second not at all.
The question
Pick, and record, a cadence for the fuzz job. Candidates:
own run. Cost grows linearly with the matrix.
schedule:at a longer budget, per-push dropped. Deeper search, cheaper overall,regression detection delayed by up to a day.
target, more YAML, and the drift guard
(
tooling/gamut-fuzz/check-targets.sh) has to reconcile two matrices instead of one.machinery, and it needs a path filter that cannot go stale — which is the same class of problem
the drift guard exists to solve.
Whether to cache and persist the engine corpus between runs (
actions/cachekeyed per target) ispart of this decision: it is what makes options 2 and 3 worth more than option 1, and it is in
tension with the "the corpus is not the regression record" rule, which is about the repository,
not about a cache.
Not urgent — nine targets at ten minutes is affordable today. It should be decided before the
matrix doubles.
Refs #264, #311, #568.