Observation
tooling/gamut-dng-oracle/build.rs links the system zlib dynamically (-lz), because the
Adobe DNG SDK includes <zlib.h> unconditionally for its Deflate reader. The linkage is dynamic
and the resulting objects carry no RUNPATH, so which libz the SDK actually calls is decided by
the loader at run time — and not only by the machine:
cargo exports every build script's native search path on the runner's library path.
gamut-dng dev-depends on libtiff-oracle, whose build script builds a stock zlib and emits
cargo:rustc-link-search=native=…/target/…/zlib-build, which contains a libz.so.1.3.1.
So cargo bench -p gamut-dng on its own — no unrelated crate involved — resolves that stock zlib
out of target/, while running the same bench binary directly resolves the platform's. On this box
those are stock zlib 1.3.1 and zlib-ng-compat 2.3.3, and zlibVersion() cannot tell them apart:
zlib-ng's compatibility build answers "1.3.1" exactly as stock zlib does. Only the path
(dladdr + realpath) discriminates.
Why it matters
The SDK's Deflate decode is the one measured path in this oracle that is neither built from source
committed here nor pinned by Cargo.lock. Measured on the same box, same fixture bytes, three
repetitions back to back at a fixed load:
decode_dng cfa/deflate arm |
stock zlib 1.3.1 (from target/) |
zlib-ng 2.3.3 (platform) |
gamut |
1.044, 0.925, 1.052 ms |
1.053, 1.051, 1.038 ms |
adobe-sdk |
0.974, 0.991, 1.113 ms |
0.830, 0.829, 0.837 ms |
gamut's arm is an unmoving control; the reference arm moves by 1.2-1.3x. That carries the published
ratio from 0.94 to 1.23-1.26 — across 1.0, with no defect in either implementation. Two
correct measurements of that row looked like a contradiction for a full review round because of it.
The same applies to the crate's other Deflate consumers of this oracle, and it will apply to
anything else that benchmarks against a -lz reference arm.
Proposal
Pin the zlib the benchmark links while keeping -lz for the conformance path, e.g. build a
known zlib from a vendored/committed source (there is already a zlib-oracle and a libtiff-oracle
in tooling/ that build one) and link the oracle against it statically for the configuration the
benchmark uses. Two things to settle when taking it:
- Conformance must not change. The reference implementation's behaviour on real DNGs should
go on being what the platform's SDK build does; only the measured configuration needs pinning.
If a single pinned link is simpler, that is a defensible answer too — but then it is a change to
what the conformance oracle is, and should be recorded as one.
- Other lanes build this crate. It is a build-system change with a wide blast radius, so it
wants its own change and its own validation rather than riding along.
Why not in #558
#558 is the benchmark harness itself. Changing the oracle's link line there would widen that PR
from "add a benchmark" to "change how every gamut-dng test links its reference implementation".
What #558 does instead: it keeps the Deflate row with an accurate label — the row is miniz_oxide
against whatever libz the loader resolved, since gamut-deflate is encoder-only — prints the
resolved library (zlibVersion() plus the dladdr path) above the divan output, and warns when
that path lies inside a build directory, because a resolution that came from the build graph is
one nobody else reproduces. That makes the row interpretable; pinning is what would make it
reproducible.
Verification
cargo bench -p gamut-dng --bench codec -- decode_dng prints the resolved library and, today, the
build-tree warning. After pinning, the same command run through cargo and the bench binary run
directly from target/release/deps/ must report the same library and the same */deflate ratios
to within run-to-run spread. mise run test -p gamut-dng (the Adobe conformance differentials) must
stay green unchanged.
Refs #163, #558.
Observation
tooling/gamut-dng-oracle/build.rslinks the system zlib dynamically (-lz), because theAdobe DNG SDK includes
<zlib.h>unconditionally for its Deflate reader. The linkage is dynamicand the resulting objects carry no
RUNPATH, so which libz the SDK actually calls is decided bythe loader at run time — and not only by the machine:
cargoexports every build script's native search path on the runner's library path.gamut-dngdev-depends onlibtiff-oracle, whose build script builds a stock zlib and emitscargo:rustc-link-search=native=…/target/…/zlib-build, which contains alibz.so.1.3.1.So
cargo bench -p gamut-dngon its own — no unrelated crate involved — resolves that stock zlibout of
target/, while running the same bench binary directly resolves the platform's. On this boxthose are stock zlib 1.3.1 and
zlib-ng-compat2.3.3, andzlibVersion()cannot tell them apart:zlib-ng's compatibility build answers
"1.3.1"exactly as stock zlib does. Only the path(
dladdr+realpath) discriminates.Why it matters
The SDK's Deflate decode is the one measured path in this oracle that is neither built from source
committed here nor pinned by
Cargo.lock. Measured on the same box, same fixture bytes, threerepetitions back to back at a fixed load:
decode_dng cfa/deflatearmtarget/)gamutadobe-sdkgamut's arm is an unmoving control; the reference arm moves by 1.2-1.3x. That carries the published
ratio from 0.94 to 1.23-1.26 — across 1.0, with no defect in either implementation. Two
correct measurements of that row looked like a contradiction for a full review round because of it.
The same applies to the crate's other Deflate consumers of this oracle, and it will apply to
anything else that benchmarks against a
-lzreference arm.Proposal
Pin the zlib the benchmark links while keeping
-lzfor the conformance path, e.g. build aknown zlib from a vendored/committed source (there is already a
zlib-oracleand alibtiff-oraclein
tooling/that build one) and link the oracle against it statically for the configuration thebenchmark uses. Two things to settle when taking it:
go on being what the platform's SDK build does; only the measured configuration needs pinning.
If a single pinned link is simpler, that is a defensible answer too — but then it is a change to
what the conformance oracle is, and should be recorded as one.
wants its own change and its own validation rather than riding along.
Why not in #558
#558 is the benchmark harness itself. Changing the oracle's link line there would widen that PR
from "add a benchmark" to "change how every gamut-dng test links its reference implementation".
What #558 does instead: it keeps the Deflate row with an accurate label — the row is
miniz_oxideagainst whatever libz the loader resolved, since
gamut-deflateis encoder-only — prints theresolved library (
zlibVersion()plus thedladdrpath) above the divan output, and warns whenthat path lies inside a build directory, because a resolution that came from the build graph is
one nobody else reproduces. That makes the row interpretable; pinning is what would make it
reproducible.
Verification
cargo bench -p gamut-dng --bench codec -- decode_dngprints the resolved library and, today, thebuild-tree warning. After pinning, the same command run through
cargoand the bench binary rundirectly from
target/release/deps/must report the same library and the same*/deflateratiosto within run-to-run spread.
mise run test -p gamut-dng(the Adobe conformance differentials) muststay green unchanged.
Refs #163, #558.