Skip to content

tooling/gamut-dng-oracle: pin the zlib the benchmark's reference arm links, keeping -lz for conformance #618

Description

@justin13888

Observation

tooling/gamut-dng-oracle/build.rs links the system zlib dynamically (-lz), because the
Adobe DNG SDK includes <zlib.h> unconditionally for its Deflate reader. The linkage is dynamic
and the resulting objects carry no RUNPATH, so which libz the SDK actually calls is decided by
the loader at run time — and not only by the machine:

  • cargo exports every build script's native search path on the runner's library path.
  • gamut-dng dev-depends on libtiff-oracle, whose build script builds a stock zlib and emits
    cargo:rustc-link-search=native=…/target/…/zlib-build, which contains a libz.so.1.3.1.

So cargo bench -p gamut-dng on its own — no unrelated crate involved — resolves that stock zlib
out of target/, while running the same bench binary directly resolves the platform's. On this box
those are stock zlib 1.3.1 and zlib-ng-compat 2.3.3, and zlibVersion() cannot tell them apart:
zlib-ng's compatibility build answers "1.3.1" exactly as stock zlib does. Only the path
(dladdr + realpath) discriminates.

Why it matters

The SDK's Deflate decode is the one measured path in this oracle that is neither built from source
committed here nor pinned by Cargo.lock. Measured on the same box, same fixture bytes, three
repetitions back to back at a fixed load:

decode_dng cfa/deflate arm stock zlib 1.3.1 (from target/) zlib-ng 2.3.3 (platform)
gamut 1.044, 0.925, 1.052 ms 1.053, 1.051, 1.038 ms
adobe-sdk 0.974, 0.991, 1.113 ms 0.830, 0.829, 0.837 ms

gamut's arm is an unmoving control; the reference arm moves by 1.2-1.3x. That carries the published
ratio from 0.94 to 1.23-1.26 — across 1.0, with no defect in either implementation. Two
correct measurements of that row looked like a contradiction for a full review round because of it.

The same applies to the crate's other Deflate consumers of this oracle, and it will apply to
anything else that benchmarks against a -lz reference arm.

Proposal

Pin the zlib the benchmark links while keeping -lz for the conformance path, e.g. build a
known zlib from a vendored/committed source (there is already a zlib-oracle and a libtiff-oracle
in tooling/ that build one) and link the oracle against it statically for the configuration the
benchmark uses. Two things to settle when taking it:

  1. Conformance must not change. The reference implementation's behaviour on real DNGs should
    go on being what the platform's SDK build does; only the measured configuration needs pinning.
    If a single pinned link is simpler, that is a defensible answer too — but then it is a change to
    what the conformance oracle is, and should be recorded as one.
  2. Other lanes build this crate. It is a build-system change with a wide blast radius, so it
    wants its own change and its own validation rather than riding along.

Why not in #558

#558 is the benchmark harness itself. Changing the oracle's link line there would widen that PR
from "add a benchmark" to "change how every gamut-dng test links its reference implementation".
What #558 does instead: it keeps the Deflate row with an accurate label — the row is miniz_oxide
against whatever libz the loader resolved, since gamut-deflate is encoder-only — prints the
resolved library (zlibVersion() plus the dladdr path) above the divan output, and warns when
that path lies inside a build directory
, because a resolution that came from the build graph is
one nobody else reproduces. That makes the row interpretable; pinning is what would make it
reproducible.

Verification

cargo bench -p gamut-dng --bench codec -- decode_dng prints the resolved library and, today, the
build-tree warning. After pinning, the same command run through cargo and the bench binary run
directly from target/release/deps/ must report the same library and the same */deflate ratios
to within run-to-run spread. mise run test -p gamut-dng (the Adobe conformance differentials) must
stay green unchanged.

Refs #163, #558.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions