Skip to content

gamut-iptc: nothing re-derives the committed IIM 4.2 dataset-name table from the standard #623

Description

@justin13888

The gap

gamut-iptc's IIM 4.2 dataset-name drift guard
(tag_table_names_match_the_standards_own_dataset_names, inline in crates/gamut-iptc/src/iim.rs)
compares every KNOWN_TAGS name against crates/gamut-iptc/tests/data/iim-4.2-dataset-names.tsv.
That .tsv is a derived artefact: crates/gamut-iptc/tests/data/extract-iim-names.py recovers
the standard's own DataSet name column from references/iptc/iim-4.2.pdf by position with
pdftotext -bbox-layout, and the output is committed because pdftotext is a system package
mise.toml does not provision.

So the guard's strength is asymmetric, and the asymmetry is worth stating exactly:

  • a name mistyped in KNOWN_TAGS alone fails the guard — this is the case the guard exists for
    and it works;
  • a name mistyped identically in KNOWN_TAGS and in the committed .tsv passes. The guard
    compares the table against the artefact, not against the PDF, and nothing in the repository
    re-derives the artefact.

What stands against the second case today is the artefact's own banner (Never hand-edit: regenerate with the extract-iim-names.py beside this file) and code review — a convention, not a
gate. gamut-exif has the same shape of gap for a different table (#544), but its extraction
scripts were never committed; here the extraction is committed and runnable, which makes this
one cheap to close.

Candidate answer

A CI step, in the extended (master/manual) tier rather than the required tier, that:

  1. checks for pdftotext (poppler-utils) and skips with an explicit "not present" line if absent —
    so a contributor without it is never blocked;
  2. re-runs crates/gamut-iptc/tests/data/extract-iim-names.py;
  3. fails if the regenerated .tsv differs from the committed one.

That turns "a two-column typo would fail" from half-true into true, and it also catches a stale
artefact after the vendored PDF is ever replaced.

Open sub-questions for whoever takes it:

  • Should pdftotext be provisioned by mise instead, so the step can be unconditional? That adds
    a system dependency to the toolchain for one table in one crate; the conditional step avoids it
    but is weaker (a runner without the tool silently proves nothing).
  • The extraction is currently reproducible: the same table came back byte-identically under a
    different PDF toolchain during review. Whether that holds across poppler versions is the thing a
    conditional step would discover, and a version pin may be needed.

Why it is filed rather than done

Raised at the close of review on #540, which delivered the guard. Adding a conditional CI step is
new tooling in a workflow file rather than a repair to the change under review, so it is left for a
human to take.

Refs #422, #540.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions