Add type conformance fixtures and validation

[nssalian]

Rationale for this change

Adds the first conformance surface: spec-derived type fixtures under table-spec/types/ (parse(input) == decoded, with valid marking accept vs reject), validated by JSON Schema. Runners that exercise the fixtures against each implementation follow in later PRs.

Are these changes tested?

Yes. dev/validate-fixtures.py validates every cases.json against the schemas and the unique-id rule in CI, and dev/check-license runs Apache RAT.

Are there any user-facing changes?

No.

AI Disclosure

Developed with Claude (Claude Code); fully reviewed by the author.

[moomindani]

Reviewed from the consumer side: I ran the cases in this PR against PyIceberg on current main to see what the surface would actually catch. Two things came out of it.

The geospatial cases work exactly as intended, and they catch a live defect. Spelling "unquoted" into the clauses was the right call. PyIceberg's GEOMETRY_REGEX requires the CRS to be quoted, so it rejects geometry(srid:4326) — the Appendix C example — and writes "geometry('srid:4326')" instead. Java's pattern captures the quotes rather than rejecting them, so it reads that CRS as 'srid:4326' with the quotes included. So this fixture would have surfaced a silent bidirectional divergence, which is the case for the corpus about as well as it can be made. I have raised it on the PyIceberg side (apache/iceberg-python#3530).

The decimal whitespace case stops one step short of the interesting input. decimal-9-2-spaced covers decimal(9, 2), which every implementation I checked already accepts. The clause the spec actually added in apache/iceberg#16798 is broader — "optional whitespace around parameters and separators" — and that is where implementations diverge: PyIceberg accepts decimal(9, 2) but rejects decimal( 9 , 2 ) and decimal(9 ,2), while Java's decimal\(\s*(\d+)\s*,\s*(\d+)\s*\) accepts all of them. The prototype in sungwy/iceberg-testing had that case, with the note "A reader stricter than that diverges here"; it is the one input on this surface that separates a conforming parser from a strict one, so it seems worth carrying over.

On the canonical field for decimal, I would leave it as you have it. The clause here says no byte-exact form is pinned, and while apache/iceberg#16798 intended to pin decimal(P, S) — Java's DecimalType.toString() emits the spaced form, and so does PyIceberg — the sentence that merged points at a table whose template still reads "decimal(<P>,<S>)" with no space. That is a spec-text question rather than something to settle in a fixture, and for read conformance accept is the field that carries the weight anyway.

[laskoviymishka]

Good first cut. The base/types schema split makes sense, expected values come from the spec rather than one implementation, and the JSON Schema validator + RAT in CI give us a useful starting point. Also good that moomindani already ran this against PyIceberg and found a real divergence.

I’d still hold the merge for a few things, mainly because this becomes the contract other implementations will validate against.

The biggest one is geospatial serialization. A few geometry/geography cases describe the canonical serialized form in the clause but don’t have a canonical field. moomindani’s run shows why that matters: PyIceberg writes the quoted CRS form, while Java reads those quotes into the CRS itself, so the two sides diverge (apache/iceberg-python#3530). I’d add canonical to those cases and settle the default-CRS form here.

The decimal whitespace case is a bit different. decimal-9-2-spaced currently uses decimal(9, 2), which all the implementations checked already accept, so it doesn’t really distinguish strict vs lenient parsing. The more useful case is decimal( 9 , 2 ) from apache/iceberg#16798: PyIceberg rejects it and Java accepts it, and both are still conformant. That probably means this needs a small normative distinction, e.g. optional normative_level with default must, so a SHOULD case isn’t encoded as MUST.

Before merge I’d fix:

• add canonical to the geospatial cases that already pin the serialized form, and decide the default-CRS representation
• change the decimal whitespace fixture to decimal( 9 , 2 ) and add optional normative level
• pin the jsonschema dependency and add one negative self-test so CI proves the validator can fail
• set additionalProperties: false on the case schema so typos don’t silently pass

The rest is in the inline comments. After these, I’m happy to take another look.

[laskoviymishka]

The clause here says the canonical form is the unquoted geometry(OGC:CRS84), but without a canonical field the write direction never gets tested — an implementation that emits geometry('OGC:CRS84') or GEOMETRY(OGC:CRS84) would pass. Since the clause already pins the spelling, I'd add canonical to each explicit case: "canonical": "geometry(OGC:CRS84)" here, "geography(OGC:CRS84, spherical)" on the geography ones, and so on. Same applies to geometry-srid and the four geography algorithm cases below.

moomindani's consumer-side run makes this concrete: PyIceberg's regex requires a quoted CRS, so it rejects geometry(srid:4326) and writes geometry('srid:4326'), while Java reads the quotes into the CRS — a silent bidirectional divergence this fixture catches once canonical pins the write side (they filed apache/iceberg-python#3530).

[laskoviymishka]

moomindani's consumer-side run reframed this one for me: decimal(9, 2) is accepted by every implementation they checked, so as written this case is inert — it doesn't separate a strict parser from a lenient one. The input that does is the broader form apache/iceberg#16798 actually added, "whitespace around parameters and separators": PyIceberg rejects decimal( 9 , 2 ) / decimal(9 ,2) while Java accepts them, and both are conformant.

That's the case I'd want on this surface (the sungwy/iceberg-testing prototype carried it, noting "a reader stricter than that diverges here"). But because both parsers are conformant on it, it can't be a plain valid: true — a MUST-accept would fail PyIceberg for behavior the spec only recommends. So I'd swap in the divergent decimal( 9 , 2 ) input and give it a normative tier: an optional normative_level on the base schema (default "must") with this case marked as the SHOULD, so runners treat it as advisory rather than a hard fail.

I'd leave canonical off the decimal cases as you have it — agreed the pinned-spelling question is spec text, not something to settle in a fixture. wdyt on the normative tier?

[nssalian]

I put in a normative_level for now, but I'd like to hear from others if that's good enough.

[moomindani]

The tier reads right to me, and the wording it encodes is already in the spec: format/spec.md:1693 says readers should accept the optional whitespace, so a reader that rejects decimal( 9 , 2 ) stays conformant and a plain MUST-accept fixture would fail it for behavior that is only recommended. Defaulting to must also means no existing case changes meaning.

One thing worth writing into the schema description while it is fresh: a runner has to report a failed SHOULD distinctly from a pass, otherwise the advisory tier is invisible in practice and contributors will assume the case is inert again.

[nssalian]

Added @szehon-ho here to add some perspective on Geo and how we should represent them. Also, CC: @huan233usc