fix(analyzer): stop promising NOT NULL for DML rows PostgreSQL stores as NULL - #61
Merged
Merged
Conversation
A soundness review found INSERT / UPDATE / DELETE / MERGE shapes where the analyzer inferred NOT NULL for a value PG 18 returns NULL for, or rejected a statement PG runs: - OVERRIDING USER VALUE: the identity value given was taken as stored, and a NULL there was rejected. USER and SYSTEM VALUE are now kept apart; under USER VALUE an identity column stores its sequence's value. - A one-row VALUES with a set-returning function made a data-modifying CTE return exactly one row. - A row written through a view of a view was narrowed by the inner view's WHERE, and its foreign keys vouched for a parent the statement's snapshot may not see: the view body is re-read with its FROM entry as the written row (inner views not narrowed, the base scan without origin). - A DO INSTEAD rule (or INSTEAD OF trigger) on the target or down its view chain was ignored when typing RETURNING, which then reads the rule's rows: nothing is NOT NULL there. - An assignment coercion through a cast that can return NULL (time(timestamp) on infinity, a user's cast) kept the value non-NULL in INSERT, UPDATE, MERGE, DEFAULT and generated columns. - A view exposing a base column twice picked one at random; every view down the chain supplies its defaults (a domain-typed view column its type's), and UPDATE ... SET col = DEFAULT through a view is the view's default or NULL. - A constant was recorded as stored although a typmod coercion changes it (numeric(2,-1) rounds 15, varchar(2) drops a trailing space). - A literal NULL into a NOT NULL column was rejected although a BEFORE ROW trigger for the event can fill it before ExecConstraints runs. Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Oct 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A soundness review of the DML area found ten ways
INSERT/UPDATE/DELETE/MERGEcould be typed NOT NULL where PostgreSQL 18 returns NULL (or rejected a statement PG runs). Each was reproduced against PG 18.6:OVERRIDING USER VALUE— the identity value given was taken as stored (INSERT INTO idt(id, a) OVERRIDING USER VALUE VALUES (-5, NULLIF(1,1)) RETURNING awithCHECK (id > 0 OR a IS NOT NULL)saidaNOT NULL), andVALUES (NULL, …)there was rejected. USER and SYSTEM VALUE had been collapsed into one flag.VALUES—WITH i AS (INSERT … VALUES (generate_series(1, 0)) RETURNING id)was taken to return exactly one row.v2(overv1 … WHERE a IS NOT NULL) was narrowed byv1's WHERE.DO INSTEADrules with RETURNING on a table (or a table under a view) were ignored; RETURNING then reads the rule's rows.time(timestamp)oninfinity, a user's cast) kept the value non-NULL in INSERT, UPDATE, MERGE, DEFAULT, INSERT … SELECT and generated columns (so plainSELECT gen_coltoo).numeric(2,-1)stores 15 as 20 andvarchar(2)drops'ab ''s trailing space, yet the literal was recorded as stored and refuted CHECK constraints.ExecConstraintsruns.While fixing 6 and 9 I found two more of the same family, now fixed and tested: a domain-typed view column takes its type's default at the view level (not the base column's
DEFAULT NULL), andUPDATE view SET col = DEFAULTis the view's own default or NULL —rewriteTargetListIUdoesn't fall back to the base default.Fix
Overriding(NotSet / UserValue / SystemValue) replaces the boolean through INSERT, MERGE and the rewriter; under USER VALUE an identity column stores its sequence value and a NULL given for it is accepted.insert_returns_one_rowrequires no set-returning function in the row.written_row_attrsre-reads a view's body with its FROM entry marked as the written row (with_written_relation): an inner view's columns recurse as written rows (no WHERE narrowing), and the base scan carries noOrigin, so no FK reasoning applies to it.returning_rewrittenwalks the target and its view chain; an INSTEAD rule for the event (or an INSTEAD OF trigger) makes every RETURNING column nullable — INSERT, UPDATE, MERGE and DELETE.ValueInfo::assignedreplacesValueInfo::of: nullability goes throughexpr::assignment_nullable(which callscast_function_can_return_null, left unchanged), and a constant is recorded only when no user cast runs andtypmod::keeps_literalproves the typmod coercion is the identity. The same check applies to INSERT … SELECT and generation expressions.WriteTargetkeeps every level of the view chain:omittedtakes the first level whose columns storing the base column have a default (or a domain type), all such columns count when a view exposes one twice, anddefault_of(UPDATE) is the target's own default.null_assignment_errorskips the column NOT NULL (not a NOT NULL domain, which fails before triggers) when a BEFORE ROW trigger for the event can rewrite the row — on the table, a partition the row is routed to, or (UPDATE) a BEFORE INSERT one on a partition a row may move to.CLAUDE.md's literal-NULL paragraph and the README's RETURNING section are updated accordingly.Tests
typedpg_analyzer/tests/query/dml_soundness.rs— 15 tests, each with the repros and the near misses that must keep narrowing (plain identity inserts, plain one-row VALUES, stored view rows and DELETE through views, ALSO rules and rules for other events, casts that can't return NULL, constants a typmod keeps, AFTER triggers / NOT NULL domains / partitions without a trigger still rejected). 14 of them fail onmain; the 15th pins near misses only.Verification
cargo nextest run --release -p typedpg_analyzer— 2260 passedscripts/run-pg-sanity.sh --no-fail-fast(PG 18) — 2264 passed, nonullability unsound; 2114 queries executed over adversarial datacargo nextest run --release --workspace— 2520 passedcargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warnings,cargo clippy -p typedpg_analyzer --all-targets --features pg_sanity -- -D warnings— cleanImplemented by Claude Opus 5.5 (claude-opus-5-5) in Claude Code via T3 Code.