Skip to content

Fix duplicate column aliases from set operations and repeated references - #435

Merged
parsonsmatt merged 6 commits into
masterfrom
matt/fix-set-op-ident-reuse
Aug 14, 2026
Merged

Fix duplicate column aliases from set operations and repeated references#435
parsonsmatt merged 6 commits into
masterfrom
matt/fix-set-op-ident-reuse

Conversation

@parsonsmatt

@parsonsmatt parsonsmatt commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Fix duplicate column aliases from set operations and repeated references

Three related fixes to alias/ident handling, found chasing 42702 column reference is ambiguous errors in production queries that layer CTEs, subqueries, and union_. Each commit stands alone.

1. Ident reuse after set operations with asymmetric branches

mkSetOperation rewinds the ident state before rendering the right-hand branch, so both branches allocate the same idents:

mkSetOperation operation lhs rhs = SqlSetOperation $ \p -> do
    state <- Q $ lift S.get
    (leftValue, leftClause) <- unSqlSetOperation (toSqlSetOperation lhs) p
    Q $ lift $ S.put state
    (_, rightClause) <- unSqlSetOperation (toSqlSetOperation rhs) p
    ...

The rewind itself is fine — the branches render as sibling SELECTs, so identical idents in them can't collide. The problem is that the ident state after the whole set operation is wherever the right branch ended. When the right branch allocates fewer idents than the left — most easily, when it only selects already-aliased references to a CTE, so its toAlias calls are no-ops — the enclosing query resumes allocating idents that already appear in leftValue, which is the value that escapes mkSetOperation.

The next aliased select list (a subquery or with wrapping the set operation) then contains two columns with the same name. Referencing such a column from one scope further out fails on PostgreSQL with 42702 (column reference "vN" is ambiguous). SQLite accepts the query and silently resolves the reference to the wrong column — the new tests demonstrate both.

Minimal shape (from the new CTE test):

select $ do
    bCte <- with $ do                      -- allocates v, v2
        b <- from $ table @B
        pure (b ^. BK, b ^. BV)
    (k, v, extra) <- from $ do
        (k, v) <- from $
            (do b <- from $ table @B
                pure (b ^. BK, b ^. BV))   -- left branch: allocates v3, v4
            `union_` from bCte             -- right branch: allocates nothing
        pure (k, v, val (42 :: Int))       -- 42 gets aliased... v3 again
    pure (k, v, extra)

which renders a subquery select list SELECT u.v3, u.v4, ? AS v3 FROM (...) AS u, and the outer SELECT sub.v3, ... is ambiguous.

This is a sibling of #299: the fix for that issue moved the state save/restore so that the state after the set operation is the right branch's end state, which fixed the symmetric case but left the asymmetric one broken. Records from deriveEsqueletoRecord are especially prone to hitting this, since their ToAlias instance allocates one ident per field while a CTE-reference branch allocates none.

Fix: after rendering the right branch, restore the ident state to the end of the left branch. Only leftValue escapes mkSetOperation, so its idents are the ones the enclosing query must not hand out again; the right branch's idents are confined to its own SELECT and never referenced from outside. For set operations whose branches allocate the same number of idents (the common case), the generated SQL is completely unchanged.

2. Repeated inner-scope references are not re-aliased

toAlias treated any expression already carrying an alias as done. That's correct for values aliased in the current scope (idempotency), but wrong for references into an inner scope: the same reference can appear more than once in a single select list, and each occurrence then exports the inner scope's column name unchanged:

select $ do
    (a, b) <- from $ do
        (x, _) <- from $ do
            b <- from $ table @B
            pure (b ^. BK, b ^. BV)
        pure (x, x)                        -- SELECT q.v, q.v FROM (...) AS q
    pure (a, b)                            -- SELECT q2.v, ... — ambiguous

This needs no set operation at all — two levels of subquery nesting and a repeated (or projected-and-repeated, e.g. pure (row, row.field)) reference suffice. Same failure modes: 42702 on PostgreSQL, silently wrong column on SQLite.

Fix: toAlias allocates a fresh alias for references (rendering "source"."inner_name" AS "fresh_name"), and remains a no-op for values aliased in the current scope.

3. A CTE declared inside a set operation branch renders as invalid SQL

with inside a union_ branch rendered a bare WITH directly after the set operator — a syntax error on every supported backend. Branches with LIMIT/ORDER BY already render parenthesized; branches with CTE clauses now do too. (SQLite doesn't accept parenthesized set-operation operands at all, so the test lives in the PostgreSQL suite.)

Also in this PR

  • CI: drop GHC 8.6 (current dependency plans no longer compile on it), fail-fast: false so one incompatible leg doesn't hide the others.

Tests

  • test/Common/Test/CTE.hs: the minimal union shape (fix 1) in both directions (CTE reference as the right and as the left branch), the repeated-reference shape (fix 2), and a layered CTE/subquery/union composite exercising both together.
  • test/Common/Record.hs: fix 2 through a record selected alongside one of its own fields (HasField on a record returns the stored reference), and fix 1 through a deriveEsqueletoRecord record (record union CTE-reference, then a new column aliased after it). Before: 42702 on PostgreSQL, PersistMarshalError on SQLite (columns shifted).
  • test/PostgreSQL/Test.hs: CTE inside a union branch (either operand) stays scoped to the branch (fix 3).

Full suite: 471 examples, 0 failures (SQLite + PostgreSQL).

Known limitation (unchanged)

Selecting the same entity twice in one select list across a subquery boundary still exports duplicate column names: entity references can't be re-aliased the way values can, because an aliased entity resolves its column names through its alias ident. Fixing that needs the expression meta to carry separate read/write aliases, which is out of scope here.

mkSetOperation rewinds the ident state before rendering the right-hand
branch so that both branches allocate the same idents. However, it then
left the ident state wherever the right-hand branch ended. When the
right branch allocates fewer idents than the left one (e.g. it selects
only already-aliased references to a CTE), the enclosing query continued
allocating idents that already appear in the escaped left-branch value.
The next aliased select list (a subquery or CTE wrapping the set
operation) then contained duplicate column names: PostgreSQL rejects
references to such columns with 42702 'column reference is ambiguous',
and SQLite silently resolves them to the wrong column.

Restore the ident state to the end of the left-hand branch instead.
Only the left branch's value escapes mkSetOperation, so its idents are
the ones the enclosing query must not reuse; the right branch's idents
are confined to its own SELECT.

deriveEsqueletoRecord makes this easy to hit because its ToAlias
instance allocates one ident per field, while reading the record back
out of a CTE allocates none.

A variant of #299.
@parsonsmatt parsonsmatt changed the title Fix alias ident reuse after set operations with asymmetric branches Fix duplicate column aliases from set operations and repeated references Aug 14, 2026
Modern persistent-era build plans no longer compile on GHC 8.6
(replace-megaparsec-1.4.5.0 requires a newer megaparsec than the 8.6
plan solves to). Disable fail-fast so one incompatible leg doesn't
cancel the rest of the matrix.
A reference into an inner scope (subquery or CTE) can appear more than
once in a single select list. toAlias treated any expression that
already carried an alias as done, so each occurrence exported the inner
scope's column name unchanged, producing duplicate column names in the
subquery's output. Any outer reference to that name then failed on
PostgreSQL with 42702 "column reference is ambiguous" (and silently
resolved to the wrong column on SQLite).

toAlias now allocates a fresh alias for references (rendering
"source"."inner_name" AS "fresh_name") while remaining idempotent
for values aliased in the current scope.
A with inside a set operation branch rendered as a bare WITH clause
directly after the set operator, which is invalid SQL. Render such
branches parenthesized, matching the existing treatment of branches
with LIMIT or ORDER BY. The branch's CTE stays scoped to the branch.

SQLite does not accept parenthesized set operation operands at all, so
the test lives in the PostgreSQL suite.
- union whose LEFT branch is the CTE reference (mirrored asymmetry):
  right-branch idents are branch-scoped, so the enclosing query may
  reuse them safely
- record selected alongside one of its own fields across a subquery
  boundary (field access on a record returns the stored reference,
  so this is the repeated-reference case in record form)
- CTE declared in the first operand of a set operation
@parsonsmatt
parsonsmatt force-pushed the matt/fix-set-op-ident-reuse branch from 6c1b6a5 to d46f47e Compare August 14, 2026 19:19
@parsonsmatt
parsonsmatt merged commit c137840 into master Aug 14, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant