Add failing repro of filter leaks across a self-joined dimension - #2452
Draft
shangyian wants to merge 2 commits into
Draft
Add failing repro of filter leaks across a self-joined dimension#2452shangyian wants to merge 2 commits into
shangyian wants to merge 2 commits into
Conversation
A dimension whose own query self-joins one table gets a filter applied to
every alias of that table, not just the one producing the filtered column.
The repro is a month-to-date bridge: `a` supplies the anchor date, `b`
enumerates the dates from the start of that month through the anchor. Orders
join the bridge on the period side and expose the anchor side, so each order
appears once per window containing it. Filtering on the anchor emits
WHERE a.date_int IN (20180208, 20180210)
AND b.date_int IN (20180208, 20180210)
and the second predicate collapses every window to just the filtered dates,
so a metric over the bridged rows returns a plausible but wrong average for
anchors that survive the filter, and drops anchors whose window becomes empty.
No error is raised.
Filter resolution is by bare column name, and both output columns trace back
to `date_int` on the same table, so nothing distinguishes the two sides. The
alias-substitution step in _resolve_pushdown_filters_for_cte then copies the
rewritten predicate onto sibling references to the same physical table, which
is deliberate for disambiguating unrelated tables that share a column name but
wrong when the siblings are two roles of one self-join.
The second test is a control: the same bridge over two different tables filters
correctly, so the trigger is the shared physical table rather than the join
shape. It passes today and should keep passing.
✅ Deploy Preview for thriving-cassata-78ae72 canceled.
|
Filter pushdown into a CTE rewrites the predicate once for the CTE's own Select, then clones it onto sibling references to the same physical table so nested subqueries and set-op arms scanning that table get the predicate too. The skip guard only excluded the primary reference itself, so a self-join -- where a second alias of the same table sits in the same Select -- received a cloned predicate on the wrong side of the join. Skip every reference in the primary Select instead. The primary rewrite resolves the filtered column through the Select's projection map, so it already constrains the side the column projects from; the other side has to stay free. Retargeting keeps reaching other scopes, which is what it was for. Fixes DataJunction#2451
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reproducing #2451
Test Plan
make checkpassesmake testshows 100% unit test coverageDeployment Plan