Skip to content

Replicas delete DROP SERIES rows when there is no FROM - #141

Open
austin-barrington wants to merge 1 commit into
refactor/lint-cleanupfrom
fix/drop-series-replicas
Open

austin-barrington wants to merge 1 commit into
refactor/lint-cleanupfrom
fix/drop-series-replicas

Conversation

@austin-barrington

Copy link
Copy Markdown
Member

Replicas now delete the rows of a DROP SERIES that has no FROM. Based on refactor/lint-cleanup, which now includes #140.

Bug

On a Raft cluster every node applies the replicated DROP SERIES. With no FROM, the apply dropped the series metadata but deleted no rows, so DROP SERIES WHERE host='h2' returned OK and every point stayed. #140 fixed only the single-node path. Its note that "the sharded path already did" was wrong, because a Raft cluster never takes that path.

Found by an e2e on the n3 e2e cluster (3 nodes, sharded, RF 3): after the statement, every pod still read the h2 rows, 14 minutes later.

Fix

With no FROM, a replica deletes the rows of the series that the replicated tag condition selects, from each of its measurements, and skips any measurement without that tag. The coordinator still sends the same condition, so older and newer nodes agree on the request.

The delete runs inside the Raft apply, so it waits for the WAL flush once, not once per measurement.

Tests

  • a_replica_applies_drop_series_without_from_to_its_rows: the coordinator (PeerQueryService) builds the mutation and a separate chDB replica applies it. Before the fix, cpu kept the host=a row (2 rows, expected 1).
  • a_replica_drop_series_without_from_waits_for_the_flush_once: before the fix, three measurements meant three flush waits (expected 1).
  • .superpowers/gate.sh: GATE-OK on f6b1e49 (lib 893, integration 380, CLI 54).
  • e2e on n3 with the f6b1e49 image: DROP SERIES WHERE host='h2' removed the h2 rows from cpu (−2) and mem (2000 → 1998) on every pod. The same statement on 1b80236 deleted nothing.

On a Raft cluster each node applies the replicated DROP SERIES. With no
FROM the apply dropped the series metadata and deleted no rows, so the
statement returned OK and every point stayed. The single-node path
already deleted them.

A replica now deletes, from each of its measurements, the rows of the
series the replicated tag condition selects, and skips a measurement
without that tag. It waits for the WAL flush once, not once per
measurement, because this runs inside the Raft apply.
@cursor

cursor Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches replicated data deletion in the Raft schema-mutation path; behavior must stay aligned with the coordinator’s tag predicate to avoid stale or over-deleted rows on cluster nodes.

Overview
Fixes replicated DROP SERIES where measurement is unset (no FROM): Raft apply previously updated series metadata only and left fact-table rows on followers.

When predicate_sql is non-empty and a points_sink is available, apply now walks every measurement in the retention policy, builds a series_id subquery from the coordinator’s tag condition, and calls delete_rows—skipping measurements that lack the referenced tag via existing is_unknown_column handling. Only the first measurement delete passes the WAL so wait_for_flush runs once inside the apply, not per measurement.

Compat tests cover end-to-end row removal across cpu/mem/disk and a CountingWal assertion of a single flush wait; TestContext exposes points_sink for direct mutation apply.

Reviewed by Cursor Bugbot for commit f6b1e49. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit f6b1e49. Configure here.

match result {
Err(e) if is_unknown_column(&e.to_string()) => {}
other => other?,
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Time predicates silently skipped in replicated DROP SERIES

Medium Severity

delete_series_rows_everywhere embeds tag_condition directly into the WHERE clause of the <series> table subquery, which only has series_id and tag columns. When a replicated DROP SERIES without FROM includes a time predicate, chDB fails with UNKNOWN_IDENTIFIER on time, and is_unknown_column silently skips every measurement, leaving all matching rows undeleted while returning OK.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit f6b1e49. Configure here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant