Repository navigation
Fix issues found reviewing the code-health stack - #140
Conversation
consensus.propose returns Ok for a committed op that the apply refused (a stale epoch). Drain ignored ClusterResponse.ok, so a refused TransferPrimary counted as handed off. Check it, as the shard scheduler does.
flushed_through started at 0 on open, while last_sequence is restored from disk. Only the flush truncates, so every entry below the oldest one left was flushed. With the mark at 0, every DELETE and DROP SERIES on a restarted node that had nothing left to flush waited the full 15 s for it, inside the Raft apply on replicated nodes.
drop_series wiped the measurement's series metadata, then built the delete predicate, which refuses a field or an unknown tag. A refused DROP SERIES returned an error and left SHOW SERIES empty. Build every predicate first. DROP SERIES with no FROM deleted rows only on a sharded node; on a single node it dropped the series and kept every row, which tombstones no longer hide. It now deletes them everywhere.
A peer deletes a DELETE's rows from its whole table after waiting up to 15 s for its WAL to flush. The coordinator sent one request per region on the 5 s peer timeout: a peer leading K regions ran K full-table deletes, and under steady ingest the coordinator timed out and reported a failed delete that the peer then carried out. Send each primary one request, with the flush wait plus the peer timeout to answer in.
is_unknown_table matched "Code: 60", which codes 600-609 also start with, and a delete that failed with one reported success. is_unknown_column matched "Code: 47" (470-479), and a replica stored such a failed delete as a tombstone. Match "Code: 60." and "Code: 47.".
The deep-dive and developer guides still pointed at adapters/cluster/sync_client.rs, which became CatchUpService in application/cluster/catch_up.rs. Also drop a repeated #[doc(hidden)] and a repeated doc line, and move apply_shard_map_op's doc comment from `mod apply;` onto the function.
PR SummaryMedium Risk Overview Drain / sharding: Graceful drain now treats a committed but refused WAL / DELETE latency: On WAL reopen, DROP SERIES / DELETE semantics: chDB error classification: Docs: Developer and deep-dive docs reference Each behavioral fix includes a regression test. Reviewed by Cursor Bugbot for commit 1b80236. Configure here. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 1b80236. Configure here.
| counter!("hyperbytedb_shard_delete_applied_total").increment(1); | ||
| outcome.applied += 1; | ||
| continue; | ||
| } |
There was a problem hiding this comment.
Primary skip misses transferred regions
Medium Severity
After one successful shard_delete to a primary, later snapshot regions with that same primary are counted applied and never sent. A transfer that finishes during that first RPC (now up to the 15s flush wait plus peer timeout) can move a remaining region to another node, so the coordinator reports a full delete while the new primary is never contacted.
Reviewed by Cursor Bugbot for commit 1b80236. Configure here.


Fixes from a review of the code-health stack (#133–#138). Based on #138.
Each fix has a test that failed before the change and passes after it.
Fixed
consensus.proposereturnsOkfor a committed op that the apply refused (for example, a stale epoch). Drain never checkedClusterResponse.ok, so a refusedTransferPrimarycounted as handed off. It now fails, as the shard scheduler does. This predates the stack. Test:a_refused_primary_handoff_is_an_error.flushed_throughto 0 but restoredlast_sequencefrom disk. Only the flush truncates the WAL, so everything below the oldest remaining entry has been flushed, and the mark now starts there. Before the fix, every DELETE and DROP SERIES on a restarted, idle node waited the full 15 s, and on replicated nodes that wait happened inside the Raft apply. Test:a_reopened_wal_reports_truncated_entries_flushed.a_refused_drop_series_keeps_the_series.drop_series_without_from_deletes_the_rows.a_delete_reaches_each_primary_once_and_waits_for_it."Code: 60"also matched codes 600–609, so a delete that failed with one of them reported success."Code: 47"also matched 470–479, so a replica stored that failure as a tombstone. Both now match the exact code. Tests:only_code_60_is_an_unknown_table,only_code_47_is_an_unknown_column.sync_client.rs; they now nameCatchUpServiceinapplication/cluster/catch_up.rs. Also removed a duplicate#[doc(hidden)]and a repeated doc line, and moved theapply_shard_map_opdoc comment offmod apply;.Not fixed
propose_viaretries a forward up to 3 times with a 10 s timeout, including after the leader may already have committed it. Schema mutations did this before the stack too. With the PeerRpc change, the scheduler and drain now use the same path. A DELETE's Raft apply can still wait up to one flush interval (10 s by default) under ingest, so a slow apply can be proposed again. The right fix is a choice between telling a timeout apart from a refused connection and not waiting for the flush inside the apply, so it is left for a decision.1for a DELETE with no WHERE, orseries_id IN (...)) as a tombstone, and that tombstone hides the matching rows permanently, including later writes. Don't run DELETE or DROP SERIES while versions are mixed.drop_generation, which clears the physical-name guard and the ingest schema cache. The next write to each measurement then rescans the catalog. The swap-then-clear is also not atomic with concurrent inserts. Both are minor.delete_series_matchingignores its predicate. It drops every series of the measurement, or of the whole RP when there is no FROM. This predates the stack and is already tracked as an open bug.Verification
.superpowers/gate.sh: fmt, clippy-D warnings, lib, integration and CLI tests: GATE-OK on 1b80236.