Repository navigation
Description
Describe the bug
When Client.Do cancels a query (the context is canceled or its deadline passes, or OnInput / OnResult returns an error), the server fails the query with a protocol error:
- A query that sends no data yet (
SELECT sleep(2), a long aggregation): code 99UNKNOWN_PACKET_FROM_CLIENT, "Unknown packet from client 0". - An
INSERTwithQuery.Input/OnInput: code 101UNEXPECTED_PACKET_FROM_CLIENT, "Unexpected packet Hello received from client".
So system.query_log and the system.errors counters record each client-side cancel as a protocol error, not as QUERY_WAS_CANCELLED_BY_CLIENT (735).
The client side is correct: Do returns the context error, and the query stops. A canceled INSERT leaves 0 rows. A streaming SELECT that the caller cancels from OnResult is not affected: it gets code 210 (NETWORK_ERROR, broken pipe), because the server fails to write the next block before it reads the Cancel.
Steps to reproduce
- Run the program below against a server on
localhost:9000. - Run
SYSTEM FLUSH LOGS, then:SELECT type, exception_code, exception FROM system.query_log WHERE query_id = 'ch-go-cancel-repro' AND type != 'QueryStart'
- The row has
exception_code99, not 735.
For the INSERT case, run INSERT INTO t VALUES with Query.Input and an OnInput that returns an error after two blocks. The query_log row has code 101.
Expected behaviour
The server gets only the Cancel packet, and logs the query as canceled by the client:
type: ExceptionWhileProcessing
exception_code: 735
exception: Code: 735. DB::Exception: Received 'Cancel' packet from the client, canceling the query. (QUERY_WAS_CANCELLED_BY_CLIENT)
We got this row for the sleep, aggregation, and INSERT cases with a local fix of the buffer in cancelQuery.
Code example
Reproduction program (Go, 29 lines)
package main
import (
"context"
"fmt"
"time"
"github.com/ClickHouse/ch-go"
"github.com/ClickHouse/ch-go/proto"
)
func main() {
ctx := context.Background()
c, err := ch.Dial(ctx, ch.Options{Address: "localhost:9000"})
if err != nil {
panic(err)
}
// Cancel the query while the server runs it.
qctx, cancel := context.WithTimeout(ctx, 500*time.Millisecond)
defer cancel()
var res proto.ColUInt8
err = c.Do(qctx, ch.Query{
Body: "SELECT sleep(2)",
QueryID: "ch-go-cancel-repro",
Result: proto.Results{{Name: "sleep(2)", Data: &res}},
})
fmt.Println("Do:", err) // context deadline exceeded: correct on the client side
}Error log
Program output:
Do: context deadline exceeded
system.query_log row of the canceled query:
type: ExceptionWhileProcessing
exception_code: 99
exception: Code: 99. DB::NetException: Unknown packet from client 0. (UNKNOWN_PACKET_FROM_CLIENT) (version 26.10.4.207 (official build))
INSERT with an OnInput that returns an error:
exception_code: 101
exception: Code: 101. DB::Exception: Unexpected packet Hello received from client. (UNEXPECTED_PACKET_FROM_CLIENT) (version 26.10.4.207 (official build))
The last Write on the connection (recorded with a net.Conn wrapper through Options.Dialer) is 0003 (2 bytes). A Cancel packet is the single byte 03.
Root cause
In cancelQuery (query.go:35-38), the buffer has length 1, so it holds one zero byte:
b := proto.Buffer{
Buf: make([]byte, 1),
}
proto.ClientCodeCancel.Encode(&b)ClientCode.Encode (proto/client_code.go:21) appends with Buffer.PutByte (proto/buffer.go:100-101), so ch-go writes [0x00, 0x03]. The server reads 0x00 (ClientCodeHello) as a Hello packet in the middle of the query.
The fix is a one-line change of this buffer.
Polyglot AI can open a fix PR if a maintainer assigns this issue to @polyglotAI-bot.
Related
- bug: UNEXPECTED_PACKET_FROM_CLIENT after context cancellation #235: the same server message; fix: close connection after query cancellation #236 fixed only its next-query failure.
- fix: close client after query transport errors #1203 (open PR): changes when
cancelQueryis called, not this buffer.
Configuration
Environment
- Client version:
mainat a53587d (the same code is in v0.76.0) - Language version: Go 1.26.11
- OS: Linux (amd64)
ClickHouse server
- ClickHouse Server version: 26.10.4.207
- ClickHouse Server non-default settings, if any: none
CREATE TABLEstatements for tables involved:INSERTcase only:CREATE TABLE t (v UInt64) ENGINE = Memory
Classification: impact: none · area: Connectivity (cancellation)
Found by automated analysis (Polyglot AI) while working on another lead on Client.Do cancellation. Verified against a live server.
Activity