Skip to content

HTTP errors: with compression, a query that fails after output started gets a wrong ErrorCode and a binary Message of up to ~1 MB #752

Open

Description

@polyglotAI-bot

Describe the bug

With compression on (the default), a SELECT that fails after the server started to write the result gives a wrong ClickHouseServerException:

  • ErrorCode is a number from a type name of the result header (Int32 gives 32, UInt64 gives 64), not the server code.
  • Message starts with the binary header and rows, and the error text is at the end. It had up to 990,928 characters.

ExecuteReaderAsync, ExecuteScalarAsync and ExecuteNonQueryAsync give the same result. An error in the first block is not affected. Workaround: UseCompression=false, or the setting wait_end_of_query=1.

Steps to reproduce

  1. Use the default ClickHouseClientSettings (compression on).
  2. Run the code example below.
  3. Compare ErrorCode with the X-ClickHouse-Exception-Code header of the response (159).

Expected behaviour

ErrorCode is the server code from X-ClickHouse-Exception-Code (159, 395), and Message is the server error text (Code: 159. DB::Exception: Timeout exceeded: ...), as with the workaround. docs/http.mdx says: "With the default timeout_overflow_mode, the command throws a ClickHouseServerException with error code 159 (TIMEOUT_EXCEEDED)." With compression on, a timeout after rows were sent gives ErrorCode 64.

Code example

using var client = new ClickHouseClient(new ClickHouseClientSettings("Host=localhost;Port=8123"));
using var connection = client.CreateConnection();
using var command = connection.CreateCommand();
command.CommandText = "SELECT number, sleepEachRow(0.001) AS z FROM numbers(100000) SETTINGS max_block_size = 100";
command.CommandTimeout = 1;
try
{
    using var reader = await command.ExecuteReaderAsync();
    while (reader.Read()) { }
}
catch (ClickHouseServerException ex)
{
    Console.WriteLine($"ErrorCode={ex.ErrorCode} Length={ex.Message.Length} CodeAt={ex.Message.IndexOf("Code: ")}");
}

Error log

Case (default settings if not stated) UseCompression ErrorCode Message.Length Code: at Message starts with
Code example (CommandTimeout = 1) true 64 8,268 8,123 \x02\x06number\x01z\x06UInt64\x05UInt8 + rows
Code example false 159 144 0 Code: 159. DB::Exception: Timeout exceeded
Code example + wait_end_of_query=1 true 159 145 0 Code: 159. ...
SELECT toInt32(number) AS n, throwIf(number = 1000000, 'boom') AS e FROM system.numbers LIMIT 10000000 true 32 324,214 323,926 \x02\x01n\x01e\x05Int32\x05UInt8 + rows
throwIf(number = 200000, ...), same query, + max_block_size=1000, http_response_buffer_size=0, wait_end_of_query=0 true 32 990,928 990,609 header + rows
Errors in the first block: throwIf(number = 5) FROM numbers(10), intDiv by zero, toInt64('abc'), unknown column true 395, 153, 6, 47 (correct) < 400 0 Code: ...

The server response, with curl: HTTP 500 with the correct X-ClickHouse-Exception-Code, and a body with the partial RowBinaryWithNamesAndTypes output before the plain error text.

$ curl -s -o body.gz -D - -H 'Accept-Encoding: gzip' \
    'http://localhost:8123/?enable_http_compression=1&max_block_size=1000&http_response_buffer_size=0&wait_end_of_query=0' \
    --data-binary "SELECT toInt32(number) AS n, throwIf(number = 200000, 'boom mid stream') AS e FROM system.numbers LIMIT 400000 FORMAT RowBinaryWithNamesAndTypes"
HTTP/1.1 500 Internal Server Error
X-ClickHouse-Format: RowBinaryWithNamesAndTypes
X-ClickHouse-Exception-Code: 395
Content-Encoding: gzip
(other headers omitted)
$ gzip -dc < body.gz | wc -c
850336
$ gzip -dc < body.gz | head -c 17 | od -c
0000000 002 001   n 001   e 005   I   n   t   3   2 005   U   I   n   t
0000020   8
$ gzip -dc < body.gz | tail -c 330 | tr -c '[:print:]' '.'
...........Code: 395. DB::Exception: boom mid stream: while executing 'FUNCTION throwIf(equals(__table1.number, 200000_UInt32) :: 4, 'boom mid stream'_String :: 2) -> throwIf(equals(__table1.number, 200000_UInt32), 'boom mid stream'_String) UInt8 : 0'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 26.10.4.207 (official build)).

The CommandTimeout query gives HTTP/1.1 408 Request Time-out with X-ClickHouse-Exception-Code: 159 and an 8,268-byte body that starts with the header and rows. In these runs the server put the partial output into the error body only with compression on. Without compression, the throwIf queries give HTTP 200 with an in-band exception block, and the client gives ErrorCode 395.

Root cause

  • ClickHouse.Driver/ClickHouseClient.cs:1195-1198 (HandleError): for a body that is not empty, the exception message is the whole decompressed body. The client reads the X-ClickHouse-Exception-Code header only for an empty body (line 1212).
  • ClickHouse.Driver/ClickHouseServerException.cs:36-48 (ParseErrorCode): the code is the first run of digits anywhere in the string, also a digit in a type name or a row byte.

Polyglot AI can open a fix PR if a maintainer assigns this issue to @polyglotAI-bot.

Related: open PR #709 (for #705) reads the header code in HandleError, but does not change Message.

Configuration

Environment

  • Client version: main at 9267d72 (after 1.5.0)
  • Language version: C#
  • .NET version: net10.0 (SDK 10.0.404)
  • OS: Linux (container)

ClickHouse server

  • ClickHouse Server version: 26.10.4.207
  • ClickHouse Server non-default settings, if any: none; the per-query settings are in the Error log table

Classification: impact: none · area: API & diagnostics (server errors)


Found by automated analysis (Polyglot AI) while working on #751. Verified against a live server.

Activity

  1. added
    bugSomething isn't working
    api-diagnosticsArea: public API shape, error reporting, logging, metrics, tracing
    on Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api-diagnosticsArea: public API shape, error reporting, logging, metrics, tracingbugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions