Repository navigation
Description
Describe the bug
When the server fails mid-stream and its exception message is longer than about 3.7 KB, ClickHouseDataReader throws the transport error (HttpIOException: The response ended prematurely, or EndOfStreamException) instead of ClickHouseServerException. The error code and the server message are lost.
ExceptionTagAwareStream keeps only the last 4096 bytes of the body (BufferCapacity = 4096). The server writes an in-band exception block of up to 16 KiB (MAX_EXCEPTION_SIZE = 16 * 1024 in src/Server/HTTP/WriteBufferFromHTTPServerResponse.h). When the block is longer than 4096 bytes, its opening marker __exception__\r\n<tag> is no longer in the ring buffer, so TryExtractMidStreamException() returns null and the reader re-throws the IOException.
Long messages occur in normal errors, for example a parse error that quotes a long string value.
Steps to reproduce
- Run the code example below with
Compression=falseagainst a server that writes the in-band block (checked with 26.10.4.207). - The server message for this query is 4,539 bytes (
Cannot parse string '9x9x…' as Int32); the whole block is 4,610 bytes. - The reader throws
HttpIOExceptioninstead ofClickHouseServerException.
Results for throwIf(number = 200000, '<message>') with the message lengths below (same settings):
| Message length | Bytes from the opening marker to the end of the body | Exception |
|---|---|---|
| 100, 3000, 3600 | up to about 4,020 | ClickHouseServerException |
| 3700, 3800, 6000, 20000 | about 4,120 and more (6000: 6,419) | HttpIOException: The response ended prematurely |
QueryAsync<T> (TryMaterializeNextRow<T>) gives the same result (6000: HttpIOException). A mock body whose block does not start at a row boundary (an Int32 column, one stray byte, then a block with a 6,000-byte message) gives EndOfStreamException for the same reason.
Expected behaviour
ClickHouseServerException with the server's code and message, for every block that the server writes (up to 16 KiB), as for a short message.
Code example
using var client = new ClickHouseClient("Host=localhost;Port=8123;Compression=false");
using var connection = client.CreateConnection();
using var command = connection.CreateCommand();
command.CustomSettings["max_block_size"] = 1000;
command.CustomSettings["http_response_buffer_size"] = 0;
command.CustomSettings["wait_end_of_query"] = 0;
command.CommandText =
"SELECT number AS n, toInt32(if(number = 200000, repeat('9x', 2000), '1')) AS v " +
"FROM system.numbers LIMIT 400000";
using var reader = command.ExecuteReader();
while (reader.Read()) { } // throws HttpIOException, expected ClickHouseServerException (Code: 6)Workaround: wait_end_of_query=1. The server then buffers the result and returns HTTP 500, and the client throws ClickHouseServerException with the full message.
Error log
System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
The server sent this block (curl, same query, end of the body):
\r\n__exception__\r\neeloxtjkfowvnctj\r\nCode: 6. DB::Exception: Cannot parse string '9x9x9x…' as Int32: … (CANNOT_PARSE_TEXT) (version 26.10.4.207 (official build))\n4539 eeloxtjkfowvnctj\r\n__exception__\r\n
Root cause
ClickHouse.Driver/Formats/ExceptionTagAwareStream.cs:14: the ring buffer has 4096 bytes.RecordByteskeeps only the last 4096 bytes received.ClickHouse.Driver/Formats/ExceptionTagAwareStream.cs:131-133:TryExtractMidStreamException()needs the opening marker in that buffer, and returnsnullwithout it.ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs:724-730(Read()) and:648-653(TryMaterializeNextRow<T>()): onnull, thecatch (IOException)re-throws the transport error.
Suggested fix
Make the ring buffer hold the largest block that the server writes: MAX_EXCEPTION_SIZE (16 KiB) plus a few bytes, because the server can add a \n after a message that it cut to fit. The buffer is allocated per reader when the response has X-ClickHouse-Exception-Tag, which the server sends for every query (checked on 26.10); renting it from ArrayPool<byte> keeps the cost low. Alternative: keep the bytes from the first opening-marker candidate on.
A test can use the query above, or throwIf with a message of 6,000 and 16,000 characters. Keep the current behavior for short blocks, and for a body with no block (EndOfStreamException).
Related: #750 (a block at the end of the body that is read as rows) changes the same detection; a fix for it that reads a block at a row boundary into its own buffer covers the server case above, but not a block that starts inside a row or the end-of-body check that uses the ring buffer. #475 / #479 use ExceptionTagAwareStream on the raw path.
Configuration
Environment
- Client version:
mainat 9267d72 - Language version: C# (net10.0)
- .NET version: 10.0.15
- OS: Linux (Docker)
ClickHouse server
- ClickHouse Server version: 26.10.4.207
- ClickHouse Server non-default settings, if any: none (query settings in the code example)
CREATE TABLEstatements for tables involved: none (system.numbers)
Found by automated analysis of the client while working on #750, and checked against a live server, not only by inspection.
Classification: impact: none · area: API & diagnostics (mid-stream errors)
Activity
polyglotAI-bot commentedon Nov 20, 2026
The fix for #750 in #753 covers the server case of this issue: a block that starts at a row boundary is now read into its own buffer, not the 4 KiB ring buffer. Against 26.10,
throwIf(number = 200000, repeat('x', N))with the settings above givesClickHouseServerExceptioncode 395 with the whole message for N = 6000 and N = 15000.Still open after #753: a block that does not start at a row boundary, on a body that the server breaks (the
catch (IOException)path), still uses the ring buffer and gives the rawHttpIOException. At a clean end of the body, #753 throwsClickHouseServerExceptionwith "Unknown error (could not parse exception message)" in that case.