Repository navigation
Fix ClickHouseDataReader: stop at the mid-stream exception block, and throw when the body ends after it - #753
polyglotAI-bot wants to merge 2 commits into
Conversation
… throw when the body ends after it
The reader looked for the server's in-band exception block only after an
IOException. When the body ended cleanly after the block at a row boundary,
the result was reported as complete; and before the error surfaced, the
bytes of the block were returned as rows.
- At the end of the body, a closing marker with the tag now throws the
ClickHouseServerException (parsed from the ring buffer when possible).
- At a row boundary, the opening marker as the server writes it
("\r\n__exception__\r\n<tag>") is matched in the read buffer, so the block
is read and thrown before it is decoded as rows.
Fixes: #750
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
🟡 Changes recommended
The release note overstates row validity, while reader documentation omits the lookahead delay, clean-EOF exception behavior, and potential invalid rows from mid-row exception blocks.
2 open issues
What changed in this PR
This PR addresses #750 in the HTTP client by detecting mid-stream server exceptions before their bytes become rows when standard framing starts at a row boundary. It also prevents a cleanly ended response from hiding a trailing server error.
Changes:
- Shares exception checks between
Read()and POCO materialization. - Adds buffered lookahead and tagged closing-marker detection.
- Expands mock, compression, and live-server test coverage, and adds a changelog entry.
| File | Description |
|---|---|
ClickHouse.Driver/Formats/PooledReadBufferStream.cs |
Adds non-consuming buffered lookahead. |
ClickHouse.Driver/Formats/ExceptionTagAwareStream.cs |
Adds opening- and closing-marker detection and buffer parsing. |
ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs |
Checks for exception blocks at row boundaries and response completion. |
ClickHouse.Driver.Tests/Http/ResponseDecompressionTests.cs |
Tests exception detection through response compression codecs. |
ClickHouse.Driver.Tests/Formats/PooledReadBufferStreamTests.cs |
Tests lookahead across refills and end of stream. |
ClickHouse.Driver.Tests/Formats/ExceptionTagAwareStreamTests.cs |
Tests marker matching and ring-buffer wraparound. |
ClickHouse.Driver.Tests/ADO/MidStreamExceptionTests.cs |
Adds boundary cases and strengthens returned-row assertions. |
changelog.d/750-midstream-exception-end-of-body.fixes.md |
Records the reader’s exception-handling fix. |
Review effort: Balanced
There was a problem hiding this comment.
Auto-approved by the AI review gate on c7ba96e. Details in the approver's comment on this pull request.
AI review gate — auto-approved
History (latest 5 heads, newest first)
|
…ngelog claim docs/http.mdx: new "Errors after the first rows" section under the ADO.NET reader (row-boundary check, clean end of the response, and its limits: blocks that do not start at a row boundary, the 4 KiB tail, and the wait for a row whose bytes so far match the start of the block), linked from POCO reads. changelog.d: limit the claim to blocks that start at a row boundary with the server's framing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Auto-approved by the AI review gate on b8b7bb7. Details in the approver's comment on this pull request.

Summary
Fixes #750.
When a query fails after the server has sent the
200 OK, the server writes an in-band exception block into the body:\r\n__exception__\r\n<tag>\r\n<message>\n<length> <tag>\r\n__exception__\r\n(WriteBufferFromHTTPServerResponse::cancelWithException), where<tag>is the value of theX-ClickHouse-Exception-Tagheader.ClickHouseDataReaderlooked for this block only incatch (IOException). So:UInt8,Int8,Bool,Enum8), and for fixed-width rows when the block length is a multiple of the row width. Today a real server breaks the chunked body after the block, soHttpClientthrowsHttpIOException, but a proxy that re-frames the body ends it cleanly.main, the reader returns 78 rows decoded from the block before the exception.Changes
ClickHouseDataReader:Read()and the POCO fast pathTryMaterializeNextRow<T>()now use one helper,HasNextRow(), at each row boundary:<tag>\r\n__exception__\r\n), it throws theClickHouseServerException, parsed from the ring buffer ofExceptionTagAwareStream. When a long message has pushed the opening marker out of the 4 KiB ring buffer, it throws with the existing "Unknown error (could not parse exception message)" text instead of reporting a complete result. This check does not allocate unless the closing marker is there.\r\n__exception__\r\n<tag>), it reads the rest of the body (at most 16 KiB, the server'sMAX_EXCEPTION_SIZE), parses it and throws, so that no row is decoded from the block. A cheap inline test (\r, then\nin the buffer) comes first, so that most rows pay no call.ExceptionTagAwareStream:MatchOpeningMarker(exact prefix compare,None/Partial/Full),EndsWithClosingMarker(ring-buffer tail, no allocation), andTryParseMidStreamException(byte[])(the existing parse, on a given buffer).PooledReadBufferStream:NextBufferedByteandPeek(minimumCount), which compacts the buffer and reads more when the opening marker continues past the buffered bytes. It reads only while the bytes still match the marker. For a tagged response,FromHttpResponseAsyncmakes the read buffer at least as long as the marker (33 bytes), so that a very smallReadBufferSizedoes not turn the check off.Streaming trade-off: when the bytes of a row received so far are a prefix of the marker (for example a single
UInt8column whose value 13 is the last byte of a server write),Read()now returns that row only when the next bytes arrive.The row-boundary check accepts only the server's exact framing. A block written in another way (for example without the leading
\r\n, as in the existing mock fixtures), or a block that does not start at a row boundary, is still found by the lenient checks at the end of the body, but the rows decoded from it before that are returned.Test
New and strengthened tests fail on
main(19 of them) and pass with this change:MidStreamExceptionMockTests(mock responses, no server):Read_ExceptionBlockAfterLastRow_ReturnsOnlyTheRowsThenThrowsServerException(parametrized):UInt8rows; alignedInt32rows; a 64-byte read buffer that splits the marker after 3 bytes; the marker's\ras the last buffered byte; one byte per read (as a chunked body can give);ReadBufferSize1; a 6000-character message (longer than the ring buffer). Each asserts exactly the real rows, thenClickHouseServerExceptionwith code 395 and the exact message. Onmain: no exception.TryMaterializeNextRow_ExceptionBlockAfterLastRow_ReturnsOnlyTheRowsThenThrowsServerException: the POCO fast path.Read_ExceptionBlockNotAtRowBoundaryAndBodyEndsCleanly_ThrowsServerException(parametrized): block 1 byte into anInt32row; block without the leading CRLF; block inside a row with a message longer than the ring buffer. Onmain: no exception.Read_RowsThatLookLikeTheMarkerWithoutTheTag_AreReturnedAsData(contrast): rows that start like the marker, and a body that ends with__exception__, but with no tag. They stay data.ResponseDecompressionTests.FromHttpResponseAsync_WithCompressedExceptionBlockAfterLastRow_ReturnsOnlyTheRows: the same through each response codec.ShouldDetectMidStreamException_AfterResponseIsCommittedandQueryAsync_MidStreamExceptionAfterResponseIsCommitted_ThrowsClickHouseServerExceptionnow also assert that every row returned is a row the server sent (added assertions only). Onmain:row 200000 of 200078 is not a row the server sent(78 rows decoded from the block).MatchOpeningMarker,EndsWithClosingMarker(also across the end of the ring buffer) andPooledReadBufferStream.Peek.Full
ClickHouse.Driver.Testssuite on net10.0 against server 26.10.4.207: 14920 passed, 0 failed. The library also builds for net6.0 and net8.0.Performance
The row-boundary check adds work to every row. Measured with an ad-hoc Stopwatch harness (not committed): a mock response (no network),
Read()loop with no accessor calls, best of 9, Release, net10.0. A/benchmark-comparerun would confirm it against a server.mainUInt8rows, all 0UInt8rows, pseudo-randomInt64rows, pseudo-randomStringrows of 13 characters (each row starts with byte0x0D)UInt8rows, all 13 (\r): worst caseRun-to-run noise was about ±3%. The cost is per row, so it is smaller for rows with more columns. The end-of-body check runs once per result. If the per-row cost is not acceptable, the end-of-body check alone fixes the lost error (point 1) with no per-row cost; the row-boundary check (point 2) is separate code in
HasNextRowand can be removed.Related
#751 (a block longer than the 4 KiB ring buffer is lost): a block that starts at a row boundary is now read into its own buffer, so the real-server case of #751 gets the full message (checked against 26.10 with
throwIf(number = 200000, repeat('x', N)): N = 6000 and 15000 giveClickHouseServerExceptioncode 395 with the whole message;maingivesHttpIOException). A block that does not start at a row boundary, on a body that the server breaks, still uses the ring buffer, so this PR does not close #751.Our open PR #479 (in-band exception on the raw streaming path,
ClickHouseRawResult) also changesExceptionTagAwareStream.csandMidStreamExceptionTests.cs. Its new mode is opt-in and not used by the reader, so the two changes do not interact; the PR that merges second may need a text merge.Checklist
changelog.d/750-midstream-exception-end-of-body.fixes.md)Pre-PR validation gate
main)IOException)AGENTS.mdandClickHouse.Driver/AGENTS.md(changelog fragment, parametrized tests, measured per-row cost)🤖 Generated with Claude Code
Note
Medium Risk
Changes core row-reading logic for all tagged responses with a per-row peek cost; behavior is well-tested but affects every
Read()on failing queries after rows start streaming.Overview
Fixes #750 by teaching
ClickHouseDataReaderto detect ClickHouse’s in-band exception block on every row boundary, not only when anIOExceptionoccurs.Read()andTryMaterializeNextRow<T>()now callHasNextRow()instead of a simple end-of-stream peek. For responses withX-ClickHouse-Exception-Tag, the reader:\r\n__exception__\r\n<tag>) and throwsClickHouseServerExceptionbefore decoding rows from the block.EndsWithClosingMarker()and throws rather than treating a truncated failed query as success (e.g. after a proxy ends the response).Supporting changes:
ExceptionTagAwareStreamgainsMatchOpeningMarker,EndsWithClosingMarker, andTryParseMidStreamException.PooledReadBufferStreamaddsPeek/NextBufferedByte. Tagged responses bump the read buffer to at least the marker length. Docs and a changelog entry describe behavior and limits; tests cover mock HTTP, compression, POCO path, and live-server mid-stream failures.Reviewed by Cursor Bugbot for commit b8b7bb7. Bugbot is set up for automated code reviews on this repo. Configure here.