Repository navigation
fix(json): consume v0 SharedData to keep native stream aligned - #2004
BetterAndBetterII wants to merge 8 commits into
Conversation
JSON deprecated object decoding only skipped SharedData offsets and left the Array(Tuple(String, String)) payload in the wire stream, which desynced later columns (ClickHouse#1854). Decode the full array and merge overflow paths into the scanned JSON object.
🤖 Claude reviewThe PR decodes the full JSON v0 SharedData column ( Key concerns:
Verified by reading:
Blind spots:
Verdict: Inline comments are attached to the relevant lines. This summary updates in place on re-review. |
AI review gate — AI review passedClean review from: Cursor Bugbot. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Reviewed by Cursor Bugbot for commit 40a6e7e. Configure here.
polyglotAI-bot
left a comment
There was a problem hiding this comment.
Approve: no blocking problems. The three inline comments are non-blocking.
TestIssue1854passes on ClickHouse 24.8, 25.3, 25.6, and 26.10. It fails onmainin all 12 cases (2 protocols, 2 APIs, 3 cases).- A read of two blocks, where only the second block has shared data, returns the first 5 rows and then the error in
rows.Err(). The next query on the same one-connection pool passes, on TCP and HTTP. - The open Bugbot finding on
TestDynamicDeprecatedSharedVariantErroris not correct. ch-goColStrwrites each string with a varint length, asPutStringdoes, and the test gets the new error (run confirmed).
|
Rejects unsupported legacy Final verdict✅ Approve Automated static review (no build or test run) by Polyglot AI (gpt-6-astra) of |
polyglotAI-bot
left a comment
There was a problem hiding this comment.
Review of a348b54 posted: see the summary comment and the inline threads.

Reject legacy Native
JSONshared data andDynamicSharedVariantvalues that the driver cannot decode. This includesDynamicvalues inside aJSONcolumn. Previously, reads could lose stream alignment or return a type-tagged binary value as a Go string.On ClickHouse 25.6 and later, set
output_format_native_use_flattened_dynamic_and_json_serialization=1to read these values. On older servers, readJSONas a string or castDynamicto a concrete type.The regression covers both Go APIs over Native TCP and HTTP. It checks the decode error, a subsequent query on the same handle, and the typed values with flattened output enabled.
Part of #1854. Enabling flattened output by default on 25.6+ is a follow-up.
Note
Medium Risk
Changes low-level Native column decode for experimental JSON/Dynamic types; behavior shifts from silent mis-reads to errors, which is safer but may surface failures in apps that previously got wrong data.
Overview
Legacy V1 Native decoding for
JSONandDynamicno longer skips or mis-handles shared storage. WhenJSONuses shared data (e.g. paths beyondmax_dynamic_paths) orDynamicuses SharedVariant, the driver returns a clear error instead of corrupting the native stream or surfacing binary garbage as strings.decodeSharedData_v1reads and validates per-row shared-data offsets; non-empty shared payloads triggerunsupportedSharedDataErrorwith guidance to enableoutput_format_native_use_flattened_dynamic_and_json_serialization=1on ClickHouse 25.6+, or toCASTin SQL on older servers. DynamicdecodeData_v1applies the same rejection when any row targets SharedVariant.Docs add a JSON and Dynamic section describing this behavior. Unit tests cover offset validation, empty roundtrips, and shared-variant paths; issue #1854 integration tests exercise both client APIs and Native/HTTP, including connection health after a failed scan and successful reads with flattened output enabled.
Reviewed by Cursor Bugbot for commit 53d2608. Bugbot is set up for automated code reviews on this repo. Configure here.