Repository navigation
fix(column): parse JSON typed paths that contain a space - #2072
spicysharkco wants to merge 3 commits into
Conversation
A backquoted typed path such as `a b` Int64 was split at its first space, giving path "a" and type "b` Int64". When the path is backquoted, split at the first space after the closing backtick. Fixes ClickHouse#1948
The server writes a backtick inside a backquoted path as \`. The closing-quote search stopped at that escaped backtick, so a path such as `c\`d e` was split inside its name, and splitWithDelimiters toggled on it too, so the entries after such a path were split wrongly. The path also kept the backslash, so a lookup by the real key c`d missed it. Skip escaped characters when looking for the closing backtick and when splitting the entries, and unescape the path as Tuple element names are.
| i++ | ||
| currentPart.WriteByte(s[i]) | ||
| } | ||
| case '`': |
There was a problem hiding this comment.
( and ) are counted into brackets even while inBackticks is true, so a quoted path containing a parenthesis stops every later top-level comma from splitting and the column fails with the same unsupported column type error this PR fixes.
- Trace:
JSON(`a(b` Int64, c String)yields one part,parsethen callsType("Int64, c String").Column(...)and returns an error. - The
SKIPbranch is affected the same way.
case '(':
- brackets++
+ if !inBackticks {
+ brackets++
+ }
currentPart.WriteByte(s[i])
case ')':
- brackets--
+ if !inBackticks {
+ brackets--
+ }Please add {"JSON(`a(b` Int64, c String)", []string{"a(b", "c"}} to TestJSONParseTypedPaths.
There was a problem hiding this comment.
Fixed in 43672cb: splitWithDelimiters now counts parentheses only outside backticks, which also covers the SKIP entries. TestJSONParseTypedPaths has your row JSON(`a(b` Int64, c String) and a second one with ) in a later path. Both failed before the change.
|
|
||
| typedPath := strings.Trim(typedPathParts[0], "`") | ||
| typeName := strings.TrimSpace(typedPathParts[1]) | ||
| typedPath := strings.Trim(typePart[:pathEnd+sep], "`") |
There was a problem hiding this comment.
💡 Nit — nit: typedPath is computed twice for quoted paths
The strings.Trim result is discarded whenever pathEnd > 0, so the quoted and unquoted cases read as one assignment that is then overwritten.
- typedPath := strings.Trim(typePart[:pathEnd+sep], "`")
- if pathEnd > 0 {
- typedPath = unescapeColName(typePart[:pathEnd+sep])
- }
+ var typedPath string
+ if pathEnd > 0 {
+ typedPath = unescapeColName(typePart[:pathEnd+sep])
+ } else {
+ typedPath = strings.Trim(typePart[:sep], "`")
+ }There was a problem hiding this comment.
Done in 43672cb: typedPath is now set in one branch.
🤖 Claude reviewFixes Key concerns (sibling paths of the same root cause):
Blind spots: the sandbox denied checking out or patching the branch, so I could not execute the new unit or integration tests; all findings rest on hand traces of the diff against the current Verdict: General findings
Inline comments are attached to the relevant lines. This summary updates in place on re-review. |
A parenthesis inside a backquoted path was counted as bracket depth in splitWithDelimiters, so no later comma split the entries and the type failed to parse, as in JSON(`a(b` Int64, c String). Count brackets only outside backticks. The SKIP branch trimmed backticks without unescaping, so SKIP `c\`d` was stored as c\`d and hasSkipPath missed the real path c`d. Unescape it as typed paths are. Build typedPath in one branch instead of overwriting it. Test rows added for each case.
|
Fixed in 43672cb: the |
Summary
Fixes #1948.
A JSON typed path with a space in its name, such as
JSON(`a b` Int64), made every query on that column fail withunsupported column type.(*JSON).parsesplit each entry at its first space, so the path becameaand the typeb` Int64. When the path is backquoted, the entry is now split at the first space after the closing backtick. An escaped backtick inside the path (\`) does not end it, and the path is unescaped the same wayTupleelement names are.Covered by
TestJSONParseTypedPathsand a native and HTTP regression test intests/issues/issue_1948_test.go. Both fail without the change.Named
Tupleelements are handled in #2074.Checklist
Note
Medium Risk
Changes core JSON type-string parsing used on every query against typed JSON columns; mistakes could mis-bind paths or types, though coverage is broad.
Overview
Fixes #1948: JSON column schemas like
JSON(`a b` Int64)no longer break withunsupported column typebecause parsing no longer splits path and type at the first space inside a backquoted path.(*JSON).parsenow finds the closing backtick via newquotedPathEnd(honoring\`escapes), splits on the first space after that quote, andunescapeColNamepaths the same way Tuple names are. SKIP entries get the same unescaping.splitWithDelimiterstreats backslashes inside quotes and ignores parentheses inside backticks so commas, dots, and nested types in path names parse correctly.Adds
TestJSONParseTypedPathsand a native/HTTP regression inissue_1948_test.gofor spaced and escaped-path JSON columns.Reviewed by Cursor Bugbot for commit 43672cb. Bugbot is set up for automated code reviews on this repo. Configure here.