Repository navigation
fix: decode control character escapes in Enum names - #2120
Merged
joe-clickhouse merged 4 commits intoNov 20, 2026
Merged
Conversation
The server writes control characters in Enum names as \b, \f, \n, \r, \t and \0 in the type name. The parser removed the backslash and kept the next byte, so 'a\nb' was read as "anb" and could not be appended with its real name. The name is now decoded into a buffer during the scan. This also removes the escape offsets, which were not reset between names and went out of range for names with more than one escape.
xmasdev
requested review from
chernser,
kavirajk and
polyglotAI-bot
as code owners
November 14, 2026 07:36
Contributor
polyglotAI-bot
approved these changes
Nov 20, 2026
polyglotAI-bot
left a comment
Collaborator
There was a problem hiding this comment.
Approve: no blocking problems. The one inline comment is non-blocking.
Test2119andTest1839fail onmain(panic:slice bounds out of range) and pass on this branch, on ClickHouse 26.10.- A probe round-trips a name with each byte 0x00–0xFF in
Enum8andEnum16, native and HTTP, and the server'shex()matches. Probes fordatabase/sql,Map,VariantandDynamicalso pass. - Not checked: servers older than 26.10. The CI test, lint and CodeQL runs wait for workflow approval.
golangci-lintv2.11 and./lib/...pass locally.
There was a problem hiding this comment.
Co-signing @polyglotAI-bot's approval of a48a18c: the AI review gate is green on this commit.
Contributor
AI review gate — Human approval in place of AI reviewApproved on the head commit by polyglotAI-bot (write access, not the author), standing in for the AI review:
What to do:
Evaluated at |
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #2119.
Fixes #1839.
The server writes control characters in
Enum8/Enum16names as escape sequences in the type name (\b,\f,\n,\r,\t,\0).extractEnumNamedValuesremoved the backslash and kept the next byte, so the name'a\nb'becameanb. A scan returned the wrong string,Append("a\nb")failed withunknown element, andAppend("anb")was accepted and storeda\nbon the server.The parser now decodes each name into a buffer while it scans the type, and it maps those six escapes to the control bytes. Other escaped bytes (
\\,\') are kept as they are, so'a\\nb'is still a backslash followed byn.Because the name is built during the scan, the
skippedValueTokensoffsets are gone. Those were not reset between names and were not adjusted after each removal, which is the bug in #1839. Onmainthe typeEnum8('a\'b\'c'=1,'d\'e'=2,'f'=3)panics withslice bounds out of range. So this change also covers what #1844 fixes. I don't want to step on that PR, and I can rebase on top of it if you prefer to merge it first.Checklist
TestExtractEnumNamedValueshas new cases for the control escapes, an escaped backslash, several escapes in one type and bytes the server does not escape.Test2119intests/issuesappends and scans the names withEnum8,Enum16,Array(Enum8)andNullable(Enum8)over native and HTTP. I ran it on 26.9. It fails onmainand passes with this change.Note
Medium Risk
Changes core Enum type parsing used for all Enum columns; behavior shifts for escaped names but is covered by new unit and integration tests.
Overview
Fixes Enum8/Enum16 type parsing so label names match what ClickHouse sends and accepts on append/scan.
extractEnumNamedValuesnow builds each name in a buffer while scanning the type string instead of slicing then stripping backslashes withskippedValueTokens. A newunescapeEnumCharmaps server escapes (\b,\f,\n,\r,\t,\0) to control bytes; other escapes like\\and\'stay literal. That corrects wrong names (e.g.'a\nb'→a\nbnotanb) and removes the buggy offset list that caused panics on enums with several escaped quotes (e.g.Enum8('a\'b\'c'=1,…)).Unit cases in
enum_test.gocover control chars, backslashes, and multi-quote types. Integration testsTest1839andTest2119exercise CAST/queries and round-trip Enum8/Enum16/Array/Nullable over native and HTTP.Reviewed by Cursor Bugbot for commit 34f7dde. Bugbot is set up for automated code reviews on this repo. Configure here.