Skip to content

perf(column): scan Array(String) into []string without reflection - #2152

Open
TheNetherWatcher wants to merge 1 commit into
ClickHouse:mainfrom
TheNetherWatcher:perf/1747-array-string-scan
Open

TheNetherWatcher wants to merge 1 commit into
ClickHouse:mainfrom
TheNetherWatcher:perf/1747-array-string-scan

Conversation

@TheNetherWatcher

@TheNetherWatcher TheNetherWatcher commented Nov 21, 2026 •

Copy link
Copy Markdown

Summary

Fixes #1747.

Array.ScanRow goes through scanSlice for every element: Row boxes the string into any, then reflect.ValueOf, reflect.New, setJSONFieldValue and reflect.Append. For Array(String) into []string that is the main cost of the scan.

This adds a fast path in ScanRow for a single-level Array(String) and a *[]string destination. It makes the slice with the exact length and reads each value from the String column directly. All other types and destinations use the old path. I also moved the start/end offset lookup into a small helper so both paths use it.

Benchmark (lib/column, 64 rows of 100 elements, Apple M-series, -count 6):

ns/op ns/elem B/op allocs/op
before ~9110 ~91 11416 602
after ~828 ~8.3 2592 101

About 11x faster, and one allocation per element (the string copy) instead of six.

Checklist

  • Unit and integration tests covering the common scenarios were added

TestArrayStringScanRow checks that the fast path returns the same values as the reflection path (scan into any), including empty rows. BenchmarkArrayStringScanRow is the benchmark above. lib/column, and the Array/String tests in tests and tests/std pass against 26.9.


Note

Low Risk
Localized performance optimization with a type-specific branch and tests; other scan destinations and array shapes are unchanged.

Overview
Adds a reflection-free fast path in Array.ScanRow when scanning single-level Array(String) into *[]string: it uses offsetRange to find element bounds, allocates a []string of the right length, and copies values directly from the underlying String column.

Extracts offsetRange so scanSlice (and the new path) share the same start/end offset logic instead of duplicating it inline.

Adds TestArrayStringScanRow (fast path vs reflection/any) and BenchmarkArrayStringScanRow to track correctness and scan performance for this case; all other array types and destinations still use the existing reflection path.

Reviewed by Cursor Bugbot for commit e30d9c5. Bugbot is set up for automated code reviews on this repo. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scanning large arrays is very expensive

1 participant