You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Perf][C++] Avoid memcpy of the RLE pattern in read_length_and_bitwidth #1076
Int64RleDecoder::read_length_and_bitwidth / Int32RleDecoder::read_length_and_bitwidth
copy the whole encoded pattern into a temporary buffer (tmp_buf_) on every
pattern, then wrap byte_cache_ over that copy. Profiling a wide-table read
showed this memcpy at ~7.5% of the query.
Proposed changes
When the caller's ByteStream is wrapped and has at least length_ bytes left,
wrap byte_cache_ directly over the caller's buffer and advance its read
position; fall back to the copy for paged or short streams. The caller's stream
outlives the pattern decode, so no lifetime issue.
Resolved by #1078 (commit 1848341). read_length_and_bitwidth in the int32 and int64 RLE decoders no longer copies the bit-packing pattern into tmp_buf_: when the input ByteStream is a wrapped in-memory buffer it now reads through a read-only view (ByteStream::is_wrapped() + wrap_from() + wrapped_buf_advance_read_pos()), so the pattern decode reads the caller's page bytes in place. The copy remains only as a fallback for paged or too-short streams.
Activity
ColinLeeo commentedon Nov 16, 2026
Resolved by #1078 (commit 1848341).
read_length_and_bitwidthin the int32 and int64 RLE decoders no longer copies the bit-packing pattern intotmp_buf_: when the input ByteStream is a wrapped in-memory buffer it now reads through a read-only view (ByteStream::is_wrapped()+wrap_from()+wrapped_buf_advance_read_pos()), so the pattern decode reads the caller's page bytes in place. The copy remains only as a fallback for paged or too-short streams.Closing as completed in develop.