You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(cpp): batch skip for SPRINTZ (skip_exact_* still decodes per value) #1115
#1110 optimised SPRINTZ read_batch_int32/int64/float/double, but skip_int32/int64/float/double still use the base Decoder implementation, which calls the virtual read_* once per skipped value and throws the value away (cpp/src/encoding/decoder.h:104 onwards). Inside a decoded block this is pure overhead: the values are already reconstructed, and a skip only has to advance current_count_.
This is on a real reader path: when a time filter rejects rows, chunk_reader.cc skips the matching values with value_decoder_->skip_exact_int32(time_count, ...) (and the corresponding skip_exact_int64/float/double calls), so filtered queries over SPRINTZ value columns still pay one virtual call per skipped value.
Suggested fix: override skip_int32/int64/float/double on SprintzDecoder, mirroring the batch state machine in SprintzDecoder::batch_read() but without the copy, or decode a block and discard it when the whole block is skipped. Other codecs already do this, e.g. Gorilla's batch_skip_raw.
Correctness: skip counts and residual block state must match the per-value loop exactly, including the block-boundary behaviour documented in batch_read() (a full block's last value is only readable while the stream has data for INT32/INT64 but not for FLOAT/DOUBLE). A differential test against a per-value skip loop, in the style of the existing batch tests in cpp/test/encoding/sprintz_codec_test.cc, would cover it.
Activity