Skip to content

perf(cpp): batch skip for SPRINTZ (skip_exact_* still decodes per value) #1115

Open

Description

@ColinLeeo

Language

C++

Description

#1110 optimised SPRINTZ read_batch_int32/int64/float/double, but skip_int32/int64/float/double still use the base Decoder implementation, which calls the virtual read_* once per skipped value and throws the value away (cpp/src/encoding/decoder.h:104 onwards). Inside a decoded block this is pure overhead: the values are already reconstructed, and a skip only has to advance current_count_.

This is on a real reader path: when a time filter rejects rows, chunk_reader.cc skips the matching values with value_decoder_->skip_exact_int32(time_count, ...) (and the corresponding skip_exact_int64/float/double calls), so filtered queries over SPRINTZ value columns still pay one virtual call per skipped value.

Suggested fix: override skip_int32/int64/float/double on SprintzDecoder, mirroring the batch state machine in SprintzDecoder::batch_read() but without the copy, or decode a block and discard it when the whole block is skipped. Other codecs already do this, e.g. Gorilla's batch_skip_raw.

Correctness: skip counts and residual block state must match the per-value loop exactly, including the block-boundary behaviour documented in batch_read() (a full block's last value is only readable while the stream has data for INT32/INT64 but not for FLOAT/DOUBLE). A differential test against a per-value skip loop, in the style of the existing batch tests in cpp/test/encoding/sprintz_codec_test.cc, would cover it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions