Skip to content

fix(cpp): bound ZigZag page length and continuation bytes - #1128

Merged
ColinLeeo merged 1 commit into
apache:developfrom
ColinLeeo:zigzag-bounds
Nov 19, 2026
Merged

ColinLeeo merged 1 commit into
apache:developfrom
ColinLeeo:zigzag-bounds

Conversation

@ColinLeeo

Copy link
Copy Markdown
Contributor

The ZigZag decoder trusted the page header's byte count: it read that many bytes into a list one at a time, wrote one 7-bit group per continuation byte into a 10-byte buffer without a bound, and indexed the byte list after it ran out. Validate both header var-ints, reject a declared length larger than the remaining input, stop at the ten groups an int64 value can carry, and treat an exhausted list as corruption. decode() now reports errors through read_int32/read_int64. Valid pages decode identically.

Tests: ZigzagDecoderTest.RejectsDeclaredLengthBeyondInput, RejectsValueRunningPastDeclaredLength, RejectsValueWithTooManyContinuationBytes, RejectsEmptyAndHeaderlessStreams. Full C++ suite passes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Malformed varint headers can trigger undefined behavior, and oversized encoded integers are silently decoded as zero instead of rejected as corruption.

2 open issues
What changed in this PR

This PR hardens Apache TsFile’s C++ ZigZag decoder against malformed page data and propagates decoding failures through read_int32 and read_int64.

Changes:

  • Checks header-read errors and declared byte lengths before loading encoded data.
  • Bounds continuation-group writes and detects exhausted value data.
  • Adds malformed-stream regression tests and updates existing tests to assert read status.
File Description
cpp/​test/​encoding/​zigzag_codec_test.cc Adds corruption cases and migrates decoding assertions to status-returning APIs.
cpp/​test/​encoding/​encoding_coverage_test.cc Updates reset coverage to check read status and decoded values.
cpp/​src/​encoding/​zigzag_decoder.h Adds input and buffer bounds checks and propagates decoding errors.

Review effort: Balanced

Comment thread cpp/src/encoding/zigzag_decoder.h Resolved
Comment thread cpp/src/encoding/zigzag_decoder.h Resolved
The ZigZag decoder trusted the page header's var-int byte count: it read
that many bytes into a list one at a time, wrote one 7-bit group per
continuation byte into a fixed-size buffer without a bound, and indexed
the byte list after it ran out. Header var-ints were also parsed with an
unbounded loop, and values wider than the target integer were silently
truncated.

Parse both header var-ints with a five-byte bound and at most four payload
bits in the fifth byte; reject a declared length larger than the remaining
input; cap groups at five for int32 and ten for int64; reject payload bits
that do not fit into the target type; and treat an exhausted byte list as
corruption. decode() now reports errors through read_int32/read_int64.
Valid pages decode identically.
@ColinLeeo
ColinLeeo merged commit 3e63eca into apache:develop Nov 19, 2026
58 of 59 checks passed
ColinLeeo added a commit that referenced this pull request Nov 20, 2026
The ZigZag decoder trusted the page header's var-int byte count: it read
that many bytes into a list one at a time, wrote one 7-bit group per
continuation byte into a fixed-size buffer without a bound, and indexed
the byte list after it ran out. Header var-ints were also parsed with an
unbounded loop, and values wider than the target integer were silently
truncated.

Parse both header var-ints with a five-byte bound and at most four payload
bits in the fifth byte; reject a declared length larger than the remaining
input; cap groups at five for int32 and ten for int64; reject payload bits
that do not fit into the target type; and treat an exhausted byte list as
corruption. decode() now reports errors through read_int32/read_int64.
Valid pages decode identically.

(cherry picked from commit 3e63eca)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants