Repository navigation
fix(cpp): bound ZigZag page length and continuation bytes - #1128
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Malformed varint headers can trigger undefined behavior, and oversized encoded integers are silently decoded as zero instead of rejected as corruption.
2 open issues
What changed in this PR
This PR hardens Apache TsFile’s C++ ZigZag decoder against malformed page data and propagates decoding failures through read_int32 and read_int64.
Changes:
- Checks header-read errors and declared byte lengths before loading encoded data.
- Bounds continuation-group writes and detects exhausted value data.
- Adds malformed-stream regression tests and updates existing tests to assert read status.
| File | Description |
|---|---|
cpp/test/encoding/zigzag_codec_test.cc |
Adds corruption cases and migrates decoding assertions to status-returning APIs. |
cpp/test/encoding/encoding_coverage_test.cc |
Updates reset coverage to check read status and decoded values. |
cpp/src/encoding/zigzag_decoder.h |
Adds input and buffer bounds checks and propagates decoding errors. |
Review effort: Balanced
The ZigZag decoder trusted the page header's var-int byte count: it read that many bytes into a list one at a time, wrote one 7-bit group per continuation byte into a fixed-size buffer without a bound, and indexed the byte list after it ran out. Header var-ints were also parsed with an unbounded loop, and values wider than the target integer were silently truncated. Parse both header var-ints with a five-byte bound and at most four payload bits in the fifth byte; reject a declared length larger than the remaining input; cap groups at five for int32 and ten for int64; reject payload bits that do not fit into the target type; and treat an exhausted byte list as corruption. decode() now reports errors through read_int32/read_int64. Valid pages decode identically.
ColinLeeo
force-pushed
the
zigzag-bounds
branch
from
November 19, 2026 07:32
3e4ef71 to
91d7283
Compare
ColinLeeo
added a commit
that referenced
this pull request
Nov 20, 2026
The ZigZag decoder trusted the page header's var-int byte count: it read that many bytes into a list one at a time, wrote one 7-bit group per continuation byte into a fixed-size buffer without a bound, and indexed the byte list after it ran out. Header var-ints were also parsed with an unbounded loop, and values wider than the target integer were silently truncated. Parse both header var-ints with a five-byte bound and at most four payload bits in the fifth byte; reject a declared length larger than the remaining input; cap groups at five for int32 and ten for int64; reject payload bits that do not fit into the target type; and treat an exhausted byte list as corruption. decode() now reports errors through read_int32/read_int64. Valid pages decode identically. (cherry picked from commit 3e63eca)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

The ZigZag decoder trusted the page header's byte count: it read that many bytes into a list one at a time, wrote one 7-bit group per continuation byte into a 10-byte buffer without a bound, and indexed the byte list after it ran out. Validate both header var-ints, reject a declared length larger than the remaining input, stop at the ten groups an int64 value can carry, and treat an exhausted list as corruption.
decode()now reports errors throughread_int32/read_int64. Valid pages decode identically.Tests:
ZigzagDecoderTest.RejectsDeclaredLengthBeyondInput,RejectsValueRunningPastDeclaredLength,RejectsValueWithTooManyContinuationBytes,RejectsEmptyAndHeaderlessStreams. Full C++ suite passes.