tests: Add zstd:chunked pull integration test - #384
Merged
cgwalters merged 2 commits intoAug 26, 2026
Conversation
zstd:chunked layers are multi-frame zstd streams (one frame per chunk, plus skippable framing metadata), and gzip layers may legally contain multiple members. The decoders stop at the first frame or member boundary, so such layers import a truncated tar and fail with "unexpected EOF reading tar entry" (bootc-dev/bootc#2408). Enable multiple_members(true) on both decoders so the full stream decodes; libzstd skips skippable frames natively at any position. This is full-pull correctness only: nothing exploits the chunked TOC for partial pulls, which remains composefs#137. Raise the async-compression floor to 0.4.43, which fixed an infinite loop on a corrupt later frame with multiple_members enabled (Nullus157/async-compression#470); older versions could hang on a malformed blob once the flag is on. Generated-by: AI Signed-off-by: Andrew Dunn <andrew@dunn.dev>
zstd:chunked layers are inherently multi-frame (one zstd frame per
chunk, plus skippable TOC framing), so the decode fix ("oci: Decode
multi-frame compressed layers") was exactly the case this format
needed, but it shipped with no zstd:chunked-specific coverage. cgwalters flagged this gap on composefs#381 ("we are definitely
missing integration tests w/ zstd:chunked"), tracked as composefs#383.
Adds test_zstd_chunked_pull: builds the same deterministic OCI layout
tests::cli::test_oci_pull_and_inspect uses, recompresses it to
zstd:chunked with skopeo (--dest-force-compress-format, so the result
can't silently stay gzip), asserts the produced manifest carries the
zstd-chunked TOC annotation (so the fixture can't silently degrade to
plain zstd either), pulls it through the same cfsctl oci pull oci:
path the other OCI-layout tests use, and asserts the resulting
composefs image ID is byte-identical to OCI_LAYOUT_COMPOSEFS_ID, the
same content pulled as plain gzip. Before the multi-frame fix this
would either fail the import or reproduce a different ID, since the
decoder stopped at the first frame boundary.
Gated on skopeo being present (mirrors tests::old_format's
have_skopeo()); also skips gracefully if the installed skopeo's copy
rejects --dest-compress-format zstd:chunked.
Generated-by: AI
Signed-off-by: Andrew Dunn <andrew@dunn.dev>
andrewdunndev
force-pushed
the
test/zstd-chunked-integration
branch
from
August 25, 2026 00:45
47d0859 to
96f87f9
Compare
cgwalters
approved these changes
Aug 25, 2026
cgwalters
enabled auto-merge
August 25, 2026 14:25
github-merge-queue
Bot
removed this pull request from the merge queue due to no response for status checks
Aug 25, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to no response for status checks
Aug 25, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Aug 25, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the zstd:chunked integration coverage flagged in review on #381: build the same deterministic OCI layout
tests::clipins, recompress it to zstd:chunked with skopeo, assert the manifest really carries the zstd:chunked TOC annotation, pull it throughcfsctl oci pull oci:, and assert the imported composefs image ID is byte-identical to the same content pulled as plain gzip. This is regression coverage for the decode failure tracked in #383. Skips when skopeo is absent or its copy rejects zstd:chunked, mirroringtests::old_format.Verified in a VM with skopeo available: the test executes the real path (not the skip), fails with the multi-frame decode fix reverted, and passes on top of #381 with no change to the rest of the suite.
Based on #381, so the diff shows its commit until it merges; only the last commit is new here. I'll rebase onto main once #381 lands.
Generated-by: AI. I am familiar with the decode change under test and reviewed this carefully.