Skip to content

Record deferred ZIP size and streaming compatibility findings - #1139

Merged
helly25 merged 9 commits into
mainfrom
docs/deferred-zip-compatibility-findings
Nov 7, 2026
Merged

helly25 merged 9 commits into
mainfrom
docs/deferred-zip-compatibility-findings

Conversation

@helly25

@helly25 helly25 commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Document why a simple streamed ZIP size fix would break successful listings, and retain exact reproductions for the remaining size defect and a separate bzip2 streaming-content bug. Production archive behavior is unchanged.

AG;DR

The current streamable ZIP reader exports zero for six-byte members whose local headers defer sizes to data descriptors. Skipping does not update the public libarchive entry size. Decoding all unknown payloads to count bytes succeeds for stored/deflated fixtures but rejects listings of an unsupported compression method, adds CRC failures, and exposes a valid bzip2 fixture that reads through the complete-memory API while failing through the streaming source API.

Six private investigative cases passed on both macOS and Linux using exact main 77aa9ef9, clang_release, Clang 23.1.4 and pinned libarchive 3.8.1. Exact eight fixture identities match across hosts. Python zipfile independently fully decodes and validates the six stored/deflate/bzip2 fixtures. The two Deflate64 fixtures have verified ZIP metadata and a verified common stored-block payload, but no independent Deflate64 decoder was run. No new decoder or licensing dependency is introduced.

The report retains the initial wrong bzip2 assumption and failure, corrected cases, exact fixture bytes in GoogleTest XML, source/configuration, both-host logs and independent validator in a 103735-byte lossless capsule of 19 hashed records. Every record was decoded and hash-checked. The temporary diagnostic is not added as a production acceptance test because it intentionally asserts current defects.

The implementation plan separates unknown versus empty sizes, exact metadata demand, identity-stable/budgeted random access, local-versus-central member association and sequential fallback. TODO keeps streamed-size correctness and O19b3 open and adds the bzip2 content issue. Release-site mappings expose the report and evidence.

Documentation pre-commit checks and Trunk passed without local clang-tidy; all 32 release-site Python tests passed on Mac. No runtime implementation, performance timings, full-suite/sanitizer validation or public-head CI result is claimed. Normal current-head publication validation remains required. No dependency on the pending publication repairs is required beyond preserving their tracking/configuration entries during integration.

Public integration: signed head 5228671a61bc291a0767dff22d17360d59e067c2 contains actual main 52c153549739546e02af7480d6d338096548528f (#1143). A normal combined merge resolved only the compatible release-site append conflict. Every main blob outside the exact five-file report/tracking/site delta is byte-identical, including all #1143 production code. Every existing main site mapping is preserved, with exactly the report page and raw-download entries added. Original report/capsule and canonical additions remain byte-exact; the private diagnostic BUILD rule and test remain excluded. Normal signed commit/push hooks passed; merge Trunk skipped normally and no actual local clang-tidy ran. All 32 integrated release-site tests passed under Python 3.13 after the synchronization push.

Fresh own Test run 42419580095 is collecting on this current post-#1143 head, including the user's requested new shipping benchmark measurements. Earlier green run 42408534169, on pre-#1143 head 1d6a623f, is preserved with its data and is not substituted for this new context. #1143's own full CI passed and its exact qualified tree/signature were verified on main; its post-merge Test 42419268471 is also monitored before the next merge.

The user's selected order is now #1139, #1140, #1141 and #1142. Each will synchronize after its predecessor merges and obtain fresh current-main benchmark measurements and required CI. The historical archive investigation capsule retains its original measured source and scope; its records are not relabeled as new runtime qualification.

Base automatically changed from fix/preserve-pending-main-benchmarks to main November 7, 2026 19:51
@helly25
helly25 merged commit 3e546db into main Nov 7, 2026
32 checks passed
@helly25
helly25 deleted the docs/deferred-zip-compatibility-findings branch November 7, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant