test: expand ecosystem assembly corpus and x86 vector coverage

[cpunion]

Summary

• Build on the external-library mechanism merged in test: discover and pin third-party assembly corpus #38 and add four exact latest module versions:
  • github.com/aead/siphash@v1.0.1
  • github.com/chain/txvm@v0.0.0-20190422181059-ff6bfbe53892
  • github.com/klauspost/crc32@v1.3.0
  • github.com/phuslu/log@v1.0.132
• Expand the strict external corpus from 24 to 28 modules, covering 11 targets and 272 target-specific assembly-file translations.
• Add x86 support found through TDD with real packages: scalar VMOVQ, immediate PSLLQ, all packed integer min/max families, and correct oversized-count PSRLQ semantics.
• Make the discovery index a module-path source: every distinct path is resolved through @latest before the exact-version seen check, including paths whose indexed historical version had no assembly.
• Replace the opaque gzip checkpoint with append-only, uncompressed JSONL runs split into 256 stable sha256(module)[0] shards. Successful no-assembly results are retained, failures remain retryable, and no .gz object remains in this PR's commit history.
• Reduce default remote ZIP tail prefetch from 8 MiB to 128 KiB while preserving the ZIP maximum-comment requirement and lazy central-directory reads.

TDD evidence

The external tests first reproduced unsupported forms in actual packages:

• siphash: PSLLQ
• crc32: VMOVQ
• phuslu/log: PMINUB

Native-Go-vs-LLVM22 conformance tests then verified the implemented behavior. The discovery changes also began with failing tests for an indexed no-assembly version whose latest version contains assembly, an indexed-version checkpoint incorrectly suppressing a latest scan, loss of reused match metadata, non-sharded output, and the 8 MiB ZIP prefetch. The prefetch regression observed 8,388,608 bytes before the fix and passes with the 128 KiB tail.

Discovery funnel

The converted historical checkpoint retains these original results. Exact-version inspections can fan out above the unique-path count because that scanner inspected indexed versions and selected resolved-latest versions.

Historical stage	Count	Result
Official default-index records read	300,000	Input window
Unique module paths represented by the scan	52,047	Deduplicated discovery paths
Completed exact module@version inspections	85,137	Includes versions without assembly
Assembly-bearing candidate versions	544	Match metadata retained
Retryable failures	121	Never marked complete

A clean replay of the same 300,000-record input size exercised the new all-path @latest behavior:

Current-latest stage	Count	Share / status
Unique module paths attempted through @latest	52,053	100% attempted
Exact latest versions successfully resolved and inspected	48,027	92.265575% of paths
Exact versions reused without another ZIP download	22,106	Subset of successful latest results
Current latest versions containing assembly	612	1.274283% of successful latest results
Newly discovered assembly-bearing modules vs. the old result	68	Caused by resolving every path to latest
Retryable latest-resolution failures	4,026	7.734425%; all are cached-only @latest 404s

Across both committed runs, exact-key deduplication leaves 111,058 successful module@version records, 52,053 module paths, 612 assembly-bearing candidates, and 4,147 unresolved exact failure keys. A later successful exact result suppresses the matching failure during ledger aggregation.

Global scope and completeness

A separate census of the official default Go module index used 33 contiguous, non-overlapping time partitions from 2019-04-10T00:00:00Z through the fixed cutoff 2026-09-13T10:49:35Z. Every partition completed, and module paths were deduplicated across partitions.

Scope	Available at cutoff	Covered by this PR	Coverage
Default-index module@version records	55,128,672	300,000	0.544181%
Unique default-index module paths	2,856,345	52,053	1.822364%

This is not a claim of complete pkg.go.dev or Go-module-universe coverage: it measures the public default index, not include=all, private modules, or modules never served by the Go services. Within the 52,053-path replay, every path was attempted, but the 4,026 cached-only 404s remain explicitly retryable. Candidate discovery is broader than the strict compilation corpus: all 612 matches are recorded, while this PR continuously compiles 28 curated issue- and coverage-driven libraries.

Issue-reported modules are covered as follows:

Issue	Exact module version	Discovery / CI status
llgo#2464	github.com/coder/websocket@v1.8.15	Not present in the first 300k index window; independently issue-seeded and green in the strict corpus
llgo#2552	github.com/klauspost/compress@v1.20.0	Discovered as amd64/arm64 assembly and green in the strict corpus
llgo#2576	github.com/tmthrgd/go-hex@v0.0.0-20190904060850-447a3041c3bc	Discovered as amd64 assembly and green in the strict corpus

Instruction coverage

Official Go assembler inputs were checked for Go 1.20 through Go 1.27 on amd64, arm64, 386, arm, and riscv64.

• 14 newly supported mnemonics: VMOVQ, PSLLQ, and the 12 signed/unsigned packed PMIN*/PMAX* byte, word, and dword families.
• 37 amd64 concrete forms move from unsupported to supported in every checked Go release.
• Supported official forms: 767 -> 804.
• Runtime-verified forms: 35 -> 51.
• Parser failures remain zero; no architecture is skipped.
• PSRLQ was already recognized, so its corrected oversized-count semantics are not counted among the 14 new mnemonics.

The external-library scan and compilation corpus intentionally uses Go 1.27 plus llc-22 only. The Go 1.20–1.27 matrix applies exclusively to the separate official Go toolchain/standard-library assembly corpus.

Validation

• go test ./... -count=1
• go test -race ./... -count=1
• go vet ./...
• nested command-module tests/vet and go mod verify in every module
• actionlint, shellcheck, and git diff --check
• total local statement coverage: 86.9%; discovery command: 77.2%
• full 28-module Go 1.27 / LLVM22 external corpus across all 11 targets (272 assembly-file translations)
• full Go 1.27 standard-library corpus across all 44 target/feature variants
• official Go 1.20–1.27 coverage reports for all five supported architectures
• native Go amd64 and LLVM22 executable semantic conformance
• committed sharded-ledger integrity, exact-key deduplication, retry preservation, and no-gzip checks

This PR intentionally remains Draft until the new CI run, Codecov patch coverage, and review all pass.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!