Skip to content

fix: export large SmolVM artifacts through native pipes - #14

Draft
hudsonaikins wants to merge 1 commit into
mainfrom
codex/smolvm-artifact-export
Draft

hudsonaikins wants to merge 1 commit into
mainfrom
codex/smolvm-artifact-export

Conversation

@hudsonaikins

Copy link
Copy Markdown
Contributor

Valid report bundles above 11 MiB were truncated by SmolVM 1.31.1's capped --stream command-output path, producing tarfile.ReadError before BackIntel's 128 MiB artifact budget was reached. The same transport also limited large read_bytes calls.

Binary downloads now use SmolVM's existing native pipe mode (-i without a PTY), with stdin explicitly closed. The constrained guest wrapper, archive validation, byte/file limits, deadlines and partial-output cleanup remain intact. Ordinary command-output limits are unchanged. This fix targets main directly and has no dependency on the pending reliability PRs.

Validation:

  • 208 tests pass, 16 optional skips; Ruff lint/format, Vulture, Gitleaks and git diff --check pass.
  • Portable regressions fail on the original read/export paths and pass after the fix.
  • Real SmolVM: 25 MiB binary read/export and three-file 24 MiB export preserve every byte. Normal PNG/SVG/interactive HTML/XLSX, source/root write denial and symlink rejection pass. A 65 MiB sparse artifact under a 64 MiB profile is rejected with cleanup.
  • Retained original report: 10 files / 4,947,856 bytes. Retained combined reports: 32 files / 19,769,798 bytes. Every source-manifest SHA-256 matches after export.

No model calls or new dependencies. The previous taxi experiment remains historically failed; its deleted report/workbook is not claimed as recovered. Root cause and reproduction.

INT-498

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant