Skip to content

sea: add vfsArchive to serve the assets from a ZIP archive - #65810

Open
mcollina wants to merge 2 commits into
nodejs:mainfrom
mcollina:sea-vfs-zip
Open

sea: add vfsArchive to serve the assets from a ZIP archive#65810
mcollina wants to merge 2 commits into
nodejs:mainfrom
mcollina:sea-vfs-zip

Conversation

@mcollina

@mcollina mcollina commented Sep 5, 2026

Copy link
Copy Markdown
Member

This adds a "vfsArchive" option to the SEA configuration that serves the bundled assets from a ZIP archive, and fixes a performance bug in the SEA asset lookup found while benchmarking it.

"vfsArchive"

Instead of listing individual "assets", the configuration can point at a prebuilt ZIP archive:

{
  "main": "main.js",
  "output": "app",
  "useVfs": true,
  "vfsArchive": "assets.zip"
}

--build-sea embeds the archive verbatim as a single reserved asset (only a PK signature sanity check happens at build time — no ZIP serialization logic is added to the build side). At runtime the SEA virtual file system mounts the existing ZipProvider over a zero-copy view of the embedded archive instead of the SEAProvider, and the main script is injected into the in-memory archive index as a stored entry, so the mounted tree looks exactly like plain "useVfs": __dirname-relative reads, relative require(), and bare specifier lookups are unchanged.

The archive can be produced with any ZIP tool or with the ZIP support in node:zlib (zlib.zipFiles()), which is also what the test does. "vfsArchive" requires "useVfs": true and cannot be combined with "assets". Since only the archive is embedded, sea.getAsset()/sea.getAssetAsBlob() do not serve the individual files; they are read through the fs APIs instead.

Perf fix: return the SEA resource by reference

FindSingleExecutableResource() returned the deserialized SeaResource by value, copying the whole assets map on every call. getAsset() calls it once per asset read, making every fs operation served by the SEA VFS pay a cost linear in the number of bundled assets: with 8192 assets, reading each of them took ~7.9s instead of ~270ms (~30x). Fixed in the first commit by returning a reference to the cached resource.

Size / speed tradeoff

Measured with a 75% text / 25% random asset mix in 16KB files (Linux x64, warm cache, medians of 15 runs; scenarios: start the binary touching no asset, read one small asset, read every asset):

Assets Binary (plain → archive) Startup +Δ Read one +Δ Read all (plain → archive)
1MB 151.3MB → 150.6MB +12ms +14ms 50ms → 64ms
8MB 158.7MB → 153.0MB +11ms +12ms 69ms → 110ms
32MB 183.9MB → 161.2MB (−22.7MB) +12ms +19ms 115ms → 213ms
128MB 284.7MB → 193.9MB (−90.8MB, −32%) +8ms +11ms 269ms → 477ms
  • The startup penalty is a flat ~10ms independent of bundle size (loading the internal/zip machinery and parsing the central directory); nothing is inflated until a file is opened.
  • Reads from the archive are ~2.5x slower than the SEAProvider memcpy path; per-file stat cost is identical (VFS dispatch dominates).
  • Max RSS is lower with the archive for workloads that read a subset of the assets (119MB vs 198MB at the 128MB tier) because fewer executable pages are touched.

Rule of thumb: the archive is a clear win above roughly 30MB of compressible assets when a run reads a subset of them; plain "useVfs" remains better for small bundles or workloads that repeatedly read large assets.

——

AI written, humanly reviewed.

FindSingleExecutableResource() returned the deserialized SeaResource by
value, copying the whole assets map on every call. getAsset() calls it
once per asset read, which made every fs operation served by the SEA
virtual file system pay a cost linear in the number of bundled assets:
with 8192 assets, reading each of them took seconds instead of
milliseconds. Return a reference to the cached resource instead.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
When "vfsArchive" names a ZIP archive in the SEA configuration (with
"useVfs": true), --build-sea embeds the archive verbatim as one
reserved asset, and at runtime the SEA virtual file system mounts the
existing ZipProvider over a zero-copy view of the embedded archive
instead of the SEAProvider. The main script is injected into the
in-memory archive index as a stored entry, so the mounted tree looks
the same as with "assets".

The archive can be produced with any ZIP tool or with the ZIP support
in node:zlib; no ZIP serialization logic is added to the build side.
This trades asset read speed (entries are inflated when opened) for a
substantially smaller executable when the assets are compressible.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/loaders
  • @nodejs/single-executable
  • @nodejs/startup
  • @nodejs/vm

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Sep 5, 2026
@mcollina
mcollina marked this pull request as ready for review September 7, 2026 06:19
@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.74576% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.19%. Comparing base (ad67159) to head (0a39ff7).
⚠️ Report is 66 commits behind head on main.

Files with missing lines Patch % Lines
src/node_sea.cc 69.49% 13 Missing and 5 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65810      +/-   ##
==========================================
+ Coverage   90.14%   90.19%   +0.04%     
==========================================
  Files         769      771       +2     
  Lines      262968   264728    +1760     
  Branches    50060    50250     +190     
==========================================
+ Hits       237057   238763    +1706     
- Misses      16932    16957      +25     
- Partials     8979     9008      +29     
Files with missing lines Coverage Δ
lib/internal/vfs/sea.js 100.00% <100.00%> (ø)
src/module_wrap.cc 74.31% <100.00%> (+0.09%) ⬆️
src/node.cc 76.75% <100.00%> (+0.12%) ⬆️
src/node_contextify.cc 81.89% <100.00%> (-0.09%) ⬇️
src/node_sea.h 100.00% <ø> (ø)
src/node_sea.cc 87.98% <69.49%> (-1.80%) ⬇️

... and 45 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 7, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 7, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pipobscure

Copy link
Copy Markdown
Contributor

The CI is possibly hitting the same bug addressed in #65814 as this means SEA is looking for a package.json, etc. at the root of the archive.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants