Skip to content

feat: expand standard library lowerings and add GCC toolchain driver support - #249

Open
mcpe500 wants to merge 48 commits into
vercel-labs:mainfrom
mcpe500:feat/stdlib-lowerings-and-gcc-support
Open

feat: expand standard library lowerings and add GCC toolchain driver support#249
mcpe500 wants to merge 48 commits into
vercel-labs:mainfrom
mcpe500:feat/stdlib-lowerings-and-gcc-support

Conversation

@mcpe500

@mcpe500 mcpe500 commented Aug 28, 2026

Copy link
Copy Markdown

Summary

This PR expands standard library lowering support across several built-in APIs and adds SCRIPTC_CC=gcc toolchain driver support for Linux environments. All extended built-ins have been verified with differential tests ensuring 100% byte-for-byte identical output against Node.js and Bun.js.

Enhancements & Built-in Lowerings

  1. Array.* & Array.prototype.*:

    • Array.isArray(x) on dynamic and computed expression values.
    • Array.prototype.splice(start, deleteCount, ...items) with variadic element insertion.
    • Array.from(Set / Array / String) native iterable conversion.
    • Array.of(...items) static constructor.
  2. Set.* & Set.prototype.*:

    • new Set(iterable) supporting Array and Set iterable initializers.
    • Nullable and optional Set<T> | null and Set<T> | undefined union arms support.
  3. Object.*:

    • Object.fromEntries(iterable) supporting tuple entries and string row arrays.
    • Object.keys(), Object.values(), Object.entries().
  4. crypto.*:

    • crypto.timingSafeEqual(a, b) constant-time XOR comparison in runtime C.
    • Bare createHash(alg).update(data).digest() returning raw Buffer/Uint8Array.
  5. process.*:

    • process.memoryUsage() record lowering (rss, heapTotal, heapUsed, external, arrayBuffers).
  6. net.* & URL:

    • net.isIP(str), net.isIPv4(str), net.isIPv6(str) IP address validation.
    • URL.port property getter extraction.
  7. Toolchain Support:

    • Added SCRIPTC_CC=gcc driver support in native-toolchain.ts.

Verification

All 12 extended test suites pass with 100% byte-for-byte parity against Node.js (v26) and Bun.js (v1.4):

  • 1200 - Array.isArray Extended
  • 1201 - Array.splice with Items
  • 1202 - crypto.timingSafeEqual
  • 1203 - crypto bare digest()
  • 1204 - net.isIP / isIPv4 / isIPv6
  • 1205 - URL.port property
  • 1206 - process.memoryUsage()
  • 1207 - Array.from(Set / Array)
  • 1208 - Nullable Set/Date in Unions
  • 1209 - Array.of static method
  • 1210 - Object.fromEntries & Object.entries
  • 1211 - new Set with Array Iterable

@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

@mcpe500 is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

@vercel vercel Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

LLVM unionNewOwned/unionExtract route date union arms through the ref-counted-pointer path, emitting a double where LLVM expects a ptr and producing invalid IR for Date | null / Date | undefined unions.

Fix on Vercel

Comment thread packages/compiler/src/frontend/lowering/lower-calls.ts Outdated
mcpe500 added 21 commits August 28, 2026 09:29
URL.origin, os.freemem/loadavg, process.version, Function type,
new Date(any), new Set(iterable), Number(any/dyn), ReadonlyArray.includes,
Object.entries/values, mixed logical operators, catch property basics.

14 compiler/runtime files + 12 differential corpus tests.
Fork-compatible: differential stdout/stderr/exit byte-identical vs Node.

Co-authored-by: internal-model
- os.arch maps to process.arch via surfaces.ts (same runtime string)
- Promise.allSettled lowers via sequential await loop helper
  (%promise.allSettled) producing honest subset {status:string}[]
  sequential helper preserves order and always fulfills
  also supports dynamic island via jsOp when marshalable
- corpus 2727-os-arch and 2728-promise-allsettled differential PASS (gcc)

ai-core scriptc:dev blockers reduced: os.arch and allSettled now pass
… + fs shim callbacks

- vendor-archives: mbedtls host-arch compile used hardcoded clang; use
  SCRIPTC_CC like the quickjs path (fixes spawn clang ENOENT on gcc-only hosts)
- scr_island: SCRIPTC_VERBOSE=1 prints the engine exception .stack before the
  uncaught bridge swallows it (default output unchanged)
- scr_island: node:fs shim gains ReadStream/WriteStream constructors and
  callback-style readFile/writeFile/appendFile/mkdir/rm/realpath/exists
Comment thread packages/runtime/src/scr_lib.c Outdated
Comment thread packages/runtime/src/scr_island.c Outdated
Comment thread packages/compiler/src/frontend/lowering/lower-exprs.ts Outdated
ctate and others added 4 commits September 1, 2026 14:26
* feat(runtime): ship precompiled macOS artifacts

- Add a versioned macOS arm64 runtime pack with deterministic feature selection and artifact verification.
- Link LLVM helper objects against release-built runtime and vendor inputs without compiling user-machine C.
- Wire package publishing, cache validation, documentation, and full-gate coverage for the new path.

* fix(runtime): harden precompiled artifact handling

* fix(runtime): make precompiled artifacts reproducible

* fix(runtime): reject opaque linkers from executable cache

* fix(runtime): normalize precompiled archive metadata

* fix(runtime): harden precompiled artifact caching

* fix(runtime): close executable cache link race

* fix(runtime): stage verified pack artifacts

* fix(runtime): bracket helper cache inputs

* fix(runtime): preserve executable link identity

* fix(runtime): trace selected linker dependencies
Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>
Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>
ctate and others added 21 commits September 1, 2026 14:26
Co-authored-by: monteslu <423800+monteslu@users.noreply.github.com>
Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>
* feat: tree-shake executable runtime sections

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* test: run runtime tree-shaking checks in sandbox

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* fix(runtime): release lazy process caches

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* test: stabilize runtime tree-shaking size guard

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

---------

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>
…s#273)

Co-authored-by: euxaristia <25621994+euxaristia@users.noreply.github.com>
* feat(net): support reusePort server listen options

Co-authored-by: Flora90001 <163703040+Flora90001@users.noreply.github.com>

* fix(net): match reusePort boolean semantics

Co-authored-by: Flora90001 <163703040+Flora90001@users.noreply.github.com>

* feat(net): support reusePort server listen options

Co-authored-by: Flora90001 <163703040+Flora90001@users.noreply.github.com>

* test(net): close listeners on successful conflict probe

Co-authored-by: Flora90001 <163703040+Flora90001@users.noreply.github.com>

---------

Co-authored-by: Flora90001 <163703040+Flora90001@users.noreply.github.com>
* fix: index large unicode strings sparsely

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* fix: retain sparse anchors for long ascii prefixes

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* fix: skip sparse index for proven ASCII strings

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* fix: complete sparse UTF-16 string indexing

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* test: rebase static size contracts for sparse index

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* test: retain sparse checkpoints through ASCII prefix end

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* fix: retain sparse anchors across threshold append

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* fix: retain sparse indexes beyond cursor cache

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* fix: bound sparse string index residency

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* fix: isolate sparse string index residency

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>

* test: restore parseInt oracle outputs

* test: verify committed string oracle

- Regenerate the string case oracle with the active Node executable.\n- Compare the checked-in fixture byte-for-byte before native runtime checks.\n- Keep the Linux parseInt ULP allowance scoped to native output.

---------

Co-authored-by: mmamedel <23098414+mmamedel@users.noreply.github.com>
* Separate native linking from C compilation

 - Split the helper-object runtime-pack path into explicit link planning and platform linking.
 - Reserve SCRIPTC_LINKER for object-only executable links while retaining a deprecated SCRIPTC_CC compatibility path.
 - Add no-legacy-C coverage and document the linker/SDK boundary.

* Preserve runtime pack link errors

* Fix linker cache invalidation and warnings

* Fix C executable cache routing

- Keep explicit C builds on compiler-backed cache identities.
- Cover linker-only environment changes across exact C repeats.
URL.origin, os.freemem/loadavg, process.version, Function type,
new Date(any), new Set(iterable), Number(any/dyn), ReadonlyArray.includes,
Object.entries/values, mixed logical operators, catch property basics.

14 compiler/runtime files + 12 differential corpus tests.
Fork-compatible: differential stdout/stderr/exit byte-identical vs Node.

Co-authored-by: internal-model
…lowlist, importStar)

- npm-static-rewrite: broaden __toESM recognizer to __importStar (tsc), __importDefault and __createRequire (babel) with per-helper shape verification and erasure; gate and ESM handling updated; JS_ONLY parity (import,node,default) documented
- npm-static: relax looksUnminified for allowlist qs/mime-db/semver (dist minified but lib readable); per-file degradedFiles map and npmStaticPerFileStatuses alongside package offenders; fs shadow now per-file island for bundler-interop degrade (with allowlist-aware package fallback for compat)
- cjs-lexer: broaden starAssign to handle __importStar/__importDefault/_interopRequireWildcard; add Object.defineProperties plural handling
- npm: broaden KNOWN_BUILTINS for Node22+ (sqlite, sea, test) and subpaths
- corpus 2730-cjs-interop.ts covering __toESM(require("express/lib/express")) broadened helpers

Owner files only: npm-static.ts, npm-static-rewrite.ts, cjs-lexer.ts, npm.ts
pnpm -r build PASS; SCRIPTC_CC=gcc corpus PASS
.gitignore 'dist/' swallowed node_modules/pg/dist so the fixture was
incomplete in git; rename to minified/ and repoint package.json main.
Replace sequential awaitExpr helpers with a concurrent promise.all.tuple
intrinsic that subscribes to every entry immediately. The first rejection
wins, slower fulfillments still run without unhandled rejections, and the
tuple record fills by input index preserving type order.

Also:
- Add scr_promise_all_tuple runtime combinator for C backend
- Add LLVM tuple thunks (store/finish/drop) via expr-callbacks
- Fix procStream scalar payload classification in C and LLVM emitters
- Add promise-all-heterogeneous-rejection differential test (2736)
- Update Promise.all fences diagnostics snapshot
- Remove the async-lifted-fn path for heterogeneous tuples
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants