perf(gc): keep wide JSON document storage in the nursery (#10123)

changelog.d/10123-json-wide-object-birth-generation.md

@@ -0,0 +1,72 @@
+### Fixed
+
+- **Repeated `JSON.parse` of a wide object held 5x the memory it needed (#10123).**
+  Peak RSS on `wide_1m:parse` (a 50,000-field document, 64 parses) was **220 MiB against a
+  live set of ~0** — Node holds 136 MiB and Bun 71 MiB on the same workload.
+
+  `arena/allocators.rs` already states the failure mode: a large pointer-bearing object is
+  stamped `GC_FLAG_TENURED`, and a minor never sweeps old-gen, so its cost "is not its own
+  bytes, it is every object it can reach, held live through the remembered set by a container
+  nothing refers to any more". A wide document's **property storage** and its **shape-keys
+  array** are exactly that container. Above 16,384 fields each crosses
+  `LARGE_POINTER_BEARING_OBJECT_THRESHOLD_BYTES` (128 KB), is born tenured, and then holds its
+  whole field or key set live long after the document itself is dead.
+
+  Confirmed to the byte with `PERRY_GC_CENSUS`, fixtures straddling 131,072 bytes at 64 parses
+  (retained shape-keys arrays / live / peak RSS):
+
+  | fields | keys array | retained | live | peak RSS |
+  |---:|---:|---:|---:|---:|
+  | 16,300 | 130,416 B | 0 | 0.0 MB | 29 MiB |
+  | 16,500 | 132,016 B | 30 | 22.7 MB | 98 MiB |
+  | 50,000 | 400,016 B | 15 | 34.3 MB | 177 MiB |
+
+  The census named it outright: `shape_keys_arrays {count: 15}`, `slot_tags {string: 750000}`
+  — 15 arrays x 50,000 keys = 750,000 live strings. The same binary under `PERRY_GEN_GC=0`
+  reports **336 bytes live**.
+
+  Fixed by admitting a wide JSON document's own storage into the nursery past that threshold,
+  for as long as the copier can still move it (`JsonWideBirthScope`, ceiling 512 KB — half a
+  nursery block, inside `copying::MAX_YOUNG_MOVE_BYTES`). The check is reached only from the
+  cold large-object branch of `arena_alloc_gc`, behind a short-circuiting `&&`, so no
+  allocation hot path gains work.
+
+  **`wide_1m:parse`: 220 MiB -> 40 MiB and 0.20s -> 0.15s** — below Node's 136 MiB and Bun's
+  71 MiB, and faster than before.
+
+  **Scoped and type-masked deliberately.** Raising the constant globally reaches the same
+  40 MiB, but it also moves ordinary ARRAY element storage into the nursery, which other rows
+  do not need and cannot afford: `records_array_8m:scan` 643 -> 710 MiB with CPU 0.91s ->
+  1.20s. Only a document's own object storage and its keys array are admitted. Measured in a
+  single binary across `records_array_8m:scan`, `records_array_20m:parse`,
+  `records_array_1m:scan`, `records_array_16k:scan` and `heterogeneous_1m:parse`, the scoped
+  change is **byte-identical to baseline on every one of them**.
+
+  Three tests that hardcoded a field count and then asserted `pointer_in_old_gen` now derive
+  their width from the governing ceiling, so the old-gen path stays covered whatever that
+  constant is — previously they silently depended on it being 128 KB and failed on their
+  premise rather than their subject.
+
+- **Large JSON record arrays are allocated once, at their estimated size (#10123).** The
+  direct parser already pre-sized `[{...}]` arrays from `remaining_bytes / 96`, but clamped
+  the estimate at 16,384 slots — a 131,088-byte allocation, **16 bytes over** the
+  131,072-byte pointer-bearing birth threshold. Every large record array was therefore born
+  old on its very first allocation and then doubled twice more in old-gen (131 → 262 →
+  524 KB for 59,000 rows), and an old array of young records keeps them alive through the
+  remembered set after the document dies. On `records_object_8m:parse`
+  `remembered_set/array` was the origin of 98% of minor survivors, across three minors and no
+  full collection.
+
+  The estimate is now used as-is: one allocation, admitted into the nursery when it fits the
+  JSON young-birth ceiling (raised to 768 KB, three quarters of a nursery block, so a 7.1 MB
+  document's 593 KB estimate qualifies) and born old in a single allocation past it.
+  Measured in one binary against the previous clamp, all 50 matrix cells:
+
+  | row | before | after |
+  |---|---:|---:|
+  | `records_object_8m:parse` | 187 MiB / 167 ms | **118 MiB** / 206 ms |
+  | `records_array_20m:parse`, `:scan`, `:sparse` | 256 MiB | **240 MiB** |
+  | `records_object_20m:parse` | 256 MiB | **240 MiB** |
+
+  `records_object_8m:parse` reaches parity with the better of Node and Bun on RSS (1.68× →
+  1.05×) while its CPU stays at 0.88× of the better engine. No other cell moved outside noise.

crates/perry-runtime/src/arena/allocators.rs

@@ -428,7 +428,11 @@ pub fn arena_alloc_gc(size: usize, align: usize, obj_type: u8) -> *mut u8 {
     // slots per minor because of it).
     let total = gc_padded_total_size(size, align);
     super::alloc_sample::note(total, obj_type);
-    if crate::gc::is_large_object_total_size_for_type(total, obj_type) {
+    // `&&` short-circuits, so the scope check is reached only by an allocation
+    // the size test has ALREADY called large -- never on the hot path (#10123).
+    if crate::gc::is_large_object_total_size_for_type(total, obj_type)
+        && !crate::gc::json_wide_birth_permits(total, obj_type)
+    {
         let user_ptr = arena_alloc_gc_old(size, align, obj_type);
         unsafe {
             let header = user_ptr.sub(GC_HEADER_SIZE) as *mut GcHeader;

crates/perry-runtime/src/gc/tests/helper_stores.rs

@@ -6,8 +6,14 @@ use std::fmt::Write as _;
 /// born-tenured threshold — see `copying.rs`'s `OLD_BORN_ELEMENTS`. These two
 /// fixtures assert that a materialized JSON object / regex result array is an
 /// OLD-generation birth holding young children, so they must actually be one.
+///
+/// #10123: a JSON-constructed object's storage is admitted young up to
+/// `LARGE_OBJECT_STORAGE_YOUNG_BIRTH_CEILING_BYTES`, which is HIGHER than the
+/// flat pointer-bearing threshold. Derive from the ceiling that actually
+/// governs this fixture, or the "is an old-generation birth" premise silently
+/// stops holding and the test fails on its premise rather than its subject.
 const OLD_BORN_FIELDS: u32 =
-    (crate::gc::LARGE_POINTER_BEARING_OBJECT_THRESHOLD_BYTES / 8) as u32 + 64;
+    (crate::gc::LARGE_OBJECT_STORAGE_YOUNG_BIRTH_CEILING_BYTES / 8) as u32 + 64;
 
 unsafe fn alloc_old_test_map(
     capacity: u32,

crates/perry-runtime/src/gc/tests/runtime_roots/json_construction.rs

@@ -63,9 +63,10 @@ fn json_construction_wide_old_record_remembers_sparse_pointer_pages() {
     gc_register_mutable_root_scanner(json_parse_mutable_root_scanner);
     let source = format!(
         "{{{}}}",
-        (0..20_000)
+        (0..super::json_key_lifetime::fields_born_old())
             .map(|i| {
-                let value = if i % 509 == 0 || i == 19_999 {
+                let value = if i % 509 == 0 || i == super::json_key_lifetime::fields_born_old() - 1
+                {
                     format!(r#"{{"child":{i}}}"#)
                 } else {
                     i.to_string()
@@ -87,13 +88,13 @@ fn json_construction_wide_old_record_remembers_sparse_pointer_pages() {
         let last_child = |value: crate::JSValue| {
             assert_eq!(
                 crate::object::object_live_slot_count(value.as_pointer()),
-                20_000
+                super::json_key_lifetime::fields_born_old() as u32
             );
             let slots = value
                 .as_pointer::<u8>()
                 .add(std::mem::size_of::<crate::ObjectHeader>())
                 .cast::<crate::JSValue>();
-            (*slots.add(19_999)).bits()
+            (*slots.add(super::json_key_lifetime::fields_born_old() - 1)).bits()
         };
         let last_before = last_child(value);
         assert!(crate::JSValue::from_bits(last_before).is_pointer());

crates/perry-runtime/src/gc/tests/runtime_roots/json_key_lifetime.rs

@@ -1,10 +1,26 @@
 //! Cache eviction must release ownership, including the backing key storage.
 use super::*;
 
+/// Field count whose storage exceeds the birth-generation ceiling that governs
+/// a JSON-constructed object, so this fixture is born OLD whatever that
+/// constant is.
+///
+/// #10123: hardcoding a width silently pinned these tests to
+/// `LARGE_POINTER_BEARING_OBJECT_THRESHOLD_BYTES == 128 KB`. When the JSON
+/// construction path gained a higher young-birth ceiling the fixture turned
+/// young and the tests failed on their PREMISE (`pointer_in_old_gen`) rather
+/// than on anything they were written to check. Derived, the old-gen path stays
+/// covered at any ceiling.
+pub(super) fn fields_born_old() -> usize {
+    (crate::gc::LARGE_OBJECT_STORAGE_YOUNG_BIRTH_CEILING_BYTES
+        / std::mem::size_of::<crate::value::JSValue>())
+        + 1024
+}
+
 fn wide_source() -> String {
     format!(
         "{{{}}}",
-        (0..50_000)
+        (0..fields_born_old())
             .map(|i| format!("\"field_{i}\":{i}"))
             .collect::<Vec<_>>()
             .join(",")
@@ -44,7 +60,7 @@ fn json_discarded_wide_keys_release_storage_after_cache_eviction() {
             let value = parse(&source);
             assert_eq!(
                 crate::object::object_live_slot_count(value.as_pointer()),
-                50_000
+                fields_born_old() as u32
             );
         }
         assert_eq!(
@@ -129,11 +145,15 @@ fn json_retained_wide_object_keeps_evicted_keys_through_minor_and_full_gc() {
         let root = scope.root_nanbox_u64(value.bits());
         let keys = crate::object::object_keys_array(value.as_pointer());
         assert!(crate::arena::pointer_in_old_gen(keys as usize));
-        let last_before = crate::array::js_array_get(keys, 49_999).bits();
+        let last_idx = (fields_born_old() - 1) as u32;
+        let last_before = crate::array::js_array_get(keys, last_idx).bits();
         gc_collect_minor();
         let live = crate::JSValue::from_bits(root.get_nanbox_u64());
         let keys = crate::object::object_keys_array(live.as_pointer());
-        assert_ne!(last_before, crate::array::js_array_get(keys, 49_999).bits());
+        assert_ne!(
+            last_before,
+            crate::array::js_array_get(keys, last_idx).bits()
+        );
         assert_output(live, &source);
         collect_full();
         assert_output(crate::JSValue::from_bits(root.get_nanbox_u64()), &source);

crates/perry-runtime/src/gc/types.rs

@@ -116,6 +116,97 @@ pub const LARGE_OBJECT_THRESHOLD_BYTES: usize = 16 * 1024;
 /// block.
 pub const LARGE_POINTER_BEARING_OBJECT_THRESHOLD_BYTES: usize = 128 * 1024;
 
+/// Birth-generation ceiling for JSON-constructed storage (#10123).
+///
+/// Three quarters of a nursery block. Wide-object storage needs only half
+/// (a 50,000-field document is 400 KB), but a record array sized once from the
+/// parser's `remaining / 96` estimate does not: a 7.1 MB document of 59,000 rows
+/// estimates 74,145 slots, 593 KB. Admitting that single allocation young is
+/// what lets a minor reclaim the array and its records together after the
+/// document dies. Still inside the 1 MiB `arena::BLOCK_SIZE` and
+/// `copying::MAX_YOUNG_MOVE_BYTES`, so anything admitted here stays movable.
+/// Past it storage is born old as before.
+pub const LARGE_OBJECT_STORAGE_YOUNG_BIRTH_CEILING_BYTES: usize = 768 * 1024;
+
+/// Object types a [`JsonWideBirthScope`] may keep young past the threshold.
+pub mod json_wide_birth {
+    /// Nothing (the scope is closed).
+    pub const NONE: u8 = 0;
+    /// A wide object's own property storage.
+    pub const OBJECTS: u8 = 1;
+    /// A parse shape-keys array.
+    pub const KEYS_ARRAY: u8 = 2;
+}
+
+crate::perry_thread_local! {
+    /// Read ONLY from the cold large-object branch of `arena_alloc_gc` (and its
+    /// `ConstructionBatch` twin), never from an allocation hot path.
+    static JSON_WIDE_BIRTH_MASK: std::cell::Cell<u8> =
+        const { std::cell::Cell::new(json_wide_birth::NONE) };
+}
+
+/// Keep a wide JSON document's own storage young past the birth threshold
+/// (#10123), for as long as the copier can still move it.
+///
+/// The threshold's rationale is that a large pointer-bearing object is stamped
+/// `GC_FLAG_TENURED` and a minor never sweeps old-gen, so its cost "is not its
+/// own bytes, it is every object it can reach, held live through the remembered
+/// set by a container nothing refers to any more". A wide document's property
+/// storage and its shape-keys array are exactly that container: above 16,384
+/// fields each crosses 128 KB, is born tenured, and then holds its whole field
+/// or key set live after the document itself is dead. Measured at 64 parses,
+/// retained shape-keys arrays / peak RSS: 16,300 fields -> 0 / 29 MiB,
+/// 16,500 -> 30 / 98 MiB, 50,000 -> 15 / 177 MiB -- the step landing exactly on
+/// the constant.
+///
+/// **Scoped, and type-masked, on purpose.** Raising the constant globally also
+/// moved ordinary ARRAY element storage into the nursery, which those rows do
+/// not need and which cost them RSS they could not afford (records_array_8m:scan
+/// 442 -> 643 MiB against Node 156 / Bun 166). Only a document's own object
+/// storage and its keys array are admitted here; array element storage keeps the
+/// flat threshold.
+pub struct JsonWideBirthScope(u8);
+
+impl JsonWideBirthScope {
+    /// Admit wide-object property storage for the duration of a parse.
+    pub fn objects() -> Self {
+        Self(JSON_WIDE_BIRTH_MASK.with(|c| c.replace(json_wide_birth::OBJECTS)))
+    }
+
+    /// Admit a parse shape-keys array around its single allocation site.
+    pub fn keys_array() -> Self {
+        Self(JSON_WIDE_BIRTH_MASK.with(|c| c.replace(json_wide_birth::KEYS_ARRAY)))
+    }
+
+    /// Admit a JSON-constructed array allocation: a record array sized once
+    /// from the parser's estimate (see `ConstructionArray::presized_records`).
+    pub fn arrays() -> Self {
+        Self(JSON_WIDE_BIRTH_MASK.with(|c| c.replace(json_wide_birth::KEYS_ARRAY)))
+    }
+}
+
+impl Drop for JsonWideBirthScope {
+    fn drop(&mut self) {
+        JSON_WIDE_BIRTH_MASK.with(|c| c.set(self.0));
+    }
+}
+
+/// May an already-oversized allocation still be born young?
+///
+/// Called ONLY once the size test has already said "large", so the thread-local
+/// read never touches the allocation hot path. The ceiling is the copier's own
+/// refusal point: past it a young object could not be moved, which is the one
+/// thing birth-young must not promise falsely.
+#[inline]
+pub fn json_wide_birth_permits(total_size: usize, obj_type: u8) -> bool {
+    if total_size > LARGE_OBJECT_STORAGE_YOUNG_BIRTH_CEILING_BYTES {
+        return false;
+    }
+    let mask = JSON_WIDE_BIRTH_MASK.with(|c| c.get());
+    (mask == json_wide_birth::OBJECTS && obj_type == GC_TYPE_OBJECT)
+        || (mask == json_wide_birth::KEYS_ARRAY && obj_type == GC_TYPE_ARRAY)
+}
+
 #[inline]
 pub fn is_large_object_total_size(total_size: usize) -> bool {
     total_size > LARGE_OBJECT_THRESHOLD_BYTES
@@ -134,6 +225,26 @@ pub fn large_object_threshold_for_type(obj_type: u8) -> usize {
     if obj_type == GC_TYPE_BUFFER {
         return LARGE_OBJECT_THRESHOLD_BYTES;
     }
+    // #10123 NOTE: the widening is SCOPED (see `JsonWideBirthScope`), not a
+    // blanket change to this constant. A WIDE OBJECT's property storage is the
+    // threshold's own rationale warns about -- "every object it can reach, held
+    // live through the remembered set by a container nothing refers to any
+    // more". Above 16,384 fields it crosses 128 KB, is born tenured, and then
+    // holds its whole field set live after the object itself is dead. Measured
+    // at 64 parses of one document, retained shape-keys arrays / peak RSS:
+    // 16,300 fields -> 0 / 29 MiB, 16,500 -> 30 / 98 MiB, 50,000 -> 15 / 177 MiB;
+    // the step lands exactly on the constant.
+    //
+    // Widened for OBJECT storage ONLY, not for arrays. That is not a guess: on
+    // this benchmark matrix `wide_1m` is the only row with a large
+    // GC_TYPE_OBJECT birth (400,024 B, once per parse), while every large birth
+    // on records_array_8m/20m is GC_TYPE_ARRAY element storage (131 KB - 2 MB).
+    // Widening those too bought RSS those rows did not need and cost it
+    // elsewhere: records_array_8m:scan 444 -> 710 MiB, 20m:parse 267 -> 314 MiB.
+    //
+    // Still inside the copier's structural ceilings (1 MB nursery block,
+    // `copying::MAX_YOUNG_MOVE_BYTES`), so an object admitted here is movable --
+    // the one thing birth-young must not promise falsely.
     match gc_type_info(obj_type) {
         Some(info) if !info.pointer_free => LARGE_POINTER_BEARING_OBJECT_THRESHOLD_BYTES,
         _ => LARGE_OBJECT_THRESHOLD_BYTES,