fix(runtime)!: refuse the stored-metadata family evaluate shapes and serve write returns at the reader-context seams

.changeset/21454-reader-context-evaluate-refusals.md

@@ -0,0 +1,15 @@
+---
+'@objectstack/metadata-protocol': minor
+'@objectstack/runtime': minor
+---
+
+fix(runtime)!: the in-process reader contexts refuse the stored-metadata-body family's EVALUATE shapes and serve what a write returns, the way the generic data door does (#21454)
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) no metadata body, authorable key, spelling, export or stored shape moves; what changes is which query shapes the in-process reader contexts accept over the two stored-metadata tables, and the form in which a write's returned row is served, so `objectstack migrate meta` has nothing to rewrite. The other categories are closed on facts: both packages publish (not `unpublished`); no ADR-0087 id covers a refused query shape (not `registered` / `already-registered`); and the change is runtime behaviour, not a declaration (not `runtime-interface-only` / `type-surface-only`). -->
+
+**BREAKING**: this narrows what an action or hook body's object API, an action handler's scoped API and an action handler's engine handle accept when they read the two stored-metadata tables. A read there that filters, sorts or groups on the stored body column or on a content-hash column, a read that names one of those columns in an explicit search-field list, and a `count` carrying such a filter, ran before this release and now answer the generic data door's `400 INVALID_FIELD` before the query runs. The route: filter, sort, group and search those tables by their scalar columns (the type, the name, the state and the like), and read the bodies with a plain list, which is served projected — the body as its type's read projection, the content hash in keyed form. A default search with no field list is not refused: it is narrowed to the columns the door serves. Every other column of the two tables, and every other object, is unchanged. It ships as `minor` under the launch-window convention for accept-set narrowings.
+
+- **`@objectstack/metadata-protocol`** now exports the generic data door's four evaluate-refusal predicates — `storedMetadataBodyGroupingRefusal`, `storedMetadataBodyPredicateRefusal`, `storedMetadataHashEvaluateRefusal` and `storedMetadataSearchRefusal` — so the `@objectstack/runtime` reader-context seam refuses the same shapes through the door's own predicates rather than a second copy. Additive: nothing that imported the package before is changed.
+- **`@objectstack/runtime`** extends the stored-metadata reader-context seam (`ctx.api.object(...)` for action and hook bodies, a handler's `ctx.api`, and `ctx.engine.find`): a filter, sort, grouping or search that would evaluate the stored body or content hash of `sys_metadata` / `sys_metadata_history` is refused with the door's `INVALID_FIELD` / 400 before the query runs (a `count` with such a predicate included); a default `$search` is narrowed to the door's served field set rather than refused; and the row a write verb returns is served projected and keyed. The engine's own action verb (`ScopedRepo.execute`) is unreachable from a served body and is left untouched.

packages/metadata-protocol/src/index.ts

@@ -207,6 +207,20 @@ export {
 } from './metadata-redaction.js';
 export type { StoredHashDigest } from './metadata-redaction.js';
 
+// [#21454] The generic data door's EVALUATE refusals on the same family
+// (#21086 grouping, #21120 body filter / sort, #21207 content-hash evaluate and
+// search). Exported so the in-process reader contexts in `@objectstack/runtime`
+// refuse the evaluate shapes the way the door does — each through the door's
+// OWN predicate, never a copy: a second definition of which shapes leak a
+// stored body or hash is exactly the drift the family's one rule exists to
+// prevent.
+export {
+  storedMetadataBodyGroupingRefusal,
+  storedMetadataBodyPredicateRefusal,
+  storedMetadataHashEvaluateRefusal,
+  storedMetadataSearchRefusal,
+} from './metadata-redaction.js';
+
 export type { MetadataHostEngine } from './host-engine.js';
 
 // [#7560] ADR-0070's read-only-package rule. The authoring path (`saveMetaItem`

packages/runtime/src/stored-metadata-reader-contexts.pin.test.ts

@@ -118,6 +118,64 @@ const PIN_APP: any = {
         { name: 'handler_engine_reads_family', label: 'Handler engine read', type: 'script' },
         { name: 'handler_engine_reads_history', label: 'Handler engine history read', type: 'script' },
         { name: 'handler_api_reads_family', label: 'Handler api read', type: 'script' },
+        // [#21454] EVALUATE shapes — each body attempts to evaluate the stored
+        // body or hash, and each must be refused before the query runs. The
+        // VALUE in every predicate is an immaterial constant: the query is
+        // refused unrun, so nothing depends on what it is.
+        {
+          name: 'body_filters_body_column',
+          label: 'Body filters the body column',
+          type: 'script',
+          body: actionBody(`return { rows: await ctx.api.object('sys_metadata').find({ where: { metadata: { $contains: 'z' } } }) };`),
+        },
+        {
+          name: 'body_sorts_body_column',
+          label: 'Body sorts by the body column',
+          type: 'script',
+          body: actionBody(`return { rows: await ctx.api.object('sys_metadata').find({ orderBy: [{ field: 'metadata', order: 'asc' }] }) };`),
+        },
+        {
+          name: 'body_groups_body_column',
+          label: 'Body groups by the body column',
+          type: 'script',
+          body: actionBody(`return { rows: await ctx.api.object('sys_metadata_history').aggregate({ groupBy: ['metadata'] }) };`),
+        },
+        {
+          name: 'body_filters_hash_column',
+          label: 'Body filters the hash column',
+          type: 'script',
+          body: actionBody(`return { rows: await ctx.api.object('sys_metadata').find({ where: { checksum: 'z' } }) };`),
+        },
+        {
+          name: 'body_counts_body_column',
+          label: 'Body counts by the body column',
+          type: 'script',
+          body: actionBody(`return { n: await ctx.api.object('sys_metadata').count({ where: { metadata: { $contains: 'z' } } }) };`),
+        },
+        {
+          name: 'body_searches_body_column',
+          label: 'Body searches an explicit body column',
+          type: 'script',
+          body: actionBody(`return { rows: await ctx.api.object('sys_metadata').find({ search: 'z', searchFields: ['metadata'] }) };`),
+        },
+        // A DEFAULT search is narrowed, not refused: a body may still search a
+        // family table by its scalar columns, served like the door.
+        {
+          name: 'body_searches_default',
+          label: 'Body default search',
+          type: 'script',
+          body: actionBody(`return { rows: await ctx.api.object('sys_metadata').find({ search: '${DS_NAME}' }) };`),
+        },
+        // The engine action verb is not on a served body's surface at all.
+        {
+          name: 'body_calls_execute',
+          label: 'Body calls execute',
+          type: 'script',
+          body: actionBody(`return { typeofExecute: typeof ctx.api.object('sys_metadata').execute };`),
+        },
+        // ② the engine handle's evaluate shape — a handler whose ctx.engine.find
+        // filters the body column is refused the same way.
+        { name: 'handler_engine_filters_body', label: 'Handler engine filters body', type: 'script' },
       ],
     },
   ],
@@ -167,6 +225,13 @@ const PIN_HANDLER_PLUGIN: Plugin = {
       async (actionCtx: any) => ({ rows: await actionCtx.api.object('sys_metadata').find({ where }) }),
       'pin.reader21454.handler',
     );
+    // [#21454] ② the engine handle's evaluate shape: a filter on the body column.
+    ql.registerAction(
+      'pin_note',
+      'handler_engine_filters_body',
+      async (actionCtx: any) => ({ rows: await actionCtx.engine.find('sys_metadata', { where: { metadata: { $contains: 'z' } } }) }),
+      'pin.reader21454.handler',
+    );
   },
 };
 
@@ -461,3 +526,73 @@ describe('[#21454] ② / ③ an action handler\'s engine handle and scoped API',
     });
   }
 });
+
+describe('[#21454] the EVALUATE shapes are refused end to end, the door\'s own refusal', () => {
+  /**
+   * Each shape answers the data door's refusal (`INVALID_FIELD` / 400), names
+   * the offending column, and carries no family content. The envelope's precise
+   * `param` / `field` are pinned directly on the door's predicate in the unit
+   * test; across the sandbox boundary only `code`, `status` and the MESSAGE are
+   * guaranteed (`SANDBOX_ERROR_PASSTHROUGH`), so the column is read from the
+   * message, which both the sandboxed-body and the host-handler paths carry.
+   */
+  async function expectRefusedAction(action: string, token: string, column: string): Promise<void> {
+    const res = await as(token, 'POST', `/actions/pin_note/${action}`, { params: {} });
+    const payload = await readJson(res);
+    const err = payload?.error ?? payload;
+    const text = JSON.stringify(payload ?? null);
+    expect(res.status, `${action}: refused with 400`).toBe(400);
+    expect(err?.code, `${action}: the data door's INVALID_FIELD`).toBe('INVALID_FIELD');
+    const named = (Array.isArray(err?.fields) && err.fields.includes(column))
+      || String(err?.message ?? '').includes(`'${column}'`);
+    expect(named, `${action}: the refusal names '${column}'`).toBe(true);
+    expect(text.includes(SENTINEL), `${action}: the stored credential reached the answer`).toBe(false);
+    for (const h of storedHashes) expect(text.includes(h), `${action}: a stored hash reached the answer`).toBe(false);
+  }
+
+  for (const [role, token] of [['administrator', () => adminToken], ['member', () => memberToken]] as const) {
+    it(`a body's filter / sort / grouping on the body column, invoked by the ${role}`, async () => {
+      await expectRefusedAction('body_filters_body_column', token(), 'metadata');
+      await expectRefusedAction('body_sorts_body_column', token(), 'metadata');
+      await expectRefusedAction('body_groups_body_column', token(), 'metadata');
+    });
+
+    it(`a body's filter on a hash column, and count as an oracle, invoked by the ${role}`, async () => {
+      await expectRefusedAction('body_filters_hash_column', token(), 'checksum');
+      await expectRefusedAction('body_counts_body_column', token(), 'metadata');
+    });
+
+    it(`a body's explicit search of the body column, invoked by the ${role}`, async () => {
+      await expectRefusedAction('body_searches_body_column', token(), 'metadata');
+    });
+
+    it(`the engine handle's filter on the body column, invoked by the ${role}`, async () => {
+      await expectRefusedAction('handler_engine_filters_body', token(), 'metadata');
+    });
+  }
+});
+
+describe('[#21454] a DEFAULT search is narrowed to the door\'s served set, not refused', () => {
+  for (const [role, token] of [['administrator', () => adminToken], ['member', () => memberToken]] as const) {
+    it(`a body's default search runs and is served like the door, invoked by the ${role}`, async () => {
+      const res = await as(token(), 'POST', '/actions/pin_note/body_searches_default', { params: {} });
+      expect(res.status).toBe(200);
+      // It ran (not refused) and answered the family served, never the stored
+      // body or hash — the body and hash columns were removed from the scan.
+      expectServedLikeTheDoor(`default search (${role})`, await readJson(res));
+    });
+  }
+});
+
+describe('[#21454] the engine action verb is unreachable from a served body', () => {
+  it('a sandboxed body sees no `execute` on ctx.api.object(...)', async () => {
+    const res = await as(adminToken, 'POST', '/actions/pin_note/body_calls_execute', { params: {} });
+    expect(res.status).toBe(200);
+    const text = JSON.stringify(await readJson(res) ?? null);
+    // The VM bridge installs only find/findOne/count/aggregate and the writes;
+    // `execute` is not a function the body can call, so no raw scoped context
+    // is ever handed to a nested action through a served body. (Read from the
+    // response text, envelope-agnostic.)
+    expect(text).toContain('"typeofExecute":"undefined"');
+  });
+});