feat(metadata-protocol)!: thread the installed-manifest lookup through every pair site; the ensure door creates no pair for a kernel package

.changeset/29594-pair-lookup-installed-kernel-verdict.md

@@ -0,0 +1,29 @@
+---
+'@objectstack/metadata-core': patch
+'@objectstack/metadata-protocol': patch
+'@objectstack/objectql': patch
+'@objectstack/runtime': patch
+'@objectstack/cli': patch
+'@objectstack/cloud-connection': patch
+'@objectstack/lint': patch
+'@objectstack/service-automation': patch
+'@objectstack/service-datasource': patch
+---
+
+feat(metadata-protocol, metadata-core)!: the ensure door creates no customization pair for a kernel package, and every pair question carries the installed manifests
+
+Clause-②: yes (narrowing) — both arms. Narrowing: `POST /api/v1/packages/:id/ensure-customization` (and the protocol's `ensureCustomizationPackage`) answers a kernel base, a package of scope `system` or `cloud`, with `409 RESOURCE_CONFLICT` and a sentence saying kernel objects are not customizable in this version, where it created `<id>.custom`; `POST /api/v1/automation/:name/clone` of a flow a kernel package ships, with no `package` named, answers that `409` where it answered `200` with the copy in `<id>.custom`; and the exported `pairOf`, `isPairOf` and `appOf` (`@objectstack/metadata-core`), `isServedOverrideRowOf` and the `installed` member of `OverrideWriteClosure` (`@objectstack/metadata-protocol`), and `resolveFlowPrecedence` / `describeFlowContender` (`@objectstack/service-automation`) take the installed-manifest lookup as a required argument. Widening: five new exports, `installedManifestsOf`, `InstalledManifestLookup`, `AppLookupManifest` and `InstalledPackageSource` (`@objectstack/metadata-core`) and `kernelPackageScope` (`@objectstack/metadata-protocol`), and one new `noticeCode` value on the permission-set clone answer, `PERMISSION_SET_CLONE_ENVIRONMENT_WIDE_KERNEL_SOURCE`, for a copy of a kernel-shipped set that now lands environment-wide.
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) no key of any schema is added, removed or re-shaped and no stored row is read differently, converted or dropped: the narrowing is one runtime refusal at one door, for one kind of base, whose remedy is that no pair exists to create; the existing pairs and the overrides in them stay served exactly as before. The retyped exports are runtime functions with no metadata surface, and the compiler names every caller that has not passed the new argument, so there is nothing for `objectstack migrate meta` to rewrite. -->
+
+**BREAKING** for a caller that ensures the customization pair of a kernel package — a platform package of scope `system` (users, organizations, security, settings and the rest of the platform suite) or `cloud` — and for a TypeScript importer of the pair helpers named above.
+
+- **Why (ruling records 6691022506 and 6691217245 on ADR-0146 D3).** The customization unit is the App, and the kernel is not an App a customer owns: it ships with the runtime, at the runtime's version (installing it whole is the goal ruling 6691022506 records, not yet what every `serve` tier does). In this version no kernel object is customizable, so the door creates no pair for one. Which kernel objects a later version opens (a whitelist) is that version's decision.
+- **What the door answers.** `409 RESOURCE_CONFLICT`, the status and code the door already answers when a base's state rules the pair out (a writable base, no namespace, a version that is not SemVer). Nothing is created or written. Every other base is answered exactly as before.
+- **A kernel pair that already exists stays.** An earlier version of the door may have created `<kernel id>.custom` on a deployment. It stays installed, the overrides inside it stay served by every read of the kernel item's name, and a further save into it is still admitted. Only the ensure call is refused.
+- **A copy of a platform permission set.** `POST /api/v1/security/permission-sets/:name/clone` with no `package` named landed the copy of a kernel-shipped set (`member_default` and the platform's other sets) in `<kernel id>.custom`. It now saves the copy environment-wide, as it already does for a set authored in the environment, with the notice code `PERMISSION_SET_CLONE_ENVIRONMENT_WIDE_KERNEL_SOURCE` and a sentence naming the kernel package and `package`. A Console that does not know the new code shows that sentence as sent. Naming a writable package with `package` still lands the copy there. The flow clone door answers a kernel-shipped flow with the ensure door's sentence.
+- **The closes stay shut on a kernel item.** A package-less write, or a write naming the kernel package, onto an item a kernel package ships is still refused `403 NOT_OVERRIDABLE` exactly where it was refused before (ADR-0146 D11). The sentence no longer names the pair as the fix, since there is none to ensure: it says kernel objects are not customizable in this version. A later version's kernel whitelist is what opens such an item.
+- **The lookup.** Which pair customizes a package is asked through the installed manifests of the registry the caller serves (`installedManifestsOf(registry)`). Today the answer is unchanged for every package: `appOf` is still the identity. The argument is threaded through every site now so that resolving a module to its App is one change in `appOf`.
+- **`os validate`'s customization-namespace advisory** now skips a package that is itself a pair through `isCustomizationPackage`, the one spelling of that question. An id ending in `.custom` that no base derives (`a.custom.custom`, a bare `.custom`) is now judged like any other package.
+- **Who moves, and the fix.** A script or client that ensured a kernel package's pair drops that call: the kernel item is served as it ships, and an existing pair keeps serving what it holds. A vendor package that declares `scope: 'system'` or `'cloud'` in its own manifest is read as kernel here, as `isWritablePackage` already reads it; its remedy is to drop the scope, which belongs to the platform's own packages. A TypeScript importer of the helpers passes `installedManifestsOf(engine.registry)`, or `installedManifestsOf(undefined)` where it holds no registry.

content/docs/api/metadata-api.mdx

@@ -111,7 +111,9 @@ capability changes only with a new version of the package that declares it. An
 sentence names that type's own route. What the reads then serve is in
 [Metadata lifecycle](/docs/concepts/metadata-lifecycle#customizing-an-installed-package-the-paired-custom-override-adr-0146).
 `POST /packages/hotcrm/ensure-customization` creates the pair when absent (Studio calls
-it on the first change), and `GET /packages/hotcrm/customizations` lists its overrides,
+it on the first change); a kernel package (scope `system` or `cloud`) has no pair in this
+version and answers `409 RESOURCE_CONFLICT`, though a pair an earlier version created keeps
+serving what it holds. `GET /packages/hotcrm/customizations` lists its overrides,
 extensions and switched-off items in one read, beside `contributions`: the fields (kind `field`)
 and flows (kind `flow`) other installed packages contribute to `hotcrm`'s objects, and the
 navigation items (kind `nav`) they contribute to `hotcrm`'s apps, each entry naming its package.

content/docs/concepts/metadata-lifecycle.mdx

@@ -182,7 +182,7 @@ document **whole** for every read that serves the base's name
 
 > **ADR-0131 D6 — the per-organization overlay axis is retired.** The `/meta` doors (REST and the runtime dispatcher) carry no organization into a metadata write or read: a Studio or `PUT /api/v1/meta/...` write of the five ✅ types below lands **environment-wide** (since ADR-0131 D7 the `sys_metadata` family declares no organization column) and is served to every organization, and an organization admin holding only the retired `manage_org_presentation` capability is refused (`403`). The protocol itself refuses every organization-scoped write, and every read — item, list, layers, cached, history, diff, audit, drafts and the commit timeline — is environment → code, whatever organization a caller names. An overlay row an earlier release stored under an organization — under the `single` posture, every Studio save of these types, filed under the Default Organization — is served by no read, and on a SQL database (SQLite, PostgreSQL, MySQL) still holding one a v18 `os serve` refuses to start until the v18 migration ceremony (ADR-0131 C7: `os migrate organization-ownership --out PLAN`, then `--apply --plan-file PLAN`) has carried such a row to the environment layer and dropped the column; nothing deletes or rewrites it before then. The anonymous form doors read the environment alone too: the ceremony carries every organization-layer public-form withdrawal onto the environment, fail-closed. The table still records which types take an overlay of a shipped item.
 
-> **ADR-0146 D11 — the tier no longer opens a write door onto a packaged item.** A write naming no package — a save, a publish, a reset (`DELETE`) or a rollback — onto a ✅-type item that one installed read-only package ships is refused `403 NOT_OVERRIDABLE`, and the sentence names the base's pair: save it into `?package=<base>.custom`, declaring `overrides` ([above](#customizing-an-installed-package-the-paired-custom-override-adr-0146)). A package-less row stored before the close is still served until `os migrate packageless-overlays` moves it into the pair as a declared override; the ✅ row is now the set that ceremony moves and the read path honours for an unmoved row. A package-less write of a name no package ships is environment metadata, and stays open. So does the package-less write of a view container (`list`, `listViews`, `form`, `formViews`): only a single view declares an override, so the ceremony lists the container and leaves it, and its views move into the pair one by one, each as a single view.
+> **ADR-0146 D11 — the tier no longer opens a write door onto a packaged item.** A write naming no package — a save, a publish, a reset (`DELETE`) or a rollback — onto a ✅-type item that one installed read-only package ships is refused `403 NOT_OVERRIDABLE`, and the sentence names the base's pair: save it into `?package=<base>.custom`, declaring `overrides` ([above](#customizing-an-installed-package-the-paired-custom-override-adr-0146)); for an item a kernel package (scope `system` or `cloud`) ships, the same write is refused with a sentence that names no pair, because kernel objects are not customizable in this version and a kernel package has no pair. A package-less row stored before the close is still served until `os migrate packageless-overlays` moves it into the pair as a declared override; the ✅ row is now the set that ceremony moves and the read path honours for an unmoved row. A package-less write of a name no package ships is environment metadata, and stays open. So does the package-less write of a view container (`list`, `listViews`, `form`, `formViews`): only a single view declares an override, so the ceremony lists the container and leaves it, and its views move into the pair one by one, each as a single view.
 
 | Type | `allowEnvironmentOverlay` | Rationale |
 | :--- | :---: | :--- |
@@ -213,9 +213,9 @@ See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/
 
 | Type | Sanctioned path the refusal names |
 | :--- | :--- |
-| `flow` | Declare an override from the base's paired `<base>.custom`, or rename it (a write from any other package is told to rename: only the pair declares an override), or switch it off (`POST /api/v1/automation/:name/toggle`, body `{enabled: false}`). The declared override is armed in the base flow's place, keeps its one activation switch, and a run suspended on the base finishes on the version it started on (ADR-0146 D9). A copy under a new name is "New similar", the clone door `POST /api/v1/automation/:name/clone` (body `{name, label}`), which the removal refusal still names. A flow that holds a credential (an inbound hook's `secret`, an `http` node's `signingSecret`) is not cloned in one step: the clone door refuses it with `409`, because a copy would share the secret, and the copy is authored as a new flow with its own. The copy is written switched off, into the source package's paired customization package `<base>.custom` (created on first use), or into the writable package the body's optional `package` names; a flow no read-only package ships has no pair, so its clone names `package`. The answer's `packageId` says where the copy landed ([ADR-0146](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0146-studio-designs-setup-administers-one-editor-per-item-paired-customize-package.md) D3, D4). |
+| `flow` | Declare an override from the base's paired `<base>.custom`, or rename it (a write from any other package is told to rename: only the pair declares an override), or switch it off (`POST /api/v1/automation/:name/toggle`, body `{enabled: false}`). The declared override is armed in the base flow's place, keeps its one activation switch, and a run suspended on the base finishes on the version it started on (ADR-0146 D9). A copy under a new name is "New similar", the clone door `POST /api/v1/automation/:name/clone` (body `{name, label}`), which the removal refusal still names. A flow that holds a credential (an inbound hook's `secret`, an `http` node's `signingSecret`) is not cloned in one step: the clone door refuses it with `409`, because a copy would share the secret, and the copy is authored as a new flow with its own. The copy is written switched off, into the source package's paired customization package `<base>.custom` (created on first use; a kernel package, of scope `system` or `cloud`, has no pair in this version, so a copy of a flow it ships answers the ensure door's `409` until the body names `package`), or into the writable package the body's optional `package` names; a flow no read-only package ships has no pair, so its clone names `package`. The answer's `packageId` says where the copy landed ([ADR-0146](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0146-studio-designs-setup-administers-one-editor-per-item-paired-customize-package.md) D3, D4). |
 | `action` | Switch it off (`POST /api/v1/actions/_activation/:object/:action`, body `{enabled: false}`; `:object` is `global` for an object-less action). No clone is offered. |
-| `permission` | Clone it under a new name: the "Clone" action on the permission set, or the clone door `POST /api/v1/security/permission-sets/:name/clone` (body `{name, label, package?}`) with a new name. The clone door copies the whole definition from the security catalog; a copy of a packaged set lands in the package's paired `<X>.custom` (ADR-0146 D3/D6). |
+| `permission` | Clone it under a new name: the "Clone" action on the permission set, or the clone door `POST /api/v1/security/permission-sets/:name/clone` (body `{name, label, package?}`) with a new name. The clone door copies the whole definition from the security catalog; a copy of a packaged set lands in the package's paired `<X>.custom`, and a copy of a set a kernel package ships lands environment-wide (ADR-0146 D3/D6). |
 
 A permission set and a position also have a switch, outside the refusal: `POST /api/v1/security/_activation/:type/:name` (`:type` = `permission` or `position`), body `{enabled: false}`, switches one off for the deployment. The refusal does not name it: a packaged position's refusal keeps the managed seal's edit remedy. The audience anchors `everyone` and `guest` are not switchable.
 

content/docs/permissions/permission-sets.mdx

@@ -453,7 +453,11 @@ the source never reach the copy.
   package `<X>.custom`, the package that holds everything you changed about X.
   The pair is created on the first copy, by the same ensure door as
   `POST /api/v1/packages/:id/ensure-customization` (its only creation path),
-  and reused after. A copy of a set already in a writable package is
+  and reused after. A copy of a set the platform itself ships (a kernel
+  package of scope `system` or `cloud`, such as `member_default`) is saved
+  environment-wide, in no package: a kernel package has no paired
+  customization package in this version, and the notice says so
+  (`noticeCode` `PERMISSION_SET_CLONE_ENVIRONMENT_WIDE_KERNEL_SOURCE`). A copy of a set already in a writable package is
   saved in that package, and a copy of a set authored in this system stays
   environment-wide, in no package. With `package`, the copy is saved there: it
   must be an installed, writable package (else `422`

packages/cli/src/utils/override-declarations.test.ts

@@ -14,6 +14,7 @@
 import { describe, expect, it } from 'vitest';
 import { judgeOverrideWrite } from '@objectstack/metadata-protocol';
 import { findOverrideDeclarationRefusals } from './override-declarations.js';
+import { installedManifestsOf } from '@objectstack/metadata-core';
 
 const BASE = 'com.acme.crm';
 const PAIR = 'com.acme.crm.custom';
@@ -33,7 +34,7 @@ const base = pkg(BASE, [], {
 });
 
 const judged = (type: string, name: string, packageId: string, item: unknown, dependencies: string[], shippers: string[]) => {
-  const v = judgeOverrideWrite({ type, name, packageId, item }, { dependencies, shippers });
+  const v = judgeOverrideWrite({ type, name, packageId, item }, { dependencies, shippers, installed: installedManifestsOf(undefined) });
   if (v.verdict !== 'refuse') throw new Error(`expected a refusal, got ${v.verdict}`);
   return v.message;
 };

packages/cli/src/utils/override-declarations.ts

@@ -38,6 +38,7 @@
 
 import { judgeOverrideWrite, overrideWriteItemsOf, type OverrideWriteItem } from '@objectstack/metadata-protocol';
 import { OverrideDeclarationSchema } from '@objectstack/spec/kernel';
+import type { InstalledManifestLookup } from '@objectstack/metadata-core';
 
 import { artifactPackages } from './artifact-packages.js';
 
@@ -56,6 +57,8 @@ export interface OverrideDeclarationRefusalRow {
 
 interface WalkedPackage {
   id: string;
+  /** The package's manifest as the artifact carries it. */
+  manifest: Record<string, unknown> | undefined;
   dependencies: string[];
   items: OverrideWriteItem[];
 }
@@ -75,14 +78,15 @@ function packagesOf(parsed: Record<string, unknown>): WalkedPackage[] {
   if (entries.length > 0) {
     return entries.map(({ index, id, body }) => ({
       id,
+      manifest: body,
       dependencies: dependencyIds(body),
       items: overrideWriteItemsOf(body, `packages[${index}].manifest.`),
     }));
   }
   const manifest = isRecord(parsed.manifest) ? parsed.manifest : undefined;
   const id = typeof manifest?.id === 'string' && manifest.id !== '' ? manifest.id : undefined;
   if (id === undefined) return [];
-  return [{ id, dependencies: dependencyIds(manifest), items: overrideWriteItemsOf(parsed, '') }];
+  return [{ id, manifest, dependencies: dependencyIds(manifest), items: overrideWriteItemsOf(parsed, '') }];
 }
 
 /** `id`'s dependency closure, followed through the packages this artifact carries. */
@@ -107,6 +111,9 @@ export function findOverrideDeclarationRefusals(parsed: Record<string, unknown>)
   const packages = packagesOf(parsed);
   const byId = new Map(packages.map((p) => [p.id, p]));
   const shipped = new Map(packages.map((p) => [p.id, new Set(p.items.map((i) => `${i.type}\u0000${i.identity}`))]));
+  // The manifests this artifact carries, as the pair lookup reads installed
+  // ones: who may declare an override of a base is its App's pair.
+  const installed: InstalledManifestLookup = (packageId) => byId.get(packageId)?.manifest;
   const rows: OverrideDeclarationRefusalRow[] = [];
   for (const pkg of packages) {
     const dependencies = closureOf(pkg.id, byId);
@@ -133,7 +140,7 @@ export function findOverrideDeclarationRefusals(parsed: Record<string, unknown>)
         (byId.get(baseId)?.items ?? []).filter((i) => i.type === walked.type).map((i) => i.item);
       const verdict = judgeOverrideWrite(
         { type: walked.type, name: walked.name, packageId: pkg.id, item: walked.item },
-        { dependencies, shippers, baseItemOf, baseItemsOf },
+        { dependencies, shippers, baseItemOf, baseItemsOf, installed },
       );
       if (verdict.verdict !== 'refuse') continue;
       rows.push({ path: walked.path, code: 'NOT_OVERRIDABLE', rule: verdict.rule, packageId: pkg.id, message: verdict.message });