fix(metadata-protocol,cli): os migrate packageless-overlays lists a package-less view container cannot_declare, and --apply never attempts it

.changeset/30261-packageless-overlays-view-container.md

@@ -0,0 +1,14 @@
+---
+'@objectstack/metadata-protocol': minor
+'@objectstack/cli': patch
+---
+
+fix(metadata-protocol,cli): `os migrate packageless-overlays` lists a package-less view container `cannot_declare` instead of `movable`, and `--apply` no longer attempts (and fails) it on every run
+
+Clause-②: yes (widening)
+Metadata-protocol: semantic
+
+- **`@objectstack/metadata-protocol`: the census gains the disposition `cannot_declare`** (`PackagelessOverlayDisposition`, reachable through the exported `PackagelessOverlayCensus`). A row lands there when the schema the save door judges its body by refuses an `overrides` key on it, so the declared save the move makes could never land. Of the five overlay types the ceremony moves, that is a view container (`list` / `listViews` / `form` / `formViews`): only a single view declares an override. Before, such a row was listed `movable` and every `--apply` reported it `failed` with `Unrecognized key(s) on this view container: overrides`, leaving the row where it was. Now `migratePackagelessOverlays` does not attempt it: no outcome, no failure, no pair ensured for it.
+- **A view container's write onto a packaged name is accepted again.** `PUT /api/v1/meta/view/<object>` with a container body (`list` / `listViews` / `form` / `formViews`) over a name one installed read-only package ships, naming no package or naming that base with `?package=<base>`, and the matching `DELETE`, answered `403 NOT_OVERRIDABLE` with a fix (a declared save into the pair) that the container schema refuses. They now succeed, as they did before the ADR-0146 D11 closes; every other shape of the five overlay types is closed exactly as before.
+- **The two closes read the same answer.** The package-less write close (`packagelessCloseSubject`) closes only what the ceremony moves, so a container's package-less save and delete stay open, as they were before the close: they are the one door such a row keeps. The close of a write naming the read-only base (`overlayPairFixAdmits`, the "the fix can be taken" predicate both closes ask) no longer names the pair as the fix for a container body, because that fix (a declared save into the pair) cannot be taken; that write keeps the answer it had before the close.
+- **`@objectstack/cli`: `os migrate packageless-overlays`** prints the way out for a `cannot_declare` row under "left as they are": save each of the container's views into the pair as a single view named `<object>.<view>` (one the base ships declaring `overrides: { package, basedOn }`, a new one declaring nothing), then delete the container row with a delete naming no package (naming the base, for a row bound to it). In `--json`, the row carries `disposition: "cannot_declare"` and `outcome: "left"`, and `--apply` exits 0 for it.

content/docs/concepts/metadata-lifecycle.mdx

@@ -173,7 +173,7 @@ document **whole** for every read that serves the base's name
 
 > **ADR-0131 D6 — the per-organization overlay axis is retired.** The `/meta` doors (REST and the runtime dispatcher) carry no organization into a metadata write or read: a Studio or `PUT /api/v1/meta/...` write of the five ✅ types below lands **environment-wide** (since ADR-0131 D7 the `sys_metadata` family declares no organization column) and is served to every organization, and an organization admin holding only the retired `manage_org_presentation` capability is refused (`403`). The protocol itself refuses every organization-scoped write, and every read — item, list, layers, cached, history, diff, audit, drafts and the commit timeline — is environment → code, whatever organization a caller names. An overlay row an earlier release stored under an organization — under the `single` posture, every Studio save of these types, filed under the Default Organization — is served by no read, and on a SQL database (SQLite, PostgreSQL, MySQL) still holding one a v18 `os serve` refuses to start until the v18 migration ceremony (ADR-0131 C7: `os migrate organization-ownership --out PLAN`, then `--apply --plan-file PLAN`) has carried such a row to the environment layer and dropped the column; nothing deletes or rewrites it before then. The anonymous form doors read the environment alone too: the ceremony carries every organization-layer public-form withdrawal onto the environment, fail-closed. The table still records which types take an overlay of a shipped item.
 
-> **ADR-0146 D11 — the tier no longer opens a write door onto a packaged item.** A write naming no package — a save, a publish, a reset (`DELETE`) or a rollback — onto a ✅-type item that one installed read-only package ships is refused `403 NOT_OVERRIDABLE`, and the sentence names the base's pair: save it into `?package=<base>.custom`, declaring `overrides` ([above](#customizing-an-installed-package-the-paired-custom-override-adr-0146)). A package-less row stored before the close is still served until `os migrate packageless-overlays` moves it into the pair as a declared override; the ✅ row is now the set that ceremony moves and the read path honours for an unmoved row. A package-less write of a name no package ships is environment metadata, and stays open.
+> **ADR-0146 D11 — the tier no longer opens a write door onto a packaged item.** A write naming no package — a save, a publish, a reset (`DELETE`) or a rollback — onto a ✅-type item that one installed read-only package ships is refused `403 NOT_OVERRIDABLE`, and the sentence names the base's pair: save it into `?package=<base>.custom`, declaring `overrides` ([above](#customizing-an-installed-package-the-paired-custom-override-adr-0146)). A package-less row stored before the close is still served until `os migrate packageless-overlays` moves it into the pair as a declared override; the ✅ row is now the set that ceremony moves and the read path honours for an unmoved row. A package-less write of a name no package ships is environment metadata, and stays open. So does the package-less write of a view container (`list`, `listViews`, `form`, `formViews`): only a single view declares an override, so the ceremony lists the container and leaves it, and its views move into the pair one by one, each as a single view.
 
 | Type | `allowEnvironmentOverlay` | Rationale |
 | :--- | :---: | :--- |

content/docs/deployment/cli.mdx

@@ -1317,7 +1317,7 @@ where the data lives.
 | `os migrate comment-parent-pair` | Stamp the parent pair (`object_name`, `record_id`) on the `sys_comment` rows written before it existed. Until it runs, a broad comment read past 2,000 candidate rows omits comments the reader may see, and its total agrees with the short list, so run it at once after upgrading. No flag; idempotent |
 | `os migrate meta --stored` | Replay the metadata conversion chain over this deployment's `sys_metadata` rows and rewrite the ones still carrying a pre-protocol shape. Hygiene, not a gate — nothing depends on it having run |
 | `os migrate security-catalog-overlays` | List the environment-wide permission-set and position rows (the legacy plurals `permissions` / `positions` included) stored over a name a configured package holds — the rows a v18 cold boot refuses — each with the package that holds its name; `--apply` deletes them, one history tombstone per row. Run it before the first v18 boot, with the flags and environment the deployment boots with: the platform security plugin's sets are held only where `os serve` composes it (an auth secret set, and the `auth` tier on), and the step takes `serve`'s `--preset` and `--dev` with `serve`'s meaning. Adopts nothing; exits 1 while it lists a row |
-| `os migrate packageless-overlays` | List the package-less view, dashboard, report, translation and email-template rows stored over a name exactly one installed package ships — the customizations Setup saved before ADR-0146 D11 closed that write — and with `--apply` move each into that package's paired `<base>.custom` (created when absent) as a declared override, `basedOn` the base's installed version; the package-less row is deleted only after the moved one reads back as served. A name no package ships is environment metadata and is not touched; a name several packages ship, or one the pair already holds a row of, is listed and left with the reason. Takes `serve`'s `--preset` and `--dev`. Idempotent: a second run moves nothing. Run it once after upgrading |
+| `os migrate packageless-overlays` | List the package-less view, dashboard, report, translation and email-template rows stored over a name exactly one installed package ships — the customizations Setup saved before ADR-0146 D11 closed that write — and with `--apply` move each into that package's paired `<base>.custom` (created when absent) as a declared override, `basedOn` the base's installed version; the package-less row is deleted only after the moved one reads back as served. A name no package ships is environment metadata and is not touched; a name several packages ship, or one the pair already holds a row of, is listed and left with the reason. So is a view container (`list`, `listViews`, `form`, `formViews`): only a single view declares an override, so it is listed `cannot_declare` and never attempted, with the way out — save each of its views into the pair as a single view, then delete the container row. Takes `serve`'s `--preset` and `--dev`. Idempotent: a second run moves nothing. Run it once after upgrading |
 | `os migrate duplicates` | Report business identifiers already minted twice across the organization partitions, and the rows blocking the boot-time NULL-safe index tightenings — a read-only inventory as JSON on stdout. Renumbers nothing and writes no row and no schema; run it before the boot-time tenancy repair, which overwrites part of the evidence |
 
 **The boot itself writes no row and no schema you did not ask for.** Each of these commands boots

packages/cli/src/commands/migrate/packageless-overlays.integration.test.ts

@@ -102,9 +102,27 @@ const BOUND_SEED: SeedRow[] = [
   { type: 'view', name: ENV_VIEW, state: 'active', body: viewBody(ENV_VIEW, STORED_COLUMNS) },
 ];
 
+/**
+ * [#30261] A database whose customization of the package's view container (a
+ * list view the package does not ship added beside its default list) was
+ * stored as a package-less CONTAINER row, beside the environment row (control).
+ */
+const CONTAINER = 'm28413_task';
+const CONTAINER_SEED: SeedRow[] = [
+  {
+    type: 'view', name: CONTAINER, state: 'active', body: {
+      object: 'm28413_task',
+      list: { label: 'All', type: 'grid', data: { provider: 'object', object: 'm28413_task' }, columns: STORED_COLUMNS.map((field) => ({ field })) },
+      listViews: { mine: { label: 'Mine', type: 'grid', data: { provider: 'object', object: 'm28413_task' }, columns: DRAFT_COLUMNS.map((field) => ({ field })) } },
+    },
+  },
+  { type: 'view', name: ENV_VIEW, state: 'active', body: viewBody(ENV_VIEW, STORED_COLUMNS) },
+];
+
 let dir: string;
 let templateDb: string;
 let boundTemplateDb: string;
+let containerTemplateDb: string;
 const savedCwd = process.cwd();
 const ENV_KEYS = ['OS_ARTIFACT_PATH', 'OS_DATABASE_URL', 'DATABASE_URL', 'OS_DATABASE_DRIVER', 'OS_HOME', 'NODE_ENV',
   'OS_AUTH_SECRET', 'AUTH_SECRET', 'BETTER_AUTH_SECRET', 'OS_ENVIRONMENT_ID'] as const;
@@ -250,8 +268,11 @@ beforeAll(async () => {
   await seed.shutdown();
   boundTemplateDb = join(dir, 'bound-template.db');
   copyFileSync(templateDb, boundTemplateDb);
+  containerTemplateDb = join(dir, 'container-template.db');
+  copyFileSync(templateDb, containerTemplateDb);
   await insertRows(templateDb, SEED);
   await insertRows(boundTemplateDb, BOUND_SEED);
+  await insertRows(containerTemplateDb, CONTAINER_SEED);
 }, CASE_TIMEOUT_MS);
 
 afterAll(() => {
@@ -384,3 +405,34 @@ describe('os migrate packageless-overlays: a row bound to the read-only package'
     expect(again.payload.rows).toEqual([]);
   }, CASE_TIMEOUT_MS * 3);
 });
+
+/**
+ * [#30261] A package-less view CONTAINER row of the package's view: its shape
+ * refuses the `overrides` key, so the declared save `--apply` makes could
+ * never land. The preview lists it `cannot_declare` (left), and `--apply`
+ * attempts nothing: no failure, exit 0, the row and the database as they were.
+ * Before, the preview listed it movable and every `--apply` failed it.
+ */
+describe('os migrate packageless-overlays: a package-less view container', () => {
+  it('the preview lists it `cannot_declare`, left; --apply attempts nothing, fails nothing, and leaves the row', async () => {
+    const db = caseDb(containerTemplateDb);
+    const before = await storedRows(db);
+    const preview = await runJson(['--database-url', `file:${db}`, '--json']);
+    expect(preview.payload.error, JSON.stringify(preview.payload).slice(0, 600)).toBeUndefined();
+    expect(preview.exitCode).toBe(0);
+    expect(preview.payload.rows.map((r: any) => `${r.type}/${r.name}/${r.state} ${r.disposition} ${r.base} → ${r.pair} ${r.outcome}`)).toEqual([
+      `view/${CONTAINER}/active cannot_declare ${PKG} → ${PAIR} left`,
+    ]);
+    expect(preview.payload.movable).toBe(0);
+    expect(preview.payload.left).toBe(1);
+
+    const { payload, exitCode } = await runJson(['--apply', '--yes', '--database-url', `file:${db}`, '--json']);
+    expect(payload.error, JSON.stringify(payload).slice(0, 600)).toBeUndefined();
+    expect(exitCode).toBe(0);
+    expect(payload.failed).toBe(0);
+    expect(payload.moved).toBe(0);
+    expect(payload.rows.map((r: any) => `${r.name} ${r.disposition} ${r.outcome}`)).toEqual([`${CONTAINER} cannot_declare left`]);
+    expect(await storedRows(db)).toEqual(before);
+    expect(await storedPackages(db)).not.toContain(PAIR);
+  }, CASE_TIMEOUT_MS * 2);
+});

packages/cli/src/commands/migrate/packageless-overlays.ts

@@ -63,6 +63,17 @@ function reasonOf(row: CensusRow): string {
         + '(served then at that package\'s address, ?package=<base>).';
     case 'no_pair':
       return `'${row.base}' has no paired .custom package id, so there is nowhere to move it. Left as it is.`;
+    case 'cannot_declare':
+      // [#30261] Its shape refuses the `overrides` key, so the move's declared save can never land.
+      return (row.type === 'view'
+        ? 'it is a view container (list/listViews/form/formViews), and only a single view can declare an override, '
+        : 'its shape cannot carry an `overrides` declaration, ')
+        + 'so it cannot be moved as one. Left as it is: '
+        + (row.type === 'view'
+          ? `save each of its views into '${row.pair}' as a single view named <object>.<view> (?package=${row.pair}) — `
+            + `one '${row.base}' ships declaring overrides: { package: '${row.base}', basedOn }, a new one declaring nothing — `
+          : `save its content into '${row.pair}' in a shape that declares overrides: { package: '${row.base}', basedOn } — `)
+        + `then delete this row with a delete naming ${row.boundToBase ? `'${row.base}'` : 'no package'}.`;
     case 'foreign_declaration':
       return `its body already carries an \`overrides\` key that does not name '${row.base}'. Left as it is: `
         + (row.boundToBase

packages/metadata-protocol/src/protocol-packaged-base-refusal.ts

@@ -46,6 +46,7 @@ import {
 import { SysMetadataRepository } from './sys-metadata-repository.js';
 import { OBJECT_EXTENSION_TYPE } from './protocol-object-extension-item.js';
 import { customizationPairEnsureRefusal, type PairRegistry } from './protocol-customization-package.js';
+import { resolveOverlaySchema } from './protocol-meta-type-schema.js';
 import { resolvePackageClosure } from '@objectstack/lint/runtime';
 import { overlayMemberField, overlayMemberOfBody, overlayMemberValue } from './migrations/overlay-index.js';
 import {
@@ -462,10 +463,72 @@ export function overlayPairFixAdmits(
         ...(body ?? { name, ...(disc && member !== null ? { [disc]: member } : {}) }),
         overrides: { package: fix.base, basedOn: fix.basedOn },
     };
+    // [#30261] The save door's schema half: a body whose shape cannot carry
+    // the declaration (a view container) is refused there, whatever the gate
+    // answers. A bare identity has no shape to judge.
+    if (body !== undefined && overrideDeclarationRefusedByShape(type, body)) return false;
     const verdict = host.overrideWriteVerdict({ type, name, packageId: fix.pair, item: declaredBody }, fix.pendingPairScope);
     return verdict.verdict !== 'refuse';
 }
 
+/**
+ * [#30261, ADR-0146 D2, D11] Whether the schema the save door judges `body`
+ * by (`resolveOverlaySchema`, the door's own resolver) refuses an `overrides`
+ * key on it as unrecognized: the body's SHAPE cannot carry the override
+ * declaration, so no save declaring it lands, whatever the override write gate
+ * answers.
+ *
+ * Measured over the five overlay types the ceremony moves, one shape answers
+ * `true`: the view CONTAINER (`ViewSchema`, the `container` member of
+ * `ViewMetadataSchema`). A single view, a list or form overlay, a dashboard, a
+ * report, a translation and an email template each spread
+ * `OverrideDeclarationFields`. Read from the parse, not from a list of shapes,
+ * so a member that gains or loses the key is answered by its own schema.
+ *
+ * Only the refusal of the KEY is read: an `unrecognized_keys` issue naming
+ * `overrides` at the body's root. Inside a union it must be named by EVERY
+ * branch still diagnosing the body — the view union keeps the branch the body
+ * claims and mutes the others to a kind mismatch at the root
+ * (`focusClaimedBranch`), which is dropped here as every consumer of a failed
+ * union drops it — so a body that claims no branch, refused by the container
+ * member and by a member that carries the key, is not answered `true`. Every
+ * other finding on the body is the save door's to answer, as before, and
+ * answers `false` here. The body's own `overrides` value, if any, is replaced
+ * by a well-formed probe: the key alone decides.
+ */
+export function overrideDeclarationRefusedByShape(type: string, body: Record<string, unknown>): boolean {
+    const schema = resolveOverlaySchema(type, body);
+    if (!schema) return false;
+    const parsed = schema.safeParse({ ...body, overrides: OVERRIDES_KEY_PROBE });
+    return !parsed.success && parsed.error.issues.some(namesOverridesAtRoot);
+}
+
+/** A well-formed declaration: the probe asks whether the KEY is carried, so its value raises nothing. */
+const OVERRIDES_KEY_PROBE = Object.freeze({ package: 'com.example.base', basedOn: '1.0.0' });
+
+type ParseIssue = { code?: unknown; path?: unknown; keys?: unknown; errors?: unknown } | null;
+
+/** An issue at the parsed value's root: a `path` of length 0. */
+const atRoot = (issue: ParseIssue): boolean => Array.isArray(issue?.path) && issue.path.length === 0;
+
+/**
+ * An `unrecognized_keys` issue naming `overrides` at the parsed body's root;
+ * for a union, named by every branch whose issues are not a bare kind
+ * mismatch at the root (a muted branch, or a member of another kind).
+ */
+function namesOverridesAtRoot(issue: unknown): boolean {
+    const candidate = issue as ParseIssue;
+    if (candidate?.code === 'unrecognized_keys') {
+        return atRoot(candidate) && Array.isArray(candidate.keys) && candidate.keys.includes('overrides');
+    }
+    if (candidate?.code === 'invalid_union' && Array.isArray(candidate.errors)) {
+        const diagnosing = candidate.errors.filter((branch): branch is unknown[] => Array.isArray(branch)
+            && !branch.every((i) => (i as ParseIssue)?.code === 'invalid_type' && atRoot(i as ParseIssue)));
+        return diagnosing.length > 0 && diagnosing.every((branch) => branch.some(namesOverridesAtRoot));
+    }
+    return false;
+}
+
 /** The class members {@link namedBaseCloseSubject} reads, typed as the class declares them. */
 export interface NamedBaseCloseHost {
     readonly engine: ObjectStackProtocolImplementation['engine'];