fix(cli): os migrate plan/apply examine the datasource a datasourceMapping default routes objects to, as os dev does

.changeset/24120-migrate-plan-declared-datasources.md

@@ -0,0 +1,15 @@
+---
+'@objectstack/cli': patch
+---
+
+fix(cli): `os migrate plan` and `os migrate apply` examine the objects a `datasourceMapping` rule routes to a declared datasource, the declared default included, in that datasource, as `os serve` connects it (#24120)
+
+Clause-②: no
+
+A project whose primary store is a declared datasource routed as the default (`datasources: [{ name: 'qa_main', … }]` with `datasourceMapping: [{ default: true, datasource: 'qa_main' }]`) got a plan that examined nothing. The plan resolved `qa_main`'s database as its primary, but it registered no driver under the name `qa_main`, so every object the rule routes there, platform objects included, was bound to no driver. The plan printed `Examined 0 managed table(s)` and a PARTIAL plan. Meanwhile `os dev` on the same project reported drift in that database and named `os migrate` as the remedy.
+
+- **Now:** `plan` and `apply` connect the project's declared **managed** datasources the way `os serve` connects them: through the same `datasource-connection` service, under the declared name, for the datasources a mapping rule routes objects to, an object binds to, or that set `autoConnect`. Each datasource that objects live in is diffed and applied beside the primary. A composition note names each one and its database, in both the human output and the `--json` output. On a `datasourceMapping`-default project, the plan's drift is the drift `os dev` reports there.
+- **Not connected:** a datasource whose schema the remote owns (`schemaMode: 'external'` or `'validate-only'`). Its objects are federated, and a plan has no table on it.
+- **Refused now:** a routed managed datasource that cannot connect fails `plan`, `apply` and `unmapped-columns` with the connect error `os serve` gives for it. It is no longer skipped with a PARTIAL plan. `OS_ALLOW_DRIVER_CONNECT_FAILURE` degrades it here as it does there.
+- **`os migrate unmapped-columns` follows:** it filters the plan's own findings, so it composes the same datasources. On such a project it reads an object's unmapped columns from the declared datasource. Before, it refused with "not in the set os migrate plan diffs: it is bound to no driver".
+- **Unchanged:** a project that declares no datasource gets the plan it got before. `os migrate security-catalog-overlays` and `organization-ownership` boot as before. The unmanaged-table sweep's `physicalTables` counts each database once, although the primary and a declared default resolved from it are two connections to one file.

packages/cli/src/commands/migrate/apply.ts

@@ -219,6 +219,10 @@ export default class MigrateApply extends Command {
         deferredDdlUnsupported: 'skip-schema-sync',
         composeHostStack: true,
         composeServedPlatform: true,
+        // [#24120] The plan's datasources too: each declared datasource a
+        // `datasourceMapping` rule routes objects to, connected as `os serve`
+        // connects it, reconciled where they live.
+        composeDeclaredDatasources: true,
       });
     } catch (error: any) {
       if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); }

packages/cli/src/commands/migrate/plan.declared-default-datasource.integration.test.ts

@@ -0,0 +1,206 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+// test-budget: 80 — spawns the real producer once per single-cause fixture: four spawns of the source CLI over one staged fixture (os dev to provision the declared datasource, os dev over the staged drift, os migrate plan --json, os migrate unmapped-columns --json); 56.1 s, the slower of two local runs on a shared container, x 1.25, no CI reading yet
+
+import { describe, it, expect, beforeAll, afterAll } from 'vitest';
+import { spawnSync } from 'node:child_process';
+import { mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';
+import { createRequire } from 'node:module';
+import { createServer } from 'node:net';
+import { tmpdir } from 'node:os';
+import { dirname, join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import Database from 'better-sqlite3';
+
+/**
+ * [#24120] On a project whose primary store is a DECLARED datasource that
+ * `datasourceMapping` routes as the default, `os migrate plan` examines the
+ * drift `os dev` reports — the same set, in the same database.
+ *
+ * ## The defect
+ *
+ * `datasources: [{ name: 'qa_main', … }]` with
+ * `datasourceMapping: [{ default: true, datasource: 'qa_main' }]`. `os dev`
+ * connects `qa_main` under its declared name (through the
+ * `datasource-connection` service `AppPlugin` connects declared datasources
+ * by), so every object — platform objects included — lives there, and its boot
+ * prints a `[schema-drift]` line per divergence, each naming `os migrate` as
+ * the remedy. `os migrate plan` resolved the same database FILE as its primary
+ * ("the project config's default datasource qa_main") but published no such
+ * service, so `qa_main` stayed metadata-only, every object routed to it was
+ * bound to no driver, and the plan printed `Examined 0 managed table(s)` and a
+ * PARTIAL plan with no drift at all.
+ *
+ * ## What is pinned
+ *
+ * Two drifts are staged in the declared datasource's database, one per kind of
+ * object the default rule routes: a column the app object no longer declares,
+ * and a column no platform object declares on `sys_user`. Then:
+ *
+ *  - the plan's drift set (`table.column` of each `changes` entry) EQUALS the
+ *    set of `[schema-drift]` lines `os dev` prints over the same database — and
+ *    holds both staged drifts, so the equality is not two empty sets;
+ *  - the plan examined every declared object (`unexaminedObjects: 0`) and
+ *    names who chose its database (`config-datasource`, `qa_main`);
+ *  - its unmanaged-table sweep counts that database's tables once, although
+ *    the primary and `qa_main` are two connections to the one file;
+ *  - `os migrate unmapped-columns`, a filter over the plan's own findings,
+ *    reads the platform object's unmapped column from the same datasource.
+ *
+ * `os dev` runs as the operator runs it, compiling the config, with
+ * `OS_MIGRATE_AND_EXIT=1` (its `serve --dev` child boots, syncs and exits) and
+ * `--no-restart` (so the parent exits with it). The datasource's filename is
+ * RELATIVE, as an author writes it: the primary resolves it against the project
+ * root, the declared connection opens it from the project directory.
+ */
+
+const HERE = dirname(fileURLToPath(import.meta.url));
+const CLI_ROOT = resolve(HERE, '..', '..', '..');
+const RUN_DEV = resolve(CLI_ROOT, 'bin', 'run-dev.js');
+const TSX = resolve(CLI_ROOT, '..', '..', 'node_modules', '.bin', 'tsx');
+const CASE_TIMEOUT_MS = 600_000;
+const SPAWN_TIMEOUT_MS = 240_000;
+
+const APP_OBJECT = 'fx24120_thing';
+const DATABASE = join('data', 'main.db');
+/** The `[schema-drift]` line the SQL driver prints at boot: `[schema-drift] TABLE.COLUMN: …`. */
+const DRIFT_LINE = /\[schema-drift\] ([a-z0-9_]+\.[a-z0-9_]+):/g;
+
+let dir: string;
+let port: number;
+
+/**
+ * The config, in the one shape `os build` accepts (`defineStack`). The project
+ * resolves `@objectstack/spec` through a link to the workspace package
+ * (`test/helpers/define-stack-fixture.ts` does the same, out of reach of this
+ * `src` program's `rootDir`).
+ */
+function writeConfig(fields: Record<string, unknown>): void {
+  const stack = {
+    manifest: { id: 'com.example.fx24120', name: 'fx24120', version: '0.0.0', type: 'app', engines: { protocol: '^18' } },
+    datasources: [{ name: 'qa_main', driver: 'sqlite', config: { filename: `./${DATABASE}` } }],
+    datasourceMapping: [{ default: true, datasource: 'qa_main' }],
+    objects: [{ name: APP_OBJECT, sharingModel: 'private', fields }],
+  };
+  writeFileSync(
+    join(dir, 'objectstack.config.ts'),
+    `import { defineStack } from '@objectstack/spec';\n\nexport default defineStack(${JSON.stringify(stack, null, 2)});\n`,
+  );
+}
+
+/**
+ * The child's environment: this process's, minus the families
+ * `test/helpers/serve-process.ts` `childEnv()` strips (its header says why),
+ * and nothing that names a database, a home, an artifact or a telemetry sibling.
+ */
+function env(extra: Record<string, string> = {}): Record<string, string | undefined> {
+  const inherited: Record<string, string | undefined> = {};
+  for (const [key, value] of Object.entries(process.env)) {
+    if (key === 'TEST' || key === 'VITEST' || key.startsWith('VITEST_') || key === 'NODE_PATH') continue;
+    inherited[key] = value;
+  }
+  return {
+    ...inherited,
+    OS_DATABASE_URL: undefined,
+    DATABASE_URL: undefined,
+    TURSO_DATABASE_URL: undefined,
+    OS_DATABASE_DRIVER: undefined,
+    OS_HOME: undefined,
+    OS_ARTIFACT_PATH: undefined,
+    OS_TELEMETRY_DB: '0',
+    OS_CLOUD_URL: 'off',
+    OS_TELEMETRY_DISABLED: '1',
+    OS_AUTH_SECRET: 'os24120-declared-default-secret-at-least-32-characters',
+    NO_COLOR: '1',
+    ...extra,
+  };
+}
+
+/** A port nothing listens on: the boot exits before it would listen, but `os dev` takes one. */
+async function freePort(): Promise<number> {
+  return new Promise((done, fail) => {
+    const server = createServer();
+    server.once('error', fail);
+    server.listen(0, '127.0.0.1', () => {
+      const address = server.address();
+      server.close(() => (address && typeof address === 'object' ? done(address.port) : fail(new Error('no port'))));
+    });
+  });
+}
+
+/**
+ * `os dev`, booted once and exited: its output. `--compile` because a config
+ * newer than its artifact is otherwise served STALE (with a warning), and the
+ * staged edit below is exactly such a config.
+ */
+function devBootOnce(): string {
+  const r = spawnSync(
+    TSX,
+    [RUN_DEV, 'dev', '--port', String(port), '--compile', '--no-watch', '--no-restart', '--no-seed-admin'],
+    { cwd: dir, env: env({ OS_MIGRATE_AND_EXIT: '1' }), encoding: 'utf8', timeout: SPAWN_TIMEOUT_MS },
+  );
+  const output = `${r.stdout}\n${r.stderr}`;
+  expect(r.status, `os dev did not boot and exit:\n${output.slice(-1500)}`).toBe(0);
+  return output;
+}
+
+/** One read-only `os migrate` door's `--json` document. */
+function migrateJson(args: string[]): any {
+  const r = spawnSync(TSX, [RUN_DEV, 'migrate', ...args, '--json'], {
+    cwd: dir, env: env(), encoding: 'utf8', timeout: SPAWN_TIMEOUT_MS,
+  });
+  expect(r.status, `os migrate ${args.join(' ')} failed:\n${`${r.stdout}\n${r.stderr}`.slice(-1500)}`).toBe(0);
+  return JSON.parse(r.stdout);
+}
+
+function tableCount(file: string): number {
+  const db = new Database(file, { readonly: true });
+  try {
+    return (db.prepare("SELECT COUNT(*) AS n FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'").get() as { n: number }).n;
+  } finally {
+    db.close();
+  }
+}
+
+beforeAll(async () => {
+  dir = mkdtempSync(join(tmpdir(), 'os-24120-declared-default-'));
+  const specRoot = dirname(createRequire(import.meta.url).resolve('@objectstack/spec/package.json'));
+  mkdirSync(join(dir, 'node_modules', '@objectstack'), { recursive: true });
+  symlinkSync(specRoot, join(dir, 'node_modules', '@objectstack', 'spec'), 'dir');
+  port = await freePort();
+});
+
+afterAll(() => {
+  if (dir) rmSync(dir, { recursive: true, force: true });
+});
+
+describe('os migrate plan on a datasourceMapping-default declared datasource (#24120)', () => {
+  it('examines the drift os dev reports there — the same set', () => {
+    // Provision: the first boot creates every table in the declared datasource.
+    writeConfig({ title: { type: 'text' }, extra: { type: 'text' } });
+    devBootOnce();
+
+    // Stage: the app object stops declaring `extra`, and `sys_user` (a platform
+    // object the same rule routes) gains a column no object declares.
+    writeConfig({ title: { type: 'text' } });
+    const db = new Database(join(dir, DATABASE));
+    try { db.exec('ALTER TABLE sys_user ADD COLUMN fx24120_legacy TEXT'); } finally { db.close(); }
+
+    const reported = new Set([...devBootOnce().matchAll(DRIFT_LINE)].map((m) => m[1]));
+    const plan = migrateJson(['plan']);
+    const planned = new Set((plan.changes as Array<{ table: string; column?: string }>).map((c) => `${c.table}.${c.column}`));
+
+    expect([...reported].sort()).toEqual(expect.arrayContaining([`${APP_OBJECT}.extra`, 'sys_user.fx24120_legacy']));
+    expect([...planned].sort()).toEqual([...reported].sort());
+    expect(plan.databaseSource).toEqual({ kind: 'config-datasource', datasource: 'qa_main' });
+    expect(plan.composition?.coverage?.unexaminedObjects, JSON.stringify(plan.composition?.coverage)).toBe(0);
+    expect(plan.managedTables).toBeGreaterThan(0);
+    expect(plan.unmanagedTables?.status, JSON.stringify(plan.unmanagedTables)).toBe('read');
+    expect(plan.unmanagedTables.physicalTables).toBe(tableCount(join(dir, DATABASE)));
+
+    // The door an unmapped column's finding leads to filters the plan's own
+    // findings over the same set: the platform object's column is read there,
+    // not refused as "not in the set os migrate plan diffs".
+    const unmapped = migrateJson(['unmapped-columns', '--object', 'sys_user']);
+    expect((unmapped.columns as Array<{ column: string }>).map((c) => c.column)).toEqual(['fx24120_legacy']);
+  }, CASE_TIMEOUT_MS);
+});

packages/cli/src/commands/migrate/plan.ts

@@ -19,6 +19,7 @@ import {
   describeWithApplyLimits,
   describeDatabaseSource,
   describeTelemetryDatabase,
+  distinctDatabaseDrivers,
   renderPlan,
   renderPendingSchemaWork,
   summarize,
@@ -185,6 +186,11 @@ export default class MigratePlan extends Command {
         // resolves — or an app declaring `requires: ['auth']` plans without
         // `sys_account`, and its retired `issuer` column is never a drop.
         composeServedPlatform: true,
+        // [#24120] …and the datasources the deployment declares, connected as
+        // `os serve` connects them: a `datasourceMapping` rule's objects are
+        // planned in the datasource it routes them to, the declared default's
+        // included. `os migrate apply` passes the same.
+        composeDeclaredDatasources: true,
       });
     } catch (error: any) {
       if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); }
@@ -221,8 +227,10 @@ export default class MigratePlan extends Command {
       // indistinguishable from "never looked at". See `../../utils/unmanaged-tables.ts`.
       const { normalizeRows } = await import('@objectstack/metadata-protocol');
       const unmanagedTables: UnmanagedTablesReport = await collectUnmanagedTables({
-        // [#22580] Every database the drift above covers, the sibling included.
-        drivers: stack.drivers,
+        // [#22580] Every database the drift above covers, the sibling included —
+        // [#24120] each read once, though a declared default and the primary
+        // resolved from it are two connections to one file.
+        drivers: distinctDatabaseDrivers(stack.drivers),
         declaredObjects: stack.allObjects(),
         composition: stack.composition,
         normalize: normalizeRows,

packages/cli/src/commands/migrate/unmapped-columns.ts

@@ -297,6 +297,10 @@ export default class MigrateUnmappedColumns extends Command {
         readOnlyProbe: true,
         composeHostStack: true,
         composeServedPlatform: true,
+        // [#24120] …and the declared datasources the plan connects, so an
+        // object a `datasourceMapping` rule routes to one — the declared
+        // default's platform objects included — is in this set as in the plan's.
+        composeDeclaredDatasources: true,
       });
     } catch (error: any) {
       if (flags.json) {
@@ -340,9 +344,10 @@ export default class MigrateUnmappedColumns extends Command {
         'objectql',
       ) as { getDriverForObject?: (name: string) => unknown } | undefined;
       const bound = engine?.getDriverForObject?.(object);
-      // [#22579] Any driver the plan diffs: the primary, or the `telemetry`
+      // [#22579] Any driver the plan diffs: the primary, the `telemetry`
       // sibling a lifecycle-classed object lives in when the serving boot
-      // keeps one. The read goes to that driver, and the document names it.
+      // keeps one, or [#24120] the declared datasource a mapping rule routes
+      // it to. The read goes to that driver, and the document names it.
       const objectDriver = stack.drivers.find((d) => d === bound) ?? null;
       const outside =
         declared.external != null
@@ -358,9 +363,9 @@ export default class MigrateUnmappedColumns extends Command {
             'for it, and that is unmeasured, not clean. Refusing rather than answering empty work.',
         );
       }
-      // The sibling is the one other planned driver, and it is planned only
-      // with its file: a driver that is not the primary has a database named.
-      const database = objectDriver === stack.driver ? stack.dbLabel : stack.telemetryDatabase!;
+      // Every planned driver has a database named (`SchemaStack.databaseOf`),
+      // and `objectDriver` is one of them by the lookup above.
+      const database = stack.databaseOf(objectDriver)!;
 
       const reader = objectDriver as unknown as Partial<UnmappedColumnReader>;
       if (typeof reader.find !== 'function') {