Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .changeset/21644-narrowed-apply-flag.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
'@objectstack/cli': patch
'@objectstack/service-storage': patch
---

`os migrate value-shapes` and `os migrate files-to-references` record the deployment-level ADR-0104 flag only from a run over every object, and every command in the `os migrate` data-migration family refuses an `--object` name the deployment does not declare (#21644).

Clause-②: no

- **A narrowed `--apply` records no deployment flag.** The flag attests the stored data of every object and turns strict enforcement on, but a run narrowed by `--object` reads only the named objects. Such a run still applies its fixes: `files-to-references` converts the named objects' values. It records no flag, whether it passes or fails, and leaves a flag that an earlier full-scope run recorded exactly as it was. Its output says why and names the run that records the flag: the same command without `--object`. The `--json` document carries `filter: { objects }`, which is `null` on a full-scope run, so a narrowed run is never mistaken for a full one. Any `--object` narrows, even a list that names every object. A full-scope `--apply` records the flag as before.
- **`runFilesToReferencesMigration`** (`@objectstack/service-storage`) skips the flag write when it is given `objects`. That includes `[]`, which walks nothing. Its `flag` result is `null` on a narrowed run.
- **The column step of `files-to-references` does not run on a narrowed run.** It retypes every single-value media column in the database on the authority of the gate, and a narrowed gate vouches only for the named objects. Before this change, a narrowed `--apply` or a misspelled one moved those columns and stamped `columns_moved_at`.
- **An unknown `--object` is an error.** This applies to `value-shapes`, `files-to-references`, `summary-nulls` and `duplicates`. A name the booted registry does not declare exits 1 with `OBJECT_NOT_FOUND`, and the error names that name and the declared objects. The check runs before anything is read or written. Until now, such a name was filtered out of the scan without a word, so a typo scanned nothing and read as a clean run. `duplicates` reports the refusal as `{ error: 'report_failed', detail, code }`. A declared object that the command has nothing to check on is still accepted.
18 changes: 16 additions & 2 deletions content/docs/deployment/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1132,11 +1132,25 @@ report no secret and no file. A read the command cannot avoid and that fails for
other reason still refuses and exits 1. Point `--database-url` at the deployment's
database, or boot the deployment once first, to see what it holds.

**`--object` narrows a run, and only a run over every object records a flag.**
`files-to-references`, `value-shapes`, `summary-nulls` and `duplicates` take
`--object` to restrict the run to the objects you name. `duplicates` takes one name,
and the others are repeatable. A name your deployment does not declare is refused
with `OBJECT_NOT_FOUND` and exit 1 before anything is read or written. The error
names the unknown name and the declared objects, so a misspelling is never answered
as a clean run over nothing. A narrowed `--apply` applies its fixes to the named
objects. `files-to-references` and `value-shapes` then record **no** deployment flag,
because the flag is a claim about every object's stored data and a narrowed run read
only some. A narrowed `files-to-references` run does not move the media columns
either. The output says so, a flag that an earlier full run recorded is left as it
was, and `--json` carries `filter: { objects }`. Any `--object` narrows, even a list
that names every object, so run the command without `--object` to record the flag.

```bash
os migrate files-to-references # Dry run: full report, writes nothing
os migrate files-to-references --apply # Convert, verify, record the flag (prompts)
os migrate files-to-references --apply --yes --json # CI / scripts
os migrate files-to-references --object product # Restrict to one object (repeatable)
os migrate files-to-references --object product # Restrict to one object (repeatable); records no flag
```

A `file` / `image` / `avatar` / `video` / `audio` field value is an opaque
Expand Down Expand Up @@ -1189,7 +1203,7 @@ The same gate for the **non-media** value classes — references (`lookup`,
os migrate value-shapes # Scan: full report, writes nothing
os migrate value-shapes --apply # Scan, then record the flag if clean (prompts)
os migrate value-shapes --apply --yes --json # CI / scripts
os migrate value-shapes --object contact # Restrict to one object (repeatable)
os migrate value-shapes --object contact # Restrict to one object (repeatable); records no flag
```

**This one converts nothing.** Its sibling rewrites legacy file values because
Expand Down
25 changes: 22 additions & 3 deletions packages/cli/src/commands/migrate/duplicates.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

import { Command, Flags } from '@oclif/core';
import { emitJson, isExitSignal } from '../../utils/format.js';
import { emitJson, errorCodeFields, isExitSignal } from '../../utils/format.js';
import { bootSchemaStack } from '../../utils/schema-migrate.js';
import { refuseUndeclaredObjects } from '../../utils/migrate-object-scope.js';
// The `objectql` slot's contract (#4251) — read the registry through it rather
// than erasing the lookup to `any`, so a rename breaks this at compile time
// instead of silently reporting zero objects.
Expand Down Expand Up @@ -874,7 +875,9 @@ export default class MigrateDuplicates extends Command {
env: 'OS_DATABASE_URL',
}),
object: Flags.string({
description: 'Restrict the scan to one object (recorded in the report, so a narrowed run cannot be mistaken for a full one)',
description:
'Restrict the scan to one object (recorded in the report, so a narrowed run cannot be mistaken for a full ' +
'one). A name the deployment does not declare is refused',
}),
};

Expand Down Expand Up @@ -906,6 +909,18 @@ export default class MigrateDuplicates extends Command {
}

try {
// [#21644] `collectScanTargets` keeps only the objects it would probe, so
// a name this registry does not declare would be dropped without a word
// and the report would read "no duplicates" over a scan of nothing.
// Refused before any probe, against the set the scan draws from.
refuseUndeclaredObjects(
flags.object === undefined ? undefined : [flags.object],
stack
.allObjects()
.map((o) => (o as { name?: unknown } | null)?.name)
.filter((name): name is string => typeof name === 'string'),
);

const {
resolveSeedTenancyExec,
normalizeRows,
Expand Down Expand Up @@ -973,7 +988,11 @@ export default class MigrateDuplicates extends Command {
await emitJson(report);
} catch (error: unknown) {
if (isExitSignal(error)) throw error;
await emitJson({ error: 'report_failed', detail: messageOf(error) }, 1, { compact: true });
await emitJson(
{ error: 'report_failed', detail: messageOf(error), ...errorCodeFields(error) },
1,
{ compact: true },
);
} finally {
await stack.shutdown();
}
Expand Down
74 changes: 67 additions & 7 deletions packages/cli/src/commands/migrate/files-to-references.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import { bootSchemaStack } from '../../utils/schema-migrate.js';
import { OCCUPANCY_HINT, probeMigrationTarget } from '../../utils/migrate-occupancy-gate.js';
import { describeOccupancy } from '../../utils/sqlite-occupancy.js';
import { buildDataMigrationPlugins } from '../../utils/data-migration-plugins.js';
import { isNarrowedRun, narrowedFlagNote, refuseUndeclaredObjects } from '../../utils/migrate-object-scope.js';
import {
describeFileColumnMoveRefusal,
runFileColumnMove,
Expand All @@ -41,6 +42,7 @@ import type { MediaColumnMoveScan, SqlDialectName } from '@objectstack/driver-sq
interface ColumnStepOutcome {
skipped:
| 'gate_not_passed'
| 'narrowed_run'
| 'no_sql_driver'
| 'no_sql_seam'
| 'driver_cannot_plan'
Expand Down Expand Up @@ -96,11 +98,19 @@ async function confirm(question: string): Promise<boolean> {
* Dry run by default, and a dry run writes NOTHING — not conversions, not the
* flag. Not run / not passed → files keep being retained forever: storage
* cost, zero data loss.
*
* [#21644] Only a run over every object records the flag or moves the
* columns. A run narrowed by `--object` converts the named objects' values and
* stops there: the producer records no flag for it, and the column step, which
* retypes every media column in the database on the strength of the gate,
* does not run. A name the deployment does not declare is refused
* (`OBJECT_NOT_FOUND`) rather than scanned as nothing.
*/
export default class MigrateFilesToReferences extends Command {
static override description =
'Migrate legacy file-field values to sys_file references and verify the ownership ledger (ADR-0104). ' +
'Dry-run by default; --apply also records the deployment-level migration flag when the self-check passes.';
'Dry-run by default; --apply also records the deployment-level migration flag when the self-check of every ' +
'object passes.';

static override examples = [
'$ os migrate files-to-references',
Expand All @@ -117,7 +127,8 @@ export default class MigrateFilesToReferences extends Command {
}),
apply: Flags.boolean({
description:
'Write the conversions and record the deployment migration flag (default is a read-only dry run)',
'Write the conversions and record the deployment migration flag (default is a read-only dry run). ' +
'Only a run without --object records the flag',
default: false,
}),
yes: Flags.boolean({ char: 'y', description: 'Skip the --apply confirmation prompt', default: false }),
Expand All @@ -126,7 +137,9 @@ export default class MigrateFilesToReferences extends Command {
default: false,
}),
object: Flags.string({
description: 'Restrict to this object (repeatable; default: every object with a file field)',
description:
'Restrict to this object (repeatable; default: every object with a file field). A narrowed run converts ' +
'but records no deployment flag and moves no column, and a name the deployment does not declare is refused',
multiple: true,
}),
'max-records': Flags.integer({
Expand All @@ -144,6 +157,7 @@ export default class MigrateFilesToReferences extends Command {
const { flags } = await this.parse(MigrateFilesToReferences);
const timer = createTimer();
const apply = flags.apply;
const narrowed = isNarrowedRun(flags.object);

if (!flags.json) {
printHeader('Migrate · files-to-references');
Expand Down Expand Up @@ -197,7 +211,12 @@ export default class MigrateFilesToReferences extends Command {
return;
}
const ok = await confirm(
chalk.bold('\nConvert legacy file values and record the migration flag on this database? [y/N] '),
chalk.bold(
narrowed
? '\nConvert legacy file values of the named object(s) on this database? ' +
'A run narrowed by --object records no deployment flag. [y/N] '
: '\nConvert legacy file values and record the migration flag on this database? [y/N] ',
),
);
if (!ok) {
printInfo('Aborted — no changes made.');
Expand Down Expand Up @@ -248,6 +267,13 @@ export default class MigrateFilesToReferences extends Command {
'Run "os build" in your project root first (the migration reads dist/objectstack.json), then re-run.',
);
}
// [#21644] Before anything is converted: the scan keeps only the
// candidates it covers, so a name this registry does not declare would
// be dropped without a word, and the run would read as clean.
refuseUndeclaredObjects(flags.object, loadedObjects);
const narrowedNote = isNarrowedRun(flags.object)
? narrowedFlagNote('files-to-references', flags.object, apply)
: null;
const getStorage = () => {
try {
// Canonical slot since #9683 (service-storage also registers the
Expand Down Expand Up @@ -293,13 +319,18 @@ export default class MigrateFilesToReferences extends Command {
engine,
apply,
gatePassed: result.gatePassed,
narrowed,
json: flags.json,
});

if (flags.json) {
if (narrowedNote) logger.info(narrowedNote);
await emitJson({
database: stack.dbLabel,
apply,
// [#21644] Recorded in the document, so a narrowed run cannot be
// mistaken for a full one (the shape `os migrate duplicates` keeps).
filter: narrowed ? { objects: flags.object } : null,
backfill: {
scannedObjects: result.backfill.scannedObjects,
scannedRecords: result.backfill.scannedRecords,
Expand Down Expand Up @@ -341,7 +372,23 @@ export default class MigrateFilesToReferences extends Command {
console.log(formatFileReferenceReport(result.verify));
console.log('');

if (result.gatePassed) {
if (narrowedNote) {
// [#21644] Every sentence below would promise the flag or the
// enforcement it turns on, and a narrowed run records neither.
if (!result.gatePassed) {
for (const failure of result.gateFailures) printError(`Gate not passed: ${failure}`);
printWarning('Fix the records listed above, then re-run.');
} else if (apply) {
printSuccess('Self-check passed over the named object(s); their conversions are written.');
} else if (result.backfill.converted > 0) {
printInfo(
`Dry run only — ${result.backfill.converted} value(s) would be converted. Re-run with --apply to convert.`,
);
} else {
printInfo('Data in the named object(s) is already in reference form.');
}
printInfo(narrowedNote);
} else if (result.gatePassed) {
if (apply) {
printSuccess(
'Self-check passed — deployment flag recorded (adr-0104-file-references). ' +
Expand Down Expand Up @@ -398,15 +445,23 @@ export default class MigrateFilesToReferences extends Command {
engine: unknown;
apply: boolean;
gatePassed: boolean;
narrowed: boolean;
json: boolean;
}): Promise<ColumnStepOutcome> {
const { stack, apply, gatePassed, json } = args;
const { stack, apply, gatePassed, narrowed, json } = args;

if (!gatePassed) {
// ⛔ The ruling's "abort unless backfill + verify report zero blocking".
// Not an error of this step's own — the gate already reported why.
return { skipped: 'gate_not_passed', failed: false, stampedAt: null, report: null };
}
if (narrowed) {
// [#21644] ⛔ The move retypes EVERY single-value media column in the
// database, and a narrowed gate vouched for the named objects only. Its
// stamp also requires the verified flag a narrowed run does not record,
// so running it here would move columns it then could not record.
return { skipped: 'narrowed_run', failed: false, stampedAt: null, report: null };
}
if (!stack.driver || typeof stack.driver.planMediaColumnMove !== 'function') {
return { skipped: 'no_sql_driver', failed: false, stampedAt: null, report: null };
}
Expand Down Expand Up @@ -503,7 +558,12 @@ export default class MigrateFilesToReferences extends Command {
/** The human-mode half of {@link runColumnStep}. JSON mode reports the same facts. */
private renderColumnStep(outcome: ColumnStepOutcome): void {
if (outcome.skipped === 'gate_not_passed' || outcome.report === null) {
if (outcome.skipped === 'no_sql_driver') {
if (outcome.skipped === 'narrowed_run') {
printInfo(
'Column step: not run — it moves every media column in the database, so only a run without ' +
'--object authorises it.',
);
} else if (outcome.skipped === 'no_sql_driver') {
printInfo(
'Column step: not applicable — the ADR-0104 file-family column move is a SQL-driver step ' +
'and no SQL driver is active here.',
Expand Down
Loading
Loading