feat(core): declarative migrations, so content queries span a family's versions

.changeset/declarative-migrations.md

@@ -0,0 +1,12 @@
+---
+'@haverstack/core': minor
+'@haverstack/wire-types': minor
+'@haverstack/conformance-fixtures': minor
+'@haverstack/adapter-conformance': minor
+'@haverstack/record-adapter-sqlite': minor
+'@haverstack/record-adapter-do-sqlite': minor
+---
+
+Migrations are declared steps. `migration(from, to, steps)` takes a list built from `rename`, `map`, `add`, `drop` and `derive`, replacing the function form, and checks the list against both handles' schemas when it runs: a step naming a field that is not there, a carried field the target would refuse, or a required field nothing produces throws `StackMigrationError`. `Migration` carries its `steps`.
+
+A collection's content filter or sort now spans the older versions its migrations translate it to, rather than only the handle's version. A key across a `rename` or `add` translates; one across a `derive`, or a sort across a `map`, narrows that version and sets `skipped.older` as before. Two query shapes carry it, each gated by a new capability that is a cap on keys: `filter.contentByType` (a content predicate per `typeId`, never beside `content` or `contentPresent`) and `sort.contentFieldByType` (a sort field or a reduced constant per `typeId`). `StackCapabilities` gains `filter.contentByType` and `sort.contentFieldByType`; local adapters declare 64, and `normalizeCapabilities()` reads an absent or malformed cap as 0. `parseQueryBody()` accepts both shapes. Conformance fixtures pin both shapes and their 400s, and the adapter conformance suite checks them.

README.md

@@ -194,7 +194,7 @@ An app has two layers, because the stack draws a line between them:
 Migrations belong to neither. They are passed to `Stack.open()` on **every startup**, so the chain is complete before the first read.
 
 ```ts
-import { Stack, migration, typeHandle } from '@haverstack/core';
+import { Stack, add, migration, typeHandle } from '@haverstack/core';
 import type { Collection, ContentOf, StackClient } from '@haverstack/core';
 
 const NoteV1 = typeHandle({
@@ -212,11 +212,9 @@ export const Note = typeHandle({
   },
 });
 
-// Startup: pass to every Stack.open(). `content` is NoteV1's, and the
-// result must be Note's.
-export const noteMigrations = [
-  migration(NoteV1, Note, (content) => ({ ...content, pinned: false })),
-];
+// Startup: pass to every Stack.open(). Steps are checked against both
+// handles' schemas when migration() runs.
+export const noteMigrations = [migration(NoteV1, Note, [add('pinned', false)])];
 
 // Data layer: whoever the stack lets in.
 export class Notes {
@@ -314,7 +312,7 @@ A relationship's `target` says which identifier space its value lives in — a R
 
 ### Migrations
 
-Types can evolve over time. A new version names the one it migrates from, and `migration()` builds the step between two handles, typed on both. Pass every step to `Stack.open()`; the library composes adjacent steps into chains:
+Types can evolve over time. A new version names the one it migrates from, and `migration()` builds the step between two handles from declared steps — `rename`, `map`, `add`, `drop` and `derive` — checked against both schemas when it runs. Pass every step to `Stack.open()`; the library composes adjacent steps into chains:
 
 ```ts
 const NoteV2 = typeHandle({
@@ -325,12 +323,12 @@ const NoteV2 = typeHandle({
 });
 
 const stack = await Stack.open(adapter, {
-  migrations: [migration(NoteV1, NoteV2, (content) => ({ ...content, title: '' }))],
+  migrations: [migration(NoteV1, NoteV2, [add('title', '')])],
 });
 await stack.defineType(NoteV2);
 ```
 
-Records stay at the version they were written at: `get()` and `query()` return them as stored, `presentAt: 'latest'` migrates them in memory for one read, and `stack.migrateAll()` commits a family to disk. Pass migrations at every startup — see [Writing an app](#writing-an-app).
+Records stay at the version they were written at: `get()` and `query()` return them as stored, `presentAt: 'latest'` migrates them in memory for one read, and `stack.migrateAll()` commits a family to disk. Because the steps are declared rather than a function, a collection carries a content filter or sort across the versions they translate to — a filter on `title` reaches records still at `@1`. Pass migrations at every startup — see [Writing an app](#writing-an-app).
 
 ### History, and watching for changes
 

docs/spec/adapters.md

@@ -154,11 +154,13 @@ type StackCapabilities = {
   filter: {
     content: 'none' | 'field' | 'path'; // how far a content filter key may reach
     contentPresent: boolean; // filter.contentPresent is honored
+    contentByType: number; // most arms filter.contentByType may hold; 0 = not honored
     search: boolean; // filter.search is honored
   };
   sort: {
     fields: ('createdAt' | 'updatedAt' | 'version')[]; // native columns this adapter can order by
     contentField: boolean; // sort.contentField is honored
+    contentFieldByType: number; // most keys sort.contentFieldByType may hold; 0 = not honored
   };
   limits: {
     attachmentBytes: number | null; // upload size ceiling, or null = client can't pre-check
@@ -179,11 +181,13 @@ One type, `StackCapabilities`, names both sides: an adapter declares it as `capa
 
 **`filter.contentPresent` sits beside the ladder rather than on it.** It gates `filter.contentPresent`, and is required `true` for local adapters on the same reasoning: asking whether a path holds a value is the same walk with a weaker predicate. It is not a rung because a server promising to match a content value has not thereby promised to answer whether one is there at all. It is meaningful only above `'none'` — presence travels in the `POST /records/query` body, which a server reaching no content does not expose — and a presence path is itself gated by the rung, so a multi-segment one needs `'path'`.
 
+**`filter.contentByType` and `sort.contentFieldByType` are caps, not flags.** Each is the most keys the [per-Type shape](./data-model.md#content-by-type) may hold, and `0` declines it. A local adapter must honor both, on the reasoning that makes `'path'` required: an arm is the content filter it already runs, scoped to one `typeId`. The first-party ones declare `64`, far above any family's version count and inside SQLite's limits on expression depth and compound statements. A server may declare any cap, `0` included, and a collection then narrows the versions it cannot reach rather than failing the query — see [Data model § Content queries across versions](./data-model.md#content-queries-across-versions). Both are meaningful only above `filter.content: 'none'`, since they travel in the `POST /records/query` body.
+
 **`filter.search` has no local-required rule.** A full-text index is a different mechanism from field matching, and neither implies the other, so a local adapter may legitimately decline it (see the JSON adapter note below).
 
 **`sort.contentField` is a boolean, and `sort.fields` names native columns only.** Content fields are app-defined and unbounded, so an adapter cannot enumerate them; and an adapter that indexes content for sorting indexes [every top-level scalar](./data-model.md#sorting-by-a-content-field), so there is nothing per-field left to declare. `sort.contentField` is required `true` for local adapters, and independent of `filter.content`: the two describe ordering and filtering, and a server may offer either without the other — filtering is answerable at whatever scale a server already scans at, where ordering wants an index. Both are enforced: `Stack.query()` refuses a sort the adapter hasn't declared rather than letting it be answered in some other order, which a caller reading one bounded page cannot detect.
 
-**Absent, malformed or unrecognized reads as the least capable value it could stand for.** One rule, for every entry: `'none'` for the reach, `false` for a flag, nothing for `sort.fields`, `null` for a limit. A discovery response that omits a group, omits `capabilities` entirely, or names a reach this client has never heard of describes a server this client will not send that query to — silence is never a claim, in either direction, and an unrecognized rung places nowhere on a ladder so it can only be read as its bottom. `normalizeCapabilities()` in `@haverstack/wire-types` applies the rule, so a client reads a foreign discovery response through one function rather than defaulting keys at each use.
+**Absent, malformed or unrecognized reads as the least capable value it could stand for.** One rule, for every entry: `'none'` for the reach, `false` for a flag, `0` for a cap, nothing for `sort.fields`, `null` for a limit. A discovery response that omits a group, omits `capabilities` entirely, or names a reach this client has never heard of describes a server this client will not send that query to — silence is never a claim, in either direction, and an unrecognized rung places nowhere on a ladder so it can only be read as its bottom. `normalizeCapabilities()` in `@haverstack/wire-types` applies the rule, so a client reads a foreign discovery response through one function rather than defaulting keys at each use.
 
 Two things this does not refuse. A query naming no sort asks for the server's own default order rather than a stated one, so it claims nothing that has to be declared and runs against an empty `sort.fields`. And a `null` limit is permissive by design: it says this client cannot pre-check, not that nothing is enforced — the server's own ceiling still answers with a `413`.
 

docs/spec/apps.md

@@ -20,7 +20,7 @@ const plan = await stack.planInstall(manifest, { did: appDid });
 await stack.installApp(plan);
 ```
 
-Migration functions are not part of a manifest. They are app code, passed to `Stack.open()` on every startup (see [Data model § Type migrations](./data-model.md#type-migrations)), and the app runs them itself — see [Migrating an installed app's types](#migrating-an-installed-apps-types).
+Migrations are not part of a manifest. They are app code, passed to `Stack.open()` on every startup (see [Data model § Type migrations](./data-model.md#type-migrations)), and the app runs them itself — see [Migrating an installed app's types](#migrating-an-installed-apps-types).
 
 `planInstall()`, `installApp()` and `uninstallApp()` live on `Stack` and are absent from `StackClient`, like `grantType()` and `defineType()`: everything they write is the owner's to write. An app reaches them only through the request it can make over the wire — see [Over the wire](#over-the-wire).
 
@@ -102,7 +102,7 @@ It refuses what no approval could make valid: a type outside the app's own names
 
 ## Migrating an installed app's types
 
-[Migration is owner-acting-alone](./access-control.md#what-a-grant-covers), and an installed app's migration functions are its own code, which the owner should not have to run with owner authority. An install closes the gap: a contained app may commit migrations within its own families. `ScopedStack.commitMigration()` — and so `migrateAll()` run by the app over `APIAdapter`, which commits through `POST /records/:id/migrate` — admits a request that is not the owner acting alone when **all** of these hold:
+[Migration is owner-acting-alone](./access-control.md#what-a-grant-covers), and an installed app's migrations are its own code, which the owner should not have to run with owner authority. An install closes the gap: a contained app may commit migrations within its own families. `ScopedStack.commitMigration()` — and so `migrateAll()` run by the app over `APIAdapter`, which commits through `POST /records/:id/migrate` — admits a request that is not the owner acting alone when **all** of these hold:
 
 1. The source and target families are both in the namespace of one live install, and both in its `defines`.
 2. The target TypeId is in that install's `defines` — a version the owner approved.