Repository navigation
Declarative migrations, so content filters and sorts can span a family's versions #448
Description
Problem
A collection presents its family at the handle's version: reads migrate each record in memory. But content filters and sorts run in the adapter, on content as stored, and a migration is an opaque function — core cannot translate a predicate written in @2's shape into @1's. A rename (author → by), a remapped enum value (done → finished) or a derived field makes a family-wide content filter silently miss older records, or a content sort misplace them.
#447 (for #410) handles this by narrowing: a content filter or sort reaches only records at the handle's version, and the page sets skipped.older / skipped.newer when records at another version match the rest of the query. That is honest but lossy, and mixed versions are a lasting state of a shared stack, not a backlog: an app at an older version may keep writing its own, so migrateAll() clears skipped.older only until the next such write.
Proposal
Declared steps
A migration step may be declared from a small vocabulary, named after what Rails, Prisma, Avro and Cambria already call these operations:
migration(TaskV1, TaskV2, [
rename('author', 'by'),
map('status', { done: 'finished' }),
add('priority', 'normal'),
drop('legacyFlag'),
]);- Declared steps compile to the existing
Migration.migrate()is built from the steps, soStack.open(), the in-memory read path andcommitMigration()are unchanged. The steps are also kept on the object, which is what query translation reads. - Checked at construction, not at the type level. Both handles' schemas are known when
migration()runs, so it refuses arenameinto a fieldtodoesn't declare, amapthat leaves afromenum value unmapped, anaddwhose default failsto's schema, or adropof a fieldtostill requires. No new type machinery; the function form keeps itsStoredContentOftyping. - The opaque function stays as the escape hatch. A field it touches is not translatable across that step.
Query translation
Core rewrites a content filter or sort into one arm per version on the older side of the chain. Newer versions stay out of reach — the handle cannot represent them either way.
| Step | Filter | Sort |
|---|---|---|
rename |
translated (path renamed in the older arm) | translated (sort key per typeId) |
add |
decided in core: the default is evaluated against the predicate, older arm kept or dropped | translated (constant per typeId) |
drop |
nothing to translate: the field is not in the handle's shape | nothing to translate |
map |
translated (value remapped in the older arm) | narrowed — remapped values don't keep their order |
| function | narrowed | narrowed |
"Narrowed" is #447's behaviour, unchanged.
Filter shape: a per-typeId content predicate, not general OR
RecordFilter is a flat AND, and it should stay that way. The rewrite needs only a bounded, flat disjunction — a content predicate keyed by typeId — not a boolean grammar:
- Generated by core from the migration chain. Callers don't write it.
- Flat. One arm per version, each an ordinary
contentfilter. No nesting. - Bounded by the family's version count; a server may cap the number of arms.
- Every arm is pinned to a typeId, and no arm may be empty, so the shape can split a query along versions but never widen it (the concern behind
data-model.md § Filterrefusing match-any forms).
It discloses nothing new: each arm is a query the caller could already run, and permission checks apply per record as today. Its cost stays within what wire-format.md § Bounding query cost already leaves to the server. The spec should say so explicitly, so the shape isn't read as a first step toward general OR.
This is a wire-contract change: spec, @haverstack/conformance-fixtures, MemoryAdapter, sqlite-shared (both SQLite adapters) and adapter-api move together. A per-typeId sort key needs the same treatment on the sort side.
skipped
Per query, from the fields the query names: a version is reported in skipped.older only when the filter or sort touches a field that does not translate across some step between it and the handle.
Questions
- Exact wire spelling of the per-typeId predicate and sort key, and whether a server advertises the arm cap through capabilities.
- Whether an adapter that doesn't support the shape falls back to feat(core): type a collection's content filters and sorts from its content type #447's narrowing (an optional capability) or whether it's required for local adapters, like
filter.content: 'path'. - Composing declared and function steps in one
migration()call, or one or the other per step.
Order
After #447. Spec section first; this changes how migrations are written and adds to the wire contract.
Activity
feat(core): narrow a collection's content filters and sorts to the ha…
feat(core): declarative migrations, so content queries span a family'…