Skip to content

Declarative migrations, so content filters and sorts can span a family's versions #448

Closed

Description

@cuibonobo

Problem

A collection presents its family at the handle's version: reads migrate each record in memory. But content filters and sorts run in the adapter, on content as stored, and a migration is an opaque function — core cannot translate a predicate written in @2's shape into @1's. A rename (author → by), a remapped enum value (done → finished) or a derived field makes a family-wide content filter silently miss older records, or a content sort misplace them.

#447 (for #410) handles this by narrowing: a content filter or sort reaches only records at the handle's version, and the page sets skipped.older / skipped.newer when records at another version match the rest of the query. That is honest but lossy, and mixed versions are a lasting state of a shared stack, not a backlog: an app at an older version may keep writing its own, so migrateAll() clears skipped.older only until the next such write.

Proposal

Declared steps

A migration step may be declared from a small vocabulary, named after what Rails, Prisma, Avro and Cambria already call these operations:

migration(TaskV1, TaskV2, [
  rename('author', 'by'),
  map('status', { done: 'finished' }),
  add('priority', 'normal'),
  drop('legacyFlag'),
]);
  • Declared steps compile to the existing Migration. migrate() is built from the steps, so Stack.open(), the in-memory read path and commitMigration() are unchanged. The steps are also kept on the object, which is what query translation reads.
  • Checked at construction, not at the type level. Both handles' schemas are known when migration() runs, so it refuses a rename into a field to doesn't declare, a map that leaves a from enum value unmapped, an add whose default fails to's schema, or a drop of a field to still requires. No new type machinery; the function form keeps its StoredContentOf typing.
  • The opaque function stays as the escape hatch. A field it touches is not translatable across that step.

Query translation

Core rewrites a content filter or sort into one arm per version on the older side of the chain. Newer versions stay out of reach — the handle cannot represent them either way.

Step Filter Sort
rename translated (path renamed in the older arm) translated (sort key per typeId)
add decided in core: the default is evaluated against the predicate, older arm kept or dropped translated (constant per typeId)
drop nothing to translate: the field is not in the handle's shape nothing to translate
map translated (value remapped in the older arm) narrowed — remapped values don't keep their order
function narrowed narrowed

"Narrowed" is #447's behaviour, unchanged.

Filter shape: a per-typeId content predicate, not general OR

RecordFilter is a flat AND, and it should stay that way. The rewrite needs only a bounded, flat disjunction — a content predicate keyed by typeId — not a boolean grammar:

  • Generated by core from the migration chain. Callers don't write it.
  • Flat. One arm per version, each an ordinary content filter. No nesting.
  • Bounded by the family's version count; a server may cap the number of arms.
  • Every arm is pinned to a typeId, and no arm may be empty, so the shape can split a query along versions but never widen it (the concern behind data-model.md § Filter refusing match-any forms).

It discloses nothing new: each arm is a query the caller could already run, and permission checks apply per record as today. Its cost stays within what wire-format.md § Bounding query cost already leaves to the server. The spec should say so explicitly, so the shape isn't read as a first step toward general OR.

This is a wire-contract change: spec, @haverstack/conformance-fixtures, MemoryAdapter, sqlite-shared (both SQLite adapters) and adapter-api move together. A per-typeId sort key needs the same treatment on the sort side.

skipped

Per query, from the fields the query names: a version is reported in skipped.older only when the filter or sort touches a field that does not translate across some step between it and the handle.

Questions

  • Exact wire spelling of the per-typeId predicate and sort key, and whether a server advertises the arm cap through capabilities.
  • Whether an adapter that doesn't support the shape falls back to feat(core): type a collection's content filters and sorts from its content type #447's narrowing (an optional capability) or whether it's required for local adapters, like filter.content: 'path'.
  • Composing declared and function steps in one migration() call, or one or the other per step.

Order

After #447. Spec section first; this changes how migrations are written and adds to the wire contract.

Activity

  1. added theissue type on Oct 11, 2026
  2. added a commit that references this issue on Oct 13, 2026
    a40cee0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions