Skip to content

Collection filters and sorts are checked against the content type #410

Closed

Description

@cuibonobo

Problem

Inside a typed query, filter.content is Record<string, unknown> and sort.contentField is string. So content: { stauts: 'want' } compiles and silently returns zero results, and so does status: 'wnat'. The handle knows every field name and value type, but none of it reaches the filter.

Decision

A collection's filter.content, filter.contentPresent and sort.contentField are typed from its content type C (#409):

  • filter.content keys are the content's paths: top-level field names, plus dotted paths through declared nested objects and arrays ('address.city', 'emails.value'). Below an open object or array (Record<string, unknown>, unknown[]), any suffix is allowed. Values are the field's type (the element type for an array, since matching is element-wise), or null.
  • filter.contentPresent takes the same paths.
  • sort.contentField takes only top-level scalar fields, which puts data-model.md § Sorting by a content field's "only top-level scalars" rule into the type.
  • Everything is derived from the content type, not the schema. Paths, enum unions, open nodes and top-level scalars are all visible in ContentOf<S>.
  • The type checks names against the handle's version, but matching still runs on the stored shape. A stack mid-migration can hold @1 records whose status is 'done', which { status: 'finished' } doesn't match. That's existing behaviour, already stated for sorting; the spec should say it for filters too.

Prototype results

A prototype built on the public StackClient API (not committed) confirmed:

  • A typo'd field, a value outside an enum, a wrong value type at a nested path, a path through a scalar, a nested sort field and a typo'd sort field all fail to compile. Paths through declared objects and arrays of objects compile, and so does any suffix below an open object.
  • Recursive path types need a depth limit. Without one, ContentPathOf fails with TS2589 ("Type instantiation is excessively deep and possibly infinite"). Typing paths to a depth of 6, and accepting any suffix below that, fixes it. Array nesting must count towards the depth too, not only objects. The spec caps paths at 32 segments, so this is a typing limit, not a query limit.
  • Internal aliases leak into errors. Typing contentField through a named alias made its error print the alias and its argument instead of the field names; spelled out inline, it prints '"title" | "status" | …'. Did you mean '"title"'?. Likewise NonNullable<C[K]> leaked into filter errors as NonNullable<"want" | …>, where Required<C>[K] doesn't.
  • Measured with tsc --pretty false: stauts in a filter is one TS2561 line of ~460 characters; 'wnat' as a status filter is one TS2322 line of ~140 characters.

Actions

  • Export ContentPathOf<C> and SortableFieldOf<C>, derived from the content type, typed to a depth of 6 with any suffix accepted below that.
  • Type the collection's filter.content, filter.contentPresent and sort.contentField with them. Spell out inline anything that appears in a compile error rather than naming an internal alias.
  • Type tests: the failing cases above are @ts-expect-error; paths through nested objects, arrays of objects and below open nodes compile; null is accepted as a filter value.
  • Spec, data-model.md § Type handles (or the collection section Typed access through collections, with records that don't fit reported rather than thrown #409 writes): filter keys are checked against the handle's version while matching runs on the stored shape.
  • Changeset: minor for @haverstack/core.

Order

Blocked by #409. Separate because it's pure types, with its own review concerns (recursion limits, error readability). Can run in parallel with #411.

Activity

  1. added theissue type on Oct 5, 2026
  2. cuibonobo commented on Oct 11, 2026

    @cuibonobo
    MemberAuthor

    A note from #446 (the #409 collection) for whoever picks this up.

    Content filters and sorts run on content as stored, but a collection returns records migrated to the handle's version. The adapter applies filter.content, filter.contentPresent and sort.contentField before any record is migrated. So if a migration renamed or derived a field, filtering or sorting an @2 collection by its @2 name silently skips the @1 records, or puts them in the wrong place.

    #446 documents this in data-model.md § Type handles ("Content filters and sorts apply to content as stored"). It doesn't solve it. Typing the keys against C makes the mismatch easier to miss, because a key the compiler accepts for @2 may not exist on older records. Options worth weighing here:

    • refuse a content filter or sort while records of the family remain at an older version
    • type the keys against every version the handle's chain covers
    • keep the documented behaviour and point at migrateAll()

    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions