Repository navigation
fix(service-analytics): the ObjectQL face echoes an offset with no limit as a statement the dialect runs - #21440
Conversation
…w the dialect runs generateSql's two window lines now call windowClauseSql (exported from native-sql-strategy.ts) with sqlDialectFor(ctx, tableName), the same dialect read the echo's read scope already makes. On SQLite an offset with no limit echoes LIMIT -1 OFFSET n, the statement the native face runs, instead of a bare OFFSET that SQLite refuses. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…atement's from ORDER BY on Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…ow; add the changeset Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…jectql-echo-window
📓 Docs Drift CheckThis PR changes 1 package(s): ⛔ 2 release-owned page(s) name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 3d428890a44add83b054db03b328c1d43525cf68 && git checkout 3d428890a44add83b054db03b328c1d43525cf68
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 3a6d92f78bb6a160b762dfee0738fd3b0b7ae6c2 b9456bff3716d8d5365ee9cf6202aa59bf09af89 && git checkout -B drift-repro 3a6d92f78bb6a160b762dfee0738fd3b0b7ae6c2 && git merge --no-ff b9456bff3716d8d5365ee9cf6202aa59bf09af89
node scripts/docs-audit/affected-docs.mjs --json 3a6d92f78bb6a160b762dfee0738fd3b0b7ae6c2
|
…string / boolean, refused in both analytics strategies in the spec's words (objectstack-ai#21000) (objectstack-ai#21452) Fixes objectstack-ai#21000 Clause-②: no (narrowing) Dispatched by the claim `5955932074` (PM loop round 1, `domain:spec` seat 1, session `session_01UtnxvdiN376GF3sgXwAw4d`), on triage's answer B `5952826307`: the enum retirement only. The row wildcard boundary is objectstack-ai#21409's (landed as `b79301000c`, merged here). ## What this does **`@objectstack/spec`: `AggregationMetricType` loses `number`, `string` and `boolean`** (ADR-0049 enforce-or-remove, grade `5923362062`). - They declared "a custom SQL expression returning a number / string / boolean": the measure's `sql` was the whole computation. Since ruling D on objectstack-ai#20943 (`5d5e679873`), a cube member's `sql` is a column reference, so the three had nothing left to compute. - The enum is declared through `enumWithRetiredValues` (`shared/retired-key.ts`), the house value-level mechanism. The six aggregates (`count`, `sum`, `avg`, `min`, `max`, `count_distinct`) are the whole vocabulary. - An authored retired type fails `tsc`, because it is gone from the type. It is refused at parse with a named prescription, at the enum, at a metric's `type` and at a cube's `measures.METRIC.type`. - The prescription names the aggregate that fits: `sum`, `avg`, `min` or `max` over the column; `count` over `'*'` or over a column; or `count_distinct`. A per-row value becomes a stored or formula field of the object that the measure aggregates. A value derived from measures is `derived: { op, of }` on an ADR-0021 dataset. - A value the enum never declared keeps zod's own message. - The prescriptions are module-private, so the export surface does not grow. - **ADR-0087:** D3 entry `cube-metric-expression-types-retired` (`migrations/entries/semantic/`), with a step-18 rationale fragment (order 62). `registry.ts` was regenerated by `gen:migration-registry` after each merge, never edited by hand. - There is no D2 conversion, by design: the column alone does not say which aggregate the author meant, and a stored cube is refused, never rewritten. - There is no `RETIRED_KEYS_BY_MAJOR` row, because no key left the shape. - **Liveness:** the `analytics_cube` row `measures.type` stays `live`, re-verified 2026-10-02. The narrowing is recorded, and the evidence now names `aggregateOfMeasure` instead of the partition. - **Generated:** only `content/docs/references/data/analytics.mdx` moved, losing the three values from the enum list and from both `type` cells. - `api-surface`, `authorable-surface`, `json-schema.manifest` and `api-surface-signatures` are byte-identical, as the playbook predicts for an enum-value narrowing. - `spec-changes.json` and the upgrade guide stay at protocol 17, and both checks are green. - The module header gained an `@module data/analytics` marker. Without it, moving the imports below the header dropped the page's opening paragraph: `lib/file-description.ts` rule 3 does not select a block inside the import list without the marker. The page's description is byte-identical to `main`. - The neighbouring `CUBE_MEMBER_SQL` docblock no longer says that the ObjectQL path refuses the partition, in the present tense. **`@objectstack/service-analytics`: the `EXPRESSION_METRIC_TYPES` partition is deleted.** It is replaced by ONE verdict both strategies call, `aggregateOfMeasure` (`strategies/native-sql-strategy.ts`). - `aggregateOfMeasure` admits a type this runtime lowers (the `AGGREGATE_SQL` keys, pinned EQUAL to the enum's options). It refuses everything else with the SPEC's own words, `AggregationMetricType.safeParse(type)`, so the runtime keeps no list of metric types, retired or otherwise. - `NativeSQLStrategy#resolveMeasureSql` asks it before anything is lowered. The verbatim emit is gone, and so is the unrecognised-type throw it replaced. - `ObjectQLStrategy#resolveMeasureAggregation` asks it at the one resolver both doors call. The `INVALID_FIELD` arm for the partition is gone. - Comments that named the partition or the three types are updated: `plugin.ts` (the bridge's comment, its docblock, and its runtime message, which said "a custom-SQL measure is refused earlier"), `preview-evaluator.ts`, `analytics-service.ts`, `cube-measure-field-type-door.ts` and `dataset-refusal.ts`. **`@objectstack/lint` (test and comment only).** objectstack-ai#21435 landed between my merges with a pin asserting that the three types sit outside the aggregate table. That is false after this retirement, so the assertion now reads `[]`. The skip-5 `silent` case is kept. No changeset is needed: a comment and a test, nothing in the published output changes. ## Clause-②, measured `node scripts/pm/check-widening-tells.mjs --declaration no --diff` (the merge-base diff against `b79301000c`) exits 0, with no widening tell. Three key lines are reported as a stated silence. They are the retired-member prescription entries `number:` / `string:` / `boolean:` in the `enumWithRetiredValues` map, which are refusals, not accept-set members. No export-listing row was added (`check:api-surface` green, byte-identical), so the line is `Clause-②: no (narrowing)`. Both changesets are BREAKING, with `!`, a **BREAKING** banner, the `(narrowing)` arm, exactly one ADR-0087 marker (`registered cube-metric-expression-types-retired`) and `minor`. ## Census (examples, packages, platform objects, objectui) The instrument is an AST walk over every object-literal member of a `measures:` record. It covered 7,654 `.ts`/`.js` files under `examples/**` and `packages/**` at `4ec505761d`, platform objects included. - 321 measure entries in total. Lit control: `count` 164, `sum` 71. - Retired-type entries: 7. All are deliberate refusal fixtures in the `service-analytics` tests, built without the parse. - Zero hits in `examples/**`, in non-test `packages/**`, in `skills/**` and in `content/docs/**` (one cube example there, `count` / `sum`). - `examples/app-showcase/src/data/analytics/showcase.cube.ts`: 3 measures (`count`, `sum`, `avg`). Its three `type: 'string'` lines (48, 53, 63) are **dimensions**. `DimensionType` is a separate enum, unchanged, and pinned in the new test file. - JSON fixtures carrying record-form `measures`: zero. - objectui at the `.objectui-sha` pin `89cad75d55`: 0 mentions of `AggregationMetricType`, and 0 record-form measure entries over 528 files that mention `measures`. Control: `clientValidation.ts` names `CubeSchema`. The Console Pin Gate is not at risk: no export left. ## Premise check (zone 2) 1. **Holds.** On `68c5ab7eba`, `AggregationMetricType` (`data/analytics.zod.ts:27`) listed the three, and `MetricSchema.type` used it. Measured through `AnalyticsService` with a column `sql`: - the raw-SQL path SERVED the column unaggregated: `SELECT status AS "status", amount AS "m" FROM "orders" GROUP BY status`; - the ObjectQL path refused the measure `INVALID_FIELD` / 400. 2. `api/analytics.zod.ts:231`: the `/analytics/meta` member's describe ("Aggregation type for a measure (`AggregationMetricType`)") is not made false by the retirement, so it is not edited. 3. That `type` is a separate `z.string()` field, deliberately not the enum, because the projection copies the value verbatim. Measured: it is not the enum. ## What a stored cube carrying a retired type meets (fail closed, never stood down) **Pinned in `cube-metric-expression-types-retirement.test.ts`:** - the artifact boot door (`ObjectStackDefinitionSchema`, the parse `MetadataPlugin` runs a built artifact through) refuses it at `analyticsCubes.0.measures.m.type` with the prescription; - `defineStack` refuses it with `STACK_SCHEMA_INVALID` / 422; - `defineCube` and the `analytics_cube` write door (`getMetadataTypeSchema('analytics_cube')`, what `PUT /api/v1/meta/analytics_cube/NAME` validates) refuse it too; - the rehydration seam (`applyConversionsToStoredItem`) replays NOTHING over it. Control: the same row's retired sub-day granularity IS rewritten, so the seam is live. The stored row reaches the parse as stored, and the parse refuses it. **Measured through the real dispatcher routes** (a temporary `packages/runtime` probe, not committed), for a cube a host registers in-process WITHOUT the parse: - `POST /api/v1/analytics/query` and `POST /api/v1/analytics/sql`, on both strategies: `500`, with `error.message` carrying the spec's prescription verbatim. Nothing executed. - `sum` control: `200`. - `GET /api/v1/analytics/meta`: `200`, listing the measure with `type: "number"` as registered (see the Acceptance notes). ## Merges (serial constraints) - `0d182f0549` merged `main` at `3a6d92f78b`, bringing PR objectstack-ai#21424 (objectstack-ai#21376, the NativeSQL filter-compile region) and PR objectstack-ai#21425 (objectstack-ai#21293, its registry entry). Clean. - `587d9d4b63` merged `main` at `d7d5b4f96a`. One hand conflict, in the `objectql-strategy.ts` import from `native-sql-strategy.js`: objectstack-ai#21440 added `windowClauseSql`, and both are kept. - `4ec505761d` merged `main` at `b79301000c`, bringing PR objectstack-ai#21431 (objectstack-ai#21409, the count-only boundary). One hand conflict, in the `analytics.zod.ts` imports: `enumWithRetiredValues` and objectstack-ai#21409's `ANALYTICS_COLUMN_PATH` / `rowWildcardOutsideCount` / `rowWildcardOutsideCountRefusal`, both kept. - Every merge went through `scripts/pm/os-regen-merge.sh`. Its step 4 was run each time, and `gen:migration-registry` afterwards wrote no diff. - **Registry reading at `4ec505761d`:** 349 semantic, 244 retired-key and 212 retired-def entries. The siblings' ids `analytics-row-wildcard-outside-count-refused` and `dashboard-widget-single-series-multi-measure-refused` appear at the same counts as on `main` (1 each). `cube-metric-expression-types-retired` appears twice: the semantic entry and its step-18 rationale fragment. - **Ledger reading at `4ec505761d`:** the `measures.sql` and `dimensions.sql` notes carry objectstack-ai#21409's count-only wording and no longer name this card. `measures.type` carries this retirement's own wording. ## Tests (head `4ec505761d`) - `@objectstack/spec`: - `vitest --project local`: 602 files, 17737 passed, 1 todo; - `--project repo`: 43 of 49 files, 705 passed (the other six are NOT MEASURED, below); - `typecheck` (`tsc`, scripts, test layer): OK. The new `@ts-expect-error` (a typed `Metric` with `type: 'number'`) sits in the compiled test program. - `@objectstack/service-analytics`: 170 files, 3846 passed, 126 skipped. `typecheck` OK. - The reverse verification happened on the way: a fixture typed `Cube` with `type: 'number'` failed `tsc` with TS2322 against the rebuilt `.d.ts` until it was cast. - `@objectstack/lint`: 119 files, 5592 passed. `typecheck` OK. ## Ablations Both run from the committed tree through `scripts/ablation-replace.mjs`. In each, the anchor hit once and the blob changed; the restore was proved by blob equal to `HEAD` and an empty `git diff HEAD`. Both subjects resolve from `src`, so no rebuild was needed. - **The runtime verdict admits every string** (`aggregateOfMeasure`'s table check removed): 28 failed, 12 passed, over `metric-type-coverage`, `measure-expression-both-strategies` and `measure-expression-sql`. - Every refusal case went red: both strategies, both doors, and the drift case. - The admitted-aggregate, cross-object-twin and coverage-equality cases stayed green, which is the predicted direction. - **The spec's `number` prescription is unmapped:** 8 failed, 17 passed in `cube-metric-expression-types-retirement.test.ts`. Every `number` door pin went red; the `string` / `boolean` pins and the controls stayed green. ## Gates `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` at `4ec505761d` derived 117 families. All 117 were run, each exit code recorded, and `--ran` answered "117 derived, 117 run, 0 NOT-MEASURED (a DERIVED zero)". Among them: - `check-adr-0087-registration` ("2 declared-breaking changeset(s), each carrying an ADR-0087 disposition"); - `check-changeset-no-major`; - `check:generated` ("All 15 generated artifacts are up to date"); - `check:liveness`, `check:doc-authoring`, `check:nul-bytes` and `check:skill-examples` (after building the client closure); - `check:type-check-debt` (325 s); - `check:dual-build-cjs-loads` (after a whole-repo build, 71 of 72 tasks cached). **NOT MEASURED** - `@objectstack/spec` repo-project files `build-schemas-check-mode`, `dist-freshness`, `dist-freshness-adoption`, `publish-smoke-boot-failure`, `publish-smoke-port-collision` and `schema-tree-freshness`. Reason: they drive whole builds and exercise build tooling this diff does not touch. - `@objectstack/cli` integration tier, declared to CI. ## Acceptance notes - **`GET /analytics/meta` still lists a host-registered unparsed cube's measure with its retired `type`**, while the query doors refuse it. This is pre-existing for any enum-invalid type (`median` read the same at base). It is reachable only by a host that registers a cube literal without `CubeSchema`, because every parsing door refuses it first. Not filed; carrier: none. - **A stored `analytics_cube` row** read at `GET /api/v1/meta/analytics_cube/NAME` comes back as stored. Stored rows keep being read (the runtime gate's D4 asymmetry), and no conversion rewrites it (pinned at the seam). - Re-saving it is refused with the prescription (the write door pin). - The analytics registry has no metadata read path (objectstack-ai#20965's measurement), so such a row reaches no query. - The HTTP read itself was NOT MEASURED through the route. - **The ObjectQL envelope for the three moves from `INVALID_FIELD` / 400 to the undeclared-500 tier.** The message is readable and carries the prescription. This is the tier `dataset-refusal.ts` assigns to a cube that never met the parse, and the changeset says so. - A never-declared type (`median`) on the ObjectQL path is now refused at the resolver in the same tier. Before, it was forwarded to `executeAggregate`: the auto-bridge refused it, a host's own executor received it, and `/analytics/sql` echoed `MEDIAN(amount)`. - `aggregate-bridge-function-vocabulary.test.ts` therefore pins both seams. The bridge is driven directly through the service's strategy context, because no cube path reaches it with a non-aggregate method any more. - **The `measures.sql` ledger note** (objectstack-ai#20943's, re-pointed by objectstack-ai#21409) was made false by this diff, as at-tier record `5959409102` ruled. It was corrected in patch round 2 (`4278601b82`): both runtime expression branches are gone (the gate's stand-down with objectstack-ai#20965, the raw-SQL verbatim emit here). What remains for a cube that reaches the service without meeting the parse is `qualifyAndRegisterJoin` passing a non-column `sql` through inside the aggregate, measured through `generateSql`. - **The prescriptions say "removed … in @objectstack/spec 17.7.0"**, assuming the next release is a minor (the label is 17.6.0, which is published). If the next release is cut as a major, those runtime strings need the new number. - **Files beyond the claim's list**, all comments or tests that named the partition or the types, or that the merges made false: - `service-analytics`: `analytics-service.ts`, `cube-measure-field-type-door.ts`, `dataset-refusal.ts`, and the tests `aggregate-bridge-function-vocabulary`, `caller-member-column-reference-gate`, `cube-authored-format-granularity`, `field-read-admission-gate` and `unlisted-refusal-envelope`; - `lint`: `validate-dataset-measure-aggregates` (source comment and test). --- _Generated by [Claude Code](https://claude.ai/code/session_01UtnxvdiN376GF3sgXwAw4d)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #21365
Clause-②: no
What this changes
This is the remainder of #21365 after PR #21399 (
6d67ad5ec, which readPart of). That PR narrowed the window contract and made the native face render an offset-only window with the dialect's no-limit spelling. The ObjectQL face's echo still wrote its own window.ObjectQLStrategy.generateSql(packages/services/service-analytics/src/strategies/objectql-strategy.ts) wroteLIMIT nwhen a limit was set, thenOFFSET nwhen an offset was. Anoffsetwith nolimittherefore echoed a bareOFFSET, and SQLite's grammar has no bareOFFSET. Those two lines are now one call:windowClauseSql(query.limit, query.offset, sqlDialectFor(ctx, tableName))windowClauseSqlis the function the native face runs, already exported fromnative-sql-strategy.ts. That file is not edited, and there is no second spelling table.The ObjectQL face's echoed
sqland thePOST /api/v1/analytics/sqlbody (both aregenerateSql) now carry the same window clause the native face executes on the same driver.Measured at the real route
The harness is a scratch copy of
packages/runtime/src/analytics-query-window-validity.test.ts. It uses the realdispatcher-pluginmount overAnalyticsServicePluginand a realObjectQLengine withSqlDriver(better-sqlite3). It boots the default composition and one narrowed to the engine aggregate (the ObjectQL face). The query isorder: { note: 'asc' }, offset: 1, with no limit. Each echoed statement was then run on the same SQLite database through knex, to read SQLite's own diagnostic. The harness was deleted and never committed.mainbdd3654f2/queryrowssql=/sqlbody… ORDER BY "note" ASC OFFSET 1… ORDER BY "note" ASC LIMIT -1 OFFSET 1near "OFFSET": syntax error/sql… ASC LIMIT -1 OFFSET 1, x y zlimit: 2, offset: 1, ObjectQL echo… LIMIT 2 OFFSET 1, runsOn this query the ObjectQL echo now equals, byte for byte, the statement the native face runs on SQLite.
The dialect source (dispatch, Zone 2 item 2)
generateSqlalready has the strategy context in scope, and it already readssqlDialectFor(ctx, tableName)for its read-scope compile. That is the samesqlDialecthook, read the same way, thatNativeSQLStrategy.generateSqluses withsqlDialectFor(ctx, this.extractObjectName(cube)).tableNamehere isthis.extractObjectName(cube)too.AnalyticsServicePluginwires that hook from the data engine's driver whatever the query capabilities are, so the narrowed composition above readssqlite. No new hook.Pin
packages/services/service-analytics/src/__tests__/objectql-echo-offset-only-window.test.tshas three tests per cell:generateSql(the/analytics/sqlbody) equals the echo, with no params. FromORDER BYon, the echo equals the statement the native face executes on the same driver. Run through the engine's raw-SQL bridge, the echo answers the face's rows.unknownarm. AnObjectQLStrategywhose context wires nosqlDialecthook echoesLIMIT 9223372036854775807 OFFSET 1, and that statement runs.limit: 2, offset: 1keeps its bytes, and the echo runs.The cells:
OS_TEST_POSTGRES_URLis set, and a named skip otherwise. No CI step provisions that variable for this package, so CI runs only the SQLite cell. I ran it locally against PostgreSQL 16.14. There the echo keepsOFFSET 1alone, byte-identical to before. The comparison with the native statement starts atORDER BYbecause on PostgreSQL the native face also casts the summed column.Ablation at committed
2cfcc44ee, throughnode scripts/ablation-replace.mjs. The anchor hit once, and the blob changed9ed31b50c50btoe8b910c3c13f. The mutation restored the two old window lines. The pin readssrc/with nodist/leg (objectql is aliased to source, and the strategy is imported fromsrc). Result, with live PostgreSQL: 3 failed, 3 passed.OFFSET 1), and theunknownarm on both engines.I predicted that direction before the run. Restore was proven: the blob after restore equals HEAD
9ed31b50c50b, andgit diff HEADis empty.Tests and gates
All of these ran at head
b9456bff3, which is this branch withorigin/main8b123c0aemerged in. That merge brought PR #21424'snative-sql-strategy.tschange.windowClauseSqlis unchanged by it.pnpm --filter @objectstack/service-analytics test, withOS_TEST_POSTGRES_URLset to a local PostgreSQL 16.14: Test Files 170 passed (170), Tests 3957 passed (3957).os-verify-lockprintedVERDICT command-exit 0.pnpm --filter @objectstack/service-analytics typecheck:VERDICT command-exit 0.tsc --noEmit --listFileslists the new pin once. The packagetsconfigincludessrcand does not exclude tests.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands(no paths) derived 63 commands. All 63 ran atb9456bff3. 62 exited 0 on the first run.pnpm check:dual-build-cjs-loadsfirst answeredPREREQUISITE NOT MET(exit 3), because the workspace was not fully built. After a fullturbo run build(72 tasks, exit 0), it exited 0.--ranreconciles 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN. The list includescheck:nul-bytes,check:changeset-gate-self-tests,check:adr-0087-registration,check:changeset-no-majorandcheck:test-source-alias. No gate touchedAGENTS.md, and the tree stayed clean.eslint --no-inline-config --format jsonon the 2 touched.tsfiles read 2 files, 0 errors, 0 warnings, none ignored.eslint.config.mjsenables no type-aware linting (noparserOptions.project), so an untouched file's verdict cannot move. The repo-widepnpm lintis left to CI.Docs
I grepped
content/docs/**outsidereleases/for the analytics echo,/analytics/sqland window rendering.api/data-api.mdxdescribes/analytics/sqlas a dry run returning{ sql, params }, andapi/client-sdk.mdxshowsclient.analytics.explain. Neither says how a window renders. No sentence is made false, so there is no docs edit.Acceptance notes
date_trunc('month', closed_on). That holds on both faces, and in the default composition too, because the native face declines granularity. SQLite refuses it withno such function: date_trunc, measured on this branch after the window fix. Onmainthe same statement failed earlier, at the bareOFFSET. The driver runsstrftime('%Y-%m', …)there (driver-sqlsql-driver.ts). The comment overdimExprsays the echo renders "the SQL shape the driver's own bucketing implements", and on SQLite that is not so. This isdimExpr, not the two window lines, so it is reported to the seat rather than fixed here.mysql. The window cell iswindowClauseSql's own (LIMIT 18446744073709551615). It is NOT MEASURED on either face here, because no MySQL server is available. That is the same declared skip PR fix(analytics)!: a query window outside the non-negative integers is refused at the door, and an offset with no limit runs on SQLite #21399 carries.sqlDialecthook names no dialect now echoesLIMIT 9223372036854775807 OFFSET nwhere it echoedOFFSET n. The native face already runs that statement for such a host. It answers the same rows on PostgreSQL and SQLite, as the pin'sunknownarm measures.Generated by Claude Code