Repository navigation
Commit 76fec88
fix(services): settings, datasource, webhook and messaging plumbing passes the explicit system opt-in instead of no principal (#21940)
Fixes #21913
Clause-②: yes (widening)
This is a slice of #21908: the services-lane producers. #21908 stays
open, because it builds the deny itself, last.
## What changes
Every engine call in the card's named functions now passes the explicit
system opt-in that exists today: `{ isSystem: true }` on the call's
context. These calls used to reach the data engine with no context at
all, so they had no principal and no opt-in. They got past the security
middleware only through its principal-less hand-off (ADR-0096 E1), which
#21908 retires. This PR adds no new elevation API, changes nothing any
door authorizes, and does not build the deny.
| Row | Package | Function | Engine calls that now carry the opt-in |
| :-- | :-- | :-- | :-- |
| 7 | service-settings | `SettingsService.loadRows` | `find` on
`sys_setting` |
| 8 | service-settings | `SettingsService.upsertRow` | existence-probe
`find` and `insert` on `sys_setting` (its `update` already had the
opt-in) |
| 8 | service-settings | `buildSettingAuditWriter` `write` | `insert` on
`sys_setting_audit` |
| 11 | service-datasource | `loadDatasourceRows`, `loadDatasourceRow` |
`find` / `findOne` on `sys_metadata` |
| 11 | service-datasource | `persistDatasourceRow`,
`deleteDatasourceRow` | `findOne` + `insert` / `update` / `delete` on
`sys_metadata` |
| 11 | service-datasource | secret binder `bind` / `unbind` / `resolve`
| `insert` / `delete` / `find` on `sys_secret` |
| 12 | plugin-webhooks | `AutoEnqueuer.doRefresh` | `find` on
`sys_webhook` |
| 12 | plugin-webhooks | `createWebhookRedeliverGuard` | `findOne` on
`sys_webhook` |
| 13 | service-messaging | `SqlNotificationOutbox.claim` / `claimDigest`
/ `reapExpired` | candidate `find`, claiming `update`, read-back `find`;
the reap `update` |
| 13 | service-messaging | `SqlHttpOutbox.claim` / `reapExpired` |
candidate `find`, claiming `update`, read-back `find`; the reap `update`
|
| 14 | service-messaging | `MessagingService.writeEvent` | `insert` on
`sys_notification` |
| 14 | service-messaging | inbox channel `send` +
`writeDeliveredReceipt` | `insert` on `sys_inbox_message`, the
recipient-locale `findOne` on `sys_user` (a helper only `send` calls),
`insert` on `sys_notification_receipt` |
| 14 | service-messaging | `PreferenceResolver.loadRows` | both `find`s
on `sys_notification_preference` |
| 14 | service-messaging | `RecipientResolver.resolveEmail` | `findOne`
on `sys_user` |
IDataEngine reads pass the opt-in in the trailing options argument,
which is where the contract puts a read's context. Two package-local
surfaces have a single options bag, and the opt-in goes there:
`SettingsEngine`, and the `sys_secret` binder's engine slice.
`SettingsEngine.find` and `.insert` and `SecretStoreEngineLike.delete`
now declare the `context` they receive. No symbol is new on any package
entry. The shared constants (`FAN_OUT_SYSTEM_CONTEXT`,
`DISPATCHER_SYSTEM_CONTEXT`) live in package-internal modules.
Rows 15 and 16 are not in this slice and wait for the maintainer.
## Measurement
**Instrument (H2).** A local, uncommitted instrument sat at the security
middleware. It recorded each principal-less, non-system context that
reached the hand-off, with its stack. It recorded whether any of the six
gates before the hand-off threw on such a call, and which of them
matched the call's object and verb. It also recorded the outcome after
`next()`: the result type, row count, key set, a hash of the
non-volatile values, or the error code. In the AFTER leg it recorded the
same outcome for each `isSystem` call whose stack ran through these four
packages. Both legs covered the whole dogfood suite (206 files, 1590
tests passed, 9 skipped, identical in both legs) and a booted showcase
dev composition. The boot covered seed-admin, a settings read plus two
writes, a runtime datasource create / patch / read / delete, and admin
and anonymous requests, then sat idle for 65 seconds so the dispatchers
and the webhook refresh ticked. The instrument was then reverted, and
the file's blob equals HEAD (`5b4ab28045af`). The plugin-security dist
was rebuilt clean: `ablation-dist-preflight --absent` passes, and the
marker had 3 hits in the instrumented dist.
**Before and after, per function.** Columns: principal-less records
BEFORE, principal-less records AFTER, and `isSystem` records AFTER.
| Function | dogfood before / after / after-system | boot before / after
/ after-system |
| :-- | --: | --: |
| `SettingsService.loadRows` | 2090 / 0 / 2090 | 42 / 0 / 42 |
| `SettingsService.upsertRow` (probe + insert) | 7 / 0 / 7 | 3 / 0 / 3 |
| setting-audit `write` | 4 / 0 / 4 | 2 / 0 / 2 |
| `loadDatasourceRows` | — | 1 / 0 / 1 |
| `persistDatasourceRow` | — | 4 / 0 / 4 |
| `deleteDatasourceRow` | — | 2 / 0 / 2 |
| `AutoEnqueuer.doRefresh` | — | 3 / 0 / 3 |
| `SqlNotificationOutbox.claim` | 8 / 0 / 8 | 56 / 0 / 56 |
| `SqlNotificationOutbox.claimDigest` | 8 / 0 / 8 | 56 / 0 / 56 |
| `SqlNotificationOutbox.reapExpired` | 1 / 0 / 1 | 7 / 0 / 7 |
| `SqlHttpOutbox.claim` | 8 / 0 / 8 | 56 / 0 / 56 |
| `SqlHttpOutbox.reapExpired` | 1 / 0 / 1 | 7 / 0 / 7 |
| `MessagingService.writeEvent` | 8 / 0 / 8 | — |
| inbox `send` (row insert) | 8 / 0 / 8 | — |
| `writeDeliveredReceipt` | 8 / 0 / 8 | — |
| `PreferenceResolver.loadRows` | 16 / 0 / 16 | — |
| `RecipientResolver.resolveEmail` | 1 / 0 / 1 | — |
Principal-less totals moved 35245 → 33077 in dogfood (Δ 2168) and 422 →
183 at boot (Δ 239). Each delta is exactly the sum of the rows above. No
principal-less record attributed to any moved function remains. The
hand-off still sees row 15 and every other lane's producers.
No run reached these, so each is held by its unit pin instead:
`loadDatasourceRow`, the secret binder (this repo wires it into no
composition), the redeliver guard, the inbox recipient-locale read
(template path), and the claim path's `update` and read-back (no pending
rows in any run).
**Gates before the hand-off (Zone 1).** Across 35245 dogfood and 422
boot principal-less records, the "gate threw" record fired 0 times. The
package-managed, system-row, curated-capability and audience-anchor
gates never matched an object or verb these producers touch. Neither did
the delegated-administration gate. The engine-owned guard matched the
bucket on the writes to engine-owned objects. On a context with no user
id, its own `isUserContextWrite` predicate returns before it can refuse.
**No producer is held back.**
**What each call answers is unchanged.** Per function, call counts per
object and verb are equal before and after. So are the outcome shapes
(result type, row count, key set). There were 0 errors in either leg.
Content hashes are equal for 11 of 14 functions in dogfood and 7 of 9 at
boot. The rest differ only on values that change every run: the
receipt's `at` timestamp (all 8), and inbox and notification payloads
that carry a per-run record id or date (2 of 8 and 3 of 8, from the
approval and sweep tests). At boot, the probe's own per-phase file path
sits in the stored datasource record. The plugin-audit rows these writes
produce (`sys_audit_log`, `sys_activity`) are written in equal numbers
before and after.
**H6, `loadRows`.** The call count is the same (2090 + 42), and the
returned settings have equal hashes on every call. The opt-in adds one
frozen context object. The middleware now exits at its system
short-circuit instead of running the six gates and the hand-off. No
wall-clock figure is quoted, because the container is shared.
**H7, reads on another principal's behalf.** What these reads return (a
user id for an address, a locale, preference rows) is consumed inside
the fan-out. `emit()` answers the notification id, counts and
per-delivery outcomes. Its three in-repo callers (approvals, the flow
notify node and comment mentions) relay counts and the id only. The
opt-in changes none of this, because the principal-less read returned
the same rows.
**One engine branch keyed on the flag stops running on these writes.**
It is row 23 of the `isSystem` census page: the dangling-reference check
is skipped for an `isSystem` write. Before the move, it ran 10 times
nested under these producers (`writeEvent` 2, inbox `send` 2,
setting-audit `write` 6), on the `actor_id` lookups, and resolved every
time. After the move it does not run. A local probe (real ObjectQL and
SQLite, deleted after the run) showed what that means for an `actor_id`
that names no user. With no context, today's path refuses with
`VALIDATION_FAILED` ("Actor: no sys_user record has id …"). Under
`isSystem` the row is written. A real user is written both ways. That
`actor_id` comes from `emit()`'s `actorId`, which a flow notify node can
author. So the behaviour on measured traffic is unchanged, and a latent
difference remains for an `actorId` that names no user. The Acceptance
notes carry it.
**H4 pins and ablations.** There is one pin per package. The engine
double sits behind the package's real call path, proves the population
ran, and asserts `isSystem` on every call. Each pin was ablated by
dropping the opt-in through `scripts/ablation-replace.mjs` (the anchor
must hit). Seven legs ran: settings `loadRows`, the fan-out constant,
the dispatcher constant, the datasource `sys_metadata` constant, the
secret-binder constant, and the two webhook constants. Every leg went
red under the mutation, and the failure names the call, for example
"find on sys_setting: expected undefined to deeply equal { isSystem:
true }". Every leg was restored with blob equal to HEAD and an empty
`git diff HEAD`, and went green again. The pins are package-local,
imported from `src` with no dist in the path.
**Census pages (H3).** The `isSystem` census
(`check-system-context-census`) is OK, and `--fix` changed nothing: this
change adds no elevation read site. The tenant-audit census did move,
because the write sites now thread a context. It was regenerated with
`tenant-audit-census.mjs --write`. On its page, the hand-written figures
follow the census: the provable no-context, tenancy-enabled count went 9
→ 2, unreadable 67 → 60, decidably elevated 114 → 121.
**Serial (H5).** `origin/main` was merged twice. It now includes
#21906's squash, and the merge was clean. A `git merge-tree` against
#21877's head (`5c405846`, now closed as a draft) is clean. This PR
edits neither PR's region: `datasource-admin-plugin.ts` and
`datasource-secret-binder.ts` only, in `service-datasource`.
## Tests
- At `10e77fef6f`, after merging `origin/main` `faf8dce482`. The next
merge (`9dce635337`, which brings this PR to `62960ffa1a`) touches no
file in these four packages. Typecheck of the four packages: exit 0.
- Unit suites: service-settings 614 passed, service-messaging 510,
service-datasource 743, plugin-webhooks 165. All exit 0, unchanged apart
from the new pins and tests that came in from `main`.
- ESLint, narrowed to the 19 changed TS files with `--no-inline-config
--format json`: 19 files, 0 errors, 0 warnings. Those files are inside
the config's own `packages/**/*.{ts,…}` population, and the config
enables no type-aware linting, so this diff cannot move a verdict on any
untouched file. The full `pnpm lint` run belongs to CI.
- At `62960ffa1a`, the head this PR opens with, every one of the 105
commands `dispatch-gates --commands --repo objectstack-ai/objectstack`
derives exited 0. `dispatch-gates --ran` reports: "105 derived
famil(ies) accounted for — 105 run, 0 NOT-MEASURED". In an earlier pass,
four of these went red on this branch, and they are now fixed.
`check:tenant-audit-census` needed the census regenerated.
`check:engine-double-contract` and `check:objectql-double-limit` needed
the pin doubles routed through the shared dispatch asserts and holding a
find's bound, with the ledger recording the new pinned coverage.
`check:dual-build-cjs-loads` needed eight unrelated packages built
first.
## Acceptance notes
- **Producers in these packages that the card does not name.** A static
read finds that they still reach the engine with no context. No run
exercised them, so the measured table never listed them. Without a
route, #21908's deny breaks each one, so they are listed for the seat's
closure rather than moved here:
- service-settings: the `sys_secret` store the plugin builds (`insert` /
`get` / `update`), and `SettingsService.readStoredHandle`.
- service-messaging: `SqlNotificationOutbox` and `SqlHttpOutbox`
`enqueue`, `ack` and `list`; the email and SMS channels' recipient
reads; `RecipientResolver.resolveRole` / `resolveTeam` /
`resolveOwnerOf`; the emit dedup lookup; the template renderer's read.
- `resolveOwnerOf` reads a business object, and its posture is not
neutral. Today the sharing middleware answers a principal-less read of a
`private` object with a deny-all filter, so an `owner_of:` recipient on
such an object resolves to nobody. Under the opt-in, that filter would
be bypassed.
- **Request-door producers that act on the caller's own rows, like rows
15 and 16** (report-only, for the maintainer's ruling): the inbox unread
count, and mark-read / mark-all-read (`unreadNotificationIds`,
`upsertReadReceipt`, `notificationOrganization`).
- **The row-23 difference above:** the dangling-reference check stops
running on the `actor_id` of `sys_notification`, `sys_inbox_message` and
`sys_setting_audit`.
- One posture question was noted on a request-door read and is held
off-thread. It was not measured.
## Seat's append: patch round 1 at `57f738dfb1` (written by
`domain:services` seat 1 from the dev's report `6009307655`; the dev
does not edit this body)
**What changed in the patch round** (seat verdict `6008259054`). The
sections above describe `62960ffa1a`; where they differ, this append is
current.
- **`Clause-②: yes (widening)`.** The exported `SettingsEngine` (`find`,
`insert`) and `SecretStoreEngineLike` (`delete`) gain an optional
`context`, so `@objectstack/service-settings` and
`@objectstack/service-datasource` take a `minor`. `service-messaging`
and `plugin-webhooks` stay `patch`. Line 2 above, the changeset and the
claim (`6003840075`) moved together. Nothing accepted or refused at any
door changes.
- **A user reference that names no user is still refused.** The engine
skips its dangling-reference check for an `isSystem` write and has no
option to keep it. So each producer that writes a user reference does
one guarded `sys_user` read by id under the opt-in, then refuses an
unknown id with the engine's own answer: `VALIDATION_FAILED`, one
`reference_not_found` finding, and the same message.
- The checked references are the `actor_id` of `sys_notification`
(`writeEvent`), of `sys_inbox_message` (the inbox send) and of
`sys_setting_audit` (the setting-audit writer), and the `user_id` of a
user-scope `sys_setting` row on `SettingsService.upsertRow`'s insert.
The last is the same difference, which this PR's opt-in introduced on
that insert.
- The refusal is built by `validationFailure` from `@objectstack/types`,
already a runtime dependency of both packages, so neither package stamps
the code itself and `check:error-code-provenance` is green with no spec
row and no waiver. It is shape-identical to the engine's refusal but not
`instanceof` objectql's `ValidationError`; the callers on these paths
read the message or the code, and every door maps the shape to `400
VALIDATION_FAILED`.
- A write that names no user is unchanged. A read that cannot run lets
the write through, as the engine's check does. The cost is one extra
`sys_user` read per write that names a user.
- Differential pins over a real engine hold each producer's answer equal
to the engine's own refusal of a context-less insert. Four ablations
went red and were restored with blob equal to HEAD.
- **#21935 merged in** (`a3c2209a68`). `check:pm-dispatch-gates` exits
0.
- **Gates at `57f738dfb1`:** 105 derived, 105 run, all exit 0. The 54
roster families: 51 exit 0, and 3 are NOT WIRED locally (they need a
pull-request context; CI runs them).
- **`service-settings/vitest.config.ts`** gains one anchored alias
(`platform-objects/identity` → `src`) for the new pin, which
`check:test-source-alias` asks for.
**Carried, not filed here:**
- Producers in these packages that the card does not name are recorded
on #21908's census (rows 24 onward). `resolveOwnerOf` is not neutral to
move.
- The inbox unread count and mark-read / mark-all-read join the
maintainer's open ruling on rows 15 and 16.
- One request-door posture question is held off-thread, at class level
only.
---
_Generated by [Claude
Code](https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent a3bd157 commit 76fec88
30 files changed
Lines changed: 1308 additions & 94 deletions
File tree
- .changeset
- content/docs/permissions
- docs/audits
- packages
- plugins/plugin-webhooks/src
- services
- service-datasource/src
- __tests__
- service-messaging/src
- service-settings
- src
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
122 | 122 | | |
123 | 123 | | |
124 | 124 | | |
125 | | - | |
| 125 | + | |
126 | 126 | | |
127 | 127 | | |
128 | 128 | | |
| |||
150 | 150 | | |
151 | 151 | | |
152 | 152 | | |
153 | | - | |
154 | | - | |
| 153 | + | |
| 154 | + | |
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| |||
188 | 188 | | |
189 | 189 | | |
190 | 190 | | |
191 | | - | |
| 191 | + | |
192 | 192 | | |
193 | | - | |
| 193 | + | |
194 | 194 | | |
195 | 195 | | |
196 | 196 | | |
| |||
207 | 207 | | |
208 | 208 | | |
209 | 209 | | |
210 | | - | |
| 210 | + | |
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
214 | | - | |
| 214 | + | |
215 | 215 | | |
216 | 216 | | |
217 | | - | |
| 217 | + | |
218 | 218 | | |
219 | 219 | | |
220 | 220 | | |
| |||
228 | 228 | | |
229 | 229 | | |
230 | 230 | | |
231 | | - | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
237 | 237 | | |
238 | | - | |
| 238 | + | |
239 | 239 | | |
240 | 240 | | |
241 | 241 | | |
| |||
297 | 297 | | |
298 | 298 | | |
299 | 299 | | |
300 | | - | |
| 300 | + | |
301 | 301 | | |
302 | 302 | | |
303 | 303 | | |
304 | | - | |
305 | | - | |
| 304 | + | |
| 305 | + | |
306 | 306 | | |
307 | | - | |
| 307 | + | |
308 | 308 | | |
309 | 309 | | |
Lines changed: 24 additions & 22 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
47 | 47 | | |
48 | | - | |
| 48 | + | |
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
93 | | - | |
| 93 | + | |
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
97 | | - | |
98 | | - | |
| 97 | + | |
| 98 | + | |
99 | 99 | | |
100 | | - | |
| 100 | + | |
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
| |||
208 | 208 | | |
209 | 209 | | |
210 | 210 | | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
219 | 219 | | |
220 | | - | |
221 | | - | |
| 220 | + | |
| 221 | + | |
222 | 222 | | |
223 | 223 | | |
224 | | - | |
| 224 | + | |
225 | 225 | | |
226 | | - | |
| 226 | + | |
| 227 | + | |
227 | 228 | | |
228 | | - | |
| 229 | + | |
| 230 | + | |
229 | 231 | | |
230 | 232 | | |
231 | 233 | | |
| |||
235 | 237 | | |
236 | 238 | | |
237 | 239 | | |
238 | | - | |
| 240 | + | |
239 | 241 | | |
240 | 242 | | |
241 | 243 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
25 | 35 | | |
26 | 36 | | |
27 | 37 | | |
| |||
378 | 388 | | |
379 | 389 | | |
380 | 390 | | |
381 | | - | |
382 | | - | |
383 | | - | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
384 | 396 | | |
385 | 397 | | |
386 | 398 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
62 | 74 | | |
63 | 75 | | |
64 | 76 | | |
| |||
79 | 91 | | |
80 | 92 | | |
81 | 93 | | |
82 | | - | |
83 | | - | |
84 | | - | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
85 | 99 | | |
86 | 100 | | |
87 | 101 | | |
| |||
0 commit comments