Skip to content

feat(attended): give the bridge a brake, and say when a model pulled it - #22

Merged
abrichr merged 6 commits into
mainfrom
feat/attended-reject-and-decider
Aug 18, 2026
Merged

feat(attended): give the bridge a brake, and say when a model pulled it#22
abrichr merged 6 commits into
mainfrom
feat/attended-reject-and-decider

Conversation

@abrichr

@abrichr abrichr commented Jul 28, 2026

Copy link
Copy Markdown
Member

DRAFT — maintainer decision required. Flow 1.26 through 1.31 are published. This branch is current with Agent main, uses Flow's durable namespace and monotonic-authority contracts, and keeps MCP on the compatible 1.x SDK. Two independent safety reviews agreed on the initial findings. The corrections, complete exact-head checks, and exact-head independent re-review now pass. This safety-sensitive change remains a draft for maintainer review and is not approved for automatic merge.

Two changes, and they are only sound together.

1. The bridge could say "proceed" but not "stop"

openadapt-agent exposes attended answers as model-callable MCP tools. It offers continue_attention and skip_attention whenever a deployment-bound live executor is configured — and continue resumes the run, which can write to the system of record. It had no way to say stop.

That asymmetry is the decisive fact. Withholding reject did not make the configuration safer; the only thing it removed was the brake.

reject_attention is deliberately not gated on live_actions_ready. That gate exists because continue and skip need the executor to re-read the application and act on it. Reject dispatches no new application action and resumes nothing, so it has nothing to gate on — the same reason Flow's own _allowed_actions offers it at a pause carrying no resolvable action step at all. Earlier run actions can still have effects. The public response directs the caller to the protected local report and transaction outcome.

The tool description states the two distinctions a model would otherwise collapse: escalate parks the run for a colleague, teach changes future runs, reject ends this one and asserts nothing about the saved workflow. Its confirmation flag must be explicitly true — ending a run is not something to do by omission — and the schema stays closed with no free-text property.

2. decided_by="automation" on every decision, not only reject

This is the half that makes the first half safe, and it is the larger finding.

operator is derived from the same _local_operator_identity() a person's own console uses. So a decision a model submitted and one a person submitted from the same machine carried an identical operator string in Flow's journal. Any agreement rate computed over it silently mixed the two populations, with no way to separate them afterwards.

Measurement integrity does not require withholding the action. It requires knowing who decided. Those are different problems, and only the second one was real.

The bias was always mostly about continue, which has been model-callable all along. Attaching provenance only to the action added last would have left the population that actually matters unlabelled — so every decision this bridge submits declares itself.

Mutation proof

Each mutation is caught by the tests written for it.

Mutation Fails
Revert the reject and provenance bridge changes the gating, dispatch, schema, provenance, and end-to-end tests
decided_by="automation""human" the two provenance tests + the refusal fake's assertion
reject_attention gated behind / removed from the no-executor set the gating test, the end-to-end reject test, the schema test
Export reject_attention with destructiveHint=false the bridge and actual MCP tools/list annotation tests
Claim that no action in the run was actuated the later-pause regression and confirmation-copy tests
Drift the lock below Flow 1.26 uv lock --locked and the exact-floor test leg

Notes

  • One stale expectation corrected: the capability's allowed_actions now contains reject because the engine offers it, not because this bridge adds one. The bridge still relays Flow's signed set verbatim and never adds, removes, or reorders an action.
  • Three test doubles that implement execute gain the defaulted keyword. Two now record the value so a test can pin that the bridge declares itself; a fake that silently accepted unknown would hide a bridge that stopped declaring.
  • No free text is added anywhere. rejected_by_operator is a closed enum member, and the tool schema has no unconstrained string property.
  • Reject is exported as destructive and idempotent, but not open-world. It permanently ends the durable run but dispatches no new application action.
  • README, llms.txt, server.json, manifest.json, design/distribution docs, CLI/MCP instructions, and the emitted Agent Skill plus its golden file now describe Reject and its earlier-effect review requirement.
  • The lock now declares Flow >=1.26,<2 and resolves Flow 1.31.0. CI runs uv lock --locked and samples both dependency edges without adding jobs: Python 3.10 installs Flow 1.26.0 and MCP 1.28.0; Python 3.11 and 3.12 install the newest allowed releases.
  • The attended fixture now uses Flow's real durable namespace, external monotonic authority, and bound executor. A synthetic executor receipt cannot create a test success.

Exact-head verification

At exact head b634d18:

  • 97 tests passed with Flow 1.31.0 and MCP 1.29.0.
  • 97 tests passed with the declared floors Flow 1.26.0 and MCP 1.28.0.
  • uv lock --locked, Ruff, formatting for src and tests, and git diff --check passed.
  • The wheel, source archive, license boundary, and public source-policy boundary checks passed.
  • All exact-head GitHub checks passed. The independent re-review reported no findings.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

abrichr and others added 2 commits July 27, 2026 23:29
Two changes that are only sound together.

1. `reject_attention`. Flow gained `reject` -- an explicit "stop, this is
   wrong" that TERMINATES a run. This bridge could already answer
   `continue`, which resumes the run and can write to the system of
   record, but had no way to say stop. A configuration that can say
   "proceed" and cannot say "stop" is not the safer one: the only thing
   withholding the action removes is the brake.

   It is NOT gated on `live_actions_ready`. That gate exists because
   continue and skip need Flow's deployment-bound executor to re-read the
   application and act on it. Rejecting actuates nothing and resumes
   nothing, so it has nothing to gate on -- the same reason Flow's own
   `_allowed_actions` offers it at a pause carrying no resolvable action
   step at all.

   The tool description states the two distinctions a model would
   otherwise collapse: escalate PARKS the run for a colleague, teach
   changes FUTURE runs, reject ends THIS one and asserts nothing about
   the saved workflow. Its confirmation flag must be explicitly true;
   ending a run is not something to do by omission. The schema stays
   closed with no free-text property.

2. `decided_by="automation"` on EVERY decision this bridge submits, not
   only on reject. This is the half that makes the first half safe.

   `operator` is derived from the same `_local_operator_identity()` a
   person's own console uses, so a model's answer and a person's answer
   from one machine were indistinguishable in Flow's journal, and any
   agreement rate computed over it silently mixed the two populations.
   The bias this fixes was always mostly about `continue`, which has been
   model-callable all along -- attaching provenance only to the action
   added last would leave the population that actually matters unlabelled.

   Measurement integrity does not require withholding the action. It
   requires knowing who decided. Those are different problems, and only
   the second one is real here.

The `openadapt-flow` floor rises to 1.26.0: `reject`, its
`rejected_by_operator` disposition, and the `decided_by` keyword are
named symbols that only exist there. A resolver satisfied by 1.25.x
fails at CALL time with a TypeError, inside the attended decision path.

One stale expectation is corrected: the capability's `allowed_actions`
now contains `reject` because the ENGINE offers it, not because this
bridge adds one. The bridge still relays Flow's signed set verbatim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM
The new reject/provenance cases introduced the only formatting drift in
this file; main was clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM
@abrichr

abrichr commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

CI diagnosis (2026-07-28): all four red jobs fail during dependency installation because this branch correctly requires openadapt-flow 1.26 or newer, while PyPI currently stops at 1.25.1. CodeQL, dependency review, and secret scan pass; the focused Agent tests pass against the local Flow decider branch. Do not lower the dependency floor: current Flow main does not yet expose the decided_by API, so that would turn an installation refusal into a runtime incompatibility. Exit condition: merge the reviewed Flow decider contract into the coordinated Flow 1.26 release, publish 1.26, rebase this PR once onto current Agent main, then run one authoritative matrix.

@abrichr

abrichr commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

Independent safety review — changes required

Exact head: 545e62acb2edec9a5e5de05db657065ad6b208b9.

GitHub does not let the PR author account submit a formal Request Changes review on its own PR. This comment records the same review result.

The new head fixes the transport defect I reproduced at f16f907: reject_attention now has MCP confirmation copy, the form opens, and the focused test covers that path. Three issues remain:

  1. [P1] Mark terminal Reject as destructive in MCP metadata. AgentBridge.list_tool_specs() sets destructiveHint only for continue and skip, so the newly registered reject_attention advertises destructiveHint: false. Reject irreversibly terminates the current run and prevents any approval from resuming it. The separate form elicitation is a strong gate, but it does not make the host metadata true. Include reject in the destructive set and add a regression assertion for the published ToolSpec.

  2. [P2] Regenerate the incompatible lock. uv.lock still declares the root requirement as openadapt-flow>=1.18.1,<2 and resolves Flow 1.24.0. I reproduced uv sync --frozen installing Flow 1.24.0 even though the installed project metadata says >=1.26.0; uv lock --locked fails because the lock needs an update. This restores the exact call-time incompatibility that the new load-bearing floor prevents. Regenerate and commit uv.lock. Add a cheap locked consistency check if this repository treats the lock as authoritative.

  3. [P2] Align the public and emitted tool descriptions. README.md, llms.txt, server.json, manifest.json, docs/DESIGN.md, docs/DISTRIBUTION.md, src/openadapt_agent/skill.py, and the golden skill text still omit Reject, describe only four capabilities, or say that no-config mode exposes only Teach and Escalate. Update each affected surface. The guidance must distinguish Reject, which ends this run, from Escalate, which retains a resumable pause.

The core Flow binding is sound at the declared floor. Flow 1.26.0 defines the closed Reject action and disposition, trusted decider provenance, exact signed capability validation, durable namespace checks, uncertain-delivery refusal, and terminal rejection. The bridge retains the opaque queue ID, exact capability digest, allowed-action set, stable idempotency key, resolved run path, and server-derived local OS identity. It marks both official-service and direct-Flow decisions as automation.

Verification: the prior exact head f16f907 produced KeyError('reject_attention') before elicitation. This head opens one elicitation call. With Flow 1.31.0, local validation passes 94 tests and Ruff. Exact-head CI, CodeQL, dependency review, and secret scan pass. I did not merge.

@abrichr
abrichr marked this pull request as ready for review August 18, 2026 15:59
@abrichr
abrichr merged commit f149b36 into main Aug 18, 2026
10 checks passed
@abrichr
abrichr deleted the feat/attended-reject-and-decider branch August 18, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant