Skip to content

test: prove the supported operator workflow through an independent MCP client #848

Open

Description

@joshrotenberg

Queue position -- 2026-10-05

Start a small executable baseline after #843 and the first #849/#850 shared projection. Add agreement/workstream reconstruction and revision-bound outcomes to the fixture journey. Expand with #846 and later client capabilities as they land; it is a continuing acceptance matrix, not a requirement to finish every platform feature before shipping the dashboard.

Goal

An independent client can carry out the complete supported operator workflow without LiveView, internal Elixir calls, or undocumented knowledge.

Use a tiny client/conformance harness as a concrete consumer of the existing single-operator API. This is the acceptance test for the five backend gaps filed alongside this issue, not a new desktop application.

Acceptance journey

  1. Authenticate, initialize, discover capabilities and reconstruct current state.
  2. Select/create a configured agent, submit a prompt with a durable identity, simulate losing the response, and retry/reconcile without duplicate execution.
  3. Observe queued/executing/progress states and distinguish request acceptance from eventual completion.
  4. Answer a question and resolve supported approval decisions, including workflow launch/budget attention. Exercise stale and duplicate decisions and unauthorized callers.
  5. Retrieve the full answer/report and supported attachment/output references with documented result shapes.
  6. Disconnect/reconnect and restart the isolated server, reauthenticate as needed, recover history via documented pagination/cursors and correlate the original message.
  7. Exercise supported queue/stopping controls and show unavailable/unknown guarantees truthfully. A pause is not evidence of physical subprocess settlement.

Contract consistency work

For the operations this journey actually uses, document/version machine-readable result and error shapes, pagination boundaries, asynchronous state meaning, stale-state behavior and safe retry/reconciliation rules. Add declared output schemas where supported and useful. Cover both MCP tool errors and JSON-RPC errors; HTTP 200 is not an operation-success signal.

Do not require every historical tool to be redesigned before the first proof. Start with current capabilities, record exact gaps, and add cases as the related issues land. Unsupported capability handling can pass its own test but must not be reported as delivery of that capability.

Proof boundaries

Use an isolated server/database and deterministic provider fixtures for repeatable CI. Record tested versions and evidence. A native-provider smoke is separate, bounded work requiring applicable authorization; no new paid runs are authorized here.

Do not use the running fleet, weaken approvals, change #554/#555, or revive #836. Verify denied MCP calls while keeping authorization distinct from execution confinement. The current observe-mode repository grant behavior is a documented limit, not an arbitrary-agent sandbox.

Existing client contract, lib/custode/cli/client.ex and docs/mcp/authorization.md provide the baseline. No UI, new transport stack or generic platform rewrite is included.

Related client-contract work

Activity

  1. added
    p2Standard queue
    area/mcpThe MCP surface, tools, identity, the mix custode CLI
    area/infraCI, deps, config, docs, tests
    on Oct 5, 2026
  2. joshrotenberg commented on Oct 9, 2026

    @joshrotenberg
    ContributorAuthor

    Independent agreement/client-reconnect proof shipped in #862 (merge 1d4bb04).

    A Python standard-library client now runs the complete agreement lifecycle separately over 2025-11-25 and 2026-07-28 HTTP: discovery, progress, exact retry, attributed checkpoint/submission, human resolution, revision and stale refusal, and paginated reconstruction by a fresh interpreter. HTTP authorization, JSON-RPC and tool errors are distinct. The isolated harness verifies no agent work is dispatched.

    Both protocol cases passed seeds 1/12345/777, all five local gates, and all CI checks on updated head d190918.

    This parent stays open. Remaining acceptance includes durable prompt/status/outcome journeys with deterministic provider fixtures, actual server restart recovery, attachments and remaining #846 actions, and supported stopping/delegation. Reconnecting the client is not restarting the server; synthetic acceptance is not independent verification of actual work. See docs/mcp/conformance.md for the exact proof and limits.

  3. added and removed on Oct 10, 2026
  4. joshrotenberg commented on Oct 12, 2026

    @joshrotenberg
    ContributorAuthor

    Workflow launch decision coverage shipped in PR #877 (merge 472cc20).

    The independent Python client discovers proposals through attention, approves and rejects over both supported protocol versions, and checks retry reconciliation, stale/opposite decisions, authority refusals, declared receipt schemas and error layers. A fresh interpreter recovers using retained proposal IDs. The isolated fixture verifies exactly one admitted run and initial job, the retained rejection reason and no unrelated message dispatch.

    Both protocol journeys passed focused seeds 1/12345/777 and all five local gates. All four CI checks passed on updated head a81e4b2 before squash merge.

    This is a bounded slice; #848 stays open. It does not inject a lost-response transport fault, restart the server or execute provider jobs. Remaining acceptance includes prompt/status/outcome journeys, actual server restart, attachments, remaining operator actions and stopping/delegation. See docs/mcp/conformance.md for the exact proof and limits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/infraCI, deps, config, docs, testsarea/mcpThe MCP surface, tools, identity, the mix custode CLIp2Standard queue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions