diff --git a/README.md b/README.md index a0a788ef..3a6d685d 100644 --- a/README.md +++ b/README.md @@ -221,7 +221,7 @@ and the [Cloud Data Contract](docs/cloud-data-contract.md). | --- | --- | | Default builders and reviewers | Claude Code + Codex | | Session hosts | Codex, Claude Code, and Cursor qualified; other identities recognized but require explicit handoff/provider transport | -| Devin | Maintained local builder and hosted release-qualification transport; local review remains informational and Devin is not yet a qualified peer orchestrator | +| Devin | Maintained local builder lane and an exact PR-bound hosted work-order library seam (`code_mower.devin_work_orders`, no packaged CLI command); local review remains informational and Devin is not yet a qualified peer orchestrator | | Organizational context | Optional Coworker delivery to approved Claude/Codex/Devin roles | | Work trackers | GitHub Issues by default; Jira Cloud is optional and guarded | | Forge and merge gate | GitHub | @@ -230,7 +230,8 @@ and the [Cloud Data Contract](docs/cloud-data-contract.md). | Slack | Command and authenticated bounded ingress foundation only; no Slack worker delivery, results, or orchestration authority | GitLab, Bitbucket, broad unattended rollout, uncalibrated merge gates, Devin -peer-orchestrator/reviewer parity, a required Graphify dependency, and Slack +peer-orchestrator/reviewer parity, a hosted work-order CLI, runtime role-policy +enforcement, a required Graphify dependency, and Slack worker delivery are not shipped in v1.4.0. The current priorities and boundaries are recorded in [Current State And Roadmap](docs/current-state-and-roadmap.md). diff --git a/docs/current-state-and-roadmap.md b/docs/current-state-and-roadmap.md index 3d5428a4..b34b6574 100644 --- a/docs/current-state-and-roadmap.md +++ b/docs/current-state-and-roadmap.md @@ -130,72 +130,130 @@ future hosted-service work. ## Near-Term Roadmap -The next three capabilities should ship as independently gated epics rather -than one cross-cutting implementation PR. - -### 1. Devin Peer Support - -Make Devin implement the same user-facing participant lifecycle as Claude and -Codex while keeping local CLI and hosted API mechanics inside separate -transports. The target includes: - -- session dispatch, progress, messaging, cancellation, result collection, and - recovery; -- qualification as a host/orchestrator; -- Coworker and later Graphify context delivery; -- structured current-head reviewer output; and -- a clean/blocked calibration campaign before any reviewer promotion. - -The maintained local builder and hosted Sessions API work are the starting -point. Reviewer authority remains evidence-based. - -### 2. Graphify Repository Context - -Treat Graphify as a repository-context provider beside Coworker, not as a -participant. Start local and code-only: - -- build the optional local provider against the conditions in the - [evaluation record](graphify-evaluation.md), which closes - [issue #876](https://github.com/codemower-ai/code-mower/issues/876) with an - adopt decision; -- add a provider registry and multiple context attachments per session; -- build and refresh graphs with commit/freshness validation — delivered by - `code-mower context-graph`, described in the - [lifecycle record](context-graph-lifecycle.md), which closes - [issue #913](https://github.com/codemower-ai/code-mower/issues/913); -- consume a pinned structured JSON contract; -- generate bounded impact, dependency, symbol, and related-test packets; and -- deliver the same packet shape to Claude, Codex, and Devin. - -Code Mower should own refresh policy and should not rely permanently on parsing -human-oriented MCP prose. - -### 3. Slack Task And Status Interaction - -Treat Slack as an interaction channel, not an orchestrator. A Slack-started -session uses the project or channel's configured default orchestrator unless the -request supplies an explicit one. The first local integration should use Socket -Mode and provide: - -- allowlisted workspace, channel, user, and repository mappings; -- idempotent task creation and Slack-thread-to-session binding; -- redacted progress and completion updates; -- clarification questions with reply-to-resume behavior; -- cancellation and restart reconciliation; and -- no raw private context or private reviewer findings in Slack. - -Slack should consume the durable session lifecycle and event surface introduced -for Devin rather than scrape terminal or Board output. +`v1.4.0` is released and its published artifacts are immutable. The agreed +sequence from the stabilization epic is: + +1. Close the seven `v1.4.0` stabilization implementation children plus the + [#974](https://github.com/codemower-ai/code-mower/issues/974) evidence + verification. These are main-only fixes. +2. Ship those fixes together with Graphify as `v1.4.1` + ([#915](https://github.com/codemower-ai/code-mower/issues/915)). +3. Ship Board as `v1.4.2`. Board work is underway; + [#935](https://github.com/codemower-ai/code-mower/issues/935) is complete. +4. Supervised Slack remains planned for `v1.5.0`; its runtime work is deferred + until the sequence above is complete. + +Devin support is a bounded builder qualification only: a maintained local CLI +builder lane and the exact PR-bound hosted work-order library seam. Hosted +dispatch has no packaged CLI command. Devin review stays informational and +Devin is not a qualified peer orchestrator. + +Role-policy enforcement +([#975](https://github.com/codemower-ai/code-mower/issues/975)) and +effective-authority rendering +([#955](https://github.com/codemower-ai/code-mower/issues/955)) are planned +runtime work, not shipped behavior. + +Each step below is an independently gated epic rather than one cross-cutting +implementation PR. + +### 1. Complete `v1.4.0` Stabilization ([#979](https://github.com/codemower-ai/code-mower/issues/979)) + +Seven main-only implementation PRs plus one evidence verification: + +- accurate advertised commands and live roadmap docs + ([#965](https://github.com/codemower-ai/code-mower/issues/965)); +- optional review defaults + ([#967](https://github.com/codemower-ai/code-mower/issues/967)); +- role-specific qualification and admission + ([#975](https://github.com/codemower-ai/code-mower/issues/975)); +- effective-authority and migration reporting + ([#955](https://github.com/codemower-ai/code-mower/issues/955)), after #975; +- quiescent, capable, consistent takeover + ([#962](https://github.com/codemower-ai/code-mower/issues/962)); +- contributor lineage and reviewer exclusion + ([#963](https://github.com/codemower-ai/code-mower/issues/963)), after #962 + and #975; and +- independent operational acceptance evidence + ([#976](https://github.com/codemower-ai/code-mower/issues/976)). + +#962 runs before #963 where handoff and provenance files overlap. #974 is +evidence-only verification of existing hosted aggregate freshness; a confirmed +hosted defect becomes a separately recorded implementation child and its own +hosted PR rather than an assumed fix. + +### 2. Graphify Repository Context — `v1.4.1` ([#902](https://github.com/codemower-ai/code-mower/issues/902) / release [#915](https://github.com/codemower-ai/code-mower/issues/915)) + +Graphify is a repository-context provider beside Coworker, not a participant. +It is adopted and partly delivered: the +[evaluation record](graphify-evaluation.md) closed +[#876](https://github.com/codemower-ai/code-mower/issues/876) with an adopt +decision, and `code-mower context-graph`, described in the +[lifecycle record](context-graph-lifecycle.md), closed +[#913](https://github.com/codemower-ai/code-mower/issues/913). Remaining work +is the query/packet PR +([#914](https://github.com/codemower-ai/code-mower/issues/914)), which consumes +a pinned structured JSON contract and generates bounded impact, dependency, +symbol, and related-test packets in one shape for Claude, Codex, and Devin. +Release #915 also carries the stabilization docs, review-default, and +operational-evidence work plus explicit freshness evidence. + +Installation stays opt-in, no command requires an index to exist, and Code +Mower owns refresh policy rather than parsing human-oriented MCP prose. + +### 3. Board Clarity And Session Visibility — `v1.4.2` ([#945](https://github.com/codemower-ai/code-mower/issues/945) / release [#952](https://github.com/codemower-ai/code-mower/issues/952)) + +Board implementation is underway rather than unstarted. +[#935](https://github.com/codemower-ai/code-mower/issues/935) is complete and +merged with [#973](https://github.com/codemower-ai/code-mower/issues/973); +[#956](https://github.com/codemower-ai/code-mower/issues/956) and +[#957](https://github.com/codemower-ai/code-mower/issues/957) are drafts behind +main that need refreshing before review. Remaining work is presentation and +producers, persistent Board services +([#961](https://github.com/codemower-ai/code-mower/issues/961)), and integrated +qualification ([#951](https://github.com/codemower-ai/code-mower/issues/951)), +then the release PR #952. #961 is required before #951 and #952, and #951 +consumes #975, #955, #962, #963, and #976 through its integration dependencies. + +Board is a read model over one closed local observation model. Missing or stale +evidence stays explicitly unknown or last-observed; Board never infers runtime +activity from a label, provider name, PID, PR author, lease, or command-line +prose. + +### 4. Supervised Slack Task And Status Interaction — `v1.5.0` ([#903](https://github.com/codemower-ai/code-mower/issues/903) / release [#923](https://github.com/codemower-ai/code-mower/issues/923)) + +Slack is an interaction channel, not an orchestrator. A real qualified Codex or +Claude supervisor controls bounded hosted work: missing supervisor readiness +blocks dispatch, and selecting a provider never promotes its role. Ingress +foundations [#916](https://github.com/codemower-ai/code-mower/issues/916) and +[#917](https://github.com/codemower-ai/code-mower/issues/917) are merged. +Remaining work is OAuth, the qualified-supervisor adapter +([#977](https://github.com/codemower-ai/code-mower/issues/977)), durable +interactions, the bridge, paired telemetry, setup +([#922](https://github.com/codemower-ai/code-mower/issues/922)), and release +acceptance #923. Slack consumes the durable session lifecycle and event surface +rather than scraping terminal or Board output, and carries no raw private +context or private reviewer findings. + +This runtime work is deferred until the sequence above is complete. Board +readiness gates only Slack's end-to-end canary and final acceptance in #923; it +does not block independent Slack OAuth, inbox, interaction, bridge, setup, or +documentation work. ## Delivery Order -1. Ship Devin lifecycle, host, context-recipient, and reviewer parity first. -2. The bounded Graphify evaluation is complete and adopted; ship the optional - local context provider after the shared context registry is stable, against - the conditions in the evaluation record. Installation stays opt-in and no - command requires an index to exist. -3. Define Slack's command and identity contract in parallel, but merge its - worker only after session lifecycle and recovery are stable. +1. Complete `v1.4.0` stabilization on main: the seven #979 implementation PRs + plus the #974 evidence verification. +2. Ship those main-only fixes together with Graphify as `v1.4.1` through #915, + after #914. +3. Ship Board as `v1.4.2` through #952, after #961 and #951. +4. Merge the supervised Slack runtime last, accepted in #923 for `v1.5.0`. + +Elapsed time, implementation difficulty, or an open draft PR never changes this +release order. An explicit evidence-backed Graphify deferral recorded in +#915/#902 may satisfy that one dependency. Merged post-`v1.4.0` fixes, including +#935/#973, count as on main until a later published package is verified to +contain them. Each child issue should produce one reviewable PR with one branch writer, independent current-head review, the normal gate, and package-level validation. diff --git a/docs/devin-peer-support-qualification.md b/docs/devin-peer-support-qualification.md index 52f888e9..177c82a0 100644 --- a/docs/devin-peer-support-qualification.md +++ b/docs/devin-peer-support-qualification.md @@ -71,9 +71,23 @@ model-quality estimate. ## Hosted API builder deliveries and recovery -Hosted Devin v3 work orders through `code-mower devin work-order`. Time is -public wall time from PR creation to merge; active provider time is -unavailable. Monetary cost is unavailable for both rows. +Hosted Devin v3 work orders were dispatched by the trusted orchestrator through +the packaged `code_mower.devin_work_orders.DevinWorkOrders` library seam; see the +[embedding example](devin-work-orders.md#embedding-example). +No packaged CLI command dispatches a hosted work order, and none is planned +here: a work-order CLI would be a separate product feature with its own issue, +not documentation written ahead of the code. Time is public wall time from PR +creation to merge; active provider time is unavailable. Monetary cost is +unavailable for both rows. + +Generic remote-session dispatch (`code-mower session`, the `RemoteSessions` +seam, and the provider transports beneath it) gives provider-neutral lifecycle +handling: create, status, message, cancel, and result collection. It does not +independently verify issue, branch, pull request, author, base, and exact head. +Only the exact PR-bound `DevinWorkOrders` path makes those observations +independently against GitHub and fails closed when any of them disagree. The +hosted rows below are evidence for that exact-bound path, not for lifecycle +dispatch alone. | Work order | ACU cap | Observed ACU | Public wall time | Active time | Work-order round | Exact-head audit cycles | Intervention | Verified PR / head | Recovery result | | --- | ---: | ---: | ---: | --- | ---: | --- | --- | --- | --- | @@ -278,6 +292,30 @@ branch, author, PR, head-SHA, and base-branch verification are unchanged, and a later valid exact-round collection clears the rejection and returns verified PR evidence. +## Post-release hosted-builder campaign conclusion + +The bounded hosted-builder campaign run after the v1.4.0 release is closed. It +is recorded here as supervised transport evidence only; it does not change the +recommendation posture below. + +- Bounded hosted-builder deliveries: 3 of 5. +- [#865](https://github.com/codemower-ai/code-mower/issues/865) and + [#935](https://github.com/codemower-ai/code-mower/issues/935) were completed + separately by Code Mower Codex in + [PR #971](https://github.com/codemower-ai/code-mower/pull/971) and + [PR #973](https://github.com/codemower-ai/code-mower/pull/973). +- The final #935 recovery attempt was cancelled by the user before delivery. It + is not a completed implementation failure and must not be counted as one. +- Correction to earlier `invalid_request` explanations: the observed failures + were caused by malformed or missing request keys, not by provider-side + rejection of the work order itself. Valid fix and cancel calls in the same + session subsequently succeeded. +- A structured logical completion does not prove that the provider session + exited. Code Mower treats the verified structured result as complete while + the remote session may remain resumable. +- Settled billing for the campaign remains unknown. Observed ACU is a + point-in-time consumption snapshot, never a final invoice. + ## Limitations - Local versus hosted: the local rows report active seconds and the hosted diff --git a/docs/devin-work-orders.md b/docs/devin-work-orders.md index 2a56b853..226eb1ea 100644 --- a/docs/devin-work-orders.md +++ b/docs/devin-work-orders.md @@ -7,12 +7,30 @@ existing `RemoteSessions`, `DevinProvider`, organization-scoped v3 client, priva It does not change campaign dispatch, release policy, review authority, or lane CLI capability declarations. No new lifecycle or automatic polling daemon is introduced. +This is a library seam, not a command. The packaged CLI exposes no hosted +work-order command; an embedding dispatcher calls the library directly, as in +the example below. A work-order CLI would be a separate product feature and +must be tracked as one before it is documented. + +Generic remote-session dispatch and this exact PR-bound path are not +interchangeable. Generic session dispatch — `code-mower session`, the +`RemoteSessions` seam, and the provider transports beneath it — provides +provider-neutral lifecycle handling: create, status, message, cancel, and +result collection. It does not independently verify issue, branch, pull +request, author, base, and exact head; a provider's own claims about those are +not observations. `DevinWorkOrders` adds exactly that independent verification: +every one of those bindings is observed against GitHub by the caller's own +credentials and fails closed on any disagreement. Use the exact-bound path +whenever hosted output becomes builder evidence. + The caller must first apply its trusted issue-author/work-order-comment policy and acquire the repository's existing builder lease. A work-order manifest is not proof of authorization. Pass the approved Markdown body explicitly; the library does not open manifest paths, load context files, or fetch an untrusted issue body. Repository, issue, branch, base, and expected GitHub author ID/login are dispatcher policy inputs. +## Embedding example + ```python from code_mower.devin_sessions import DevinClient from code_mower.devin_work_orders import DevinWorkOrders, WorkOrder diff --git a/docs/public-release-checklist.md b/docs/public-release-checklist.md index ac176c75..0adaaadb 100644 --- a/docs/public-release-checklist.md +++ b/docs/public-release-checklist.md @@ -183,7 +183,8 @@ interpreter. release-hygiene integration tests. - Add static-analysis gates in stages: broaden Ruff for stable subpackages first, then add scoped type checking as module contracts stabilize. -- Add a zero-config first-value experiment, such as `code-mower try OWNER/REPO`, +- Add a zero-config first-value experiment — a future "try this repository" + entry point, not a command that exists today — that can auto-discover recent PR history, generate a draft corpus, and produce a value report without asking a new user to understand the full calibration model first. diff --git a/tests/test_documented_commands.py b/tests/test_documented_commands.py new file mode 100644 index 00000000..54f34df0 --- /dev/null +++ b/tests/test_documented_commands.py @@ -0,0 +1,200 @@ +"""Documentation contract: advertised top-level commands must exist. + +Current guidance is only useful if a cold operator can run what it names. +These checks are fully offline: command handlers are replaced with recording +stubs, so nothing launches a provider, touches the network, or writes state. + +Historical records are excluded on purpose. The changelog and per-version +release notes describe commands as they were at that release; correcting them +would falsify the record. +""" + +from __future__ import annotations + +import re +import sys +import textwrap +import unittest +from pathlib import Path +from unittest import mock + +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src")) + +from code_mower import cli + + +ROOT = Path(__file__).resolve().parents[1] + +# `code-mower ` at a shell command position: the start of a line or +# code span, after a prompt marker, or after an operator that begins a new +# command. Prose that merely names the product mid-sentence -- "report +# command -v code-mower and code-mower --version" -- is not an invocation, so +# the following word is not a command. The command itself must be +# command-shaped and not part of a longer token such as `code-mower==1.4.0`. +COMMAND_POSITION = r"(?:^|\$\(|[;&|(])[ \t]*(?:[$>#][ \t]+)?" +INVOCATION_RE = re.compile( + COMMAND_POSITION + r"code-mower[ \t]+([a-z][a-z0-9-]*)(?![\w.=-])" +) +# Only a fence with no info string or a shell info string holds executable +# examples. `text` carries agent prompts, and `json`/`yaml`/`python` carry +# data, so a product name inside them is prose or payload, not a command. +FENCE_RE = re.compile(r"^ {0,3}(```|~~~)[ \t]*([^\s`]*)") +SHELL_FENCE_INFO = frozenset( + {"", "sh", "bash", "shell", "zsh", "console", "shell-session", "terminal"} +) +INLINE_CODE_RE = re.compile(r"`([^`\n]+)`") + +HISTORICAL_RE = re.compile(r"(^|/)(CHANGELOG\.md|release-history\.md|v[\d]+-release-notes\.md)$") + + +def _current_guidance_files() -> list[Path]: + paths = [ROOT / "README.md"] + paths.extend( + path + for path in sorted((ROOT / "docs").rglob("*.md")) + if not HISTORICAL_RE.search(path.as_posix()) + ) + return [path for path in paths if path.is_file()] + + +def _code_segments(text: str) -> list[str]: + """Return shell-block lines and inline code spans, ignoring prose. + + Two kinds of text name the product without running it: prose outside code + spans, and prompt or data blocks fenced as something other than a shell. + Neither contributes segments. Inside a fence, backticks are literal text + rather than markdown inline code, so no spans are read from one. + """ + segments: list[str] = [] + fence = "" + shell_block = False + for line in text.splitlines(): + match = FENCE_RE.match(line) + if match: + marker, info = match.group(1), match.group(2).lower() + if not fence: + fence = marker + shell_block = info in SHELL_FENCE_INFO + elif marker == fence: + fence = "" + shell_block = False + continue + if fence: + if shell_block: + segments.append(line) + continue + segments.extend(INLINE_CODE_RE.findall(line)) + return segments + + +def _commands_in(text: str) -> set[str]: + """Return the top-level commands one document advertises as runnable.""" + return { + command + for segment in _code_segments(text) + for command in INVOCATION_RE.findall(segment) + } + + +def _advertised_commands() -> dict[str, set[str]]: + """Map each advertised command to the files advertising it.""" + advertised: dict[str, set[str]] = {} + for path in _current_guidance_files(): + relative = path.relative_to(ROOT).as_posix() + for command in _commands_in(path.read_text(encoding="utf-8")): + advertised.setdefault(command, set()).add(relative) + return advertised + + +class DocumentedCommandTests(unittest.TestCase): + def test_extractor_finds_the_commands_readme_actually_shows(self) -> None: + # Guards the extractor itself: a silently empty scan would make every + # other check in this file vacuous. + advertised = set(_advertised_commands()) + expected = {"init", "doctor", "session", "lanes", "board", "productivity"} + self.assertTrue(expected.issubset(advertised), msg=sorted(advertised)) + + def test_extractor_reads_invocations_and_not_prose(self) -> None: + # Prose that names the product mid-sentence, and prompt blocks fenced + # as `text`, are not invocations: the word after the product name is + # English, not a command. The rule is positional rather than a list of + # excluded words, so an unseen prose word cannot slip through and an + # unseen command is still caught. + document = textwrap.dedent( + """\ + Run `code-mower inline-fabricated --json` first, and before and + after the install report command -v code-mower and code-mower + --version. + + ```text + Before and after the install, report command -v code-mower and + code-mower --version. Then run code-mower prompt-fabricated. + ``` + + ```bash + code-mower shell-fabricated --repo OWNER/REPO + cd repo && code-mower chained-fabricated + ``` + """ + ) + self.assertEqual( + _commands_in(document), + {"inline-fabricated", "shell-fabricated", "chained-fabricated"}, + ) + + def test_prose_in_the_prompt_pack_is_not_read_as_a_command(self) -> None: + # Regression for the shipped corpus: docs/orchestrator-prompt-pack.md + # tells an agent to "report command -v code-mower and code-mower + # --version" inside a `text` prompt block. + advertised = _advertised_commands() + self.assertNotIn("and", advertised) + self.assertIn("doctor", advertised) + + def test_every_advertised_command_parses_under_the_packaged_cli(self) -> None: + advertised = _advertised_commands() + self.assertTrue(advertised) + calls: list[list[str]] = [] + + def handler(argv: list[str]) -> int: + calls.append(argv) + return 0 + + for command, sources in sorted(advertised.items()): + with self.subTest(command=command): + calls.clear() + self.assertIn( + command, + cli.COMMAND_HANDLERS, + msg=f"{command} is documented in {sorted(sources)} but is not a command", + ) + with mock.patch.dict( + cli.COMMAND_HANDLERS, {command: handler}, clear=False + ): + self.assertEqual(cli.main([command, "--offline-contract-probe"]), 0) + self.assertEqual(calls, [["--offline-contract-probe"]]) + + def test_every_advertised_command_has_a_help_description(self) -> None: + for command in sorted(_advertised_commands()): + with self.subTest(command=command): + self.assertTrue(cli.COMMAND_DESCRIPTIONS.get(command)) + + def test_fabricated_command_is_rejected(self) -> None: + # `code-mower devin work-order` was documented for hosted work orders + # and never existed; hosted dispatch is the DevinWorkOrders library + # seam. A fabricated command must fail to parse rather than dispatch. + for argv in (["devin", "work-order"], ["work-orders"], ["devin-work-order"]): + with self.subTest(argv=argv): + self.assertNotIn(argv[0], cli.COMMAND_HANDLERS) + with self.assertRaises(SystemExit) as raised: + cli.main(argv) + self.assertEqual(raised.exception.code, 2) + + def test_no_document_advertises_a_hosted_work_order_command(self) -> None: + pattern = re.compile(r"code-mower[ \t]+devin[ \t]+work-order") + for path in _current_guidance_files(): + with self.subTest(path=path.relative_to(ROOT).as_posix()): + self.assertIsNone(pattern.search(path.read_text(encoding="utf-8"))) + + +if __name__ == "__main__": + unittest.main()