Skip to content

Docs: replace the nonexistent hosted-Devin work-order command - #981

Merged
jeffhuber merged 3 commits into
mainfrom
claude/965-docs-work-order-command
Sep 14, 2026
Merged

Docs: replace the nonexistent hosted-Devin work-order command#981
jeffhuber merged 3 commits into
mainfrom
claude/965-docs-work-order-command

Conversation

@jeffhuber

@jeffhuber jeffhuber commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Closes #965.

Hosted-Devin qualification guidance now directs operators to the packaged DevinWorkOrders embedding example and explains the difference between generic remote-session lifecycle commands and independently verified, exact PR-bound work orders. An offline documentation contract rejects nonexistent advertised top-level CLI commands.

The current roadmap now follows completion of v1.4.0 stabilization, shipment of those fixes with Graphify v1.4.1, then Board v1.4.2. Supervised Slack remains planned for v1.5.0. Historical qualification evidence is preserved, including the bounded 3/5 hosted-builder result, user-cancelled final recovery, corrected request-key explanation, and unknown settled billing.

Builder: Code Mower Claude through the maintained local lane runner. Independent reviewer: Code Mower Codex. The draft hold remains for roadmap integration; no release artifacts are changed.

Validation at 11e2b8d6c7ed5812d6ef301490450b82dfe0dc34: 41 focused tests and 289 subtests passed in the canonical repository environment, covering the new command contract, documentation links, current-release hygiene, and the merged review-default regressions. Ruff, public privacy scan, and whitespace checks passed. The independent exact-head Codex audit passed with no findings; its own environment lacked the test dependency, so test results are recorded separately. Normal CI passed, including all three Python package matrices and both graph-containment jobs. The authoritative code-mower/gate status passed on this head.

Hosted Devin work orders were documented as `code-mower devin work-order`,
which the packaged CLI never exposed. Point the qualification guide at the
maintained DevinWorkOrders library seam and its embedding example, and state
what generic session dispatch does and does not verify.

Add tests/test_documented_commands.py: every top-level command advertised in
current guidance must parse and dispatch under the packaged CLI, and a
fabricated command must fail to parse. The check is offline; handlers are
replaced with recording stubs.

Closes #965

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jeffhuber jeffhuber added builder:claude Code Mower generated label needs-codex-audit documentation Improvements or additions to documentation labels Sep 14, 2026
@jeffhuber

Copy link
Copy Markdown
Contributor Author

Code Mower validation: changes required at 6c431ad

Two blocking corrections are required before the independent exact-head audit:

  1. The new focused test run failed: tests/test_documented_commands.py plus tests/test_documentation.py produced 2 failed, 7 passed, 170 subtests passed. _code_segments treats prose in the text-fenced prompt at docs/orchestrator-prompt-pack.md:68 as a shell invocation, so it extracts and from prose after the product name. Fix the extractor to distinguish executable command examples and inline code from prose/text prompts. Add an extractor regression proving the prose case is ignored while a fabricated inline/shell command is still detected. Do not add an arbitrary allowlist of non-command English words.

  2. docs/current-state-and-roadmap.md still contradicts its new checkpoint: the later Near-Term Roadmap and Delivery Order still promise host/reviewer parity first, omit Board, and prescribe the old Slack/local Socket Mode sequence. This document is current guidance, not an archival release record. Rewrite those obsolete sections to the current Roadmap through v1.5.0: bounded builders, Graphify, Board, and supervised Slack #900/Epic: qualification, handoff reliability, and operational evidence through v1.5.0 #979 sequence: fully complete v1.4.0 stabilization, ship its main-only fixes with Graphify v1.4.1, then Board v1.4.2, and keep qualified-supervisor Slack v1.5.0 runtime planned/deferred. Include the real Board implementation state and existing PR mapping. Retain the release-baseline sentence and historical qualification tables, but do not retain contradictory current instructions.

The docs/public-release-checklist.md wording-only change is approved. Do not broaden source ownership. Keep this same draft PR and builder:claude provenance, resolve the corrections, push the new head, and update the validation statement truthfully. The orchestrator will run canonical-environment tests and an independent Code Mower Codex audit. Do not merge.

Fix round for the two blocking corrections at 6c431ad.

The documentation contract extractor treated every fenced line as a shell
invocation, so prose inside the text-fenced prompt block at
docs/orchestrator-prompt-pack.md:68 advertised "and" as a command.
Extraction is now positional rather than an allowlist of English words:
only fences with no info string or a shell info string hold executable
examples, and an invocation must sit at a shell command position. A new
regression proves the prose case is ignored while fabricated inline and
shell commands are still detected, plus a corpus regression pinning the
prompt-pack line.

docs/current-state-and-roadmap.md no longer contradicts its own checkpoint.
The Near-Term Roadmap sections and Delivery Order now carry the #900/#979
sequence: complete v1.4.0 stabilization on main, ship those fixes with
Graphify as v1.4.1, then Board as v1.4.2, with supervised Slack deferred to
v1.5.0. Board's real implementation state and the existing PR mapping are
recorded. The release-baseline sentence and qualification tables are
unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jeffhuber

Copy link
Copy Markdown
Contributor Author

Fix round pushed: 7cec22b

Both blocking corrections from the validation comment at 6c431ad are addressed.

1. Extractor treated prose as a shell invocation

_code_segments appended every line of every fenced block, so the text-fenced
agent prompt at docs/orchestrator-prompt-pack.md:68 ("report command -v
code-mower and code-mower --version") yielded and as an advertised command,
failing both the parse check and the help-description check.

The fix is structural, not an allowlist of English words:

  • Fence classification. Only a fence with no info string or a shell info
    string (sh, bash, shell, zsh, console, shell-session, terminal)
    holds executable examples. text prompt blocks and json/yaml/python
    data blocks contribute nothing. Inside any fence, backticks are literal text
    rather than markdown inline code, so no spans are read from one.
  • Command position. An invocation must sit where a shell command starts: the
    beginning of a line or inline code span, after a $/>/# prompt marker, or
    after ;, &, |, (, or $(. A product name mid-sentence is a mention,
    not an invocation, so the following word is never treated as a command.

Either rule alone kills the reported false positive; both are in place because
bare ``` fences are the most common fence in this corpus and also carry prose.

Two regressions added:

  • test_extractor_reads_invocations_and_not_prose — a fixture document in which
    the prose sentence and the text-fenced prompt (both containing the exact
    reported phrasing) are ignored, while a fabricated inline command, a fabricated
    shell-block command, and a fabricated command after && are all still
    detected. Asserted as set equality, so a silent over- or under-match fails.
  • test_prose_in_the_prompt_pack_is_not_read_as_a_command — pins the shipped
    corpus: and is not advertised, doctor still is.

Extraction only narrows, so no new command can enter the advertised set from
this change. The six commands asserted by the existing extractor guard
(init, doctor, session, lanes, board, productivity) all appear at
line start inside bash fences in README, so the guard against a vacuous scan
still holds.

2. Roadmap sections contradicted the new checkpoint

docs/current-state-and-roadmap.md kept a Near-Term Roadmap body and
Delivery Order that promised Devin host/reviewer parity first, omitted Board,
and prescribed the local Socket Mode Slack sequence. Those sections are rewritten
to the current #900/#979 sequence:

  1. Complete v1.4.0 stabilization (Epic: qualification, handoff reliability, and operational evidence through v1.5.0 #979) — the seven main-only implementation
    PRs (Docs: replace the nonexistent hosted-Devin work-order command #965, Work orders: keep optional Gitar out of the default review lanes #967, Participants: enforce role-specific qualification for hosted builders #975, Adoption polish: report effective review authority and superseded Devin bridge drift #955, Local builder handoff: make takeover quiescent, Git-write capable, and runner-consistent #962, Builder takeover: make provenance, labels, and reviewer exclusion handoff-aware #963, Release evidence: separate completion, provider exit, usage settlement, and cloud freshness #976) with their real ordering
    constraints (Adoption polish: report effective review authority and superseded Devin bridge drift #955 after Participants: enforce role-specific qualification for hosted builders #975; Local builder handoff: make takeover quiescent, Git-write capable, and runner-consistent #962 before Builder takeover: make provenance, labels, and reviewer exclusion handoff-aware #963), plus Cloud: verify aggregate freshness after accepted release metadata #974 as evidence-only
    verification whose confirmed defect would become a separately recorded child.
  2. Graphify v1.4.1 (Epic: Graphify local repository context for v1.4.1 #902 / release Release: qualify and publish optional Graphify context in v1.4.1 #915) — adopted and partly delivered
    (Context: evaluate official Graphify as a local repository provider for v1.4.1 #876/context: adopt Graphify as an optional bounded local provider and prove the graph contract offline #924 and Graphify: private revision-bound local graph lifecycle #913/context: bind a local graph to its revision and fail closed on everything else #926 merged); remaining work is the Graphify: bounded query normalization and context packets #914 query/packet PR,
    and Release: qualify and publish optional Graphify context in v1.4.1 #915 also carries the stabilization docs, review-default, and
    operational-evidence work plus freshness evidence.
  3. Board v1.4.2 (Epic: Board clarity and provider-neutral session visibility for v1.4.2 #945 / release Release: publish and qualify Board clarity v1.4.2 #952) — real implementation state recorded:
    Sessions: discover the current local brief and lease from a checkout #935 complete and merged with Sessions: discover and validate the current checkout brief #973, Board: freeze provider-neutral observation contract #956/Board: correct existing-data truth and hierarchy #957 drafts behind main needing a
    refresh, remaining presentation/producers, persistent services Board: manage persistent services and reject stale keepalive bindings during release restart #961, and
    integrated qualification Board: integrate head-bound evidence and qualify local and hosted session visibility #951 (which consumes Participants: enforce role-specific qualification for hosted builders #975, Adoption polish: report effective review authority and superseded Devin bridge drift #955, Local builder handoff: make takeover quiescent, Git-write capable, and runner-consistent #962, Builder takeover: make provenance, labels, and reviewer exclusion handoff-aware #963, Release evidence: separate completion, provider exit, usage settlement, and cloud freshness #976),
    then Release: publish and qualify Board clarity v1.4.2 #952.
  4. Supervised Slack v1.5.0 (Epic: Slack session ingress and supervision for v1.5.0 #903 / release Release: qualify and publish Slack session ingress in v1.5.0 #923) — qualified-supervisor
    framing (missing supervisor readiness blocks dispatch; selecting a provider
    never promotes its role), Slack: freeze command, identity, lifecycle, privacy, and event contracts #916/Slack: define session ingress and supervision contract #929 and Slack: app manifest and authenticated bounded ingress envelope #917/Slack: authenticate and durably receive bounded ingress #933 merged, remaining OAuth,
    supervisor adapter Slack: connect authorized tasks to a qualified Code Mower supervisor #977, durable interactions, bridge, paired telemetry,
    setup Slack: optional integration setup, doctor, and operating guidance #922, acceptance Release: qualify and publish Slack session ingress in v1.5.0 #923. Runtime work stays deferred; Board readiness
    gates only the end-to-end canary and final acceptance.

Delivery Order now states the same order and records that elapsed time,
implementation difficulty, or an open draft PR never changes it; that an
explicit evidence-backed Graphify deferral in #915/#902 may satisfy that one
dependency; and that merged post-v1.4.0 fixes count as on main until a later
published package is verified to contain them.

Retained unchanged: the release-baseline sentence and pinned install spec, the
Capability Matrix and qualification tables, Known Limits, and the bounded-Devin
paragraph. No speculative CLI is documented anywhere in the diff.

Validation

Python execution is denied in this lane's sandbox (/usr/bin/python3 -c "print(1)" is refused), so the focused suites were again not run locally. The
change is documentation plus one test file, so CI on this head is the gate:
tests/test_documented_commands.py, tests/test_documentation.py,
tests/test_release_hygiene.py, Ruff, and the full suite.

Checked by inspection against the shipped corpus rather than by execution:
fence info strings across README and docs/ (584 bare, 498 bash, 28 text,
18 json, 15 yaml, 15 sh, 2 python, 2 mermaid); the prompt-pack fence
spanning lines 48–98 that contains line 68; every mid-line code-mower <word>
occurrence in current guidance; the six guard commands at line start in README
bash fences; and the test_release_hygiene.py assertions on this document
(baseline sentence, current-status paragraph, stale-phrase list) — none of which
the rewrite touches.

Remaining

Nothing outstanding from the two corrections. If CI on 7cec22b disagrees with
the extractor reasoning above, that is the one place this round could still be
wrong, since it was verified by inspection rather than by a local run.
Re-requesting the independent exact-head audit with needs-codex-audit.

@jeffhuber

Copy link
Copy Markdown
Contributor Author

Codex audit (merge-authority lane)

Head SHA: 7cec22b0ffc54500d0165d4bcea2e86e3abd259d
Findings: P0=0, P1=0, P2=0, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: PASS

Summary:

No actionable defects were found. Tests could not be fully validated because the environment was missing the packaging dependency.

Findings: none.

@jeffhuber

Copy link
Copy Markdown
Contributor Author

Code Mower maintenance: refresh #981 for integration

The orchestrator verified that #980 merged into main as 529b70697908c4748106740fb241698783609fc9, leaving this draft behind. Code Mower Claude remains the sole branch writer.

Starting from the current PR head 7cec22b0ffc54500d0165d4bcea2e86e3abd259d, merge current origin/main into this same branch and resolve any routine conflicts while preserving both accepted changes. Do not rewrite or remove the work-order review-default fix from #980. Keep this PR draft and keep builder:claude attribution; do not merge the PR or release anything. This is a maintenance refresh, not a new feature or a new PR.

Run available relevant checks, push the resulting head, and request a fresh independent Codex audit. The orchestrator will run the canonical environment tests and Code Mower audit wrapper against the new immutable head. Prior head reviews are historical after this refresh. Do not begin #975 implementation; that remains dependent on #965 merge.

@jeffhuber

Copy link
Copy Markdown
Contributor Author

Maintenance refresh pushed: 11e2b8d

PR: #981
Previous head: 7cec22b0ffc54500d0165d4bcea2e86e3abd259d
New head: 11e2b8d6c7ed5812d6ef301490450b82dfe0dc34

What changed

Merged current origin/main (529b70697908c4748106740fb241698783609fc9, #980)
into this same branch with a merge commit. No conflicts: #980 touches
docs/planning-work-orders.md, src/code_mower/work_orders.py, and
tests/test_work_orders.py, and this branch touches none of those files.
The work-order review-default fix from #980 is carried in unmodified, and no
commit on this branch was rewritten. The PR stays draft with builder:claude;
nothing was merged or released.

Contract review of the merged content

docs/planning-work-orders.md gains a bash fence from #980 whose lines are
continuation flags (--review-lane codex-audit --review-lane claude-audit --review-lane gitar) with no code-mower token, so the documented-command
extractor reads no new advertised command from it. The changed
attach-delivery example swaps a --reviewer-check value only. The roadmap
document does not reference #980, so no section went stale from this merge.
This is a static review, not a substitute for the test run below.

Tests

Not run on this head from the lane runner: every Python invocation was denied
by the sandbox in this session, including the exact commands
./scripts/dev-python -m pytest tests/test_documented_commands.py tests/test_documentation.py tests/test_work_orders.py -q and
/opt/homebrew/bin/python3.12 -m pytest tests/test_documented_commands.py -q.
git diff --check origin/main...HEAD is clean. Per the maintenance request,
the orchestrator runs the canonical environment tests and the Code Mower audit
wrapper against this immutable head; the 13 focused tests and 286 subtests
recorded at 7cec22b are historical after this refresh.

Remaining

  1. Fresh independent exact-head Codex audit at
    11e2b8d6c7ed5812d6ef301490450b82dfe0dc34 (label swapped:
    codex-audit-done removed, needs-codex-audit added).
  2. Canonical-environment test run at this head by the orchestrator.
  3. Draft hold stays for roadmap integration. Participants: enforce role-specific qualification for hosted builders #975 implementation not started;
    it remains dependent on Docs: replace the nonexistent hosted-Devin work-order command #965 merge.

@jeffhuber

Copy link
Copy Markdown
Contributor Author

Codex audit (merge-authority lane)

Head SHA: 11e2b8d6c7ed5812d6ef301490450b82dfe0dc34
Findings: P0=0, P1=0, P2=0, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: PASS

Summary:

No actionable defects were identified. Test execution was blocked by a missing packaging dependency under Python 3.12.

Findings: none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

builder:claude Code Mower generated label codex-audit-done documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs: replace the nonexistent hosted-Devin work-order command

1 participant