diff --git a/.env.example b/.env.example index 643748c..e4efd63 100644 --- a/.env.example +++ b/.env.example @@ -9,9 +9,15 @@ TYPESAFE_API_KEY= # Set TYPESAFE_API_KEY above to your OpenRouter key and uncomment this base URL. # TYPESAFE_BASE_URL=https://openrouter.ai/api -# Codex is the default worker backend. OpenCode is available as an unsandboxed alternative. +# Backends: codex (default), opencode, hermes, or deepagents. # FOREMAN_WORKER_BACKEND=codex +# Deep Agents Code requires a separately installed dcode CLI (deepagents-code >=0.1.83). +# FOREMAN_DEEPAGENTS_EXECUTABLE=dcode +# FOREMAN_DEEPAGENTS_MODEL=openai:your-model +# FOREMAN_DEEPAGENTS_MAX_TURNS=200 +# FOREMAN_DEEPAGENTS_SHELL_ALLOW_LIST=recommended + # Live steering uses Codex App Server by default. Use `exec` as a non-steerable fallback. # FOREMAN_CODEX_BACKEND=app-server # FOREMAN_STEERING_ENABLED=true diff --git a/README.md b/README.md index b7131f1..b8e6a0a 100644 --- a/README.md +++ b/README.md @@ -130,7 +130,7 @@ than Codex-specific types. ### Worker backends The semantic-supervision loop is agent-agnostic. Select the worker backend with -`FOREMAN_WORKER_BACKEND` (`codex`, the default, `opencode`, or `hermes`): +`FOREMAN_WORKER_BACKEND` (`codex`, the default, `opencode`, `hermes`, or `deepagents`): ```bash FOREMAN_WORKER_BACKEND=opencode foreman run --repo ./my-project --job "Add request retries" @@ -138,10 +138,16 @@ FOREMAN_WORKER_BACKEND=opencode foreman run --repo ./my-project --job "Add reque The OpenCode backend runs `opencode run` non-interactively and streams its output like the Codex exec backend. The Hermes backend runs `hermes chat` in headless single-query mode and decodes its -newline-delimited JSON events into tool-activity summaries. Live steering into an active turn is only +newline-delimited JSON events into tool-activity summaries. The Deep Agents backend runs the +separately installed `dcode` CLI headlessly with bounded output and turn/time budgets. +Live steering into an active turn is only available with the Codex App Server backend; other backends degrade to stop/retry. See [Worker backends](docs/workers.md) for the `Worker` protocol and how to add your own. +Deep Agents also supports interactive attached supervision through native hooks. The core +`foreman deepagents setup` command merges those hooks while preserving existing configuration. +See the [Deep Agents setup guide](integrations/deepagents/README.md) for both paths. + ## What Foreman watches Each observation is compact and bounded. It contains: @@ -284,6 +290,8 @@ invent one. Its minimum assessment interval defaults to five seconds and is conf - For the OpenCode backend: the [OpenCode CLI](https://opencode.ai) on `PATH` with an available provider and model. - For the Hermes backend: the Hermes Agent CLI (`hermes`) on `PATH`. +- For the Deep Agents backend: `dcode` on `PATH` (`deepagents-code >= 0.1.83`, which needs + Python 3.12+) and credentials for the selected model provider. ## Installation @@ -382,7 +390,7 @@ printf '%s\n' '{"session_id":"thr_123","cwd":"/workspace/project","hook_event_na | foreman hook --client codex ``` -The explicit `--client` selects `codex`, `pi`, or `pi-durable`; `codex` remains +The explicit `--client` selects `codex`, `deepagents`, `pi`, or `pi-durable`; `codex` remains the compatibility default. Foreman does not infer a client from arbitrary JSON, and unknown clients fail closed. The adapter normalizes events before they reach the shared attached-worker runtime and translates semantic outcomes back into client-specific hook JSON. @@ -447,7 +455,11 @@ The most useful environment overrides are: | `FOREMAN_MAX_ITERATIONS` | `20` | Semantic decision ceiling | | `FOREMAN_MAX_CONSECUTIVE_ASSESSMENT_FAILURES` | `3` | Tolerated supervisor failures before escalation | | `FOREMAN_CODEX_BACKEND` | `app-server` | `app-server` for steering or `exec` fallback | -| `FOREMAN_WORKER_BACKEND` | `codex` | `codex`, `opencode`, or `hermes` worker backend | +| `FOREMAN_WORKER_BACKEND` | `codex` | `codex`, `opencode`, `hermes`, or `deepagents` worker backend | +| `FOREMAN_DEEPAGENTS_EXECUTABLE` | `dcode` | Installed Deep Agents Code executable | +| `FOREMAN_DEEPAGENTS_MODEL` | unset | Native provider/model identifier | +| `FOREMAN_DEEPAGENTS_MAX_TURNS` | `200` | Headless agentic turn limit | +| `FOREMAN_DEEPAGENTS_SHELL_ALLOW_LIST` | `recommended` | Native shell preset or command allow-list | | `FOREMAN_STEERING_ENABLED` | `true` | Allow Jev-informed active-turn guidance | | `FOREMAN_MAX_STEERS_PER_WORKER` | `1` | Steering attempts before stop/retry | | `FOREMAN_STEERING_GRACE_SECONDS` | `30` | Time to recover before another intervention | diff --git a/docs/README.md b/docs/README.md index f8e9838..78aa620 100644 --- a/docs/README.md +++ b/docs/README.md @@ -10,6 +10,8 @@ These notes explain Foreman's design and implementation: - [Responsibility configuration and routing](routing.md) explains central definitions and matching. - [Extensions](extensions.md) defines registration, lifecycle, snapshots, and hierarchical routing. - [Coding-assistant hooks](hooks.md) describes the adapter-neutral attached-worker protocol. +- [Worker backends](workers.md) describes owned coding-agent transports. +- [Deep Agents setup](../integrations/deepagents/README.md) covers dcode workers and native hooks. - [Live steering](steering.md) explains how Jev guidance reaches an active Codex turn. - [Releasing](releasing.md) documents the build and trusted-publishing process. diff --git a/docs/hooks.md b/docs/hooks.md index 3663b5c..a7e9dcf 100644 --- a/docs/hooks.md +++ b/docs/hooks.md @@ -29,7 +29,8 @@ target repository. ## Client adapters The `--client` option selects the protocol adapter. Foreman does not guess from arbitrary input -JSON. It ships `codex`, `pi`, and `pi-durable` adapters. Codex remains the default for compatibility: +JSON. It ships `codex`, `deepagents`, `pi`, and `pi-durable` adapters. +Codex remains the default for compatibility: ```bash foreman hook --client codex @@ -41,6 +42,15 @@ its assistant's event names and fields into `HookEvent`, then renders Foreman's session code only sees those normalized types. Adding another assistant therefore requires a new adapter and registry entry, not another supervision runtime. +Deep Agents Code uses native Hooks v2 command handlers. Run `foreman deepagents setup` +to merge the lifecycle handlers into its user configuration, then start a fresh `dcode` +session. `foreman hook --client deepagents` normalizes native session, prompt, tool, +failure, and stop events; it preserves `prompt_id` and `stop_hook_active`. Post-tool +stop directives become native feedback, while pre-tool and completion hooks provide +operation blocking and continuation control. See the +[Deep Agents setup guide](../integrations/deepagents/README.md) for credentials, +responsibilities, configuration preservation, native limitations, and verification. + Pi uses the TypeScript [bridge package](../integrations/pi/README.md), which wraps native events in a session/cwd envelope and applies semantic response objects through Pi's native APIs. Its events are `session_start`, `input`, `tool_call`, `tool_result`, `agent_before_settle`, and diff --git a/docs/releases/0.5.0.md b/docs/releases/0.5.0.md new file mode 100644 index 0000000..75dd140 --- /dev/null +++ b/docs/releases/0.5.0.md @@ -0,0 +1,38 @@ +# Foreman 0.5.0 + +This release adds LangChain Deep Agents Code (`dcode`) to Foreman core as both a +headless worker backend and an agent harness supervised through native hooks. + +- Select `FOREMAN_WORKER_BACKEND=deepagents` to run coding and verification jobs + through the separately installed CLI. Configure its executable, model, turn + limit, and shell allow-list through Foreman's environment settings. +- Stream bounded output from both pipes, enforce worker deadlines, recognize + native budget exhaustion, and stop descendant process groups, including the + separate LangGraph server, during supervisory termination. +- Use `foreman hook --client deepagents` for seven native lifecycle events, + including tool failures, completion checks, and session cleanup. Existing + responsibilities, routing, evidence, and completion allowances apply. +- Run `foreman deepagents setup` to merge native Hooks v2 handlers atomically + while preserving other settings. Repeating setup with the same executable + does not duplicate handlers. +- Follow the [installation and setup guide](../../integrations/deepagents/README.md) + for provider credentials, repository scope, custom paths, and troubleshooting. + +Foreman keeps Python 3.11+ support and gains no LangChain SDK dependency. Install +`deepagents-code >= 0.1.83` separately with Python 3.12+. Native hooks and the +worker were tested with CLI 0.1.83 and SDK 0.7.23. + +Live OpenAI runs on isolated copies of the sample calculator repository passed: +Foreman launched a coding worker and independent verifier, reached `FINISHED`, +and passed 36 tests; the separate autonomous native-hook run passed 34 tests and +all 26 hook invocations. The original sample repository remained unchanged. +See the [live test report](../../integrations/deepagents/live-test-results.md) +for scope and limitations. + +The automated suite passes 430 Python tests. Native post-tool stop directives +provide model feedback; pre-tool denial and Stop handle their respective control +boundaries. The headless worker uses stop/retry steering. This integration does +not configure a sandbox, and its shell allow-list should match the project. + +Since 0.4.4, the repository also documents OpenRouter setup through the TypeSafe +SDK and pins GitHub Actions to immutable commits for CI and publishing. diff --git a/docs/releasing.md b/docs/releasing.md index 2e0ca5e..9c6e927 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -3,7 +3,7 @@ Foreman's PyPI distribution is `foreman-core`. It provides the `foreman` import package and `foreman` command. -Prepared release notes: [Foreman 0.4.4](releases/0.4.4.md). +Prepared release notes: [Foreman 0.5.0](releases/0.5.0.md). ## One-time setup diff --git a/docs/workers.md b/docs/workers.md index a1aca32..97174b8 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -23,6 +23,7 @@ one capability and three coroutines: | Codex exec | `FOREMAN_CODEX_BACKEND=exec` | No — stop/retry only | | OpenCode | `FOREMAN_WORKER_BACKEND=opencode` | No — stop/retry only | | Hermes | `FOREMAN_WORKER_BACKEND=hermes` | No — stop/retry only | +| Deep Agents Code | `FOREMAN_WORKER_BACKEND=deepagents` | No — stop/retry only | The OpenCode backend shells out to `opencode run` in non-interactive mode. The prompt is passed positionally and `--auto` keeps the headless run from @@ -46,6 +47,14 @@ stream. Command flags can be tuned per worker non-interactive CLIs, Hermes has no live-turn input channel: `steer` always returns `False` and the policy falls back to stop/retry. +The Deep Agents backend uses the separately installed `dcode` CLI +(`deepagents-code >= 0.1.83`, Python 3.12+). It runs `--non-interactive` with +`--max-turns`, `--timeout`, and a configurable `--shell-allow-list` (default +`recommended`). It streams bounded text chunks from both pipes and maps native +budget exit 124 to a timed-out worker. It does not opt into project hook trust +or configure a sandbox. See the [Deep Agents setup guide](../integrations/deepagents/README.md) +for model credentials, backend settings, and the separate native-hook path. + ## Adding a backend 1. Implement the `Worker` protocol (see `OpenCodeWorker` for the diff --git a/integrations/README.md b/integrations/README.md index d89b3c4..5d05708 100644 --- a/integrations/README.md +++ b/integrations/README.md @@ -7,6 +7,9 @@ are available to marketplace discovery before installation. - The [Foreman Codex plugin](https://github.com/thruwire/marketplace/tree/main/plugins/foreman) invokes the core `foreman hook --client codex` protocol. +- [`deepagents/`](deepagents/) documents the core dcode worker and native hook adapter. + `foreman deepagents setup` merges native hook configuration; no separate bridge or plugin + package is required. - [`pi/`](pi/) supplies buildable Pi extension and Pi Durable task-hook bridges, invoking `foreman hook --client pi` and `foreman hook --client pi-durable` respectively. This is a local integration package with installation documentation and offline tests. Pi supervision requires diff --git a/integrations/deepagents/README.md b/integrations/deepagents/README.md new file mode 100644 index 0000000..81f5bb0 --- /dev/null +++ b/integrations/deepagents/README.md @@ -0,0 +1,180 @@ +# Deep Agents Code integration + +Foreman core supports both headless workers and native hooks for LangChain's +[Deep Agents Code](https://github.com/langchain-ai/deepagents/tree/main/libs/code) +(`dcode`). The CLI supplies the agent harness; Foreman owns responsibility routing, +evidence collection, interventions, verification, and completion checks. No Foreman +plugin or LangChain SDK dependency is required. + +## Requirements and installation + +- `foreman-core >= 0.5.0` needs Python 3.11+ and includes + `foreman deepagents setup` and the `deepagents` worker backend. +- `deepagents-code >= 0.1.83` needs Python 3.12+. The CLI flags and Hooks v2 contract + were checked against the published 0.1.83 package; older versions are unsupported. + Foreman's automated tests use offline subprocesses and protocol fixtures. Both + integration paths were also exercised with live model calls against a sample + repository; see the [live test results](live-test-results.md). +- A TypeSafe API key for Foreman, plus credentials for the worker's selected model provider. + +Install the commands in separate tool environments, for example with `uv`: + +```bash +uv tool install 'foreman-core>=0.5.0' +uv tool install --python 3.12 'deepagents-code>=0.1.83' +foreman --version +foreman deepagents setup --help +dcode --version +``` + +For development or before 0.5.0 is published, run `uv tool install --force .` +from this Foreman checkout instead of installing the published core package. + +Configure the model provider following the +[Deep Agents documentation](https://docs.langchain.com/oss/python/deepagents/overview). +For example, export `OPENAI_API_KEY` for an OpenAI model or `ANTHROPIC_API_KEY` for +an Anthropic model. Foreman inherits these variables into the worker, while removing +its own `TYPESAFE_*` variables. Do not put supervisor credentials in the repository. + +Export `TYPESAFE_API_KEY` in the terminal running `foreman run`. For native hooks, +it may be supplied by the `dcode` environment or by Foreman's protected central +`${FOREMAN_DATA_DIR:-~/.foreman}/.env`: + +```bash +mkdir -p ~/.foreman +chmod 700 ~/.foreman +touch ~/.foreman/.env +chmod 600 ~/.foreman/.env +``` + +Edit that file to add `TYPESAFE_API_KEY=your-key`. Existing process variables take +precedence. When using both paths, keep the central file configured too: a headless +worker does not inherit Foreman's TypeSafe credentials, but its installed user hooks +may still call Foreman. The `foreman run` command requires the exported key. + +## Foreman-launched jobs + +Select the backend and optionally the model, then run a job: + +```bash +export FOREMAN_WORKER_BACKEND=deepagents +export FOREMAN_DEEPAGENTS_MODEL=openai:your-model +foreman run --repo /absolute/path/to/project --job "Add request retries and verify them" +``` + +Foreman runs `dcode --non-interactive ` in the repository with a turn limit, +a wall-clock timeout, and a shell allow-list. Both output pipes are streamed in bounded +chunks, including output without newlines. No hook setup is necessary for this path. +Native user hooks already installed in `dcode` still run; project hooks are not +automatically trusted. Choose either supervision path initially to avoid duplicate +assessments when user hooks and an owned worker are both active. + +| Variable | Default | Meaning | +| --- | --- | --- | +| `FOREMAN_DEEPAGENTS_EXECUTABLE` | `dcode` | Executable name or absolute path | +| `FOREMAN_DEEPAGENTS_MODEL` | unset | Provider/model identifier; otherwise use dcode configuration | +| `FOREMAN_DEEPAGENTS_MAX_TURNS` | `200` | Headless agentic turn limit | +| `FOREMAN_DEEPAGENTS_SHELL_ALLOW_LIST` | `recommended` | Native shell allow-list: preset, comma-separated commands, or `all` | +| `FOREMAN_WORKER_TIMEOUT_SECONDS` | `3600` | Foreman's deadline, also passed to dcode rounded up to whole seconds | + +The recommended shell preset may exclude commands needed by your project. Set an +explicit list (for example `git,python,pytest,uv`) when appropriate. `all` allows arbitrary +shell commands on the host; this worker does not configure a sandbox. The CLI's native +managed configuration may further restrict execution. + +Headless mode has no live input channel: Foreman uses stop/retry for steering. Native +exit code 124 means a timeout or turn budget was exhausted. A zero exit is a worker +result; Foreman still decides whether the job meets its completion and verification +requirements. On POSIX, shutdown collects descendant process groups so the separate +LangGraph server is stopped as well. This collection requires the system `ps` command. + +## Interactive sessions with native hooks + +First configure Foreman's central credentials and responsibilities. Packaged responsibilities +are enabled by default. To restrict supervision to selected repositories, merge a `[hooks]` +section into `${FOREMAN_CONFIG:-~/.foreman/config.toml}`: + +```toml +[hooks] +repositories = ["/absolute/path/to/project"] +``` + +Repository identities include subdirectories and linked Git worktrees. Omit `repositories` +for global supervision or use `[]` to disable it. Central responsibility overrides, +custom criteria, and extensions work just as with Codex and Pi; see +[responsibility configuration](../../docs/hooks.md#responsibility-configuration). + +Install the native lifecycle handlers: + +```bash +foreman deepagents setup +``` + +This merges seven wildcard event handlers into `~/.deepagents/hooks.json`, preserving +existing handlers and other settings. Repeating the same setup does not duplicate entries. +Writes are atomic. The command refuses malformed or legacy list-shaped configurations; +convert those to Hooks v2 using the native hook documentation before retrying. +It resolves the installed Foreman executable +to an absolute path when available, which avoids dependence on the coding agent's PATH. + +To select another hook file or executable: + +```bash +foreman deepagents setup \ + --hooks-file /absolute/path/to/hooks.json \ + --foreman-executable /absolute/path/to/foreman +``` + +For project-scoped configuration, select `/project/.deepagents/hooks.json`. Interactive +`dcode` requires workspace trust before loading project hooks; headless runs require +the native `--trust-project-hooks` opt-in. Foreman's worker deliberately does not add it. +User hooks load without project trust. Setup configures hooks only; it does not install +the CLI, change credentials, or rewrite Foreman's responsibilities. + +Start a fresh `dcode` session in the configured repository and submit your task. Handlers +invoke `foreman hook --client deepagents` directly through native command hooks. + +| Native event | Foreman boundary | +| --- | --- | +| `SessionStart` | Create or refresh attached-session state | +| `UserPromptSubmit` | Route work; native `prompt_id` identifies repeat submissions | +| `PreToolUse` | Assess proposed tool call; deny or inject guidance | +| `PostToolUse` | Record result and assess progress | +| `PostToolUseFailure` | Record error/interruption and assess progress | +| `Stop` | Check completion; request one continuation or finish/halt | +| `SessionEnd` | Remove attached-session state | + +Native `stop_hook_active` and Foreman's saved continuation allowance prevent repeated +completion requests. Tool events emitted inside subagents contribute to the parent +session's evidence; `SubagentStop` is not treated as parent completion. Native failures +are normalized with `isError` and the error text. + +The native response contract has limits: pre-tool denial blocks the proposed operation, +while post-tool block/stop requests become feedback in the model's next context, rather +than immediately aborting the whole run. Prompt hooks can prevent work submission; +`Stop` controls continuation and settling. Hook errors with exit code 2 block pre-tool +operations or prompt submission; other nonzero errors and hook timeouts are diagnostics +in dcode and do not guarantee blocking. Each installed handler has a 120-second deadline. +See the [native hook contract](https://github.com/langchain-ai/deepagents/blob/main/libs/code/HOOKS.md). + +## Verify setup and troubleshoot + +- Check `dcode --help` includes `--non-interactive`, `--max-turns`, `--timeout`, and + `--shell-allow-list`. Upgrade `deepagents-code` if these are absent. +- Inspect `~/.deepagents/hooks.json`: the seven handlers should invoke your Foreman + executable with `hook --client deepagents`. Repeat setup with the same executable + to verify it reports `Already installed`. +- During a task in a fresh interactive session, look for a `client: "deepagents"` + record under `${FOREMAN_DATA_DIR:-~/.foreman}/sessions/`. Session-end removes it. + An excluded repository or disabled responsibility registry produces no session state. + `jev-usage.jsonl` in the same central data directory records real supervision requests. +- A missing executable needs PATH repair or `FOREMAN_DEEPAGENTS_EXECUTABLE`; provider + errors need worker credentials/model configuration. Missing supervisor credentials + need the protected central `.env` or the launching process environment. +- `arrived before work_submitted` means tools ran without the initial prompt hook. + Install all seven events and start a fresh session. Unsupported events fail explicitly. +- Repeated shell denials need a suitable allow-list. Exit 124 needs a larger turn/time + budget or a narrower task. Neither is reported as successful worker completion. +- Existing project/user hook decisions may take precedence over Foreman's responses. + Native matching handlers all execute; avoid duplicate Foreman registration in multiple + scopes. Remove only Foreman's handler entries to uninstall, preserving other hooks. diff --git a/integrations/deepagents/live-test-results.md b/integrations/deepagents/live-test-results.md new file mode 100644 index 0000000..a674ed2 --- /dev/null +++ b/integrations/deepagents/live-test-results.md @@ -0,0 +1,76 @@ +# Deep Agents live integration test + +Tested on 2026-10-08 with the unpublished Foreman checkout, Python 3.12, +`deepagents-code==0.1.83`, `deepagents==0.7.23`, and +`openai:gpt-5.4-mini`. Foreman used real TypeSafe Jev requests for routing, +assessment, intervention, and completion. + +## Sample and isolation + +The source was `/Users/dev/Documents/GitHub/foreman-test-app`, including its +existing uncommitted changes. Its baseline had 25 passing calculator tests. +Two disposable Git copies were used for independent coding runs, each asked to +add `Calculator.modulo(a, b)`, Python remainder semantics, a specific zero-divisor +error, regression tests, and README usage. + +The source repository's `AGENTS.md` prohibits tools and changes. Only the coding +copies received ordinary instructions to implement the feature and run tests. +A separate copy retained the original instructions for the hook intervention +test. SHA-256 checks confirmed all six original source files were unchanged. +The coding runs therefore validate the integration on the sample's code, rather +than claiming permission to change the original repository under its smoke-test +instructions. + +An isolated Python environment, Deep Agents profiles, and Foreman data/config +directories kept these tests separate from the user's normal installations. +Credentials were loaded into the process environment and were not copied into +the sample repositories or test report. The shell allow-list was +`git,python,pytest,cat`. + +## Foreman-owned worker + +The installed `foreman run` command launched the actual `dcode` executable using +the new `deepagents` worker backend. Its profile had no Foreman hooks installed. + +- The coding worker added the operation, tests, and documentation, then exited 0. +- Foreman launched a separate Deep Agents verifier, which inspected the changes + and reran pytest, then exited 0. +- Foreman reached `FINISHED` and returned 0. +- An independent pytest run confirmed **36 tests passed**. + +## Native hooks with an autonomous agent + +The installed `dcode --non-interactive` command ran the same feature request in +the second copy. `foreman deepagents setup` installed the seven native handlers +into its isolated user profile. A recording wrapper passed each input unchanged +to the installed Foreman command and recorded its response and exit status. + +| Event | Calls | +| --- | ---: | +| SessionStart | 1 | +| UserPromptSubmit | 1 | +| PreToolUse | 11 | +| PostToolUse | 11 | +| Stop | 1 | +| SessionEnd | 1 | + +All 26 handlers exited 0 without stderr diagnostics. Foreman attached its +responsibilities at prompt submission, accepted completion at Stop, and removed +the session state at SessionEnd. The autonomous agent added the requested +feature, tests, and README examples, then exited 0. An independent pytest run +confirmed **34 tests passed**; the two agents produced different test matrices. + +The successful coding run did not trigger a tool failure or Stop continuation. +Separate tests using the installed native Hooks v2 runtime and real Foreman/Jev +calls covered `PostToolUseFailure`, continuation followed by the one-continuation +limit, and repeated tool denial under the original prohibitive instructions. +Those tests explicitly dispatched native lifecycle events; they were not +autonomous model runs. + +## Final checks + +Both autonomous runs shut down their LangGraph servers; no server processes +remained. Foreman's automated suite passed **430 tests**, Ruff passed, and +`git diff --check` passed. These results cover the tested versions, model, and +local macOS environment; other providers and interactive terminal sessions +were not exercised in this test. diff --git a/pyproject.toml b/pyproject.toml index aaac4e7..fad3877 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "foreman-core" -version = "0.4.4" +version = "0.5.0" description = "An agent supervisor and software factory foreman powered by TypeSafe’s Jev model" readme = "README.md" requires-python = ">=3.11" diff --git a/src/foreman/cli.py b/src/foreman/cli.py index 9403d2b..c5ce829 100644 --- a/src/foreman/cli.py +++ b/src/foreman/cli.py @@ -3,6 +3,7 @@ import asyncio import json import os +import shutil import sys from datetime import UTC from pathlib import Path @@ -15,6 +16,7 @@ from foreman import __version__ from foreman.config import FactoryConfig +from foreman.deepagents_setup import install_deepagents_hooks from foreman.extensions import ( ExtensionError, ExtensionIdentity, @@ -58,6 +60,8 @@ ) extension_app = typer.Typer(help="Manage configured Foreman extensions.", no_args_is_help=True) app.add_typer(extension_app, name="extension") +deepagents_app = typer.Typer(help="Set up native Deep Agents Code hooks.", no_args_is_help=True) +app.add_typer(deepagents_app, name="deepagents") console = Console() @@ -91,6 +95,32 @@ def _extension_manager(data_dir: Path | None) -> ExtensionManager: return ExtensionManager(data_dir=data_dir) +@deepagents_app.command("setup") +def deepagents_setup( + hooks_file: Annotated[ + Path | None, + typer.Option("--hooks-file", dir_okay=False, + help="Hooks v2 file (default: ~/.deepagents/hooks.json)"), + ] = None, + foreman_executable: Annotated[ + str | None, + typer.Option("--foreman-executable", help="Foreman command to invoke from dcode"), + ] = None, +) -> None: + """Add Foreman lifecycle hooks, preserving existing handlers and settings.""" + path = (hooks_file or Path.home() / ".deepagents" / "hooks.json").expanduser().resolve() + executable = foreman_executable or shutil.which("foreman") or "foreman" + try: + changed = install_deepagents_hooks(path, executable) + except (OSError, ValueError) as error: + typer.echo(f"Deep Agents hook setup failed: {error}", err=True) + raise typer.Exit(code=2) from error + typer.echo(f"{'Installed' if changed else 'Already installed'} Foreman hooks: {path}") + typer.echo( + "Configure Foreman credentials and responsibilities, then start a fresh dcode session." + ) + + @extension_app.command("login") def extension_login( extension_id: Annotated[str, typer.Argument(help="Configured extension id")], @@ -355,7 +385,7 @@ def run( ), ] = None, ) -> None: - """Launch a real Codex worker supervised by TypeSafe AI Jev.""" + """Launch a coding worker supervised by TypeSafe AI Jev.""" # Capture the process-level factory setting before reading the target repository's # local environment. A managed repository cannot select Foreman's responsibilities. diff --git a/src/foreman/config.py b/src/foreman/config.py index 664395a..ad08e14 100644 --- a/src/foreman/config.py +++ b/src/foreman/config.py @@ -64,7 +64,11 @@ class FactoryConfig(BaseModel): max_workers: int = Field(default=3, ge=1) max_retries: int = Field(default=1, ge=0) max_iterations: int = Field(default=20, ge=1) - worker_backend: Literal["codex", "opencode", "hermes"] = "codex" + worker_backend: Literal["codex", "opencode", "hermes", "deepagents"] = "codex" + deepagents_executable: str = "dcode" + deepagents_model: str | None = None + deepagents_max_turns: int = Field(default=200, ge=1) + deepagents_shell_allow_list: str = "recommended" hermes_executable: str = "hermes" hermes_model: str | None = None hermes_provider: str | None = None @@ -112,6 +116,10 @@ def from_environment(cls) -> FactoryConfig: "FOREMAN_MAX_RETRIES": ("max_retries", int), "FOREMAN_MAX_ITERATIONS": ("max_iterations", int), "FOREMAN_WORKER_BACKEND": ("worker_backend", str), + "FOREMAN_DEEPAGENTS_EXECUTABLE": ("deepagents_executable", str), + "FOREMAN_DEEPAGENTS_MODEL": ("deepagents_model", str), + "FOREMAN_DEEPAGENTS_MAX_TURNS": ("deepagents_max_turns", int), + "FOREMAN_DEEPAGENTS_SHELL_ALLOW_LIST": ("deepagents_shell_allow_list", str), "FOREMAN_HERMES_EXECUTABLE": ("hermes_executable", str), "FOREMAN_HERMES_MODEL": ("hermes_model", str), "FOREMAN_HERMES_PROVIDER": ("hermes_provider", str), diff --git a/src/foreman/deepagents_setup.py b/src/foreman/deepagents_setup.py new file mode 100644 index 0000000..343e32e --- /dev/null +++ b/src/foreman/deepagents_setup.py @@ -0,0 +1,76 @@ +from __future__ import annotations + +import json +import os +import shlex +import tempfile +from pathlib import Path +from typing import Any + +DEEPAGENTS_HOOK_EVENTS = ( + "SessionStart", "UserPromptSubmit", "PreToolUse", "PostToolUse", + "PostToolUseFailure", "Stop", "SessionEnd", +) + + +def install_deepagents_hooks(path: Path, executable: str) -> bool: + """Merge Foreman's handlers into a Hooks v2 document atomically. + + Existing settings and handlers are retained. Refuse unsupported legacy or + malformed configurations before writing; repeat setup is a no-op. Returns + whether the file changed. The caller chooses the configuration destination. + """ + document: dict[str, Any] = {} + if path.exists(): + document = json.loads(path.read_text(encoding="utf-8")) + if not isinstance(document, dict): + raise ValueError("hooks configuration must be a Hooks v2 JSON object") + hooks = document.setdefault("hooks", {}) + if not isinstance(hooks, dict): + raise ValueError("hooks must be an object; convert legacy hooks to Hooks v2 first") + argv = [executable, "hook", "--client", "deepagents"] + command = shlex.join(argv) + changed = False + for event in DEEPAGENTS_HOOK_EVENTS: + groups = hooks.setdefault(event, []) + if not isinstance(groups, list) or any( + not isinstance(group, dict) or not isinstance(group.get("hooks"), list) + or any(not isinstance(handler, dict) for handler in group["hooks"]) + for group in groups + ): + raise ValueError(f"{event} must contain matcher groups with command handler lists") + installed = any( + group.get("matcher", "*") == "*" and any( + handler.get("type") == "command" and ( + handler.get("argv") == argv + or ("argv" not in handler and handler.get("command") == command) + ) + for handler in group["hooks"] + ) + for group in groups + ) + if not installed: + groups.append({"hooks": [{ + "type": "command", "command": command, "argv": argv, "timeout": 120, + }]}) + changed = True + if not changed: + return False + path.parent.mkdir(parents=True, exist_ok=True) + mode = path.stat().st_mode & 0o777 if path.exists() else 0o600 + temporary: str | None = None + try: + with tempfile.NamedTemporaryFile( + mode="w", encoding="utf-8", dir=path.parent, delete=False, + ) as stream: + temporary = stream.name + json.dump(document, stream, indent=2, ensure_ascii=False) + stream.write("\n") + stream.flush() + os.fsync(stream.fileno()) + os.chmod(temporary, mode) + os.replace(temporary, path) + finally: + if temporary is not None and os.path.exists(temporary): + os.unlink(temporary) + return True diff --git a/src/foreman/hook_adapters.py b/src/foreman/hook_adapters.py index a283a30..50a4b3f 100644 --- a/src/foreman/hook_adapters.py +++ b/src/foreman/hook_adapters.py @@ -1,7 +1,7 @@ from __future__ import annotations from enum import StrEnum -from typing import Any, Protocol +from typing import Any, Literal, Protocol from pydantic import BaseModel, ConfigDict, Field, ValidationError, model_validator @@ -126,6 +126,81 @@ def render(self, event: HookEvent, outcome: HookOutcome) -> dict[str, Any]: } +class DeepAgentsHookInput(BaseModel): + """Published dcode Hooks v2 wire subset, including failed tool calls.""" + + model_config = ConfigDict(extra="allow") + + session_id: str = Field(min_length=1, max_length=1_000) + cwd: str = Field(min_length=1) + hook_event_name: Literal[ + "SessionStart", "UserPromptSubmit", "PreToolUse", "PostToolUse", + "PostToolUseFailure", "Stop", "SessionEnd", + ] + prompt_id: str | None = Field(default=None, min_length=1, max_length=1_000) + prompt: str | None = None + tool_name: str | None = None + tool_use_id: str | None = None + tool_input: Any = None + tool_response: Any = None + stop_hook_active: bool = False + last_assistant_message: str | None = None + error: str | None = None + is_interrupt: bool | None = None + + @model_validator(mode="after") + def required_deepagents_fields(self) -> DeepAgentsHookInput: + if self.hook_event_name == "UserPromptSubmit" and not ( + self.prompt and self.prompt.strip() + ): + raise ValueError("UserPromptSubmit requires a non-empty prompt") + if self.hook_event_name in {"PreToolUse", "PostToolUse", "PostToolUseFailure"}: + if not self.tool_name or not self.tool_use_id: + raise ValueError("tool events require tool_name and tool_use_id") + if self.hook_event_name == "PostToolUseFailure" and self.error is None: + raise ValueError("PostToolUseFailure requires error") + return self + + +class DeepAgentsHookAdapter(CodexHookAdapter): + # dcode accepts the same decision/context output vocabulary as Codex. + # Post-tool stop directives are feedback in dcode; Stop and prompt hooks + # provide the actionable turn boundary (see the integration guide). + client = "deepagents" + + def parse(self, payload: dict[str, Any]) -> HookEvent: + try: + source = DeepAgentsHookInput.model_validate(payload) + kind = ( + HookEventKind.AFTER_TOOL + if source.hook_event_name == "PostToolUseFailure" + else _CODEX_EVENT_KINDS[source.hook_event_name] + ) + response = source.tool_response + if source.hook_event_name == "PostToolUseFailure": + response = {"isError": True, "error": source.error} + if source.is_interrupt is not None: + response["is_interrupt"] = source.is_interrupt + return HookEvent( + client=self.client, + session_id=source.session_id, + cwd=source.cwd, + kind=kind, + source_event_name=source.hook_event_name, + work_id=source.prompt_id, + turn_id=source.prompt_id, + prompt=source.prompt, + tool_name=source.tool_name, + tool_use_id=source.tool_use_id, + tool_input=source.tool_input, + tool_response=response, + continuation_active=source.stop_hook_active, + last_assistant_message=source.last_assistant_message, + ) + except (ValidationError, ValueError) as error: + raise HookError(f"invalid Deep Agents hook input: {error}") from error + + class PiHookInput(BaseModel): """Bridge envelope around Pi's native events; identities come from its host.""" @@ -209,6 +284,7 @@ class PiDurableHookAdapter(PiHookAdapter): _ADAPTERS: dict[str, HookAdapter] = { CodexHookAdapter.client: CodexHookAdapter(), + DeepAgentsHookAdapter.client: DeepAgentsHookAdapter(), PiHookAdapter.client: PiHookAdapter(), PiDurableHookAdapter.client: PiDurableHookAdapter(), } diff --git a/src/foreman/runtime.py b/src/foreman/runtime.py index 0893fd1..df2968d 100644 --- a/src/foreman/runtime.py +++ b/src/foreman/runtime.py @@ -35,7 +35,14 @@ resolve_hierarchical_routing, ) from foreman.steering import build_steering_message -from foreman.workers import CodexAppServerWorker, CodexWorker, HermesWorker, OpenCodeWorker, Worker +from foreman.workers import ( + CodexAppServerWorker, + CodexWorker, + DeepAgentsWorker, + HermesWorker, + OpenCodeWorker, + Worker, +) from foreman.workers.codex import mission_for EventSink = Callable[[FactoryEvent], object] @@ -159,6 +166,15 @@ async def _route_work(self) -> None: def _default_worker_factory(self, worker_type: WorkerType) -> Worker: del worker_type + if self.config.worker_backend == "deepagents": + return DeepAgentsWorker( + executable=self.config.deepagents_executable, + model=self.config.deepagents_model, + max_turns=self.config.deepagents_max_turns, + shell_allow_list=self.config.deepagents_shell_allow_list, + output_limit=self.config.output_limit, + graceful_termination_seconds=self.config.graceful_termination_seconds, + ) if self.config.worker_backend == "opencode": return OpenCodeWorker( output_limit=self.config.output_limit, diff --git a/src/foreman/terminal.py b/src/foreman/terminal.py index d4d38e5..e3fb316 100644 --- a/src/foreman/terminal.py +++ b/src/foreman/terminal.py @@ -28,7 +28,7 @@ def __call__(self, event: FactoryEvent) -> None: else "Worker" ) self.console.print(f"{label} {payload.get('worker_id')} started.") - self.console.print("Codex is working... Foreman is watching independently.") + self.console.print("Worker is running... Foreman is watching independently.") elif event.event_type in { EventType.WORKER_COMPLETED, EventType.WORKER_FAILED, diff --git a/src/foreman/workers/__init__.py b/src/foreman/workers/__init__.py index 4c79019..80a96b1 100644 --- a/src/foreman/workers/__init__.py +++ b/src/foreman/workers/__init__.py @@ -1,6 +1,7 @@ from foreman.workers.base import EventCallback, Worker, codex_environment, worker_environment from foreman.workers.codex import CodexWorker, coding_mission, verification_mission from foreman.workers.codex_app_server import CodexAppServerWorker +from foreman.workers.deepagents import DeepAgentsWorker from foreman.workers.hermes import HermesWorker from foreman.workers.opencode import OpenCodeWorker from foreman.workers.simulation import FakeWorker @@ -8,6 +9,7 @@ __all__ = [ "CodexWorker", "CodexAppServerWorker", + "DeepAgentsWorker", "EventCallback", "FakeWorker", "HermesWorker", diff --git a/src/foreman/workers/deepagents.py b/src/foreman/workers/deepagents.py new file mode 100644 index 0000000..3a6a36f --- /dev/null +++ b/src/foreman/workers/deepagents.py @@ -0,0 +1,244 @@ +from __future__ import annotations + +import asyncio +import codecs +import math +import os +import signal +from datetime import UTC, datetime +from pathlib import Path + +from foreman.models import EventType, WorkerRecord, WorkerStatus +from foreman.workers.base import EventCallback, worker_environment + + +class DeepAgentsWorker: + """Run the separately installed Deep Agents Code CLI (deepagents-code >=0.1.83). + + Headless dcode streams text rather than a machine event stream. Read bounded + chunks so output without newlines still reaches supervision. Native hooks + are a separate attached-session integration; this transport has no live + steering channel and uses Foreman's stop/retry policy. + """ + + supports_steering = False + + def __init__( + self, + *, + executable: str = "dcode", + model: str | None = None, + max_turns: int = 200, + shell_allow_list: str = "recommended", + output_limit: int = 50_000, + graceful_termination_seconds: float = 5.0, + ) -> None: + self.executable = executable + self.model = model + self.max_turns = max_turns + self.shell_allow_list = shell_allow_list + self.output_limit = output_limit + self.graceful_termination_seconds = graceful_termination_seconds + self.process: asyncio.subprocess.Process | None = None + self._termination_reason: str | None = None + self._process_groups: set[int] = set() + + def command(self, mission: str, timeout_seconds: float) -> list[str]: + command = [ + self.executable, + "--max-turns", str(self.max_turns), + "--timeout", str(max(1, math.ceil(timeout_seconds))), + "--shell-allow-list", self.shell_allow_list, + ] + if self.model is not None: + command.extend(["--model", self.model]) + command.extend(["--non-interactive", mission]) + return command + + async def _stream( + self, + stream: asyncio.StreamReader | None, + stream_name: str, + record: WorkerRecord, + emit: EventCallback, + ) -> None: + if stream is None: + return + decoder = codecs.getincrementaldecoder("utf-8")(errors="replace") + while True: + chunk = await stream.read(4_096) + text = decoder.decode(chunk, final=not chunk) + if text: + current = getattr(record, stream_name) + setattr(record, stream_name, (current + text)[-self.output_limit:]) + await emit(EventType.WORKER_OUTPUT, { + "worker_id": record.worker_id, + "stream": stream_name, + "line": text.rstrip()[-4_000:], + }) + if not chunk: + break + + async def run( + self, + record: WorkerRecord, + repository: Path, + emit: EventCallback, + timeout_seconds: float, + ) -> WorkerRecord: + self._termination_reason = None + self.process = None + self._process_groups.clear() + record.client = "deepagents" + record.supports_steering = False + record.status = WorkerStatus.RUNNING + record.started_at = datetime.now(UTC) + environment = worker_environment() + environment["PYTHONUNBUFFERED"] = "1" + try: + self.process = await asyncio.create_subprocess_exec( + *self.command(record.mission, timeout_seconds), + cwd=repository, + env=environment, + stdin=asyncio.subprocess.DEVNULL, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + start_new_session=True, + ) + except Exception as error: + record.status = WorkerStatus.FAILED + record.finished_at = datetime.now(UTC) + record.stderr = ( + f"failed to launch Deep Agents Code: {error}. Install deepagents-code>=0.1.83 " + "with Python 3.12+ and ensure dcode is on PATH, or set " + "FOREMAN_DEEPAGENTS_EXECUTABLE." + )[-self.output_limit:] + record.termination_reason = "launch_failed" + return record + + streams = [ + asyncio.create_task(self._stream(self.process.stdout, "stdout", record, emit)), + asyncio.create_task(self._stream(self.process.stderr, "stderr", record, emit)), + ] + try: + # Include pipe draining in the deadline: descendants can keep pipes open + # after the CLI exits, and emit can trigger supervisory intervention. + async def wait_and_drain() -> None: + await asyncio.gather(self.process.wait(), *streams) + + await asyncio.wait_for(wait_and_drain(), timeout=timeout_seconds) + if self._termination_reason: + record.status = WorkerStatus.STOPPED + record.termination_reason = self._termination_reason + elif self.process.returncode == 124: + record.status = WorkerStatus.TIMED_OUT + record.termination_reason = "deepagents_budget_exhausted" + elif self.process.returncode == 0: + # A CLI exit is a worker result; Foreman still verifies completion. + record.status = WorkerStatus.COMPLETED + else: + record.status = WorkerStatus.FAILED + record.termination_reason = "nonzero_exit" + except TimeoutError: + record.status = WorkerStatus.TIMED_OUT + record.termination_reason = "worker_timeout" + await self.terminate("worker_timeout") + except asyncio.CancelledError: + record.status = WorkerStatus.CANCELLED + record.termination_reason = "cancelled" + await self.terminate("cancelled") + raise + except Exception: + await self.terminate("stream_failed") + record.status = WorkerStatus.FAILED + record.termination_reason = "stream_failed" + raise + finally: + for task in streams: + if not task.done(): + task.cancel() + await asyncio.gather(*streams, return_exceptions=True) + record.exit_code = self.process.returncode + record.finished_at = datetime.now(UTC) + return record + + def _signal_group(self, sig: signal.Signals) -> None: + process = self.process + if process is None: + return + for group in self._process_groups | {process.pid}: + try: + os.killpg(group, sig) + except (ProcessLookupError, PermissionError): + if group == process.pid and process.returncode is None: + if sig == signal.SIGKILL: + process.kill() + else: + process.terminate() + + async def _collect_process_groups(self) -> None: + # dcode launches its LangGraph server in a separate POSIX session. + # Collect descendant groups before signaling the CLI, while ancestry + # is still available; killing only the CLI's group would miss the server. + process = self.process + if process is None: + return + probe = None + try: + probe = await asyncio.create_subprocess_exec( + "ps", "-e", "-o", "pid=", "-o", "ppid=", + stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.DEVNULL, + env=worker_environment(), + ) + output, _ = await asyncio.wait_for(probe.communicate(), timeout=2) + children: dict[int, list[int]] = {} + for line in output.decode().splitlines(): + pid, parent = map(int, line.split()) + children.setdefault(parent, []).append(pid) + pending = [process.pid] + own_group = os.getpgrp() + while pending: + pid = pending.pop() + pending.extend(children.get(pid, ())) + try: + group = os.getpgid(pid) + except ProcessLookupError: + continue + if group != own_group: + self._process_groups.add(group) + except (OSError, ValueError, TimeoutError): + # Minimal systems may lack ps; the CLI's own shutdown still owns + # server cleanup, and its original group remains terminable. + pass + finally: + if probe is not None and probe.returncode is None: + probe.kill() + await probe.wait() + + async def terminate(self, reason: str) -> None: + self._termination_reason = reason + process = self.process + if process is None: + return + if os.name == "nt": + killer = await asyncio.create_subprocess_exec( + "taskkill", "/PID", str(process.pid), "/T", "/F", + stdout=asyncio.subprocess.DEVNULL, stderr=asyncio.subprocess.DEVNULL, + ) + await killer.wait() + await process.wait() + return + await self._collect_process_groups() + self._signal_group(signal.SIGTERM) + try: + await asyncio.wait_for(process.wait(), timeout=self.graceful_termination_seconds) + except TimeoutError: + self._signal_group(signal.SIGKILL) + await process.wait() + # The CLI may exit before its server or shell child does. Kill any + # remaining members of this worker's process group as well. + self._signal_group(signal.SIGKILL) + + async def steer(self, message: str) -> bool: + del message + return False diff --git a/tests/test_deepagents.py b/tests/test_deepagents.py new file mode 100644 index 0000000..9fd9406 --- /dev/null +++ b/tests/test_deepagents.py @@ -0,0 +1,217 @@ +from __future__ import annotations + +import asyncio +import os +import signal +import sys + +import pytest + +from foreman.config import FactoryConfig +from foreman.foreman import FakeForemanModel +from foreman.models import EventType, WorkerRecord, WorkerStatus, WorkerType +from foreman.runtime import FactoryRuntime +from foreman.workers import DeepAgentsWorker + + +def record(): + return WorkerRecord(worker_id="d1", worker_type=WorkerType.CODING, mission="Fix tests") + + +async def discard(*args): + pass + + +def scripted_worker(monkeypatch, script, **kwargs): + worker = DeepAgentsWorker(**kwargs) + monkeypatch.setattr(worker, "command", lambda *args: [sys.executable, "-c", script]) + return worker + + +def test_command_and_configuration(monkeypatch, tmp_path): + monkeypatch.setenv("FOREMAN_WORKER_BACKEND", "deepagents") + monkeypatch.setenv("FOREMAN_DEEPAGENTS_EXECUTABLE", "/opt/my tools/dcode") + monkeypatch.setenv("FOREMAN_DEEPAGENTS_MODEL", "openai:test-model") + monkeypatch.setenv("FOREMAN_DEEPAGENTS_MAX_TURNS", "42") + monkeypatch.setenv("FOREMAN_DEEPAGENTS_SHELL_ALLOW_LIST", "git,python,uv") + config = FactoryConfig.from_environment() + runtime = FactoryRuntime( + repository=tmp_path, job="job", model=FakeForemanModel([]), config=config, + ) + worker = runtime.worker_factory(WorkerType.CODING) + assert isinstance(worker, DeepAgentsWorker) + assert worker.command("--mission with spaces", 3.2) == [ + "/opt/my tools/dcode", "--max-turns", "42", "--timeout", "4", + "--shell-allow-list", "git,python,uv", "--model", "openai:test-model", + "--non-interactive", "--mission with spaces", + ] + assert worker.supports_steering is False + + +def test_default_command_does_not_opt_in_project_hooks(): + command = DeepAgentsWorker().command("mission", 10) + assert command == [ + "dcode", "--max-turns", "200", "--timeout", "10", + "--shell-allow-list", "recommended", "--non-interactive", "mission", + ] + + +@pytest.mark.asyncio +async def test_streams_without_newlines_and_bounds_both_pipes(monkeypatch, tmp_path): + worker = scripted_worker( + monkeypatch, + "import sys; sys.stdout.write('x'*100000); sys.stderr.write('y'*100000)", + output_limit=120, + ) + events = [] + + async def emit(kind, payload): + assert kind is EventType.WORKER_OUTPUT + events.append(payload) + + result = await worker.run(record(), tmp_path, emit, 10) + assert result.status is WorkerStatus.COMPLETED + assert result.client == "deepagents" + assert result.stdout == "x" * 120 and result.stderr == "y" * 120 + assert {event["stream"] for event in events} == {"stdout", "stderr"} + assert all(len(event["line"]) <= 4_000 for event in events) + + +@pytest.mark.asyncio +async def test_utf8_split_across_reads(tmp_path): + worker = DeepAgentsWorker() + stream = asyncio.StreamReader() + rec = record() + events = [] + + async def emit(kind, payload): + events.append(payload["line"]) + + task = asyncio.create_task(worker._stream(stream, "stdout", rec, emit)) + stream.feed_data(b"hello \xe2") + await asyncio.sleep(0) + stream.feed_data(b"\x82\xac") + stream.feed_eof() + await task + assert rec.stdout == "hello €" + assert "�" not in "".join(events) + + +@pytest.mark.asyncio +async def test_workspace_and_environment(monkeypatch, tmp_path): + monkeypatch.setenv("TYPESAFE_API_KEY", "supervisor-secret") + monkeypatch.setenv("TYPESAFE_BASE_URL", "supervisor-url") + monkeypatch.setenv("OPENAI_API_KEY", "worker-key") + worker = scripted_worker(monkeypatch, + "import os; print(os.getcwd()); " + "print(any(k.startswith('TYPESAFE_') for k in os.environ)); " + "print(os.environ['OPENAI_API_KEY']); print(os.environ['PYTHONUNBUFFERED'])") + result = await worker.run(record(), tmp_path, discard, 10) + assert result.stdout.splitlines() == [str(tmp_path), "False", "worker-key", "1"] + + +@pytest.mark.parametrize("code,status,reason", [ + (0, WorkerStatus.COMPLETED, None), + (1, WorkerStatus.FAILED, "nonzero_exit"), + (124, WorkerStatus.TIMED_OUT, "deepagents_budget_exhausted"), +]) +@pytest.mark.asyncio +async def test_exit_status(monkeypatch, tmp_path, code, status, reason): + worker = scripted_worker(monkeypatch, f"import sys; sys.exit({code})") + result = await worker.run(record(), tmp_path, discard, 10) + assert result.status is status and result.termination_reason == reason + assert result.exit_code == code and result.finished_at is not None + + +@pytest.mark.asyncio +async def test_missing_executable_has_setup_guidance(tmp_path): + worker = DeepAgentsWorker(executable=str(tmp_path / "missing-dcode")) + result = await worker.run(record(), tmp_path, discard, 1) + assert result.status is WorkerStatus.FAILED + assert result.termination_reason == "launch_failed" + assert "deepagents-code>=0.1.83" in result.stderr + assert "FOREMAN_DEEPAGENTS_EXECUTABLE" in result.stderr + + +@pytest.mark.asyncio +async def test_timeout_and_reuse_clear_termination(monkeypatch, tmp_path): + worker = scripted_worker(monkeypatch, "import time; time.sleep(30)", + graceful_termination_seconds=0.1) + result = await worker.run(record(), tmp_path, discard, 0.1) + assert result.status is WorkerStatus.TIMED_OUT + assert worker.process.returncode is not None + monkeypatch.setattr(worker, "command", lambda *args: [sys.executable, "-c", "print('ok')"]) + result = await worker.run(record(), tmp_path, discard, 10) + assert result.status is WorkerStatus.COMPLETED and result.termination_reason is None + + +@pytest.mark.asyncio +async def test_supervisory_stop_and_no_live_steering(monkeypatch, tmp_path): + worker = scripted_worker(monkeypatch, "import time; print('ready', flush=True); time.sleep(30)") + + async def emit(*args): + await worker.terminate("off_track") + + result = await worker.run(record(), tmp_path, emit, 10) + assert result.status is WorkerStatus.STOPPED + assert result.termination_reason == "off_track" + assert await worker.steer("change course") is False + + +@pytest.mark.asyncio +async def test_cancellation_stops_worker(monkeypatch, tmp_path): + worker = scripted_worker(monkeypatch, "import time; print('ready', flush=True); time.sleep(30)") + ready = asyncio.Event() + + async def emit(*args): + ready.set() + + rec = record() + task = asyncio.create_task(worker.run(rec, tmp_path, emit, 10)) + await asyncio.wait_for(ready.wait(), 5) + task.cancel() + with pytest.raises(asyncio.CancelledError): + await task + assert rec.status is WorkerStatus.CANCELLED + assert rec.finished_at is not None and worker.process.returncode is not None + + +@pytest.mark.skipif(os.name == "nt", reason="POSIX process group test") +@pytest.mark.asyncio +async def test_termination_escalates_and_kills_descendants(monkeypatch, tmp_path): + script = ( + "import subprocess,sys,signal,time; " + "signal.signal(signal.SIGTERM, signal.SIG_IGN); " + "p=subprocess.Popen([sys.executable,'-c'," + "'import signal,time; signal.signal(signal.SIGTERM,signal.SIG_IGN); time.sleep(30)'" + "],start_new_session=True); " + "print(p.pid,flush=True); time.sleep(30)" + ) + worker = scripted_worker(monkeypatch, script, graceful_termination_seconds=0.05) + result = await worker.run(record(), tmp_path, discard, 0.3) + assert result.status is WorkerStatus.TIMED_OUT + assert worker.process.returncode == -signal.SIGKILL + with pytest.raises(ProcessLookupError): + os.killpg(worker.process.pid, 0) + child = int(result.stdout.strip()) + # Linux can retain an orphan as a zombie until init reaps it. Verify the + # child cannot keep working without requiring immediate PID disappearance. + probe = await asyncio.create_subprocess_exec( + "ps", "-p", str(child), "-o", "stat=", stdout=asyncio.subprocess.PIPE, + ) + state, _ = await probe.communicate() + assert not state.strip() or state.strip().startswith(b"Z") + + +@pytest.mark.asyncio +async def test_emit_failure_cleans_up_subprocess(monkeypatch, tmp_path): + worker = scripted_worker(monkeypatch, "import time; print('ready', flush=True); time.sleep(30)") + + async def emit(*args): + raise RuntimeError("observer failed") + + rec = record() + with pytest.raises(RuntimeError, match="observer failed"): + await worker.run(rec, tmp_path, emit, 10) + assert rec.status is WorkerStatus.FAILED + assert worker.process.returncode is not None diff --git a/tests/test_deepagents_hooks.py b/tests/test_deepagents_hooks.py new file mode 100644 index 0000000..5cf13c2 --- /dev/null +++ b/tests/test_deepagents_hooks.py @@ -0,0 +1,151 @@ +from __future__ import annotations + +import json +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from foreman.cli import app +from foreman.deepagents_setup import DEEPAGENTS_HOOK_EVENTS, install_deepagents_hooks +from foreman.hook_adapters import hook_adapter +from foreman.hooks import HookAction, HookError, HookEventKind, HookOutcome + + +def payload(tmp_path, event="UserPromptSubmit", **kwargs): + return { + "session_id": "dcode-thread", "cwd": str(tmp_path), "hook_event_name": event, + "prompt_id": "dcode-prompt", "prompt": "Fix tests", **kwargs, + } + + +@pytest.mark.parametrize("name,kind", [ + ("SessionStart", HookEventKind.SESSION_STARTED), + ("UserPromptSubmit", HookEventKind.WORK_SUBMITTED), + ("PreToolUse", HookEventKind.BEFORE_TOOL), + ("PostToolUse", HookEventKind.AFTER_TOOL), + ("PostToolUseFailure", HookEventKind.AFTER_TOOL), + ("Stop", HookEventKind.WORKER_STOPPING), + ("SessionEnd", HookEventKind.SESSION_ENDED), +]) +def test_native_events(tmp_path, name, kind): + event = hook_adapter("deepagents").parse(payload( + tmp_path, name, tool_name="Bash", tool_use_id="call-1", + tool_input={"command": "pytest"}, tool_response="passed", error="failed", + is_interrupt=False, stop_hook_active=True, last_assistant_message="done", + transcript_path="/unused.jsonl", agent_id="child-id", + )) + assert event.client == "deepagents" and event.kind is kind + assert event.session_id == "dcode-thread" + assert event.work_id == event.turn_id == "dcode-prompt" + assert event.continuation_active and event.last_assistant_message == "done" + if name == "PostToolUseFailure": + assert event.tool_response == {"isError": True, "error": "failed", "is_interrupt": False} + elif name == "PostToolUse": + assert event.tool_response == "passed" + + +@pytest.mark.parametrize("extra", [ + {"hook_event_name": "unknown"}, + {"prompt": " "}, + {"hook_event_name": "PreToolUse"}, + {"hook_event_name": "PostToolUseFailure", "tool_name": "Bash", "tool_use_id": "c"}, + {"session_id": ""}, + {"hook_event_name": "SubagentStop"}, +]) +def test_invalid_input_does_not_create_state(tmp_path, extra): + with pytest.raises(HookError): + hook_adapter("deepagents").parse(payload(tmp_path, **extra)) + assert not (tmp_path / "sessions").exists() + + +@pytest.mark.parametrize("name", DEEPAGENTS_HOOK_EVENTS) +def test_response_vocabulary(tmp_path, name): + adapter = hook_adapter("deepagents") + event = adapter.parse(payload( + tmp_path, name, tool_name="Bash", tool_use_id="c", error="failed", + )) + assert adapter.render(event, HookOutcome()) == {} + guidance = HookOutcome(action=HookAction.INJECT_CONTEXT, reason="guidance") + assert adapter.render(event, guidance) == { + "hookSpecificOutput": {"hookEventName": name, "additionalContext": "guidance"}, + } + block = adapter.render(event, HookOutcome(action=HookAction.BLOCK, reason="unfinished")) + if name == "PreToolUse": + assert block == {"hookSpecificOutput": { + "hookEventName": name, "permissionDecision": "deny", + "permissionDecisionReason": "unfinished", + }} + else: + assert block == {"decision": "block", "reason": "unfinished"} + assert adapter.render(event, HookOutcome(action=HookAction.HALT, reason="stop")) == { + "continue": False, "stopReason": "stop", "systemMessage": "stop", + } + + +def test_cli_excluded_repository_and_invalid_event(tmp_path, monkeypatch): + config = tmp_path / "config.toml" + config.write_text("[hooks]\nrepositories = []\n") + monkeypatch.setenv("FOREMAN_CONFIG", str(config)) + runner = CliRunner() + args = ["hook", "--client", "deepagents", "--data-dir", str(tmp_path)] + result = runner.invoke(app, args, input=json.dumps(payload(tmp_path))) + assert result.exit_code == 0 and json.loads(result.stdout) == {} + result = runner.invoke(app, args, input=json.dumps(payload(tmp_path, "unknown"))) + assert result.exit_code == 2 and "Deep Agents hook input" in result.output + assert not (tmp_path / "sessions").exists() + + +def test_setup_merges_preserves_and_is_idempotent(tmp_path): + path = tmp_path / "hooks.json" + existing = { + "custom": {"keep": True}, + "hooks": { + "PreToolUse": [{"matcher": "Bash", "hooks": [{ + "type": "command", "command": "existing-policy", "timeout": 20, + }]}], + "Notification": [{"hooks": [{"type": "command", "command": "notify"}]}], + }, + } + path.write_text(json.dumps(existing)) + path.chmod(0o640) + assert install_deepagents_hooks(path, "/my tools/foreman") + merged = json.loads(path.read_text()) + assert merged["custom"] == existing["custom"] + assert merged["hooks"]["Notification"] == existing["hooks"]["Notification"] + assert merged["hooks"]["PreToolUse"][0] == existing["hooks"]["PreToolUse"][0] + for event in DEEPAGENTS_HOOK_EVENTS: + handler = merged["hooks"][event][-1]["hooks"][0] + assert handler["argv"] == ["/my tools/foreman", "hook", "--client", "deepagents"] + assert handler["command"] == "'/my tools/foreman' hook --client deepagents" + before = path.read_bytes() + assert install_deepagents_hooks(path, "/my tools/foreman") is False + assert path.read_bytes() == before + assert path.stat().st_mode & 0o777 == 0o640 + + +@pytest.mark.parametrize("contents", [ + "{not-json", "[]", '{"hooks": []}', + '{"hooks": {"Stop": "bad"}}', '{"hooks": {"Stop": [{"hooks": ["bad"]}]}}', +]) +def test_setup_refuses_malformed_config_without_writing(tmp_path, contents): + path = tmp_path / "hooks.json" + path.write_text(contents) + with pytest.raises(ValueError): + install_deepagents_hooks(path, "foreman") + assert path.read_text() == contents + + +def test_setup_cli_default_and_custom_path(tmp_path, monkeypatch): + monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path)) + runner = CliRunner() + result = runner.invoke(app, ["deepagents", "setup", "--foreman-executable", "/bin/foreman"]) + assert result.exit_code == 0, result.output + assert (tmp_path / ".deepagents" / "hooks.json").exists() + custom = tmp_path / "project" / ".deepagents" / "hooks.json" + result = runner.invoke(app, ["deepagents", "setup", "--hooks-file", str(custom), + "--foreman-executable", "/bin/foreman"]) + assert result.exit_code == 0 and custom.exists() + result = runner.invoke(app, ["deepagents", "setup", "--hooks-file", str(custom), + "--foreman-executable", "/bin/foreman"]) + assert result.exit_code == 0 and "Already installed" in result.output diff --git a/tests/test_hooks.py b/tests/test_hooks.py index b9cd019..d7d259f 100644 --- a/tests/test_hooks.py +++ b/tests/test_hooks.py @@ -213,6 +213,57 @@ def runtime(tmp_path, model=None, runtime_config=None, responsibilities_dir=None return supervisor, store, router +@pytest.mark.asyncio +async def test_deepagents_routing_steering_failure_evidence_and_completion(tmp_path) -> None: + model = StubModel(work_off_track=0.96) + supervisor, store, router = runtime(tmp_path, model) + adapter = hook_adapter("deepagents") + base = {"session_id": "dcode-session", "cwd": str(tmp_path), "prompt_id": "p1"} + + async def native(name, **fields): + normalized = adapter.parse({**base, "hook_event_name": name, **fields}) + return adapter.render(normalized, await supervisor.handle(normalized)) + + assert await native("SessionStart") == {} + await native("UserPromptSubmit", prompt="Implement the feature") + await native("UserPromptSubmit", prompt="Implement the feature") + assert router.calls == ["Implement the feature"] + output = await native("PostToolUseFailure", tool_name="Bash", tool_use_id="c1", + tool_input={"command": "pytest"}, error="tests failed") + assert output["hookSpecificOutput"] == { + "hookEventName": "PostToolUseFailure", + "additionalContext": "active worker appears off track", + } + session = store.load("dcode-session", "deepagents") + assert session.state.workers[0].steer_count == 1 + assert "tests failed" in model.calls[0][0].latest_worker_output + # Let completion checks determine the next outcome independently of drift. + model.scores["work_off_track"] = 0 + output = await native("Stop", stop_hook_active=False, last_assistant_message="Started") + assert output["decision"] == "block" + output = await native("Stop", stop_hook_active=True, last_assistant_message="Still incomplete") + assert output["continue"] is False + assert "one automatic continuation" in output["systemMessage"] + assert await native("SessionEnd") == {} + assert store.load("dcode-session", "deepagents") is None + + +@pytest.mark.asyncio +async def test_deepagents_pre_tool_block(tmp_path) -> None: + supervisor, _, _ = runtime(tmp_path, StubModel(needs_human=0.99)) + adapter = hook_adapter("deepagents") + base = {"session_id": "dcode-session", "cwd": str(tmp_path), "prompt_id": "p1"} + submitted = adapter.parse({ + **base, "hook_event_name": "UserPromptSubmit", "prompt": "Fix tests", + }) + await supervisor.handle(submitted) + before = adapter.parse({**base, "hook_event_name": "PreToolUse", + "tool_name": "Bash", "tool_use_id": "c1", + "tool_input": {"command": "pytest"}}) + output = adapter.render(before, await supervisor.handle(before)) + assert output["hookSpecificOutput"]["permissionDecision"] == "deny" + + @pytest.mark.asyncio async def test_prompt_routes_multiple_built_in_toml_responsibilities(tmp_path) -> None: supervisor, store, router = runtime(tmp_path) diff --git a/tests/test_integration.py b/tests/test_integration.py index 6cc39a5..0b65f41 100644 --- a/tests/test_integration.py +++ b/tests/test_integration.py @@ -206,7 +206,10 @@ async def test_noisy_events_are_coalesced(tmp_path) -> None: repository=tmp_path, job="Small job", model=model, - config=config(assessment_min_interval_seconds=0.1), + # Completion bypasses debounce. Keep routine assessments outside the + # worker's lifetime so slower event persistence cannot consume the + # scripted ready result while the output burst is still in progress. + config=config(assessment_min_interval_seconds=5, periodic_assessment_seconds=5), worker_factory=lambda _: FakeWorker( output_lines=[str(number) for number in range(50)], delay_seconds=0 ), @@ -219,6 +222,8 @@ async def test_noisy_events_are_coalesced(tmp_path) -> None: ] assert len(output_events) == 50 assert len(model.calls) == 2 + assert state.status is FactoryStatus.FINISHED + assert not model.calls[-1].active_workers @pytest.mark.asyncio diff --git a/tests/test_pi_hooks.py b/tests/test_pi_hooks.py index 1811bd4..ca125c0 100644 --- a/tests/test_pi_hooks.py +++ b/tests/test_pi_hooks.py @@ -88,4 +88,4 @@ def test_pi_cli_excluded_repository_is_noop(client, native, tmp_path, monkeypatc def test_pi_clients_registered(): - assert available_hook_clients() == ("codex", "pi", "pi-durable") + assert available_hook_clients() == ("codex", "deepagents", "pi", "pi-durable")