fix(coding-agent): preserve task services through verification

evals/terminal-bench/README.md

@@ -7,6 +7,9 @@ ATIF export and usage accounting run after evidence recovery, including interrup
 The [staged benchmark release gate](RELEASE_GATE.md) pins dataset inputs and verifies retained evidence before larger Daytona runs.
 Use the [offline job exporter](RELEASE_GATE.md#export-retained-trials-for-harbor-hub) to package existing stage results for Harbor Hub without rerunning evaluations.
 This integration is not yet ready for leaderboard runs.
+The adapter now requires a release with `--background-process-lifetime` and `--background-process-ownership-dir` support.
+The checked-in Diffex 0.3.0 pin predates these flags; installation deliberately fails its capability check until a corrected release and integrity are published and pinned.
+Source-only lifecycle tests do not qualify the installed release or a live benchmark trial.
 
 ## Host setup
 
@@ -121,14 +124,43 @@ Generated evidence is retained under `results/diffex-execution-*/`.
 Each execution smoke validates ATIF with Harbor's validator and reconciles nonzero input, output, cached-token, and cost totals against every root and child response.
 The outcome smoke exercises success, authentication failure, exhausted retry, recovered retry, interrupted JSON output, and a Harbor-style execution timeout as both root and a non-root user.
 It uses the pinned npm release with a loopback provider and checks nonzero terminal-error exits and rejection of truncated output even when the process exits zero.
-Success and timeout cases create a detached tool descendant that ignores TERM, then verify that cleanup stops it before the next trial.
+Success, provider-failure, forced-CLI-death, and timeout cases create a TERM-resistant background service and verify that its PID and start time are unchanged across adapter cleanup.
+The verifier phase queries that same service and checks that the launcher, CLI, and provider requests have stopped.
+The Linux cleanup regression independently creates a detached service with durable handoff evidence, and covers PID reuse, delayed launch, malformed records, and TERM-resistant active work.
 Evidence is retained under `results/diffex-outcomes-*/`, including partial stdout, stderr, native sessions, process status, cleanup, and outcomes.
 The outcome smoke also validates recoverable ATIF steps and checks that interrupted runs report incomplete accounting.
 The packaging check packs every public npm workspace into temporary tarballs with lifecycle scripts disabled, inspects their contents for benchmark paths, and records file counts and SHA-256 hashes in `results/production-contents.json`.
 It requires existing production build output, including `packages/coding-agent/dist/cli.js`, and fails if that CLI entrypoint is missing.
 It does not build, publish, or regenerate release artifacts.
 Run `npm run check` from the repository root after changing code.
 
+### Source lifecycle validation
+
+Until the corrected release is published, build the development-only image from the repository root:
+
+```bash
+docker build --platform linux/arm64 -f evals/terminal-bench/tests/fixtures/Dockerfile.source -t diffex-adapter-tests:arm64 .
+cd evals/terminal-bench
+uv run --locked python scripts/smoke_outcomes.py --platform linux/arm64 --source-image diffex-adapter-tests:arm64
+uv run --locked python scripts/smoke_outcomes.py --platform linux/arm64 --source-image diffex-adapter-tests:arm64 --grpc-fixture
+docker run --rm --init --user 65534:65534 -v "$PWD:/eval:ro" diffex-adapter-tests:arm64 node --test /eval/tests/cleanup.test.cjs
+```
+
+Repeat with `linux/amd64` and an `amd64` image tag for emulated coverage on an arm64 host.
+Run the cleanup regression without `--user` for root coverage.
+The development image installs dependencies without lifecycle scripts and runs the source CLI through `tsx`.
+The gRPC fixture uses a known-correct implementation and all seven unmodified verifier tests extracted from the SHA-256-verified dataset archive.
+It checks the original process identity before and after verifier RPCs and confirms container deletion afterward.
+Every development smoke report is marked `qualification: source-only`.
+After publication, update `pins.toml`, rerun both pinned-release Linux smokes, and start a fresh live benchmark trial before declaring integration qualification complete.
+
+On 2026-09-20, source-only validation passed 16 controlled outcome cases on each architecture, including root and UID 65534 execution.
+Both architectures also passed all seven pinned gRPC verifier tests with unchanged service identity and confirmed container deletion.
+The Linux cleanup suite passed seven deterministic cases, including forced death before CLI identity publication, and all 142 Python tests and `npm run check` passed.
+The retained outcome reports are `results/diffex-outcomes-8dab466d6f5c/smoke.json` (arm64) and `results/diffex-outcomes-284ee6f0c224/smoke.json` (amd64).
+The gRPC reports are `results/diffex-outcomes-8d83db9627fb/smoke.json` (arm64) and `results/diffex-outcomes-da1f6f5534a3/smoke.json` (amd64).
+These local source checks leave pinned-release smokes and a fresh live benchmark trial pending.
+
 ### Recorded validation
 
 The following historical smoke records used Diffex 0.2.2.
@@ -270,7 +302,9 @@ Each run directory retains:
 - `effective.json`: resolved model, thinking, tools, working directory, session path, prompt hashes, and release-context verification.
 - `system-prompt.txt`: the effective system prompt, including release and task context.
 - `stdout.jsonl`, `stderr.log`, and `process.json`: native JSON events, diagnostics, and process status.
-- `launcher.json` and `cleanup.json`: launcher process identity and verified cleanup status.
+- `launcher.json` and `cli.json`: launcher and root CLI PID/start-time identities.
+- `ownership/*.json`: durable shell ownership records published by the CLI.
+- `stop-requested` and `cleanup.json`: the launch barrier, verified control shutdown, and explicitly retained environment-owned processes.
 - `launcher.stdout.log` and `launcher.stderr.log`: launcher diagnostics, including failures before CLI startup.
 - `outcome.json`: execution classification, stream completeness, final assistant stop reason, retry counts, interruption state, and recovery errors.
 - `accounting.json`: observed usage, nullable totals, per-model observations, and reasons accounting is incomplete.
@@ -283,7 +317,8 @@ Provider usage fields in `AgentContext` contain reconciled sums of recorded usag
 
 ### Execution outcomes and interruption recovery
 
-A completed execution requires zero CLI and launcher exit codes, no terminating signal or spawn error, matching effective configuration, and verified cleanup.
+A completed execution requires zero CLI and launcher exit codes, no terminating signal or spawn error, matching effective configuration, and verified cessation of agent execution.
+Environment-owned services may remain alive for shared-environment verification.
 The parser reads JSONL incrementally and requires matching assistant messages, a final `stop` response, closed message/run boundaries, resolved retries, and the native `agent_settled` event.
 Malformed or truncated lines, missing completion events, pending tool use, token-limit termination, and unresolved retries cannot count as completed runs.
 Earlier provider errors are recoverable when retry events and the subsequent terminal assistant state establish success.
@@ -297,7 +332,8 @@ Tool errors alone do not fail execution: the assistant can inspect and recover f
 | Nonzero process, signal, or spawn error | `process_failure` | Raise `DiffexExecutionError` |
 | Missing, truncated, or unfinished native stream | `incomplete` | Raise `DiffexExecutionError` |
 | Effective configuration mismatch | `configuration_failure` | Raise `DiffexExecutionError` |
-| Unverified cleanup or failed evidence collection | `cleanup_failure` / `infrastructure_failure` | Raise `DiffexExecutionError` |
+| Unverified agent shutdown or missing shutdown evidence | `cleanup_failure` / `infrastructure_failure` | Raise `DiffexControlStopError`; block verification |
+| Failed evidence collection with verified shutdown | `infrastructure_failure` | Raise `DiffexExecutionError` |
 | Harbor task deadline cancels `run()` | `cancelled` | Preserve cancellation; Harbor raises `AgentTimeoutError` |
 | Environment transport timeout or failure | `timeout` / `transport_failure` | Preserve the original exception |
 
@@ -308,13 +344,23 @@ The pinned `0.3.0` release returns nonzero for terminal assistant errors and abo
 The adapter also parses native events to classify provider failures and reject incomplete output independently of process exit status.
 
 The launcher writes stdout and stderr directly to files and retains native sessions during execution.
-On success, failure, or cancellation, an independent Harbor exec requests launcher shutdown, stops the CLI process group, and scans Linux `/proc` for same-user descendants carrying the trial's unique environment marker.
-Process start times protect against signaling a reused PID.
-Cleanup escalates from TERM to KILL after five seconds, waits up to two more seconds, and records whether any live tracked processes remain.
+On success, failure, or cancellation, an independent Harbor exec writes the stop barrier and signals identified launcher and CLI processes individually.
+A private pipe prevents CLI startup until the launcher has recorded its PID and start time; launcher death before publication closes the pipe without starting agent execution.
+No launcher process-group signal can include a retained service.
+After control processes exit, cleanup reads durable ownership records and scans Linux `/proc` for trial-marked processes and identified shell groups.
+Linux boot ID, PID, start time, process group, and shell execution tokens distinguish transferred groups from reused process IDs.
+Environment-owned groups survive; active or otherwise untransferred work receives TERM, escalating to KILL within a seven-second recovery budget.
+Control escalation begins after three seconds, reserving time for active descendants; active-work escalation begins after five seconds.
 Already exited zombies cannot execute and are left for sandbox init to reap.
-A stop-request file prevents delayed launch after transport cancellation.
-Task commands must preserve the inherited trial environment marker for detached descendants to remain discoverable.
-Harbor must still delete the sandbox after the trial for complete environment isolation.
+The report records `agent_execution_stopped`, `ownership_validated`, `remaining_agent_pids`, control identities, retained service identities, and the consumed ownership records.
+Outcome parsing and gate validation reconcile these records with retained files.
+`cleanup_confirmed` in the outcome means pre-verifier agent shutdown, not an empty sandbox.
+Missing, malformed, contradictory, or unverified shutdown evidence raises `DiffexControlStopError`, which is outside Harbor's verifier-eligible timeout and nonzero-exit exceptions.
+This exception overrides a pending cancellation while retaining its interrupted outcome in the evidence.
+Verified shutdown preserves Harbor's existing verification behavior for deadlines and nonzero agent exits without changing their failed classifications.
+Task commands must preserve the inherited trial marker or remain in an identified shell group for unfinished descendants to remain discoverable.
+Harbor stops the shared environment after verification, including verifier failure or timeout recovery; separate-environment verification keeps Harbor's artifact-based lifecycle.
+The benchmark runner independently confirms sandbox deletion after every trial, including abandonment and transport failures.
 
 Recovery is shielded against repeated cancellation and runs before `run()` returns or re-raises the interruption.
 It allows up to 15 seconds for the cleanup transport and 15 seconds for mounted-log preparation or 30 seconds for remote log download.

evals/terminal-bench/RELEASE_GATE.md

@@ -2,7 +2,8 @@
 
 This gate qualifies a specific Diffex adapter, dataset, model, and harness configuration for a larger benchmark run.
 It does not publish a product release or submit a leaderboard entry.
-The implementation supports all qualification stages; live Diffex qualification remains pending a model-provider credential.
+The implementation supports all qualification stages; live Diffex qualification requires provider credentials and a published release with environment-owned process support.
+The checked-in Diffex 0.3.0 pin lacks that support and deliberately fails installation capability validation until a corrected release and integrity are pinned.
 
 ## Configuration and prerequisites
 
@@ -102,7 +103,8 @@ The Harbor authentication and upload commands below also apply to the exported s
 
 The adapter runs from the local uv project, and a newly started command reads the current Python source and `benchmark.toml`.
 An already running command keeps its loaded code and policy; let it finish or interrupt it once with Ctrl-C and wait for cleanup before restarting.
-No Diffex npm release or product rebuild is needed for these runner and policy changes; new sandboxes install the existing pinned release with the updated tool allowlist.
+The environment-owned process policy requires a corrected Diffex npm release and a reviewed pin update before new sandboxes can qualify.
+Development source images exercise the lifecycle without qualifying the installed release.
 
 After updating the checkout, run the following in your Mac's zsh terminal with `DAYTONA_API_KEY` and `OPENAI_API_KEY` exported.
 `git status --short` must produce no output.
@@ -245,6 +247,12 @@ Diffex installation evidence includes the three CLI checks and verified npm/Node
 The oracle does not install Diffex or call a model, so its Diffex installation, model usage, and ATIF checks are not applicable.
 
 The gate re-parses native process/stream evidence instead of trusting the outcome summary alone.
+The execution policy includes `background_process_lifetime: environment` in the requested/effective configuration and benchmark identity.
+Pre-verifier evidence must establish that all agent execution stopped, with every retained task service backed by a durable environment ownership record.
+The gate reconciles control identities, ownership files, cleanup reports, outcome summaries, and trajectory metadata.
+Retained services do not count as cleanup failures, and verifier reward cannot convert a deadline or provider failure into completed execution.
+An unverified control stop raises `DiffexControlStopError` and prevents Harbor from entering verification.
+Final cleanup remains Harbor's environment stop followed by independent sandbox-deletion confirmation.
 It regenerates ATIF into a temporary directory, validates it with Harbor, and compares it with retained trajectories and image attachments.
 Native totals, per-model totals, `accounting.json`, and `AgentContext` must reconcile.
 Unknown telemetry remains unknown and cannot satisfy complete accounting.

evals/terminal-bench/benchmark.toml

@@ -30,3 +30,4 @@ thinking = "max"
 tools = "read,bash,edit,write,update_plan"
 subagents = "enabled"
 project_resources = "context-only"
+background_process_lifetime = "environment"

evals/terminal-bench/execution.yaml

@@ -13,3 +13,4 @@ agents:
       tools: read,bash,edit,write,update_plan
       subagents: enabled
       project_resources: context-only
+      background_process_lifetime: environment

evals/terminal-bench/full.yaml

@@ -14,6 +14,7 @@ agents:
       tools: read,bash,edit,write,update_plan
       subagents: enabled
       project_resources: context-only
+      background_process_lifetime: environment
     env:
       OPENAI_API_KEY: ${OPENAI_API_KEY}
 datasets: