[Pepps233]

Summary

Separate shutdown verification from bulk artifact collection so a log-download timeout does not suppress the Terminal-Bench verifier after the agent has stopped.

Keep task packages and caches on sandbox disk while persisting logs and transcripts in a shared Daytona volume.

Automatically reclaim completed trial prefixes after verification and local artifact validation.

Reasoning

The completed 89 x 5 run had 55 errors, including 21 bulk-download timeouts classified as shutdown failures.

Eighteen of those trials later retained evidence that passes the current shutdown validator; exported state reached 3.5 GB in one trial.

The adapter now validates a small independent evidence bundle before downloading logs and continues to block verification when shutdown cannot be established.

Daytona recommends shared volumes with isolated subpaths and limits organizations to 100 volumes, so each sandbox gets a unique prefix in one project volume.

Completed trials release volume storage only after their remote files match retained local copies and all original sandboxes have been deleted.

Failed or incomplete trials retain remote recovery data, while zero-reward completed trials are eligible for cleanup.

The implementation follows the pinned SDK and Harbor interfaces and documents Tier 4 capacity and retention requirements using the volume and limits documentation.

Changes

• Fetch shutdown reports, stop markers, identities, and ownership records first, with bounded transport retries and time for delayed report publication.
• Record bulk-download, trajectory, and local persistence failures as artifact errors while preserving conclusive execution failures and interruption semantics.
• Store agent state outside exported logs and retain it through task verification.
• Mount a shared volume at /logs, wait for readiness, isolate sandbox subpaths, and retain all session recovery locations.
• Route the native full-job config through the volume backend and document migration from --env daytona.
• Automatically prune eligible trial prefixes in the staged runner; enable the same END hook in the documented native command with --plugin diffex_terminal_bench.retention:VolumeCleanupPlugin.
• Use one disposable cleanup sandbox per eligible trial to compare SHA-256 hashes and prune only its mounted prefixes, including separate verifier prefixes, with bounded retries and shielded sandbox deletion.
• Keep local artifacts and record cleanup status in volume-cleanup.json; account for the extra sandbox creation within the existing worker slot.
• Include terminal subagent failure handling, safe evidence symlink support, and disabled benchmark subagent defaults from the branch's existing commits.

Testing

• Reproduced the bulk-timeout shutdown failure through Harbor's SingleStepTrial flow before the fix; the regression now reaches the verifier.
• .venv/bin/python -m unittest discover -s tests -v passed, followed by focused retention and recovery tests after final retry adjustments.
• Cleanup tests execute the uploaded pruning script on real temporary files and exercise Harbor's finalize hook, delayed deletion, separate verifiers, hash mismatches, missing local artifacts, symlinks, lost responses, and cancellation.
• node --test tests/launcher.test.cjs.
• harbor run --config full.yaml --plugin diffex_terminal_bench.retention:VolumeCleanupPlugin --print-config.
• npm run check.
• ./test.sh.

Live Daytona and Docker smoke tests were not run: this session has no Daytona/provider credentials and Docker is not running.
The 445-trial evaluation was inspected, not rerun.