[rappdw]

Summary

Copied launch logic, weak success checks, and ambiguous suite output made regressions easy to miss and failures difficult to diagnose. This change exercises production paths, requires successful exit codes and positive completion evidence, and reports an explicit overall suite verdict.

• Split the main runner into selectable harness, production-contract, host, container, and Go groups. Show named Python tests, numbered group progress, per-group results, and a final pass/failure status; preserve failure details in logs.
• Replace duplicated implementations and obsolete assertions with production-function contracts and a real-launch container fixture. Three launches verify persistence, workspace isolation, protected mounts, startup behavior, and local package installation without live provider credentials.
• Isolate host fixtures and block accidental Docker/tmux access there. Keep logs and structured results in ignored workspace .test_output/, rotate to at most three runs, and provide a host/container-compatible latest link.
• Strengthen live-provider integration checks with actual exit status, positive answer/completion markers, process-group timeouts, image freshness checks, and a network positive control.
• Separate CI groups, add macOS contract coverage, retain failure artifacts, and schedule full integration coverage on Sundays and by default for manual runs.
• Fix screenshot-directory validation to reject HOME even when its path contains symlinks, and document the behavior and test coverage.

Linked issue

None.

Checklist

• Ran both test suites on the host: bash test/run-tests.sh and bash test/run-integration-tests.sh. The main suite passed on macOS with Docker; the live-provider integration suite has not been rerun.
• Ran the regen --check scripts (test/regen-config-docs.sh --check, test/regen-template.sh --check) — no drift.
• Updated SPECIFICATION.md, CLAUDE.md, configuration guidance, and the testing plan for the behavior changes.
• Config metadata/tier updates: not applicable; no config keys added or re-tiered.
• sandy is still a single file with no new production runtime dependency.
• Commit is focused and branched from main.

Notes for the reviewer

The latest macOS host run passed all 14 harness tests, 10 production-contract tests, 2,869 host checks, the real-launch container test, and Go tests with the race detector. Two host groups explicitly skipped runtime assertions covered by the container group; the runner reports those skips without treating them as executed checks. ShellCheck and generated-file drift checks passed in that run.

The separate live-provider integration suite and the updated GitHub Actions workflows still require execution. The testing plan records replacement coverage and remaining limits. The branch merges cleanly with current main.