✨ Repeat browser workflows with reproducible seeds

.github/actions/test/action.yml

@@ -2,7 +2,7 @@ name: Run Dyno tests
 description: Run one suite and retain its console and JUnit output.
 inputs:
   suite:
-    description: core, browser, execution, project, reporter, package, capture, or native.
+    description: core, browser, execution, repeat, project, reporter, package, capture, or native.
     required: true
   browser:
     description: Playwright engine for browser suites.

.github/workflows/ci.yml

@@ -74,7 +74,7 @@ jobs:
       fail-fast: false
       matrix:
         engine: [chromium, firefox, webkit]
-        suite: [browser, execution, project, reporter, capture, native]
+        suite: [browser, execution, repeat, project, reporter, capture, native]
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
@@ -95,7 +95,7 @@ jobs:
       fail-fast: false
       matrix:
         engine: [chromium, firefox, webkit]
-        suite: [browser, execution, project, reporter, capture, native]
+        suite: [browser, execution, repeat, project, reporter, capture, native]
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:

docs/architecture.md

@@ -147,6 +147,12 @@ The CLI, live reporter and saved report expose the seed. `schedule.json` records
 
 ## Shared coordination
 
+Managed headless `--repeat` composes ordinary runs inside one CLI session. `repeat.js` folds accepted round outcomes into immutable per-test counts; the existing run model still accepts one execution per stable test ID. Each seeded round uses a versioned root-derived seed, with the literal root preserved for round one. Queue replay stays fixed. The coordinator's common replacement path receives the next seed and reuses the pinned test bundle, worker contexts, servers and project resources.
+
+Automatic repeated executions park their final app document before project draining and release, including when there is no later real test. The next round waits for terminal persistence and the current serialized operation to settle. Each round has its own browser-result deadline; lifecycle and storage failures stop advancement. Ordinary interactive completion keeps the final app inspectable as before. Storage and backend reset remain project-owned.
+
+`repeat-reports.js` saves compact round outcomes and schedules independently of the coordinator's 20-run cache. Failed/interrupted and final rounds use the ordinary portable formatter. Successful intermediate captures are removed after evidence persistence. A 256 MiB evidence limit stops further rounds explicitly; accepted counts and aggregate/session metadata remain available outside that budget. The test fingerprint does not identify the running app or backend, and rerunning a batch prefix does not replay asynchronous interleaving.
+
 `coordinator.js` owns run snapshots, processes results in order, and coordinates assignment waiting, message replays, draining and final saves. `run.js` contains pure ownership/resource transitions. Private gateways bind requests to their worker ID; posted worker IDs cannot borrow a peer execution. Cross-gateway image reads resolve accepted UUIDs from the same canonical outcomes.
 
 Project setup runs once per CLI session and optionally returns `createWorker`. Both receive one frozen session identity and immediate cleanup registration. Registration seals when acquisition settles; partial failures use the same cleanup stack as returned teardown. Every started acquisition settles before worker resources or shared parents are destroyed. The factory supplies private server/env/fixtures/beforeExecution/drain/teardown descriptors. Startup joins every worker task before cleanup, including on partial failure. Cleanup finishes the coordinator and gateways before stopping app servers, worker projects and shared setup. Browser shutdown follows.

docs/ci.md

@@ -10,6 +10,7 @@
 | Package | `npm run test:package` | Fresh tarball and checkout installs, init, and a real Chromium test on Node 26 |
 | Browser | `npm run test:browser -- firefox` | Real app, gateway, viewport, and process workflows in Chromium, Firefox, and WebKit on Linux and macOS, using Node 24 |
 | Execution | `npm run test:execution -- firefox` | Native concurrency, ownership, queue replay, seeded order, suite hooks, cleanup, pause, watch and replacement workflows in all three engines on Linux and macOS |
+| Repeat | `npm run test:repeat -- firefox` | Repeated selections, seeded batches, resource reuse, cancellation and cleanup in all three engines on Linux and macOS |
 | Project | `npm run test:project -- firefox` | Project resource acquisition, fixture transport and per-execution preparation in all three engines on Linux and macOS |
 | Reporter | `npm run test:reporter -- firefox` | Inspection, output, timings, reruns, and portable saved reports in all three engines on Linux and macOS |
 | Native | `npm run test:native -- firefox` | Native defaults, capabilities, state restoration, dialogs, diagnostics, and cancellation in all three engines on Linux and macOS |
@@ -22,11 +23,11 @@ node bin/dyno.js install --browser firefox
 npm run test:browser -- firefox
 ```
 
-Tests live under `tests/core`, `tests/package`, `tests/browser`, `tests/execution`, `tests/project`, `tests/reporter`, `tests/capture`, and `tests/native`. The runner discovers every `.test.js` file in the selected directory and rejects test files outside these suites. Keep browser-free checks in core and browser workflows in the relevant feature suite. Tests use real CLI, server, and Playwright browser boundaries; these groups only decide where they run.
+Tests live under `tests/core`, `tests/package`, `tests/browser`, `tests/execution`, `tests/repeat`, `tests/project`, `tests/reporter`, `tests/capture`, and `tests/native`. The runner discovers every `.test.js` file in the selected directory and rejects test files outside these suites. Keep browser-free checks in core and browser workflows in the relevant feature suite. Tests use real CLI, server, and Playwright browser boundaries; these groups only decide where they run.
 
-Core checks and actionlint workflow validation run first. Browser and package suites run in parallel once core passes. Documentation runs separately. Browser, execution, project, reporter, native, and capture suites have separate jobs so each complete suite has its own budget. Each test job has a hard five-minute cap, including setup. Browser and execution checks run up to three independent files together; each owns its app, gateway, browser profiles, and fixture directory. Tests inside a file stay sequential. Project, reporter, native, capture, and package suites stay sequential. The macOS process-group check runs once in core instead of repeating for each browser. There are no retries or extra sleeps.
+Core checks and actionlint workflow validation run first. Browser and package suites run in parallel once core passes. Documentation runs separately. Browser, execution, repeat, project, reporter, native, and capture suites have separate jobs so each complete suite has its own budget. Each test job has a hard five-minute cap, including setup. Browser and execution checks run up to three independent files together; each owns its app, gateway, browser profiles, and fixture directory. Tests inside a file stay sequential. Repeat, project, reporter, native, capture, and package suites stay sequential. The macOS process-group check runs once in core instead of repeating for each browser. There are no retries or extra sleeps.
 
-Execution and reporter checks have their own suites because the combined macOS Firefox browser suite reached 288–294 seconds. Project setup and preparation now have their own suite after execution exhausted the same cap. Every test and engine/OS combination remains covered. Product suites and benchmarks keep their configured execution; this grouping only changes Dyno's own CI.
+Execution and reporter checks have their own suites because the combined macOS Firefox browser suite reached 288–294 seconds. Project setup and preparation now have their own suite after execution exhausted the same cap. Repeated runs have a separate suite after the added real-browser rounds pushed macOS Firefox execution past that cap. Every test and engine/OS combination remains covered. Product suites and benchmarks keep their configured execution; this grouping only changes Dyno's own CI.
 
 ## Results and setup
 

docs/guide.md

@@ -21,6 +21,16 @@ npx dyno install
 
 `init` adds a config and an example test without overwriting existing files. Commit your lockfile so local runs and CI use the same Dyno revision.
 
+Dyno currently pins Interactor fixes that are not yet on the registry. pnpm 11 blocks Git subdependencies and their prepare builds by default. Projects that permit this Git source need the following in `pnpm-workspace.yaml`, alongside their existing build entries:
+
+```yaml
+blockExoticSubdeps: false
+allowBuilds:
+  interactor.js: true
+```
+
+`blockExoticSubdeps` changes the project's whole dependency-source policy; the build permission above only allows Interactor. See [pnpm's source policy](https://pnpm.io/settings#blockexoticsubdeps) before choosing that setting.
+
 ### Point it at your app
 
 In `dyno.config.js`, set the command that starts your app and the URL it serves:
@@ -431,6 +441,27 @@ Turn on **Pause on failure** in Run options to inspect a failing app. Continue h
 
 The CLI also prints a separate reporter URL. Open it in another tab if you'd rather keep the app uncovered. A completed interactive run keeps each worker's last app available; rerun an earlier test to inspect it again.
 
+### Repeat while developing
+
+Exercise a workflow repeatedly, or vary the order of a whole selection:
+
+```sh
+dyno --test 'checkout' --repeat 100
+dyno --suite Checkout --repeat 100 --random
+dyno --suite Checkout --repeat 100 --seed checkout-race
+dyno --repeat 100 --replay .dyno/reports/<run-id>/schedule.json
+```
+
+`--repeat N` runs the complete selection N times, including the first run. It supports managed headless runs; combining it with `--ui`, `--manual`, or `--watch` fails before startup. Tests keep their normal hooks and per-execution fixture preparation. Setup runs once, and browser storage, servers, and project worker resources survive between rounds. Keep scenario reset and backend cleanup in your project.
+
+Each automatic execution removes its test runtime and parks the app before project draining, including the final test. A round must finish cleanup and save its results before the next starts. Each round gets its own `runTimeout`. A failed test stays in the batch result while later rounds continue; cleanup, browser, preparation, persistence, or deadline failures stop repetition. Ctrl-C saves accepted partial results and stops further rounds.
+
+Without a seed, rounds use declared priority. With `--seed`, round one uses that literal seed and later rounds derive repeatable seeds from it. `--random` generates and prints a root seed; pass it to `--seed` to reproduce the series. The count does not affect seed derivation, so rerunning through the first failing round preserves the prefix. Repeated `--replay` keeps the recorded worker queues fixed.
+
+The summary prints commands to replay the first failing round and rerun the original selection through it. A focused rerun can remove the predecessor that caused a failure. A round-only replay also omits state accumulated in earlier rounds. Seeds reproduce priority, and schedules reproduce queues; neither controls app randomness, backend data, or asynchronous timing. Keep app code and data stable while collecting results. Dyno pins the test bundle, not the running application.
+
+See [saved reports](guide.md#saved-reports) for batch counts, schedules, images, and evidence limits. Passing many times is evidence under those conditions; it does not prove a test can never fail.
+
 ### Bring your own browser
 
 ```sh
@@ -501,10 +532,18 @@ Each folder includes `run.json`, `junit.xml`, `schedule.json`, and `session.json
 
 Runs are `completed` or `interrupted`; tests without accepted results are `not-run`. Failures retain their phase and source context. JUnit adds an error case for interruption, so an unfinished run can't look green.
 
-Dyno saves results before reporting completion, including accepted outcomes on signals, browser loss, and app failure. A force kill or machine failure can't flush results. Storage errors fail the CLI, and partially written folders aren't published as reports. A source image removed before saving appears as **Screenshot unavailable**.
+Dyno saves results before reporting completion, including accepted outcomes on signals, browser loss, and app failure. A force kill or machine failure can't flush results. Storage errors fail the CLI, and partially written run folders aren't published as reports. A source image removed before saving appears as **Screenshot unavailable**.
 
 Reports contain test output and source excerpts. Dyno doesn't save config, fixtures, connection keys, or launch tokens in them.
 
+### Repeated-run reports
+
+`--repeat` writes `.dyno/reports/repeat-<session-id>/`. Open its `index.html` for aggregate counts and the first failure. `batch.json` records the root seed, test-bundle fingerprint, browser identity, original selection, and observed outcomes. Skipped and unfinished tests stay separate from passes. The batch's `junit.xml` gives each test execution its round number, so a later pass cannot replace an earlier failure.
+
+`rounds.jsonl` records completed and interrupted rounds, per-test outcomes, and paths to their worker schedules under `schedules/`. Full portable reports and images are retained under `runs/` for failed or interrupted rounds and the final round. Successful intermediate rounds keep compact evidence; their screenshots are discarded. Session timing lives once in the batch folder.
+
+Round evidence and retained reports share a 256 MiB budget. Reaching it stops repetition, marks unavailable evidence explicitly, and preserves accepted counts in `batch.json`. The aggregate summary, final compact outcome record, and session timing are saved outside that budget so a limit cannot hide a failure or stop reason. Batch metadata updates after each saved round; HTML and JUnit are finalized on shutdown. Storage failures also fail the command. Copy the whole batch folder to keep its relative report and image links; printed commands refer to the original local paths.
+
 ## Understand where time goes
 
 Open **Slowest tests** in the reporter, then a test's **Timings** tab. Dyno always records document startup, hooks, the test body, queued errors, and operations such as navigation, requests, screenshots, and native calls.

package.json

@@ -40,6 +40,7 @@
     "test:core": "node scripts/test.js core",
     "test:browser": "node scripts/test.js browser",
     "test:execution": "node scripts/test.js execution",
+    "test:repeat": "node scripts/test.js repeat",
     "test:project": "node scripts/test.js project",
     "test:reporter": "node scripts/test.js reporter",
     "test:package": "node scripts/test.js package",

scripts/test.js

@@ -7,11 +7,11 @@ import { fileURLToPath } from 'node:url';
 import { browserEngine } from '../src/browsers.js';
 
 let [suite = 'all', browser = 'chromium', ...extra] = process.argv.slice(2);
-let directories = ['core', 'browser', 'execution', 'project', 'reporter', 'package', 'capture', 'native'];
+let directories = ['core', 'browser', 'execution', 'repeat', 'project', 'reporter', 'package', 'capture', 'native'];
 let suites = ['all', ...directories];
 let needsBrowser = suite === 'all' || (directories.includes(suite) && suite !== 'core');
 if (!suites.includes(suite) || extra.length || (!needsBrowser && process.argv[3]))
-  throw new Error('Usage: node scripts/test.js [all|core|browser|execution|project|reporter|package|capture|native] [chromium|firefox|webkit for browser suites]');
+  throw new Error('Usage: node scripts/test.js [all|core|browser|execution|repeat|project|reporter|package|capture|native] [chromium|firefox|webkit for browser suites]');
 
 let root = fileURLToPath(new URL('../', import.meta.url));
 let files = [];

src/browser/shell.js

@@ -833,7 +833,7 @@ try {
       (state.phase === 'pausing' || (state.pauseOnFailure && status === 'failed'))
     )
       await paused();
-    if (reported.workRemaining) await parkApp(runner.id);
+    if (reported.workRemaining || settings.releaseFinal) await parkApp(runner.id);
   }
   let failed = state.tests.filter((test) => test.status === 'failed').length;
   let skipped = state.tests.filter((test) => test.status === 'skipped').length;

src/cli-output.js

@@ -65,6 +65,24 @@ export function createCLIOutput({ json, stdout = process.stdout, stderr = proces
           break;
         case 'fatal': log(`Browser run failed: ${event.error?.message ?? 'Unknown error'}`); break;
         case 'watch': log(`Watch: ${event.message ?? event.status}`); break;
+        case 'repeat-start':
+          line(`Repeating ${event.requested} rounds; browser storage and project worker resources are reused.`);
+          if (event.seed !== null) line(`Root seed: ${event.seed}`);
+          line(`Reproduce: ${event.command}`);
+          break;
+        case 'repeat-round': line(`\nRound ${event.round}/${event.requested}`); break;
+        case 'repeat-complete':
+          line(`\nRepeated runs: ${event.status} · ${event.completed}/${event.requested} rounds completed · ${event.failedRounds} with failures`);
+          for (let test of event.tests)
+            line(`${testTitle(test)}: ${test.passed} passed, ${test.failed} failed, ${test.skipped} skipped, ${test.notRun} not run`);
+          if (event.firstFailure) {
+            line(`First failure: round ${event.firstFailure.round} · ${event.firstFailure.error?.message ?? 'Run interrupted'}`);
+            if (event.firstFailureReplay) line(`Replay this round: ${event.firstFailureReplay}`);
+            line(`Rerun through failure: ${event.prefixCommand}`);
+          }
+          if (event.reason) log(`Repeated runs stopped: ${event.reason}`);
+          line(`Repeated-run report: ${event.html}`);
+          break;
         case 'report-saved': line(`Report: ${event.html}`); break;
         case 'browser-stopped':
           if (!event.graceful) log('Browser required forced cleanup. See session.json for timings.');