Skip to content

✨ Repeat browser workflows with reproducible seeds - #33

Merged
Robdel12 merged 3 commits into
mainfrom
rd/repeat-runs
Nov 12, 2026
Merged

Robdel12 merged 3 commits into
mainfrom
rd/repeat-runs

Conversation

@Robdel12

@Robdel12 Robdel12 commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Acceptance tests can pass once while depending on test order or state left by earlier workflows. dyno --suite Checkout --repeat 100 --random exercises complete selections in one reused managed session, preserves every observed failure, and prints commands to replay a failing round or rerun its prefix.

Each round remains an ordinary Dyno run. Final documents park before project draining, advancement waits for cleanup and persistence, and each round has its own deadline. Backend reset stays project-owned, and Interactor owns convergence. Seeds reproduce priority; schedules reproduce worker queues, not asynchronous interleaving. UI, manual and watch combinations are rejected explicitly.

Batch HTML, JSON, JUnit and schedules retain failures independently of live history. Successful intermediate screenshots are discarded; failed, interrupted and final rounds keep portable reports within a 256 MiB evidence budget. Replay commands retain config, browser and report overrides. Storage errors preserve accepted outcomes when the remaining output paths are writable, and terminal metadata is published after the final artifacts. Reporter reruns cannot bypass automatic batch ownership.

The complete local framework suite passed with 262 passes and two skips. Real CLI coverage includes resource reuse, seeded replay, storage failures, cleanup timeouts, all-skipped selections, signals and offline reports. Focused Chromium and WebKit checks passed, as did the docs build and seven docs tests. Repeat execution has a separate CI suite so all browser engines stay within the existing job limit. Local Firefox could not open its installed profile; the CI matrix verifies Firefox independently.

Consumer archive checks passed Vizzly’s 355 workflows, BackChannel’s 31 desktop/mobile workflows, and the docs app’s 40 workflows. Installation guidance also calls out pnpm 11’s Git dependency policy, which must be an explicit project choice while the required Interactor fixes remain unreleased.

Exercise complete selections within one managed session, retiring final documents before project draining and advancing only after persistence. Keep failures visible across rounds and save bounded replay evidence, aggregate reports, and source-generated guides.
Preserve explicit execution options in replay commands, retain accepted outcomes through artifact write failures, and publish terminal batch metadata last. Keep automatic batch replacement under one owner and give repeated browser workflows their own CI budget without dropping engine coverage.
Document the source and build policies needed by the pinned Interactor fixes so projects can make that installation choice explicitly.
@Robdel12
Robdel12 merged commit ae58510 into main Nov 12, 2026
46 of 48 checks passed
@Robdel12
Robdel12 deleted the rd/repeat-runs branch November 12, 2026 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant