Skip to content

Provision an isolated browser for the nightly reliability probe - #58

Merged
wolfiesch merged 9 commits into
mainfrom
fix/nightly-ready
Sep 25, 2026
Merged

wolfiesch merged 9 commits into
mainfrom
fix/nightly-ready

Conversation

@wolfiesch

Copy link
Copy Markdown
Owner

The nightly background-tab reliability probe assumed an AgentTab extension and native host were already installed in the runner's browser. They never were after the v2 cutover, so every nightly since then failed.

The probe can now launch its own Chrome for Testing with --launch-chrome:

  • starts it in the background through LaunchServices with the freshly built development extension
  • writes a native messaging manifest for the freshly built host into a persistent provisioned profile
  • keeps host state in a temporary directory and quits the browser afterwards
  • reads the tab inventory from the AgentTab service worker's chrome.tabs.query over a loopback DevTools endpoint, because the runner cannot answer the macOS Automation consent that Apple events need

Without --launch-chrome, the probe inspects an already running Chrome through Apple events as before. Runner provisioning is documented in docs/verification.md.

Verification: a dispatched nightly on this branch passed (31 samples, no focus violations, clean teardown).

The nightly probe assumed an AgentTab extension and native host were
already provisioned in the runner's Chrome for Testing profile. The
runner was never provisioned for v2, so every nightly since the v2
cutover failed with a missing IPC socket.

The probe can now launch Chrome with a disposable profile that loads
the freshly built development extension and a native messaging
manifest for the freshly built host, then address that instance by
process ID and remove it afterwards.
A browser executed directly by the runner is not registered with the
login session and never answers Apple events, so start it with open(1)
and find its process by profile. Enabling automation is a user-gesture
grant, so the probe now uses a persistent provisioned profile and keeps
only host state in a temporary directory.
The runner cannot address a LaunchServices application by process ID,
so address the Chrome for Testing bundle and refuse to launch when
another instance would make that address ambiguous.
JXA reports a Chrome instance started with open -n as not running even
though its windows are scriptable, so the launched probe skips that guard.
The runner cannot answer the macOS Automation consent that Apple events
need, so the launched browser exposes a loopback DevTools endpoint and the
probe reads chrome.tabs.query from the AgentTab service worker. Inspecting
an already running Chrome still uses Apple events.
@wolfiesch
wolfiesch merged commit 788e8e9 into main Sep 25, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant