Skip to content

[FEATURE] Pause a flow at its failing step so a person or an agent can fix it on the live device #246

Open

Description

Problem Statement

When a step fails, the run moves on (or ends) and leaves a screenshot and a hierarchy dump. Finding out why means changing the flow and running it again from the start: 40 to 150 s per attempt for the flows we run, plus whatever state the app had to be brought into. That loop is slow for a person, and slower for an AI agent fixing a flow, which needs many small probes ("is this id on screen now?", "does this tap do it?").

Proposed Solution

An opt-in debugger: with MAESTRO_RUNNER_DEBUG_ADDR=127.0.0.1:<port> set, a step that would fail the flow pauses the flow instead, with the device, the app and the flow's context (variables, output.*, copiedText) left exactly as they were, and serves:

GET  /debug/state       {"state": "running"|"paused", "flow", "step": {type, describe, label, error, depth}, "pausedAt"}
GET  /debug/screenshot  the screen now (paused only)
GET  /debug/hierarchy   the driver's hierarchy now (paused only)
POST /debug/exec        body: YAML steps, run in the paused flow's context; their results
POST /debug/retry       body: YAML steps to run instead of the failed step, or empty to run it again;
                        on success the flow continues, on failure it stays paused
POST /debug/skip        the step counts as passed and the flow continues
POST /debug/fail        the step fails as it would have without the debugger

It pauses only where a failure would fail the flow: not on an optional step, not on an attempt a retry: will repeat, and not on the runFlow / retry / repeat that merely passes on a failure inside it (the inner step paused already). The device is driven from the flow's own goroutine only; an HTTP request hands it a command and waits. Without the variable nothing changes.

We have this working in our fork (about 340 lines in pkg/executor/debugger.go and a few in flow_runner.go, with a test): https://github.com/luka-vladika-happening-xyz/maestro-runner/tree/simector

Use Case

A real one from 2026-10-21, on an emulator, in a run nobody had prepared for debugging: a forgot-password flow failed at assertVisible: {id: forgotPasswordSubmitButton} after an app update. Paused there, /debug/screenshot showed a new intermediate "account access" screen and /debug/hierarchy showed why the flow could not get past it (its two buttons share one resource id). One POST /debug/exec of - tapOn: "Forgot password?" (the button's text) got past it, an empty POST /debug/retry ran the failed assertion again, which passed, and the flow carried on to the end (46.5 s in all). The fix went into the flow file afterwards; finding it took seconds instead of several full reruns.

We drive it from a device tool (a dashboard with the live screen next to the paused step, and a CLI an agent uses), but curl is enough.

Example

MAESTRO_RUNNER_DEBUG_ADDR=127.0.0.1:7001 maestro-runner --device emulator-5554 test flows/forgot_password.yaml
# ⏸  Paused: assertVisible: id="forgotPasswordSubmitButton" — Element not visible ...
#      http://127.0.0.1:7001/debug/state  (retry, skip or fail it there)
curl -s 127.0.0.1:7001/debug/state
curl -s 127.0.0.1:7001/debug/screenshot > now.png
curl -s -X POST 127.0.0.1:7001/debug/exec --data-binary '- tapOn: "Forgot password?"'
curl -s -X POST 127.0.0.1:7001/debug/retry

Alternatives Considered

  • feat: port session-based REST API and JSON step protocol #230's session REST API drives a fresh session step by step, which is great for bindings, but it does not stop a running flow at its failure with the flow's context intact. The two could share the step-from-JSON/YAML parsing and the screenshot/hierarchy handlers; if you'd rather see the pause built on feat: port session-based REST API and JSON step protocol #230's server once it lands, that works for us.
  • Rerunning with extra takeScreenshot / evalScript steps: what everyone does today; slow, and the app state is gone.
  • An interactive terminal prompt on failure: doesn't work for CI-like harnesses or agents; HTTP does both.

Additional Context

Would you be interested in this upstream? If yes, I'll open a PR from the fork's commit, shaped however you prefer (env variable vs. a --debug-addr flag, route names, whether exec takes YAML or #230's JSON steps). If not, we'll keep it in our fork, no problem.

Activity

  1. luka-vladika-happening-xyz commented on Nov 6, 2026

    @luka-vladika-happening-xyz
    ContributorAuthor

    Thanks for building --pause-on-failure from this, and for the credit in 1.1.30.

    We tried it against how we use our pause, where a tool drives it instead of a person at a terminal. Four things keep us on our own patch for now:

    1. It can't be driven without a terminal. The flag is ignored when stdin is not a TTY, and the terminal pauser is the only FailurePauser the CLI wires up. A tool or an agent needs a machine interface. One option is an HTTP pauser on a loopback address behind a flag, answering the state, a screenshot, the hierarchy, steps to run, retry and continue as JSON.
    2. It pauses on the top-level step. Most of our flows call shared subflows (runFlow for login and navigation), so a failure pauses on the runFlow, and r runs the whole subflow again. Pausing on the failed step inside it, with its depth and file, lets a retry run just that step.
    3. There is no skip. Sometimes the screen gets fixed by hand, and the step should count as passed so the flow can go on.
    4. There is no retry with other steps. When the step itself is wrong (a changed id), running other steps in its place and going on lets you try a fix before editing the flow file.

    Would you take a PR that adds these on top of FailurePauser? It would add a skip answer, a retry with other steps, an option to pause inside compound steps, and an HTTP pauser behind a flag. Our version of the same idea is pkg/executor/debugger.go on our fork (about 350 lines plus tests). We would reshape it around your interface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions