Repository navigation
fix(wda): leave a prompt only to the step aimed at it, as DeviceLab's driver does - #265
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
I pushed a second commit after running this on simulators and five real iPhones. Answering a prompt only when a lookup misses came too late, in two ways:
The commit leaves a prompt only to the step aimed at it, as DeviceLab's driver does ( On five real iPhones (iOS 18.3, 18.4, 26.2, 27.0 and 27.2), 20 flows of our suite, three rounds alternating with a build that has the first commit only (the build with this commit also had the settle fix of #269):
On simulators with |
…hers answered (#265) Since a flow that taps a prompt's button turns WDA's alert monitor off for the whole session (so WDA cannot beat the flow to it), a prompt the flow never meant to answer stayed up over the app: a flow that taps "Ask App Not to Track" left the notification prompt open, and every later lookup failed. A lookup that finds nothing now reads the open alert and, when none of its buttons is one the flow's taps name and the flow has no acceptAlert or dismissAlert, answers it as WDA's monitor would have (launchApp's permissions: the "Allow…" button, or "Don't Allow…" / "Ask App Not…"), then looks again. The prompts the flow names are still left to it. Refs #264
… driver does When the flow answers some permission prompts itself (WDA's monitor off), a prompt is now left only to the step aimed at it (a tap on one of its buttons, a check of its text), not to every step of a flow that names one of its buttons somewhere, and it is asked for by a tap after its find, in a watch after launchApp (15 s) and after an answered prompt (3 s), as well as by a lookup that misses (at most once a second). A tap under the notification prompt was lost, since its element was found in the app beneath the prompt, and the tracking prompt held the onboarding's 10 s wait for the screen beneath it until the flow's retry tapped it. Asked before the find, the check could hold WebDriverAgent for 19 s while the app was still launching.
79f1e8a to
779c67b
Compare
|
Rebased onto |
This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [maestro-runner](https://open.devicelab.dev/maestro-runner) ([source](https://github.com/devicelab-dev/maestro-runner)) | [`1.1.29` → `1.1.31`](https://renovatebot.com/diffs/npm/maestro-runner/1.1.29/1.1.31) |  |  | --- >⚠️ **Warning** > > Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/2097) for more information. --- ### Release Notes <details> <summary>devicelab-dev/maestro-runner (maestro-runner)</summary> ### [`v1.1.31`](https://github.com/devicelab-dev/maestro-runner/blob/HEAD/CHANGELOG.md#1131---2026-11-11) [Compare Source](devicelab-dev/maestro-runner@v1.1.29...v1.1.31) A fix release for 1.1.30, whose GitHub release was withdrawn. Everything listed under 1.1.30 below ships in 1.1.31. On top of it: WDA answers the permission prompts a flow does not tap (a 1.1.30 regression), taps after a check wait for the screen to settle, WDA page-source lookups on real iPhones are much faster, and a few smaller fixes. ##### Changed **WDA: page-source lookups are much faster on real iPhones ([#​266](devicelab-dev/maestro-runner#266 A lookup that falls back to the page source (a selector not on screen yet, an optional step, `when: visible`, relative selectors, `index:`, `copyTextFrom`) now asks for it without the `visible` and `accessible` attributes, which WebDriverAgent computes for every element with an accessibility call each and the matcher never reads: on a heavy screen a page source went from about 25 s to 2.4 s. The report and the `hierarchy` command still get every attribute. **WDA: `--wait-for-idle-timeout` above 0 no longer sends settings WebDriverAgent ignores ([#​267](devicelab-dev/maestro-runner#267 The value never changed anything on WDA: WebDriverAgent reads `shouldWaitForQuiescence` only when a session starts, takes `waitForIdleTimeout` in seconds, and every `launchApp` set it back to 0. On WDA only `0` has an effect, turning off the settle before each action, and the flag's help now says so. ##### Fixed **WDA: a flow that taps one permission prompt's button still gets the other prompts answered ([#​264](devicelab-dev/maestro-runner#264), [#​265](devicelab-dev/maestro-runner#265 Since 1.1.30, a flow that taps a prompt's button itself (`tapOn: "Ask App Not to Track"`) turned off WDA's automatic answer for the whole session, so a later prompt the flow never meant to answer (notifications) stayed up over the app and every later step failed behind it. A lookup that finds nothing now answers such a prompt as WDA would have (`Allow…`, or `Don't Allow…` with `permissions: {all: deny}`) and looks again. The prompts the flow taps are still left to it, and a flow with `acceptAlert` or `dismissAlert` is unchanged. **WDA: `kAXErrorServerNotFound` fails the step at once and says what it is ([#​268](devicelab-dev/maestro-runner#268 On real iPhones, after a `clearState` reinstall, the app sometimes runs without its accessibility server, and every page source fails with this error. A lookup polled to its timeout, each poll a page source of 8 to 10 s, and failed with "not visible within 30s", which read like a slow screen. The condition does not clear within a flow, so the step now fails at the first such read with "the app's accessibility server is not running (kAXErrorServerNotFound): WDA can reach the device but not the app; a relaunch does not help, restarting the device does". **WDA: a tap after a check waits for the screen to settle ([#​269](devicelab-dev/maestro-runner#269 The driver settles before an action when the step before it may have moved the screen, but any other step cleared that, so a check or a script between two taps made the second tap skip it. When a tap opened a sheet, the check passed while the sheet was still sliding up (WebDriverAgent reports its final frame), and the next tap on the sheet went out mid-slide and was lost. A step that neither acts nor moves the screen (an assert, a script, a condition) now leaves it as it was. **WDA: `eraseText` on a simulator no longer hides the keyboard for the rest of the run ([#​270](devicelab-dev/maestro-runner#270 WebDriverAgent cleared a field with a hardware keyboard's Clear key. On a simulator iOS then took a hardware keyboard as attached and kept the software keyboard off screen whenever a field had focus, so anything an app shows with the keyboard never appeared. On a simulator the field is now cleared with typed backspaces, as Maestro does; a real iPhone keeps the Clear key. **DeviceLab iOS: `launchApp` with `stopApp: false` keeps the running app ([#​261](devicelab-dev/maestro-runner#261 The launch applied the default permissions first, and iOS kills an app whose permissions change, so the app restarted and lost its state. With `stopApp: false`, permissions and `clearKeychain` are no longer applied (a log line says so when the flow gives them). **DeviceLab iOS: `hideKeyboard` no longer swipes on the keyboard while it is still sliding in.** The iOS agent took the screen size from the keyboard's frame. Right after a field took focus, that frame still lay below the screen, so the dismiss swipes started at the centre of a screen twice as tall: on the keys, where they dismiss nothing. Since 1.1.30 fails the step when the keyboard stays up ([#​259](devicelab-dev/maestro-runner#259)), flows that passed in 1.1.29 failed there (React Native's RNTester `text.yml` on iOS 27, 4 of 6 runs). The agent now reads the screen size from the app, waits for the keyboard to stop moving, and never starts the swipes below the keyboard. ##### Contributors [@​luka-vladika-happening-xyz](https://github.com/luka-vladika-happening-xyz) 1. WDA answers the permission prompts a flow does not tap ([#​264](devicelab-dev/maestro-runner#264), [#​265](devicelab-dev/maestro-runner#265)) 2. WDA lookups read the page source without `visible` and `accessible` ([#​266](devicelab-dev/maestro-runner#266)) 3. WDA settles before a tap that follows a check ([#​269](devicelab-dev/maestro-runner#269)) 4. WDA `eraseText` on a simulator keeps the keyboard on screen ([#​270](devicelab-dev/maestro-runner#270)) 5. Reported `--wait-for-idle-timeout` doing nothing on WDA ([#​267](devicelab-dev/maestro-runner#267)) and `kAXErrorServerNotFound` ([#​268](devicelab-dev/maestro-runner#268)) [@​benalbrecht](https://github.com/benalbrecht) 1. DeviceLab iOS `launchApp` with `stopApp: false` keeps the running app ([#​261](devicelab-dev/maestro-runner#261)) [@​emil-doychev-bijenkorf](https://github.com/emil-doychev-bijenkorf) 1. Reported `copyTextFrom` ignoring a `childOf` anchor's index on 1.1.29 ([#​262](devicelab-dev/maestro-runner#262)), fixed by [#​241](devicelab-dev/maestro-runner#241) in 1.1.30 </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "before 9am on tuesday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4yMjcuNCIsInVwZGF0ZWRJblZlciI6IjQ0LjIyNy40IiwidGFyZ2V0QnJhbmNoIjoiZGV2ZWxvcCIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZSJdfQ==--> Reviewed-on: https://code.futo.org/harbor/harbor/pulls/2341
Follows #264, fixed in 1.1.31 by
d75c50d(this PR's first commit). Rebased ontomain, so this is the second commit alone.With 1.1.31 a lookup that finds nothing answers a prompt the flow does not tap, so the notification prompt no longer stays up for the whole flow. It still answers late: a tap that meets the prompt is lost under it, and the flow then waits out a lookup for what the lost tap should have opened (about 20 s on an onboarding).
The change: when the flow turned WDA's alert monitor off, the driver answers prompts as DeviceLab's driver does: a prompt is left only to the step aimed at it, and answered before any other step acts, as the monitor would have.
stepAimsAtAlert): a tap, double tap or long press on one of its buttons (pickAlertButton, as a tap would pick it), a check of one of its buttons or of its text (assertVisible,assertNotVisible,copyTextFrom,extendedWaitUntil). A flow withacceptAlert/dismissAlertkeeps every prompt to itself, as today.answerUnnamedPrompt): the button the monitor'sautoClickAlertSelectorwould have tapped (Allow…; orDon't Allow…/Ask App Not…forall: deny), checked closed (answerAlert), one log line.launchApp, 3 s after an answered prompt, at most every 0.5 s), answers, and finds its element again. A lookup that finds nothing asks at most once a second. Asking after the find matters: an alert check sent while the app was still launching held WebDriverAgent for 19 s on a busy simulator, and a find answers only once the app does.What stays: a flow with no prompt tap keeps WDA's monitor and pays nothing. The common "if the prompt shows, tap Don't Allow" (
runFlowwithwhen: visible) still answers the prompt itself when it runs while the prompt is up: its condition finds the prompt's button, and its tap is aimed at it. A tap on the app that meets the prompt first gets it answered as the permissions say, as with DeviceLab's driver.Cost: only in flows that tap a prompt's button: one
/alert/text(60-80 ms on a light page, about 0.3 s on a heavy one), plus/wda/alert/buttonswhen an alert is up, at the taps in the watch and on lookups that miss.Tests:
TestLookupLeavesThePromptToTheStepAimedAtIt,TestLookupAnswersAPromptTheFlowAnswersElsewhere,TestTapAnswersAnUnnamedPromptFirst,TestTapAsksForNoAlertWhenWDAAnswers,TestTapAsksForPromptsOnlyInTheWatch.go test -race ./pkg/driver/wda/ ./pkg/executor/ ./pkg/flow/passes; golangci-lint v2.13.2: 0 issues.On devices: five real iPhones (iOS 18.3 to 27.2), 20 flows of our suite that tap the tracking prompt's button themselves and leave the notification prompt to the runner: no prompt stayed up in 300 flow runs, and the three onboarding flows took 24 s at the median, against 44 s with the first commit (now
d75c50d) alone. On simulators (--driver wda, iOS 27, six at once) the onboarding flows take 16-17 s, against 34-46 s before and 14-15 s with DeviceLab's driver.