Repository navigation
fix: scrollUntilVisible with a timeout of its own looks until it passes on the mobile drivers, as in Maestro - #283
Conversation
…es, as in Maestro A step that sets `timeout:` now keeps looking for its element after the screen stops moving, swiping every round, and fails only when that timeout passes. Maestro's scrollUntilVisible swipes until its timeout whatever the screen does (Orchestra.scrollUntilVisible). A list that fills late reads to ScrollProgress exactly like the end of the content (an empty page while its rows load; on the drivers that compare the page source, a spinner's node too, while wda's settled screenshot sees a turning spinner move), so the step gave up after two or three scrolls where Maestro waited and found the row. A step without a timeout keeps the early stop, so a target that is not in the list still does not cost the whole default timeout. The end-of-content answers for an element already found (on screen but not yet centred, or partly past its container's edge) are unchanged, and so is devicelab iOS's answer to a permission prompt over a list that looks stuck. A step whose timeout passes now fails in Maestro's words, "No visible element found: <selector>", naming the timeout. Its cause says "not found" in every driver but appium's partly visible case (an element found but never sufficiently visible), so the report files it as element_not_found (devicelab iOS's "not visible" was filed as an assertion) and that appium case as timeout. One that runs out of maxScrolls keeps its message. The no-progress tests of uiautomator2, devicelab, appium, wda and devicelab_ios_legacy gave their step a 60 s (legacy 20 s) timeout; they now give none, and still check the early stop.
…efault 20 scrolls On uiautomator2, devicelab and appium a step that sets `timeout:` and no `maxScrolls` is now bounded by its timeout alone, as the iOS drivers have done since 6ef5fc7 and c9fc359 ("the timeout is the budget; a scroll count caps it only when the flow sets one") and as Maestro does. The default 20 scrolls could end such a step before its timeout, before a row that came late could show. A step without a timeout keeps the cap of 20, and a flow's `maxScrolls` still caps a step with one. TestScrollUntilVisibleDefaultMaxScrolls gave its step a 60 s timeout; it now gives none (its 20 mock rounds take about 26 s, inside the default 30 s) and still checks the cap.
…e at the deadline
The after-loop branch that fails in Maestro's words when the deadline has
passed was not gated: a step with no timeout of its own that ran into the
default 30 s (a list that keeps moving, a maxScrolls it never reaches) got
"No visible element found: ... (scrolling down until the 30000ms timeout)"
where it had its driver's own message ("Element not found after 100
scrolls" on uiautomator2 and devicelab, "... not visible after scrolling
down" on devicelab iOS, whose cause also moved it from an assertion to
element_not_found, and appium's partly visible case from unknown to
timeout).
That branch now takes the same predicate as the early stop
(core.ScrollLooksUntilTimeout), so a step without a timeout of its own
keeps its message and its cause exactly when its loop ends at the
deadline or at maxScrolls, in all six drivers. Each driver has a test of
such a step at the deadline; it passes on main, fails before this change
and passes after it.
A step with a timeout of its own looks until that timeout on a still screen, so a run cancelled meanwhile (a runFlow's timeout, a caller's cancel) made the uiautomator2, devicelab and wda drivers swipe on until the step's timeout: 17 swipes in 5 s for `timeout: 5000`. Their loops now check the run's context at the start of each round, as the appium driver's does, and end with the cancel as the cause. A cancel that comes during a round still lets that round's swipe go out; the step ends before the next one. The devicelab iOS drivers already stop there: devicelab iOS checks the context and ends with "not visible after scrolling", and the legacy driver's requests carry it, so its next one fails with the aborted request as the cause. This check is the one exit not gated on the step's own timeout: a step without one stops at the next round too, where it went on to the no-progress check, its cap or the default 30 s.
…ent at the end of the content A step with its own timeout keeps looking after the screen stops moving only while its element was never found: an element on screen that centerElement cannot bring to the middle (the last row of a list) is still taken there. A test of that in uiautomator2, devicelab, wda and devicelab iOS, each asserting the end-of-content exit and its scroll count; they fail when the timeout's gate is put before that answer. The growingList comment said a partly visible match ends the find at once; in these mocks every round spends the find's 1s budget and the scroll's settle, about 1.35s.
… its list's edge at the end of the content The end-of-content answer for a row hanging past its scroll container's leading edge is reached when the find did not see the row while the list still coasted and the captures after it show the list at rest, the row hanging. A test of it with a timeout set in uiautomator2 and devicelab: the mock answers each round's first read (the find's) with the row not in yet for two rounds, and every other read with the list at rest. It passes on main and fails when the timeout's gate is put before that answer (the step then ends a scroll later, on another exit).
The comment beside the lifted 20-scroll cap said 20 scrolls ended a 30s step at about 20s. No run measured that: a report's "after 20 scrolls" prints maxScrolls whatever ended the loop, and a round's time varies with the device and the screen. The comment now says only that the default count could end a step before its own timeout, in uiautomator2, devicelab and appium. No code change.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Thanks Luka, this is a clear and careful change. The rule (a step's own timeout is its budget; with no timeout the early stop stays) is the one we want, and the behaviour checks out on every driver, including the centred and partly-visible answers and the permission prompt on DeviceLab iOS. One thing before merging: the new tests add about 5 minutes of real time across the driver packages (uiautomator2 alone goes from ~167 s to ~246 s). Most of it is the six Two small notes, no change needed unless you want to: on DeviceLab iOS, a step with a timeout on a still screen now checks for a system alert every round until the timeout; and the cap choice and the final "timed out" check could each become a small helper in |
The tests added for a step's own timeout spent about 5 minutes of real time across the driver packages: the six tests of a step without a timeout that runs into the default waited its whole 30s, and the test that the 20-scroll cap does not cut a step with a timeout of its own ran 21 rounds of about 1.3s each on uiautomator2 and devicelab. Both defaults are now package variables in pkg/core, ScrollUntilVisibleDefaultTimeout (30s) and ScrollUntilVisibleDefaultMaxScrolls (20, the cap of uiautomator2, devicelab and appium), which the six drivers read where they had the literals; their values are unchanged. The deadline tests set the default timeout to 1s, the cap tests the cap to 2 with the target after the third scroll, each restored by t.Cleanup (no test in these packages runs in parallel). The still-screen tests with a timeout of their own go from 5s to 3s on uiautomator2, devicelab and appium and to 3.5s on wda: a round that finds nothing costs about 1.3s there (1s on wda), and the test needs one round past the no-progress stop to tell the two apart. The iOS drivers' were already at 1s.
TestScrollUntilVisibleDefaultMaxScrolls (uiautomator2, devicelab) runs its 20 scrolls with no timeout of its own, so the default 30s timeout was about 4s from ending it first (20 rounds of about 1.3s each). It now sets the default timeout to a minute for itself, so a slower round cannot turn a check of the cap into a check of the deadline. It still expects 20 scrolls.
|
Thanks! Two new commits on the branch; the behaviour code is unchanged.
Times with
Across the six driver packages the PR now adds about 70 s instead of about 5 minutes (470 s on To check that the shorter tests still catch what they are for, I ran three mutants on the branch: with the after-loop gate taken out, all six deadline tests fail ("No visible element found: … until the 1000ms timeout"); with the 20-scroll cap put back for a step with its own timeout, the three cap tests fail ("Element not found after 2 scrolls", on appium "element found but never sufficiently visible after scrolling"); with the no-progress stop ungated, all six still-screen tests fail. golangci-lint v2.13.2: 0 issues. On the DeviceLab iOS alert check: yes. Once the list reads as stuck, each further round on a still screen sends one On the helpers: agreed, the cap choice and the final timed-out check are the same few lines in six drivers. I'd rather send them as a small follow-up PR once this one is in, so this one stays what you checked, but I can fold them in here if you prefer. |
|
Thanks Luka. The behaviour code is unchanged, and the added test time is now about 73 s in total instead of about 5 minutes; |
Summary
On uiautomator2, devicelab, appium, wda and both devicelab iOS drivers, a
scrollUntilVisiblethat setstimeout:now keeps looking for its element after the screen stops moving and fails only when that timeout passes, as Maestro's does, so a list that fills late is found. A step without a timeout keeps the early stop of af473dd, the 20-scroll cap and its messages. On uiautomator2, devicelab and appium a step with its own timeout and nomaxScrollsis no longer cut at the default 20 scrolls, as on the iOS drivers. A step whose own timeout passes fails with Maestro's "No visible element found: …". The scroll loops of uiautomator2, devicelab and wda now stop at their next round when the run is cancelled, as appium's does. The browser driver is unchanged. Proposed in #282, which asks which shape you prefer; this PR is its first shape, opened so the change can be read, and I am happy to reshape it.Type of Change
timeout:whose element is not there,optional: trueones included, now takes that whole timeout instead of stopping at the end of the contentChanges Made
pkg/core:ScrollLooksUntilTimeout(step)(the step set its own timeout) andScrollTimeoutMessage, besideScrollProgress; the step's two defaults, its 30 s timeout and the 20-scroll cap of uiautomator2, devicelab and appium, as the package variablesScrollUntilVisibleDefaultTimeoutandScrollUntilVisibleDefaultMaxScrolls, which tests shorten (the values are unchanged).Observestill runs, so an element already found (not yet centred, or partly past its container's edge) is still accepted at the end of the content, and devicelab iOS still answers a permission prompt over a list that looks stuck.maxScrolls(a cancel, below, is the one exit that does not depend on the step's timeout); one with its own timeout that ends atmaxScrollskeeps its message.maxScrollshas no scroll cap, as on wda and devicelab iOS.runFlowtimeout, a library caller's cancel) does not swipe on until the step's own timeout; a cancel during a round still lets that round's swipe go out. This is the one exit not gated on the step's own timeout: a step without one stops at the next round too, where it went on to the no-progress check, its cap or the default 30 s.Related Issues
Fixes #282
Testing
go test -race ./...on this branch with go1.26.7, no device attached; left out here:pkg/driver/browser/cdp, whose tests hang launching Chrome on this machine, andTestCreateAndDeleteSimulator_Integration, which creates and deletes a real simulator)mainand Maestro 2.12.1: the numbers are in scrollUntilVisible gives up when the screen stops moving, where Maestro scrolls until its timeout #282)In each of the six drivers:
…WithTimeoutWaitsForListThatFillsLate(the screen does not change for four scrolls, then the element is there; a 20 s timeout) passes after at least four scrolls, and…WithTimeoutRunsToItOnStillScreen(a screen that never changes) fails no sooner than its timeout, with more scrolls than the stop takes, in Maestro's words; both fail onmainwith the early stop's message. On uiautomator2, devicelab and appium,…WithTimeoutIsNotCutAtDefaultMaxScrollssets the default cap to 2 and finds an element that comes on screen after the third scroll; with the cap put back for a step with its own timeout it fails with the cap's message. In each of the six,…WithoutTimeoutKeepsItsMessageAtTheDeadline(a step without a timeout that runs into the default timeout, set to 1 s for the test, on a list that keeps moving) gets the driver's own message and cause, exactly as onmain. On uiautomator2, devicelab and wda,TestScrollUntilVisibleStopsWhenRunIsCancelled(a cancelled context, a still screen, a 5 s timeout) ends without a swipe, with the cancel as the cause. On uiautomator2, devicelab, wda and devicelab iOS, acenterElementstep with a timeout still takes the last row at the end of the content, and on uiautomator2 and devicelab a step with a timeout still takes a row partly past its list's edge there. The no-progress tests (…StopsWhenScreenStopsMovingin uiautomator2, devicelab, appium and wda,TestScrollUntilVisibleStopsWhenNothingMovesin the legacy driver) gave their step a 60 s (legacy 20 s) timeout, under which the stop came after about 3.6 s; with this change such a step looks until its timeout, so they now give none and still check the stop.TestScrollUntilVisibleDefaultMaxScrolls(uiautomator2, devicelab) likewise gives none and still checks the cap; it sets the default timeout to a minute for itself, so its 20 scrolls (26 s here, as onmain) stay the limit. With the defaults shortened in the tests, the PR adds about 70 s to the six driver packages'go test -race(470 s onmain, 540 s on this branch, one package at a time).Checklist
Additional Notes
Proposed CHANGELOG entry: