Skip to content

fix: scrollUntilVisible with a timeout of its own looks until it passes on the mobile drivers, as in Maestro - #283

Merged
omnarayan merged 9 commits into
devicelab-dev:mainfrom
luka-vladika-happening-xyz:fix/scroll-until-visible-keeps-looking-until-timeout-2
Nov 19, 2026
Merged

omnarayan merged 9 commits into
devicelab-dev:mainfrom
luka-vladika-happening-xyz:fix/scroll-until-visible-keeps-looking-until-timeout-2

Conversation

@luka-vladika-happening-xyz

@luka-vladika-happening-xyz luka-vladika-happening-xyz commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

On uiautomator2, devicelab, appium, wda and both devicelab iOS drivers, a scrollUntilVisible that sets timeout: now keeps looking for its element after the screen stops moving and fails only when that timeout passes, as Maestro's does, so a list that fills late is found. A step without a timeout keeps the early stop of af473dd, the 20-scroll cap and its messages. On uiautomator2, devicelab and appium a step with its own timeout and no maxScrolls is no longer cut at the default 20 scrolls, as on the iOS drivers. A step whose own timeout passes fails with Maestro's "No visible element found: …". The scroll loops of uiautomator2, devicelab and wda now stop at their next round when the run is cancelled, as appium's does. The browser driver is unchanged. Proposed in #282, which asks which shape you prefer; this PR is its first shape, opened so the change can be read, and I am happy to reshape it.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change): a step with a timeout: whose element is not there, optional: true ones included, now takes that whole timeout instead of stopping at the end of the content
  • Documentation update
  • Refactoring (no functional changes)

Changes Made

  • pkg/core: ScrollLooksUntilTimeout(step) (the step set its own timeout) and ScrollTimeoutMessage, beside ScrollProgress; the step's two defaults, its 30 s timeout and the 20-scroll cap of uiautomator2, devicelab and appium, as the package variables ScrollUntilVisibleDefaultTimeout and ScrollUntilVisibleDefaultMaxScrolls, which tests shorten (the values are unchanged).
  • uiautomator2, devicelab, appium, wda, devicelab iOS and the legacy devicelab iOS driver: on a screen that stopped moving, the not-found failure is returned only for a step without its own timeout; Observe still runs, so an element already found (not yet centred, or partly past its container's edge) is still accepted at the end of the content, and devicelab iOS still answers a permission prompt over a list that looks stuck.
  • The same drivers: a step with its own timeout that runs out of it says "No visible element found: (scrolling until the ms timeout)"; the cause says "not found" everywhere but appium's partly visible case, which keeps its "never sufficiently visible" cause (devicelab iOS's cause said "not visible", so the report now files it as element_not_found). A step without its own timeout keeps its message and cause when its loop ends at the deadline or at maxScrolls (a cancel, below, is the one exit that does not depend on the step's timeout); one with its own timeout that ends at maxScrolls keeps its message.
  • uiautomator2, devicelab and appium (second commit): a step with its own timeout and no maxScrolls has no scroll cap, as on wda and devicelab iOS.
  • uiautomator2, devicelab and wda: the scroll loop checks the run's context at the start of each round, as appium's does, so a cancelled run (a runFlow timeout, a library caller's cancel) does not swipe on until the step's own timeout; a cancel during a round still lets that round's swipe go out. This is the one exit not gated on the step's own timeout: a step without one stops at the next round too, where it went on to the no-progress check, its cap or the default 30 s.
  • Behaviour change: a step with a timeout whose element is truly absent now takes that whole timeout, as on Maestro.

Related Issues

Fixes #282

Testing

  • Tests pass locally (go test -race ./... on this branch with go1.26.7, no device attached; left out here: pkg/driver/browser/cdp, whose tests hang launching Chrome on this machine, and TestCreateAndDeleteSimulator_Integration, which creates and deletes a real simulator)
  • Linting passes (golangci-lint v2.13.2: 0 issues)
  • Added tests for new functionality
  • Tested manually with sample flows (an Android emulator, Chrome, a list that fills 10 s late, against main and Maestro 2.12.1: the numbers are in scrollUntilVisible gives up when the screen stops moving, where Maestro scrolls until its timeout #282)

In each of the six drivers: …WithTimeoutWaitsForListThatFillsLate (the screen does not change for four scrolls, then the element is there; a 20 s timeout) passes after at least four scrolls, and …WithTimeoutRunsToItOnStillScreen (a screen that never changes) fails no sooner than its timeout, with more scrolls than the stop takes, in Maestro's words; both fail on main with the early stop's message. On uiautomator2, devicelab and appium, …WithTimeoutIsNotCutAtDefaultMaxScrolls sets the default cap to 2 and finds an element that comes on screen after the third scroll; with the cap put back for a step with its own timeout it fails with the cap's message. In each of the six, …WithoutTimeoutKeepsItsMessageAtTheDeadline (a step without a timeout that runs into the default timeout, set to 1 s for the test, on a list that keeps moving) gets the driver's own message and cause, exactly as on main. On uiautomator2, devicelab and wda, TestScrollUntilVisibleStopsWhenRunIsCancelled (a cancelled context, a still screen, a 5 s timeout) ends without a swipe, with the cancel as the cause. On uiautomator2, devicelab, wda and devicelab iOS, a centerElement step with a timeout still takes the last row at the end of the content, and on uiautomator2 and devicelab a step with a timeout still takes a row partly past its list's edge there. The no-progress tests (…StopsWhenScreenStopsMoving in uiautomator2, devicelab, appium and wda, TestScrollUntilVisibleStopsWhenNothingMoves in the legacy driver) gave their step a 60 s (legacy 20 s) timeout, under which the stop came after about 3.6 s; with this change such a step looks until its timeout, so they now give none and still check the stop. TestScrollUntilVisibleDefaultMaxScrolls (uiautomator2, devicelab) likewise gives none and still checks the cap; it sets the default timeout to a minute for itself, so its 20 scrolls (26 s here, as on main) stay the limit. With the defaults shortened in the tests, the PR adds about 70 s to the six driver packages' go test -race (470 s on main, 540 s on this branch, one package at a time).

Checklist

Additional Notes

Proposed CHANGELOG entry:

scrollUntilVisible with a timeout: of its own looks until that timeout, as in Maestro (#283). A list that fills late looks like a list at its end, so the step gave up after two or three scrolls; it now keeps scrolling and fails only when its timeout passes, with Maestro's "No visible element found". A step without a timeout still stops when the screen stops moving. On uiautomator2, devicelab and appium a step with a timeout is no longer cut at the default 20 scrolls. A cancelled run ends the scroll at its next round on every mobile driver.

…es, as in Maestro

A step that sets `timeout:` now keeps looking for its element after the
screen stops moving, swiping every round, and fails only when that timeout
passes. Maestro's scrollUntilVisible swipes until its timeout whatever the
screen does (Orchestra.scrollUntilVisible). A list that fills late reads to
ScrollProgress exactly like the end of the content (an empty page while its
rows load; on the drivers that compare the page source, a spinner's node
too, while wda's settled screenshot sees a turning spinner move), so the
step gave up after two or three scrolls where Maestro waited and found the
row.

A step without a timeout keeps the early stop, so a target that is not in
the list still does not cost the whole default timeout. The end-of-content
answers for an element already found (on screen but not yet centred, or
partly past its container's edge) are unchanged, and so is devicelab iOS's
answer to a permission prompt over a list that looks stuck.

A step whose timeout passes now fails in Maestro's words, "No visible
element found: <selector>", naming the timeout. Its cause says "not found"
in every driver but appium's partly visible case (an element found but
never sufficiently visible), so the report files it as element_not_found
(devicelab iOS's "not visible" was filed as an assertion) and that appium
case as timeout. One that runs out of maxScrolls keeps its message.

The no-progress tests of uiautomator2, devicelab, appium, wda and
devicelab_ios_legacy gave their step a 60 s (legacy 20 s) timeout; they now
give none, and still check the early stop.
…efault 20 scrolls

On uiautomator2, devicelab and appium a step that sets `timeout:` and no
`maxScrolls` is now bounded by its timeout alone, as the iOS drivers have
done since 6ef5fc7 and c9fc359 ("the timeout is the budget; a scroll count
caps it only when the flow sets one") and as Maestro does. The default 20
scrolls could end such a step before its timeout, before a row that came
late could show.

A step without a timeout keeps the cap of 20, and a flow's `maxScrolls`
still caps a step with one. TestScrollUntilVisibleDefaultMaxScrolls gave
its step a 60 s timeout; it now gives none (its 20 mock rounds take about
26 s, inside the default 30 s) and still checks the cap.
…e at the deadline

The after-loop branch that fails in Maestro's words when the deadline has
passed was not gated: a step with no timeout of its own that ran into the
default 30 s (a list that keeps moving, a maxScrolls it never reaches) got
"No visible element found: ... (scrolling down until the 30000ms timeout)"
where it had its driver's own message ("Element not found after 100
scrolls" on uiautomator2 and devicelab, "... not visible after scrolling
down" on devicelab iOS, whose cause also moved it from an assertion to
element_not_found, and appium's partly visible case from unknown to
timeout).

That branch now takes the same predicate as the early stop
(core.ScrollLooksUntilTimeout), so a step without a timeout of its own
keeps its message and its cause exactly when its loop ends at the
deadline or at maxScrolls, in all six drivers. Each driver has a test of
such a step at the deadline; it passes on main, fails before this change
and passes after it.
A step with a timeout of its own looks until that timeout on a still
screen, so a run cancelled meanwhile (a runFlow's timeout, a caller's
cancel) made the uiautomator2, devicelab and wda drivers swipe on until
the step's timeout: 17 swipes in 5 s for `timeout: 5000`. Their loops
now check the run's context at the start of each round, as the appium
driver's does, and end with the cancel as the cause. A cancel that comes
during a round still lets that round's swipe go out; the step ends before
the next one. The devicelab iOS drivers already stop there: devicelab iOS
checks the context and ends with "not visible after scrolling", and the
legacy driver's requests carry it, so its next one fails with the aborted
request as the cause.

This check is the one exit not gated on the step's own timeout: a step
without one stops at the next round too, where it went on to the
no-progress check, its cap or the default 30 s.
…ent at the end of the content

A step with its own timeout keeps looking after the screen stops moving
only while its element was never found: an element on screen that
centerElement cannot bring to the middle (the last row of a list) is
still taken there. A test of that in uiautomator2, devicelab, wda and
devicelab iOS, each asserting the end-of-content exit and its scroll
count; they fail when the timeout's gate is put before that answer.

The growingList comment said a partly visible match ends the find at
once; in these mocks every round spends the find's 1s budget and the
scroll's settle, about 1.35s.
… its list's edge at the end of the content

The end-of-content answer for a row hanging past its scroll container's
leading edge is reached when the find did not see the row while the list
still coasted and the captures after it show the list at rest, the row
hanging. A test of it with a timeout set in uiautomator2 and devicelab:
the mock answers each round's first read (the find's) with the row not
in yet for two rounds, and every other read with the list at rest. It
passes on main and fails when the timeout's gate is put before that
answer (the step then ends a scroll later, on another exit).
The comment beside the lifted 20-scroll cap said 20 scrolls ended a 30s
step at about 20s. No run measured that: a report's "after 20 scrolls"
prints maxScrolls whatever ended the loop, and a round's time varies with
the device and the screen. The comment now says only that the default
count could end a step before its own timeout, in uiautomator2, devicelab
and appium. No code change.
@codecov

codecov Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.36364% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pkg/driver/appium/commands.go 85.71% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@omnarayan

Copy link
Copy Markdown
Contributor

Thanks Luka, this is a clear and careful change. The rule (a step's own timeout is its budget; with no timeout the early stop stays) is the one we want, and the behaviour checks out on every driver, including the centred and partly-visible answers and the permission prompt on DeviceLab iOS.

One thing before merging: the new tests add about 5 minutes of real time across the driver packages (uiautomator2 alone goes from ~167 s to ~246 s). Most of it is the six …WithoutTimeoutKeepsItsMessageAtTheDeadline tests, which each wait out the built-in 30 s default, plus the 21-round …IsNotCutAtDefaultMaxScrolls. Could you make the 30 s default a package-level variable that tests can shorten, and cut the still-screen timeouts to about 1 s? The behaviour code can stay as it is.

Two small notes, no change needed unless you want to: on DeviceLab iOS, a step with a timeout on a still screen now checks for a system alert every round until the timeout; and the cap choice and the final "timed out" check could each become a small helper in pkg/core, next to ScrollLooksUntilTimeout, since the same few lines repeat in six drivers.

The tests added for a step's own timeout spent about 5 minutes of real
time across the driver packages: the six tests of a step without a
timeout that runs into the default waited its whole 30s, and the test
that the 20-scroll cap does not cut a step with a timeout of its own ran
21 rounds of about 1.3s each on uiautomator2 and devicelab.

Both defaults are now package variables in pkg/core,
ScrollUntilVisibleDefaultTimeout (30s) and
ScrollUntilVisibleDefaultMaxScrolls (20, the cap of uiautomator2,
devicelab and appium), which the six drivers read where they had the
literals; their values are unchanged. The deadline tests set the default
timeout to 1s, the cap tests the cap to 2 with the target after the
third scroll, each restored by t.Cleanup (no test in these packages runs
in parallel). The still-screen tests with a timeout of their own go from
5s to 3s on uiautomator2, devicelab and appium and to 3.5s on wda: a
round that finds nothing costs about 1.3s there (1s on wda), and the
test needs one round past the no-progress stop to tell the two apart.
The iOS drivers' were already at 1s.
TestScrollUntilVisibleDefaultMaxScrolls (uiautomator2, devicelab) runs
its 20 scrolls with no timeout of its own, so the default 30s timeout
was about 4s from ending it first (20 rounds of about 1.3s each). It now
sets the default timeout to a minute for itself, so a slower round
cannot turn a check of the cap into a check of the deadline. It still
expects 20 scrolls.
@luka-vladika-happening-xyz

Copy link
Copy Markdown
Contributor Author

Thanks! Two new commits on the branch; the behaviour code is unchanged.

  • ff6f148: both defaults are package variables in pkg/core, ScrollUntilVisibleDefaultTimeout (30 s) and ScrollUntilVisibleDefaultMaxScrolls (20, the cap on uiautomator2, devicelab and appium), which the six drivers read where the literals were; their values are unchanged. The six …WithoutTimeoutKeepsItsMessageAtTheDeadline tests set the default timeout to 1 s. The three …IsNotCutAtDefaultMaxScrolls tests set the cap to 2, with the element after the third scroll: their cost was 21 rounds of about 1.3 s each (the find's 1 s budget for a missing element and the 300 ms settle), which a shorter timeout does not touch. Each test restores the value with t.Cleanup; nothing in these packages calls t.Parallel.
  • The still-screen tests with a timeout of their own (…WithTimeoutRunsToItOnStillScreen) go from 5 s to 3 s on uiautomator2, devicelab and appium and to 3.5 s on wda, rather than 1 s: a round that finds nothing costs about 1.3 s there (1 s on wda), and the test has to run one round past the no-progress stop to tell the two apart. At 1 s they run a single round. The two iOS drivers' were already at 1 s.
  • f7ec738: TestScrollUntilVisibleDefaultMaxScrolls (uiautomator2, devicelab) runs its 20 scrolls with no timeout of its own since this PR, and they ended about 4 s inside the 30 s default. It now sets the default timeout to a minute for itself, so a slow round on CI can't turn the cap check into a deadline check. It still takes its 26 s, as on main.

Times with go test -race -count=1, one package at a time on this machine:

Package main before (691fd01) now (f7ec738)
uiautomator2 168 s 246 s 191 s
devicelab 119 s 197 s 142 s
devicelab_ios 25 s 56 s 27 s
devicelab_ios_legacy 11 s 42 s 13 s
appium 31 s 78 s 42 s
wda 117 s 157 s 125 s

Across the six driver packages the PR now adds about 70 s instead of about 5 minutes (470 s on main, 540 s now). What is left is the new tests' own rounds (the list that fills late, the end-of-content cases, the still screen) at each driver's normal pace.

To check that the shorter tests still catch what they are for, I ran three mutants on the branch: with the after-loop gate taken out, all six deadline tests fail ("No visible element found: … until the 1000ms timeout"); with the 20-scroll cap put back for a step with its own timeout, the three cap tests fail ("Element not found after 2 scrolls", on appium "element found but never sufficiently visible after scrolling"); with the no-progress stop ungated, all six still-screen tests fail. golangci-lint v2.13.2: 0 issues.

On the DeviceLab iOS alert check: yes. Once the list reads as stuck, each further round on a still screen sends one alert read before its next scroll, and only when the auto-answer is on (launchApp's permissions). We think that is right as it is: it is one agent call next to the settle and the lookup the round already makes, and a permission prompt that comes up while the step waits on a still list gets answered, as one that came up before the stop did.

On the helpers: agreed, the cap choice and the final timed-out check are the same few lines in six drivers. I'd rather send them as a small follow-up PR once this one is in, so this one stays what you checked, but I can fold them in here if you prefer.

@omnarayan

Copy link
Copy Markdown
Contributor

Thanks Luka. The behaviour code is unchanged, and the added test time is now about 73 s in total instead of about 5 minutes; -race is clean. Merging. A follow-up moving the cap and timed-out checks into a helper in pkg/core is welcome. We'll add the grace period for steps without a timeout: on top, and I'll ping you for your three emulator cases when it's up.

@omnarayan
omnarayan merged commit 3b39d3a into devicelab-dev:main Nov 19, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

scrollUntilVisible gives up when the screen stops moving, where Maestro scrolls until its timeout

3 participants