Skip to content

fix(flow): a timeout or a scroll speed may be a ${...} expression, as in Maestro - #279

Merged
omnarayan merged 1 commit into
devicelab-dev:mainfrom
luka-vladika-happening-xyz:fix/variable-in-number-field
Nov 18, 2026
Merged

omnarayan merged 1 commit into
devicelab-dev:mainfrom
luka-vladika-happening-xyz:fix/variable-in-number-field

Conversation

@luka-vladika-happening-xyz

@luka-vladika-happening-xyz luka-vladika-happening-xyz commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A flow that sets extendedWaitUntil's timeout from a variable fails validation, before any device is touched:

appId: com.example
env:
  PAUSE_MS: 3000
---
- launchApp
- extendedWaitUntil:
    visible: "Nothing"
    timeout: ${PAUSE_MS}
Validation errors:
  - flow.yaml: parse error: flow.yaml:3: yaml: unmarshal errors:
  line 4: cannot unmarshal !!str `${PAUSE...` into int
Error: validation failed with 1 error(s)

The validation covers every flow of the run, so none of them starts, including the ones without a variable. The same happens with waitForAnimationToEnd's timeout, scrollUntilVisible's timeout and speed, and with those numbers quoted (timeout: "3000").

Maestro's YAML model reads these four as strings (YamlExtendedWaitUntil.timeout, YamlWaitForAnimationToEndCommand.timeout, YamlScrollUntilVisible.timeout and .speed). Each command evaluates them with its other scripts when it runs (evaluateScripts of AssertConditionCommand, WaitForAnimationToEndCommand and ScrollUntilVisibleCommand in Commands.kt), then reads the number (parseTimeoutMs, NumericFields.parseScrollSpeed). Maestro 2.12.1's check-syntax accepts ${N} in each of them, and in the other numbers it reads as strings (repeat.times, retry.maxRetries, assertScreenshot.thresholdPercentage, a selector's index), which this runner already takes. It refuses ${N} where its model types the number as Int, Long, Double or Boolean (tapOn's repeat, delay and waitToSettleTimeoutMs, a selector's width, scrollUntilVisible's visibilityPercentage, swipe's duration, travel's speed, optional, launchApp's clearState), as this runner does.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)

Changes Made

  • The parser takes a ${...} value of those four fields out of the step's mapping before decoding and keeps it as written (BaseStep.TimeoutExpr, ScrollUntilVisibleStep.SpeedExpr). A literal that is not a number is still refused at parse time.
  • A number in those fields, quoted or not, is read as Maestro reads it: in decimal, so timeout: 0100 is 100 where YAML reads the unquoted 0100 as octal 64 (and timeout: "0100", quoted, was refused before this change, as any quoted number was), and a timeout's underscores dropped, as Maestro's extendedWaitUntil drops them (parseTimeoutMs). Maestro's scrollUntilVisible and waitForAnimationToEnd fail on an underscore when the step runs, so for those two this is more lenient.
  • ExpandStep resolves the expression into TimeoutMs or Speed each time the step runs, in the copy that expansion makes, so a sub-flow run twice with different env:, or a step in a repeat, gets the values of that moment.
  • A value that is not a number fails the step without running it, naming what it resolved to. As in Maestro (Orchestra.kt, the command loop), an optional extendedWaitUntil or scrollUntilVisible lets the flow go on, since their timeout error is an InvalidCommand, which optional downgrades: Maestro reports the step as a warning, the runner as failed, as for any optional step that fails. A bad scroll speed (InvalidNumericFieldValue) or waitForAnimationToEnd timeout (a NumberFormatException) fails even an optional step. The speed is resolved before the timeout, as ScrollUntilVisibleCommand.evaluateScripts evaluates them, so with both bad the speed's error is the step's.

Related Issues

None open; found running a suite written for Maestro (searched for "timeout variable", "cannot unmarshal", "into int").

Testing

  • Tests pass locally (go test -race ./..., pkg/driver/browser/cdp left out: it launches Chrome)
  • Linting passes (golangci-lint v2.13.2: 0 issues)
  • Added tests for new functionality
  • Tested manually with sample flows

pkg/flow/number_expression_test.go: the four fields keep a ${...}; a number, quoted or not, is read in decimal ("0100" and an unquoted 0100 are both 100, 08 is 8) and a timeout's underscores dropped (for scrollUntilVisible's "3_000" a leniency, as above); a word or an out-of-range speed is still refused.

pkg/executor/number_expression_test.go: the driver gets the values at the top level, in a sub-flow run twice with different env:, and in each pass of a repeat. A value that is not a number fails the step without reaching the driver, with optional honoured per field as above, also inside a runFlow, a repeat, a retry and an onFlowStart; with a bad speed and a bad timeout the speed's error wins; a ${...} that throws fails even an optional step. Both tests fail on main, where the flows do not parse.

On an Android emulator (API 36), with Maestro 2.12.1 for reference: the repro above (with an installed app's id) waits its 3 s and fails at the wait, as on Maestro (main refuses the file), and an optional wait whose timeout comes from -e waits that long and lets the flow pass, as on Maestro.

Checklist

Additional Notes

Proposed CHANGELOG entry:

timeout: of extendedWaitUntil, waitForAnimationToEnd and scrollUntilVisible, and scrollUntilVisible's speed:, may come from a variable, as in Maestro (#279). timeout: ${PAUSE_MS} failed the flow's validation ("cannot unmarshal !!str … into int"), and with it every flow of the run, before a device was touched. Maestro reads these four as text and evaluates them when the step runs; so does the runner now. A number there, quoted or not, is read in decimal. A value that is not a number fails the step; an optional extendedWaitUntil or scrollUntilVisible is reported failed and the flow goes on, as Maestro lets it go on.

… in Maestro

Maestro reads extendedWaitUntil's, waitForAnimationToEnd's and
scrollUntilVisible's timeout, and scrollUntilVisible's speed, as text
(Strings in its YAML model) and evaluates them with the step's other
scripts when the step runs (evaluateScripts in Commands.kt). Here they are
ints, so a flow that writes `timeout: ${PAUSE_MS}`, or quotes the number,
failed to parse ("cannot unmarshal !!str `${PAUSE...` into int"), and the
validation stopped every flow of the run before a device was touched.

Such an expression is now kept as written (TimeoutExpr, SpeedExpr) and
resolved into the int each time the step runs, in the copy expansion
makes, so a sub-flow run twice or a repeat gets its variables' values of
the moment. A number, quoted or not, is read as Maestro reads it: in
decimal ("0100" is 100, where YAML reads octal 64), a timeout's
underscores dropped as Maestro's extendedWaitUntil drops them (its
scrollUntilVisible and waitForAnimationToEnd fail on one when the step
runs, so there this is lenient). A value that is not a number
fails the step without running it, naming what it resolved to; as in
Maestro (Orchestra.kt executeCommands), an optional extendedWaitUntil or
scrollUntilVisible lets the flow go on (reported failed here, a warning
there), since its timeout error is an InvalidCommand, while a bad scroll speed (InvalidNumericFieldValue) or
waitForAnimationToEnd timeout fails even an optional step; the speed is
resolved before the timeout, as Maestro evaluates it. A literal that is
not a number is still refused at parse time.

The other number fields Maestro reads as text (repeat's times, retry's
maxRetries, assertScreenshot's thresholdPercentage, a selector's index,
the points) already took an expression here, and the ones it reads as
numbers (tapOn's repeat and delay, scrollUntilVisible's
visibilityPercentage, swipe's duration) refuse one there too.
@codecov

codecov Bot commented Nov 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.92308% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pkg/executor/scripting.go 95.12% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@omnarayan
omnarayan merged commit fc78a51 into devicelab-dev:main Nov 18, 2026
5 checks passed
omnarayan added a commit that referenced this pull request Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants