fix(plugin-e2e): correct dashboard toolbar version thresholds - #2836
Conversation
sunker
left a comment
There was a problem hiding this comment.
Thanks for raising this PR @adamyeats! What we usually do in situations like these is to temporarily uncomment this line to run the internal plugin-e2e tests in all supported version of Grafana. Would you mind doing that and verify all tests still pass?
|
|
…a version" This reverts commit 3900788.
|
All 28 versions ran. 24 passed. Four failed: 9.5.21, 10.0.13, 10.1.10 and 11.5.10. None of the four failed on the toolbar. I checked every errored action in their traces, and each one stalls on panel options groups, alert rule steps, query editor rows or panel menus. No errored action uses The four failures look pre-existing on versions the matrix has never included. I can raise a separate issue with the details if that is useful.
|
What this PR does / why we need it:
DashboardPage.toolbarmaps a Grafana version to the toolbar that holds the time range controls.Each threshold was the version at which the selector first existed, per
@grafana/e2e-selectors.At each of those versions the element is still behind a default-off feature toggle, so the getter
returns a locator for an element Grafana never renders:
Dashboard.ControlsdashboardScene)NavToolbar.containertopnav)Anything scoping to
toolbarthen waits for a root that never appears. InTimeRange.setthescoped click consumes the whole test budget, so its
catchonly runs during teardown and reportslocator.click: Target page, context or browser has been closed— which hides the real error,Test timeout of 30000ms exceeded.This moves both thresholds to the version at which each toolbar became the default. 9.5.0 is
already the threshold
addPaneluses for the same top nav change, two branches down the same file.Which issue(s) this PR fixes:
None filed — found while investigating the red
9.4.17and11.1.13legs on #2832.Special notes for your reviewer:
These versions reached CI for the first time this week.
playwright.ymlruns the full matrix onlyon PRs that touch
packages/plugin-e2e/**, and the resolver picks the latest patch of each minorwhen it runs, so the set moves as Grafana publishes releases. The last full-matrix run before this
week resolved 9.3.16 and 11.0.11, which sit either side of both broken ranges. The nightly uses
limit: 1, so it only covers the latest stable and grafana-dev. Every plugin-e2e PR is affecteduntil this lands, including #2819.
Behaviour changes only for 9.4.x and 11.1.x–11.2.x, which are exactly the broken ranges. Every
other version resolves to the same selector as before. Verified against each version with the
compose environment from
playwright.yml:.page-toolbar.page-toolbarNav toolbarNav toolbarNav toolbardashboard controlsDashboardPage.test.tslocks all three branches and fails on 9.4.17, 11.1.13 and 11.2.10 with theold thresholds.