update test plans to use template-id instead of regex (Infra) - #2843
clairlin53 wants to merge 2 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2843 +/- ##
==========================================
- Coverage 61.33% 61.33% -0.01%
==========================================
Files 499 499
Lines 50395 50391 -4
Branches 8144 8143 -1
==========================================
- Hits 30911 30908 -3
+ Misses 18644 18643 -1
Partials 840 840
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🟡 Changes recommended
Multiple after-suspend selectors/overrides use after-suspend-…/<template-id> identifiers that won’t match any jobs (and several non-after-suspend plans will unintentionally select after-suspend sibling jobs) under current template-id/sibling behavior.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates provider test plans under providers/base/units/** to reference template-instantiated jobs via template-id-style stable identifiers instead of broad regex patterns, aligning with the follow-ups to #1137 / #1127.
Changes:
- Replace many
include:regex selectors (e.g.foo_.*) with fixed template identifiers (e.g.foo_name,foo_interface). - Update
certification_status_overridesentries similarly (notably in usb/touchscreen/bluetooth plans). - Add missing
template-idfields for a couple of camera attachment templates.
File summaries
| File | Description |
|---|---|
| providers/base/units/usb/test-plan.pxu | Switch usb storage selections/overrides from regex to template-id-like identifiers |
| providers/base/units/touchscreen/test-plan.pxu | Update multitouch rotate blocker overrides to template-id identifiers |
| providers/base/units/touchpad/test-plan.pxu | Replace palm-rejection template regex selectors with template-id identifiers |
| providers/base/units/stress/test-plan.pxu | Replace iperf3 selector/override with template-id identifier |
| providers/base/units/snapd/test-plan.pxu | Replace snap store config selector with template-id identifier |
| providers/base/units/optical/test-plan.pxu | Replace optical automated read selectors with template-id identifiers |
| providers/base/units/networking/test-plan.pxu | Replace info_device selector with template-id identifier |
| providers/base/units/mediacard/test-plan.pxu | Replace storage-preinserted selectors with template-id identifiers |
| providers/base/units/led/test-plan.pxu | Replace camera/led selector with template-id identifier |
| providers/base/units/input/test-plan.pxu | Replace pointing/clicking/fixed-orientation selectors with template-id identifiers |
| providers/base/units/info/test-plan.pxu | Replace hdparm attachment selector with template-id identifier |
| providers/base/units/ethernet/test-plan.pxu | Replace hotplug selector with template-id identifier |
| providers/base/units/disk/test-plan.pxu | Replace disk/benchmarks selectors with template-id identifiers |
| providers/base/units/cpu/test-plan.pxu | Replace VFP support selectors with template-id identifiers |
| providers/base/units/camera/test-plan.pxu | Replace multiple camera selectors with template-id identifiers |
| providers/base/units/camera/jobs.pxu | Add explicit template-id fields for attachment templates |
| providers/base/units/bluetooth/test-plan.pxu | Replace beacon eddystone selectors/overrides with template-id identifiers |
Review details
Suppressed comments (11)
providers/base/units/usb/test-plan.pxu:416
- These overrides target
after-suspend-…/<template-id>, but after-suspend sibling jobs retain the originaltemplate-id(without theafter-suspend-prefix). As written, the overrides will not apply to the after-suspend jobs; revert to the previous regex for those sibling ids (or change sibling generation to rewrite template-id).
apply blocker to com.canonical.certification::usb/storage-preinserted-symlink_uuid
apply blocker to com.canonical.certification::after-suspend-usb/storage-detect
apply blocker to com.canonical.certification::after-suspend-usb/storage-preinserted-symlink_uuid
providers/base/units/led/test-plan.pxu:38
after-suspend-…/<template-id>will not match any jobs because after-suspend sibling jobs keep the originaltemplate-id(noafter-suspend-prefix). Revert to the previous id-regex to select the after-suspend instances.
after-suspend-camera/led_name certification-status=blocker
providers/base/units/bluetooth/test-plan.pxu:216
- These overrides use exact
after-suspend-…/<template-id>identifiers, but after-suspend sibling jobs keep the original template-id. As a result, these overrides won't apply to the after-suspend jobs; revert to the previous regex patterns.
apply non-blocker to com.canonical.certification::bluetooth4/beacon_eddystone_url_interface
apply non-blocker to com.canonical.certification::after-suspend-bluetooth4/beacon_eddystone_url_interface
providers/base/units/optical/test-plan.pxu:50
after-suspend-…/<template-id>won't match any jobs because after-suspend sibling jobs keep the originaltemplate-id(noafter-suspend-prefix). Revert to the previous id-regex here so the after-suspend automated read jobs are included.
after-suspend-optical/read-automated_name certification-status=blocker
providers/base/units/mediacard/test-plan.pxu:42
after-suspend-…/<template-id>won't match any jobs because after-suspend sibling jobs keep the originaltemplate-id. Revert to the previous id-regex so the after-suspend mediacard storage-preinserted jobs are included.
after-suspend-mediacard/storage-preinserted-symlink_uuid
providers/base/units/touchpad/test-plan.pxu:52
after-suspend-…/<template-id>won't match any jobs because after-suspend sibling jobs keep the originaltemplate-id(noafter-suspend-prefix). Revert to the previous id-regex here so the after-suspend palm-rejection jobs are selected.
after-suspend-touchpad/palm-rejection-firmware-labeling_product_slug certification-status=non-blocker
providers/base/units/ethernet/test-plan.pxu:26
after-suspend-…/<template-id>won't match any jobs because after-suspend sibling jobs keep the originaltemplate-id. Revert to the previous id-regex so the after-suspend hotplug jobs are selected.
after-suspend-ethernet/hotplug-interface certification-status=blocker
providers/base/units/networking/test-plan.pxu:26
after-suspend-…/<template-id>won't match any jobs because after-suspend sibling jobs retain the originaltemplate-id. Revert to the previous id-regex so the after-suspend networking info job is selected.
after-suspend-networking/info_device__index___interface certification-status=blocker
providers/base/units/disk/test-plan.pxu:43
after-suspend-…/<template-id>won't match any jobs because after-suspend sibling jobs keep the originaltemplate-id(noafter-suspend-prefix). Revert to the previous id-regex patterns so the after-suspend disk jobs are selected.
after-suspend-disk/stats_name
after-suspend-disk/read_performance_name certification-status=blocker
after-suspend-disk/storage_device_name certification-status=blocker
after-suspend-benchmarks/disk/hdparm-read_name
after-suspend-benchmarks/disk/hdparm-cache-read_name
providers/base/units/input/test-plan.pxu:48
after-suspend-…/<template-id>won't match any jobs because after-suspend sibling jobs keep the originaltemplate-id(noafter-suspend-prefix). Revert to the previous id-regex so the after-suspend fixed-orientation jobs are selected.
after-suspend-input/fixed_screen_orientation_on_product___index__ certification-status=non-blocker
providers/base/units/input/test-plan.pxu:29
after-suspend-…/<template-id>won't match any jobs because after-suspend sibling jobs keep the originaltemplate-id(noafter-suspend-prefix). Revert to the previous id-regex patterns so the after-suspend pointing/clicking jobs are selected.
after-suspend-input/pointing_product_slug_category___index__ certification-status=blocker
after-suspend-input/clicking_product_slug_category___index__ certification-status=blocker
- Files reviewed: 17/17 changed files
- Comments generated: 13
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Description
WARNING: This modifies com.canonical.certification::sru-server
Update test plans to use
template-idinstead of the previous format,following up on #1137 and #1127.
Resolved issues
https://warthogs.atlassian.net/browse/OEMQA-6553
Documentation
N/A
Tests
Iperf: https://certification.canonical.com/hardware/202511-38065/submission/509888/
Auto: https://certification.canonical.com/hardware/202511-38065/submission/509873/
Manual: https://certification.canonical.com/hardware/202511-38065/submission/509768/
09/10
remove redundant: https://certification.canonical.com/hardware/202511-38064/submission/510250/