Skip to content

update test plans to use template-id instead of regex (Infra) - #2843

Open
clairlin53 wants to merge 2 commits into
mainfrom
modify_templateid
Open

clairlin53 wants to merge 2 commits into
mainfrom
modify_templateid

Conversation

@clairlin53

@clairlin53 clairlin53 commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Description

WARNING: This modifies com.canonical.certification::sru-server

Update test plans to use template-id instead of the previous format,
following up on #1137 and #1127.

Resolved issues

https://warthogs.atlassian.net/browse/OEMQA-6553

Documentation

N/A

Tests

Iperf: https://certification.canonical.com/hardware/202511-38065/submission/509888/

Auto: https://certification.canonical.com/hardware/202511-38065/submission/509873/

Manual: https://certification.canonical.com/hardware/202511-38065/submission/509768/

09/10
remove redundant: https://certification.canonical.com/hardware/202511-38064/submission/510250/

@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.33%. Comparing base (7e814fc) to head (98ca7b6).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2843      +/-   ##
==========================================
- Coverage   61.33%   61.33%   -0.01%     
==========================================
  Files         499      499              
  Lines       50395    50391       -4     
  Branches     8144     8143       -1     
==========================================
- Hits        30911    30908       -3     
+ Misses      18644    18643       -1     
  Partials      840      840              
Flag Coverage Δ
provider-base 39.81% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@clairlin53
clairlin53 marked this pull request as ready for review September 9, 2026 03:44
Copilot AI lite review requested due to automatic review settings September 9, 2026 03:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Multiple after-suspend selectors/overrides use after-suspend-…/<template-id> identifiers that won’t match any jobs (and several non-after-suspend plans will unintentionally select after-suspend sibling jobs) under current template-id/sibling behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates provider test plans under providers/base/units/** to reference template-instantiated jobs via template-id-style stable identifiers instead of broad regex patterns, aligning with the follow-ups to #1137 / #1127.

Changes:

  • Replace many include: regex selectors (e.g. foo_.*) with fixed template identifiers (e.g. foo_name, foo_interface).
  • Update certification_status_overrides entries similarly (notably in usb/touchscreen/bluetooth plans).
  • Add missing template-id fields for a couple of camera attachment templates.
File summaries
File Description
providers/base/units/usb/test-plan.pxu Switch usb storage selections/overrides from regex to template-id-like identifiers
providers/base/units/touchscreen/test-plan.pxu Update multitouch rotate blocker overrides to template-id identifiers
providers/base/units/touchpad/test-plan.pxu Replace palm-rejection template regex selectors with template-id identifiers
providers/base/units/stress/test-plan.pxu Replace iperf3 selector/override with template-id identifier
providers/base/units/snapd/test-plan.pxu Replace snap store config selector with template-id identifier
providers/base/units/optical/test-plan.pxu Replace optical automated read selectors with template-id identifiers
providers/base/units/networking/test-plan.pxu Replace info_device selector with template-id identifier
providers/base/units/mediacard/test-plan.pxu Replace storage-preinserted selectors with template-id identifiers
providers/base/units/led/test-plan.pxu Replace camera/led selector with template-id identifier
providers/base/units/input/test-plan.pxu Replace pointing/clicking/fixed-orientation selectors with template-id identifiers
providers/base/units/info/test-plan.pxu Replace hdparm attachment selector with template-id identifier
providers/base/units/ethernet/test-plan.pxu Replace hotplug selector with template-id identifier
providers/base/units/disk/test-plan.pxu Replace disk/benchmarks selectors with template-id identifiers
providers/base/units/cpu/test-plan.pxu Replace VFP support selectors with template-id identifiers
providers/base/units/camera/test-plan.pxu Replace multiple camera selectors with template-id identifiers
providers/base/units/camera/jobs.pxu Add explicit template-id fields for attachment templates
providers/base/units/bluetooth/test-plan.pxu Replace beacon eddystone selectors/overrides with template-id identifiers
Review details

Suppressed comments (11)

providers/base/units/usb/test-plan.pxu:416

  • These overrides target after-suspend-…/<template-id>, but after-suspend sibling jobs retain the original template-id (without the after-suspend- prefix). As written, the overrides will not apply to the after-suspend jobs; revert to the previous regex for those sibling ids (or change sibling generation to rewrite template-id).
    apply blocker to com.canonical.certification::usb/storage-preinserted-symlink_uuid
    apply blocker to com.canonical.certification::after-suspend-usb/storage-detect
    apply blocker to com.canonical.certification::after-suspend-usb/storage-preinserted-symlink_uuid

providers/base/units/led/test-plan.pxu:38

  • after-suspend-…/<template-id> will not match any jobs because after-suspend sibling jobs keep the original template-id (no after-suspend- prefix). Revert to the previous id-regex to select the after-suspend instances.
    after-suspend-camera/led_name      certification-status=blocker

providers/base/units/bluetooth/test-plan.pxu:216

  • These overrides use exact after-suspend-…/<template-id> identifiers, but after-suspend sibling jobs keep the original template-id. As a result, these overrides won't apply to the after-suspend jobs; revert to the previous regex patterns.
    apply non-blocker to com.canonical.certification::bluetooth4/beacon_eddystone_url_interface
    apply non-blocker to com.canonical.certification::after-suspend-bluetooth4/beacon_eddystone_url_interface

providers/base/units/optical/test-plan.pxu:50

  • after-suspend-…/<template-id> won't match any jobs because after-suspend sibling jobs keep the original template-id (no after-suspend- prefix). Revert to the previous id-regex here so the after-suspend automated read jobs are included.
 after-suspend-optical/read-automated_name       certification-status=blocker

providers/base/units/mediacard/test-plan.pxu:42

  • after-suspend-…/<template-id> won't match any jobs because after-suspend sibling jobs keep the original template-id. Revert to the previous id-regex so the after-suspend mediacard storage-preinserted jobs are included.
    after-suspend-mediacard/storage-preinserted-symlink_uuid

providers/base/units/touchpad/test-plan.pxu:52

  • after-suspend-…/<template-id> won't match any jobs because after-suspend sibling jobs keep the original template-id (no after-suspend- prefix). Revert to the previous id-regex here so the after-suspend palm-rejection jobs are selected.
 after-suspend-touchpad/palm-rejection-firmware-labeling_product_slug   certification-status=non-blocker

providers/base/units/ethernet/test-plan.pxu:26

  • after-suspend-…/<template-id> won't match any jobs because after-suspend sibling jobs keep the original template-id. Revert to the previous id-regex so the after-suspend hotplug jobs are selected.
    after-suspend-ethernet/hotplug-interface       certification-status=blocker

providers/base/units/networking/test-plan.pxu:26

  • after-suspend-…/<template-id> won't match any jobs because after-suspend sibling jobs retain the original template-id. Revert to the previous id-regex so the after-suspend networking info job is selected.
    after-suspend-networking/info_device__index___interface      certification-status=blocker

providers/base/units/disk/test-plan.pxu:43

  • after-suspend-…/<template-id> won't match any jobs because after-suspend sibling jobs keep the original template-id (no after-suspend- prefix). Revert to the previous id-regex patterns so the after-suspend disk jobs are selected.
    after-suspend-disk/stats_name
    after-suspend-disk/read_performance_name                 certification-status=blocker
    after-suspend-disk/storage_device_name                   certification-status=blocker
    after-suspend-benchmarks/disk/hdparm-read_name
    after-suspend-benchmarks/disk/hdparm-cache-read_name

providers/base/units/input/test-plan.pxu:48

  • after-suspend-…/<template-id> won't match any jobs because after-suspend sibling jobs keep the original template-id (no after-suspend- prefix). Revert to the previous id-regex so the after-suspend fixed-orientation jobs are selected.
 after-suspend-input/fixed_screen_orientation_on_product___index__       certification-status=non-blocker

providers/base/units/input/test-plan.pxu:29

  • after-suspend-…/<template-id> won't match any jobs because after-suspend sibling jobs keep the original template-id (no after-suspend- prefix). Revert to the previous id-regex patterns so the after-suspend pointing/clicking jobs are selected.
    after-suspend-input/pointing_product_slug_category___index__     certification-status=blocker
    after-suspend-input/clicking_product_slug_category___index__     certification-status=blocker
  • Files reviewed: 17/17 changed files
  • Comments generated: 13
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread providers/base/units/bluetooth/test-plan.pxu
Comment thread providers/base/units/disk/test-plan.pxu
Comment thread providers/base/units/ethernet/test-plan.pxu
Comment thread providers/base/units/input/test-plan.pxu
Comment thread providers/base/units/input/test-plan.pxu
Comment thread providers/base/units/optical/test-plan.pxu
Comment thread providers/base/units/touchpad/test-plan.pxu
Comment thread providers/base/units/usb/test-plan.pxu
Comment thread providers/base/units/usb/test-plan.pxu
Comment thread providers/base/units/touchscreen/test-plan.pxu Outdated
@clairlin53
clairlin53 marked this pull request as draft September 9, 2026 04:41
@clairlin53
clairlin53 marked this pull request as ready for review September 10, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants