Repository navigation
feat(skills): add the aws-support-managed tier - #244
Conversation
Skills that AWS Support maintains should show as theirs in catalogs, separate from partner-managed and community skills. metadata.tier says who maintains a skill, and the publishing check accepted only aws-managed and partner-managed. - skill_checks.py accepts aws-support-managed. Its error lists every allowed value, built from TIERS so the two can't drift apart. - A conformance case covers the new value. The unit tests cover it and a few near misses. - CONTRIBUTING.md and the agent conventions say the tier is only for skills that AWS Support maintains. The check can't verify that, so the PR template asks reviewers to check that the pull request's author and the named owner are part of AWS Support. Every skill on main still passes. Connector tiers (connector.schema.json) are separate and unchanged.
There was a problem hiding this comment.
What needs to change
- Define how a maintainer verifies the AWS Support ownership claim.
- Checking whether the PR author appears to work in AWS Support is unreliable and doesn't prove that Support has accepted ongoing ownership of the skill. A maintainer can also open a PR on the owner's behalf.
- Use the existing internal skill-submission workflow (project nebula) as the source of truth. Its record should identify the skill and public PR, name the author and ongoing Support owner/team, record that the owner accepted maintenance responsibility, and show that testing and SME review gates passed.
- The public PR must not expose an internal task number or link. Use the public PR URL as the lookup key in the internal record. After checking it, a maintainer can apply an
aws-support-managed-verifiedlabel or leave a public verification comment. That is the signal future maintainers can trust. - Fix the PR template's reviewer note to describe that verification step. Use one rule in
CONTRIBUTING.md,.claude/CLAUDE.mdand.kiro/steering/project-conventions.md: when a PR setsaws-support-managed, a maintainer verifies through the internal ownership process that the named owner/team has accepted ongoing AWS Support maintenance. Don't make the PR author's affiliation the gate.
Nits (non-blocking)
- Add a fail case for
aws-supporttoconformance-cases.json. That file is the contract for other tools, but the near misses are covered only intests/test_skill_checks.py. - #246 (draft, same author) changes 7 of these 8 files, including
skill_checks.pyandconformance-cases.json. Whichever merges second needs a rebase and a re-run of the self-check. - Before the first skill uses aws-support-managed, confirm that the publisher and catalogs accept and display the value.
Thanks
@miscreantmoogly, building the error message from TIERS means it can't fall behind the list again. The tests cover the exact message and the case and prefix near misses, and both agent mirrors carry identical text.
What I verified, on a trial merge with current main (which includes #235):
- The merge is clean.
python3 -I -m unittest discover -s tests: 271 tests, OK.validate_skills.py --self-check-onlypasses, and againstmainit finds 100 skills with 0 errors.- No code in the repo outside
skill_checks.pyreads skill tier values. The connector checks are untouched. - The two agent mirrors differ only by a pre-existing mkdocs line, not by anything this PR changes.
A maintainer confirms through AWS Support's internal ownership process that the named owner or team has accepted ongoing maintenance, then adds the aws-support-managed-verified label or a verification comment. The pull request author's affiliation is no longer the test, and the same rule now reads the same in CONTRIBUTING.md, .claude/CLAUDE.md and the steering file. Also add a conformance case for the aws-support near miss.
|
Thanks @ams-thakkar! Done in 11b2740:
Checks: 258 tests pass, self-check passes, |
|
Your reply mentions a hold line in the description, but I don't see it. Could you add it back? A skill that sets |
|
Thanks @yeremyjt, it got dropped when I edited the description. It's back at the top, with your point: once this merges, a skill that sets |
|
@yeremyjt the service-side support for |
yeremyjt
left a comment
There was a problem hiding this comment.
Thanks for the follow-up! I checked that the service-side change is live in every prod Region, so a skill that sets aws-support-managed can now be added and synced. On a trial merge with current main, the unit tests, the conformance self-check and validate_skills.py against main all pass (102 skills, 0 errors).
Description
Ready to merge: the service-side change that accepts this tier is live in every region, so a skill that sets
aws-support-managedcan now be added and synced.Adds
aws-support-managedas a value for a skill'smetadata.tier, for skills that AWS Support maintains, so catalogs can label them apart from partner-managed and community skills. Until now the publishing check accepted onlyaws-managedandpartner-managed.skill_checks.pyaccepts the new value. Its error now lists every allowed value, built fromTIERS, so the message can't fall behind the list.aws-supportnear miss. The unit tests cover it and a few more near misses (wrong case, prefixes).CONTRIBUTING.md,.claude/CLAUDE.mdand.kiro/steering/project-conventions.mdlist the new value and say it is only for skills that AWS Support maintains.aws-support-managed, a maintainer confirms through AWS Support's internal ownership process that the named owner or team has accepted ongoing maintenance, then adds theaws-support-managed-verifiedlabel or a verification comment. Who opened the pull request isn't the test. CONTRIBUTING.md,.claude/CLAUDE.mdand the steering file state the same rule.No skill uses the new tier yet. Skills that do will come in their own PRs.
Connector tiers (
connector.schema.json) are separate and unchanged.For reviewers: the part to check is the wording of the new reviewer note and of the
metadata.tierrow inCONTRIBUTING.md. The eval rules don't change, and they are the same for every tier.Type of change
Required before review
.github/,tests/,CONTRIBUTING.md,.claude/CLAUDE.mdand.kiro/steering/project-conventions.md./.github/and/tests/.Testing
python3 -I -m unittest discover -s tests -v: all 258 tests pass.python3 .github/scripts/validate_skills.py --self-check-onlypasses with the new conformance case, and that case fails against the old tier list.python3 .github/scripts/validate_skills.py --base-ref origin/main --head-ref HEAD: 100 skills checked, 0 with errors.--self-check, then againstmain) reports nothing.License confirmation