Skip to content

feat(skills): add the aws-support-managed tier - #244

Merged
miscreantmoogly merged 2 commits into
mainfrom
skills/aws-support-managed-tier
Nov 16, 2026
Merged

miscreantmoogly merged 2 commits into
mainfrom
skills/aws-support-managed-tier

Conversation

@miscreantmoogly

@miscreantmoogly miscreantmoogly commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Description

Ready to merge: the service-side change that accepts this tier is live in every region, so a skill that sets aws-support-managed can now be added and synced.

Adds aws-support-managed as a value for a skill's metadata.tier, for skills that AWS Support maintains, so catalogs can label them apart from partner-managed and community skills. Until now the publishing check accepted only aws-managed and partner-managed.

  • skill_checks.py accepts the new value. Its error now lists every allowed value, built from TIERS, so the message can't fall behind the list.
  • Conformance cases cover the new value and the aws-support near miss. The unit tests cover it and a few more near misses (wrong case, prefixes).
  • CONTRIBUTING.md, .claude/CLAUDE.md and .kiro/steering/project-conventions.md list the new value and say it is only for skills that AWS Support maintains.
  • The PR template gets a reviewer note: when a PR sets aws-support-managed, a maintainer confirms through AWS Support's internal ownership process that the named owner or team has accepted ongoing maintenance, then adds the aws-support-managed-verified label or a verification comment. Who opened the pull request isn't the test. CONTRIBUTING.md, .claude/CLAUDE.md and the steering file state the same rule.

No skill uses the new tier yet. Skills that do will come in their own PRs.

Connector tiers (connector.schema.json) are separate and unchanged.

For reviewers: the part to check is the wording of the new reviewer note and of the metadata.tier row in CONTRIBUTING.md. The eval rules don't change, and they are the same for every tier.

Type of change

  • New skill
  • New custom agent
  • New MCP server
  • Update to an existing skill, agent, or MCP server
  • Documentation or infrastructure change

Required before review

  • Repository layout is correct: this PR changes only .github/, tests/, CONTRIBUTING.md, .claude/CLAUDE.md and .kiro/steering/project-conventions.md.
  • A domain SME has reviewed and signed off. n/a: a check and docs change that changes no technical claim a tool makes.
  • Evaluation evidence is included. n/a: no skill, agent, or MCP server changes.
  • No internal-only or proprietary content: no internal hostnames, account IDs, customer names, ticket numbers, internal URLs, or secrets.
  • README is complete. n/a: no tool directory changes.
  • CHANGELOG updated and the version bumped. n/a: no tool directory changes.
  • Named owner for ongoing maintenance. n/a: the check is owned by the code owners of /.github/ and /tests/.

Testing

  • python3 -I -m unittest discover -s tests -v: all 258 tests pass.
  • python3 .github/scripts/validate_skills.py --self-check-only passes with the new conformance case, and that case fails against the old tier list.
  • python3 .github/scripts/validate_skills.py --base-ref origin/main --head-ref HEAD: 100 skills checked, 0 with errors.
  • The AWS identifier scan (--self-check, then against main) reports nothing.

License confirmation

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache License 2.0.

Skills that AWS Support maintains should show as theirs in catalogs,
separate from partner-managed and community skills. metadata.tier says
who maintains a skill, and the publishing check accepted only
aws-managed and partner-managed.

- skill_checks.py accepts aws-support-managed. Its error lists every
  allowed value, built from TIERS so the two can't drift apart.
- A conformance case covers the new value. The unit tests cover it and
  a few near misses.
- CONTRIBUTING.md and the agent conventions say the tier is only for
  skills that AWS Support maintains. The check can't verify that, so
  the PR template asks reviewers to check that the pull request's
  author and the named owner are part of AWS Support.

Every skill on main still passes.

Connector tiers (connector.schema.json) are separate and unchanged.
Galaoaoa
Galaoaoa previously approved these changes Nov 12, 2026

@ams-thakkar ams-thakkar left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What needs to change

  1. Define how a maintainer verifies the AWS Support ownership claim.
    • Checking whether the PR author appears to work in AWS Support is unreliable and doesn't prove that Support has accepted ongoing ownership of the skill. A maintainer can also open a PR on the owner's behalf.
    • Use the existing internal skill-submission workflow (project nebula) as the source of truth. Its record should identify the skill and public PR, name the author and ongoing Support owner/team, record that the owner accepted maintenance responsibility, and show that testing and SME review gates passed.
    • The public PR must not expose an internal task number or link. Use the public PR URL as the lookup key in the internal record. After checking it, a maintainer can apply an aws-support-managed-verified label or leave a public verification comment. That is the signal future maintainers can trust.
    • Fix the PR template's reviewer note to describe that verification step. Use one rule in CONTRIBUTING.md, .claude/CLAUDE.md and .kiro/steering/project-conventions.md: when a PR sets aws-support-managed, a maintainer verifies through the internal ownership process that the named owner/team has accepted ongoing AWS Support maintenance. Don't make the PR author's affiliation the gate.

Nits (non-blocking)

  • Add a fail case for aws-support to conformance-cases.json. That file is the contract for other tools, but the near misses are covered only in tests/test_skill_checks.py.
  • #246 (draft, same author) changes 7 of these 8 files, including skill_checks.py and conformance-cases.json. Whichever merges second needs a rebase and a re-run of the self-check.
  • Before the first skill uses aws-support-managed, confirm that the publisher and catalogs accept and display the value.

Thanks

@miscreantmoogly, building the error message from TIERS means it can't fall behind the list again. The tests cover the exact message and the case and prefix near misses, and both agent mirrors carry identical text.

What I verified, on a trial merge with current main (which includes #235):

  • The merge is clean.
  • python3 -I -m unittest discover -s tests: 271 tests, OK.
  • validate_skills.py --self-check-only passes, and against main it finds 100 skills with 0 errors.
  • No code in the repo outside skill_checks.py reads skill tier values. The connector checks are untouched.
  • The two agent mirrors differ only by a pre-existing mkdocs line, not by anything this PR changes.

A maintainer confirms through AWS Support's internal ownership process
that the named owner or team has accepted ongoing maintenance, then adds
the aws-support-managed-verified label or a verification comment. The
pull request author's affiliation is no longer the test, and the same
rule now reads the same in CONTRIBUTING.md, .claude/CLAUDE.md and the
steering file. Also add a conformance case for the aws-support near miss.
@miscreantmoogly

Copy link
Copy Markdown
Contributor Author

Thanks @ams-thakkar! Done in 11b2740:

  • The reviewer note in the PR template now says a maintainer confirms through AWS Support's internal ownership process that the named owner or team has accepted ongoing maintenance, looks the record up by the PR's URL without posting internal task numbers or links, and then adds the new aws-support-managed-verified label or a verification comment. Who opened the PR isn't the test.
  • CONTRIBUTING.md, .claude/CLAUDE.md and the steering file now carry the same rule, word for word.
  • Added an aws-support fail case to conformance-cases.json; the self-check fails if that value is ever allowed.
  • On your last nit: I've added a hold line to the description. No skill should use the tier until service support for it is in place, so please don't merge this until then.

Checks: 258 tests pass, self-check passes, validate_skills reports 0 errors.

@yeremyjt

yeremyjt commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Your reply mentions a hold line in the description, but I don't see it. Could you add it back? A skill that sets aws-support-managed must not merge until the service-side change that accepts the tier is in every prod region, or its Add and Sync calls get a 500.

@miscreantmoogly

Copy link
Copy Markdown
Contributor Author

Thanks @yeremyjt, it got dropped when I edited the description. It's back at the top, with your point: once this merges, a skill that sets aws-support-managed passes every check but its Add and Sync calls get a 500 until the service-side change is in every prod region, so this PR and any skill that uses the tier wait for that change.

@miscreantmoogly

Copy link
Copy Markdown
Contributor Author

@yeremyjt the service-side support for aws-support-managed is now live in every region, so I've taken the hold off the description. Could you take another look and approve when you get a chance? Thanks!

@yeremyjt yeremyjt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up! I checked that the service-side change is live in every prod Region, so a skill that sets aws-support-managed can now be added and synced. On a trial merge with current main, the unit tests, the conformance self-check and validate_skills.py against main all pass (102 skills, 0 errors).

@miscreantmoogly
miscreantmoogly merged commit 571e93d into main Nov 16, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants