Skip to content

fix(skills): clean up filter tags and author names - #245

Open
miscreantmoogly wants to merge 5 commits into
mainfrom
fix/bug-bash-content
Open

miscreantmoogly wants to merge 5 commits into
mainfrom
fix/bug-bash-content

Conversation

@miscreantmoogly

@miscreantmoogly miscreantmoogly commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Description

Skill catalogs build their filters from each skill's aws-services and technical-domains values, exactly as written, and link metadata.author to the GitHub profile of that name. Some of these values are wrong: an All service, Database next to Databases, skills with no tags, and authors whose links 404 or open the wrong profile. Parts 2 to 5 below are separate commits, so each can be reviewed on its own. Parts 3, 4 and 4b can each be dropped too (4b with a one-line CHANGELOG fix); part 2 can't, because parts 4, 4b and 5 reuse its version bumps.

1. Connectors: already fixed in #234. Skills and connectors on main are released together, and one change that breaks the release rules holds back everything else. Two connector changes had done that: Elastic's logos changed at a version that was already released, and Coralogix changed an auth field's label without a major version bump. So the connector fix had to come first; until it merged, nothing here could be released. #234 moved Elastic to 1.0.1 and Coralogix to 2.0.0, so this PR leaves connectors alone, and its changes are released once it merges.

2. Filter values (fix(skills): clean up AWS service and domain values)

Skill Change Version Owner
investigation-cost-guardrail Remove aws-services: "All". It isn't a service, and a skill that applies to every service leaves the key out. 2.1.1 → 2.1.2 @inesttia @tipuq
devops-agent-cost-insights CloudWatch, CloudTrail, AIDevOps → AWS DevOps Agent, Amazon CloudWatch, AWS CloudTrail 1.0.1 → 1.0.2 @inesttia @tipuq
database-migration-service-expertise Domain Database → Databases 1.0.3 → 1.0.4 @abhit-cloud
aiml-access-diagnostics Domains Security → Security, AI/ML; Amazon SageMaker → Amazon SageMaker AI 1.2.4 → 1.2.5 @ams-thakkar
aws-eks-operations-review Add Amazon EKS and Containers, Operations 1.9.6 → 1.9.7 @shyamkulkarni
ecs-operation-review Add Containers, Operations 2.5.3 → 2.5.4 @shyamkulkarni
aws-eks-healthdashboard Add Amazon EKS and Containers, Observability 1.0.0 → 1.0.1 @kuntshah
database-rds-devops Add Amazon RDS and Databases 1.0.2 → 1.0.3 @kiranmam
analytics-opensearch-expertise Add Amazon OpenSearch Service and Analytics 2.6.2 → 2.6.3 @genedragon
live-streaming-event-readiness-review Amazon Kinesis → Amazon Kinesis Data Streams, the service it reads 1.0.0 → 1.0.1 @awsmanishon

Owners, please say if a value is wrong for your skill.

3. Vocabulary (chore(vocabulary): add Amazon EBS, Amazon FSx for NetApp ONTAP, Amazon Keyspaces and Amazon Kinesis Data Streams). Adds Amazon EBS, Amazon FSx for NetApp ONTAP, Amazon Keyspaces and Amazon Kinesis Data Streams, which skills already use, plus an Amazon Kinesis → Amazon Kinesis Data Streams alias that part 5 removes again. Part 5 also adds AWS DevOps Agent, so the net change is five values and no alias. Amazon CloudFront and AWS Elemental MediaTailor are left to #188, which adds them. The file isn't published, so no versions change. It's under /.github/, so it needs an admin's approval.

4. Authors (fix(skills): set each skill's author to their GitHub username). Each new name is the account that opened the skill's original pull request. Please check your row, and say so here if you'd rather be listed differently.

Skill Author Version
agentcore-observability-setup vggargav → @vipulg1993 1.0.2 → 1.0.3
analytics-opensearch-expertise genealpe → @genedragon, in the README's Maintainers list too 2.6.3, from part 2
cloudtrail-cost-optimization holmalla → @kaiserschmarm 1.3.2 → 1.3.3
config-cost-optimization holmalla → @kaiserschmarm 1.5.3 → 1.5.4
guardduty-cost-optimization holmalla → @kaiserschmarm 1.3.2 → 1.3.3
msk-operations kjjanaki → @kalyanjanaki 1.0.5 → 1.0.6
firehose-operation-review stharolz → @shinutharol2ext 1.4.3 → 1.4.4
aiml-access-diagnostics tamrish → @ams-thakkar 1.2.5, from part 2
storage-s3-resiliency-expertise hokang → @howard-m-k 1.0.3 → 1.0.4
bedrock-operation-review smnixon → @sean-nixon 1.0.2 → 1.0.3
database-migration-service-expertise tabhiman → @abhit-cloud 1.0.4, from part 2

4b. Authors still to confirm (fix(skills): set four more skills' authors to their GitHub usernames). It's a separate commit, so it can be dropped, or a row changed, if the answer is no. Dropping it needs a one-line fix in devops-agent-cost-insights' 1.0.2 CHANGELOG. A no on the first row means inesttia alone, not tqquresh again, because that account doesn't exist.

Skill Author Version Question
devops-agent-cost-insights, investigation-cost-guardrail tqquresh, inesttia → tipuq, inesttia 1.0.2 and 2.1.2, from part 2 @inesttia @tipuq: should tipuq stay a co-author on these two skills?
service-quota-check yuriypr → @yprikhodko 1.0.1 → 1.0.2 @yprikhodko: is this skill yours, and should it list yprikhodko rather than YuriyPr?
redshift-support-specialist aws-samples → @andrehass 1.8.2 → 1.8.3 @andrehass: can aws-samples be replaced with your username?

5. Review follow-up (fix(skills): tag devops-agent-cost-insights with AWS DevOps Agent). From ams-thakkar's review: devops-agent-cost-insights now leads with AWS DevOps Agent, which joins the vocabulary, and part 3's Amazon Kinesis alias is dropped. It reuses part 2's 1.0.2.

Every skill moves one version step from main, even when two parts change it, and gets a CHANGELOG line. Organizations and teams stay as authors (coralogix, honeycombio/agentic-intelligence), because their links work.

Review. @ams-thakkar and @yprikhodko, please confirm your own rows in a comment. Since this writes both of your names into bylines, the approving review should come from another admin, such as @shyamkulkarni, @SimonCMoore or @udid-aws.

Open PRs that overlap. Please rebase after this merges and re-run the checks:

Not in this PR.

  • aws-routing still has no tags, since docs(aws-routing): correct DXGW/TGW ECMP and LP guidance #189 merged without them. @midakin, which services and domains should it have?
  • crm-production-investigation-guidelines and skip-scheduled-maintenance still have no tags. @udid-aws, which services and domains should they have?
  • Picking one AI domain (AI/ML or Machine Learning), and checks that stop these values coming back. Both are separate follow-ups.

Type of change

  • New skill
  • New custom agent
  • New MCP server
  • Update to an existing skill, agent, or MCP server
  • Documentation or infrastructure change

Required before review

  • Repository layout is correct — skills in /skills, MCP servers in /mcp, custom agents in /custom-agents; skills and MCP servers are never mixed in one directory.
  • A domain SME has reviewed and signed off. A named subject-matter expert for this domain — someone other than the author — has reviewed the tool for technical correctness and approved it in this PR. Required for any change to what the tool tells the agent; an edit that changes no technical claim can be marked n/a with the reason, as CONTRIBUTING.md describes. n/a: only catalog tags, versions and author names change; no skill's instructions or technical claims change.
  • Evaluation evidence is included (see Testing). n/a: only catalog metadata changes, not what any skill tells the agent, so there's no new behavior to evaluate; validate-skill-evals passes.
  • No internal-only or proprietary content — no internal hostnames, account IDs, customer names, ticket numbers, internal URLs, or secrets in the tool, its examples, or its tests.
  • README is complete — prerequisites, setup/deployment, and at least one sample prompt or usage example. n/a: READMEs are unchanged apart from one Maintainers name in analytics-opensearch-expertise.
  • CHANGELOG updated and the version bumped.
  • Named owner for ongoing maintenance.

Testing

Run on this branch against main, with the commands the workflows use:

  • python3 .github/scripts/validate_skills.py --base-ref origin/main --head-ref HEAD: 100 skills, 0 errors. On the skills this touches, the warnings left are older ones (missing metadata.summary, msk-operations' body length, a <script> in a redshift-support-specialist template) and Amazon CloudFront and AWS Elemental MediaTailor, which Add the AWS Elemental Media Services and CloudFront to the skill vocabulary #188 adds. Each commit passes on its own as well.

  • python3 -I -m unittest discover -s tests -v: 258 tests pass.

  • python3 .github/scripts/validate_skill_evals.py --base-ref origin/main --head-ref HEAD: 0 failures. 13 of the 20 skills are legacy and only warn.

  • python3 .github/scripts/scan_aws_identifiers.py --base-ref origin/main --head-ref HEAD: no findings.

  • mkdocs build --strict: passes.

  • gh api users/<name> finds every new author.

  • structure checks pass. n/a: metadata-only change; validate-skills checks the frontmatter, and the instructions the evals test don't change.

  • best-practices run across multiple iterations, with consistent results. n/a: no skill's instructions change.

  • functional run with and without the change, so the difference in the agent's behavior is visible. n/a: no skill's instructions change, so the agent's behavior doesn't either.

  • evals/evals.json includes positive cases (should trigger) and at least one negative case (should not trigger on unrelated input). n/a: no evals.json changes.

Skills — complete if this PR adds or changes a skill

  • SKILL.md frontmatter is valid: name matches the directory, description present and within length limits, body at most 500 lines. n/a: the frontmatter is valid in every skill here (validate-skills: 0 errors), but msk-operations' body was already 569 lines, and this PR changes no skill body.
  • The description is written from the agent's point of view — scenarios, services, error types, symptoms. n/a: no description changes.
  • Scoped to a specific agent type in metadata.aws-devops-agent-skills.agent-types (INCIDENT_RCA, INCIDENT_TRIAGE, INCIDENT_MITIGATION, …), not left GENERIC without reason. n/a: no agent types change.
  • Procedures are structured as decision trees on real conditions, not flat prose. n/a: no skill body changes.
  • Reference material lives in references/ or assets/ and loads on demand. n/a: no files are added.
  • Non-executable content only — no scripts or code in the skill directory. Deterministic compute belongs in an MCP server. n/a: no files are added. One redshift-support-specialist HTML template already has a <script> element (a warning on main), which this PR doesn't change.

Security and safety

  • Read-only by default; any action that changes customer state is explicit, documented, and approval-gated. n/a: no instructions or actions change.
  • Customer data stays in the customer's account. n/a: no instructions or actions change.
  • No destructive operations without an explicit, documented confirmation step. n/a: no instructions or actions change.

License confirmation

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache License 2.0.

Catalogs show each aws-services and technical-domains value as written,
so a placeholder or a near-duplicate becomes its own filter option, and
a skill without tags drops out of the filters that should list it.

- investigation-cost-guardrail 2.1.2: remove aws-services "All". It
  isn't a service; a skill that applies to every service leaves the key
  out.
- devops-agent-cost-insights 1.0.2: use "Amazon CloudWatch, AWS
  CloudTrail", and drop "AIDevOps".
- database-migration-service-expertise 1.0.4: "Database" becomes the
  approved "Databases".
- aiml-access-diagnostics 1.2.5: add the AI/ML domain, and name Amazon
  SageMaker AI, whose training jobs and endpoints the skill diagnoses.
- aws-eks-operations-review 1.9.7, aws-eks-healthdashboard 1.0.1,
  ecs-operation-review 2.5.4, database-rds-devops 1.0.3 and
  analytics-opensearch-expertise 2.6.3: add the missing service and
  domain tags.
- live-streaming-event-readiness-review 1.0.1: "Amazon Kinesis" becomes
  "Amazon Kinesis Data Streams", the service the skill reads.

Each skill gets one patch bump and a CHANGELOG entry. Keys are removed
rather than left empty, because an empty value is an error.
…n Keyspaces and Amazon Kinesis Data Streams

Add Amazon EBS, Amazon FSx for NetApp ONTAP, Amazon Keyspaces and Amazon
Kinesis Data Streams to the approved aws-services values. Skills already
use all four (live-streaming-event-readiness-review uses Kinesis Data
Streams after the previous commit), so the check stops warning about
them.

Add the alias "Amazon Kinesis" -> "Amazon Kinesis Data Streams", so the
check suggests the specific service name instead of the family name.

Amazon CloudFront and AWS Elemental MediaTailor are left to #188, which
adds them. The file isn't published, so no skill version changes.
Catalogs link metadata.author to the GitHub profile of that name. These
eleven skills name an account that doesn't exist, or a different
account from the one that contributed the skill, so the link 404s or
opens the wrong profile. Each now names the account that opened the
skill's original pull request:

- agentcore-observability-setup 1.0.3: vipulg1993
- analytics-opensearch-expertise 2.6.3: genedragon, in the README's
  Maintainers list too
- cloudtrail-cost-optimization 1.3.3, config-cost-optimization 1.5.4
  and guardduty-cost-optimization 1.3.3: kaiserschmarm
- msk-operations 1.0.6: kalyanjanaki
- firehose-operation-review 1.4.4: shinutharol2ext
- aiml-access-diagnostics 1.2.5: ams-thakkar
- storage-s3-resiliency-expertise 1.0.4: howard-m-k
- bedrock-operation-review 1.0.3: sean-nixon
- database-migration-service-expertise 1.0.4: abhit-cloud

analytics-opensearch-expertise, aiml-access-diagnostics and
database-migration-service-expertise keep the version the tag cleanup
gave them, so each skill moves one step in this change. The others get a
patch bump. Every skill gets a CHANGELOG line.
As in the previous commit, each skill now names the GitHub account of
the person who maintains it:

- devops-agent-cost-insights and investigation-cost-guardrail: the
  co-author tqquresh becomes tipuq. No GitHub account is named
  tqquresh; tipuq is the co-author's account, and merged #40 and #61 on
  the cost guardrail. Both skills keep the version the tag cleanup gave
  them.
- service-quota-check 1.0.2: yuriypr becomes yprikhodko, the account
  that opened the skill's pull request.
- redshift-support-specialist 1.8.3: aws-samples becomes andrehass.
  aws-samples is an organization; andrehass opened the skill's pull
  request and wrote nearly all of its commits.

Each skill gets a CHANGELOG line.
@ams-thakkar
ams-thakkar self-requested a review November 12, 2026 02:07
@miscreantmoogly
miscreantmoogly marked this pull request as ready for review November 12, 2026 04:13
@miscreantmoogly
miscreantmoogly requested a review from a team as a code owner November 12, 2026 04:13

@ams-thakkar ams-thakkar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What needs to change

  1. Keep the skill's primary service in its catalog tags. skills/devops-agent-cost-insights/SKILL.md:3 is specifically about the cost of AWS DevOps Agent, but line 8 now lists only CloudWatch and CloudTrail. Removing AIDevOps is right because it is a metric namespace, not the service name. Add AWS DevOps Agent to vocabulary.json and use AWS DevOps Agent, Amazon CloudWatch, AWS CloudTrail here, so the catalog can find this skill by its actual subject.

Nits (non-blocking)

  • Drop the Amazon Kinesis → Amazon Kinesis Data Streams alias from vocabulary.json:67. The changed skill now uses the precise name directly, and the broad alias could give the wrong correction for a family-level or Kinesis Video Streams skill later.
  • The open-overlap list misses #25, #230 and the stacked draft #246, which also touch vocabulary.json. Whichever merges second needs a rebase and another self-check.

Thanks

I confirm the aiml-access-diagnostics row you asked me about: author: ams-thakkar, service Amazon SageMaker AI, and domains Security, AI/ML are all correct.

The Redshift change to andrehass is reasonable: aws-samples identifies the source project/organization, not an individual owner.

@miscreantmoogly, the remaining filter cleanups are consistent with the skills' scopes, and the author mappings match the original PR openers. I also confirmed that the catalog renders tipuq, inesttia as two separate profile links.

What I verified on a trial merge with current main:

  • All 271 unit tests pass.
  • validate_skills, validate_skill_evals, scan_aws_identifiers, and mkdocs build --strict pass.
  • All 13 proposed GitHub author accounts resolve.
  • Each of the 20 changed skills advances one patch version and has a matching changelog entry.

This is feedback and confirmation only. Another admin should provide the approval, as the PR description requests.

devops-agent-cost-insights reports on what AWS DevOps Agent itself costs,
so its aws-services value now leads with AWS DevOps Agent, which joins the
approved vocabulary. Also drop the Amazon Kinesis alias: the one skill that
used the broad name now uses Amazon Kinesis Data Streams directly, and the
alias could suggest the wrong name for a skill about another Kinesis service.
@miscreantmoogly

Copy link
Copy Markdown
Contributor Author

Thanks @ams-thakkar! Added AWS DevOps Agent to the vocabulary and to devops-agent-cost-insights' services (5bd891d), and dropped the Amazon Kinesis alias. I've also added #25, #230 and #246 to the overlap list. Checks: validate_skills 0 errors, self-check passes, 258 tests pass.

@miscreantmoogly
miscreantmoogly requested review from a team and ams-thakkar November 13, 2026 14:18

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants