Repository navigation
fix(skills): clean up filter tags and author names - #245
miscreantmoogly wants to merge 5 commits into
Conversation
Catalogs show each aws-services and technical-domains value as written, so a placeholder or a near-duplicate becomes its own filter option, and a skill without tags drops out of the filters that should list it. - investigation-cost-guardrail 2.1.2: remove aws-services "All". It isn't a service; a skill that applies to every service leaves the key out. - devops-agent-cost-insights 1.0.2: use "Amazon CloudWatch, AWS CloudTrail", and drop "AIDevOps". - database-migration-service-expertise 1.0.4: "Database" becomes the approved "Databases". - aiml-access-diagnostics 1.2.5: add the AI/ML domain, and name Amazon SageMaker AI, whose training jobs and endpoints the skill diagnoses. - aws-eks-operations-review 1.9.7, aws-eks-healthdashboard 1.0.1, ecs-operation-review 2.5.4, database-rds-devops 1.0.3 and analytics-opensearch-expertise 2.6.3: add the missing service and domain tags. - live-streaming-event-readiness-review 1.0.1: "Amazon Kinesis" becomes "Amazon Kinesis Data Streams", the service the skill reads. Each skill gets one patch bump and a CHANGELOG entry. Keys are removed rather than left empty, because an empty value is an error.
…n Keyspaces and Amazon Kinesis Data Streams Add Amazon EBS, Amazon FSx for NetApp ONTAP, Amazon Keyspaces and Amazon Kinesis Data Streams to the approved aws-services values. Skills already use all four (live-streaming-event-readiness-review uses Kinesis Data Streams after the previous commit), so the check stops warning about them. Add the alias "Amazon Kinesis" -> "Amazon Kinesis Data Streams", so the check suggests the specific service name instead of the family name. Amazon CloudFront and AWS Elemental MediaTailor are left to #188, which adds them. The file isn't published, so no skill version changes.
Catalogs link metadata.author to the GitHub profile of that name. These eleven skills name an account that doesn't exist, or a different account from the one that contributed the skill, so the link 404s or opens the wrong profile. Each now names the account that opened the skill's original pull request: - agentcore-observability-setup 1.0.3: vipulg1993 - analytics-opensearch-expertise 2.6.3: genedragon, in the README's Maintainers list too - cloudtrail-cost-optimization 1.3.3, config-cost-optimization 1.5.4 and guardduty-cost-optimization 1.3.3: kaiserschmarm - msk-operations 1.0.6: kalyanjanaki - firehose-operation-review 1.4.4: shinutharol2ext - aiml-access-diagnostics 1.2.5: ams-thakkar - storage-s3-resiliency-expertise 1.0.4: howard-m-k - bedrock-operation-review 1.0.3: sean-nixon - database-migration-service-expertise 1.0.4: abhit-cloud analytics-opensearch-expertise, aiml-access-diagnostics and database-migration-service-expertise keep the version the tag cleanup gave them, so each skill moves one step in this change. The others get a patch bump. Every skill gets a CHANGELOG line.
As in the previous commit, each skill now names the GitHub account of the person who maintains it: - devops-agent-cost-insights and investigation-cost-guardrail: the co-author tqquresh becomes tipuq. No GitHub account is named tqquresh; tipuq is the co-author's account, and merged #40 and #61 on the cost guardrail. Both skills keep the version the tag cleanup gave them. - service-quota-check 1.0.2: yuriypr becomes yprikhodko, the account that opened the skill's pull request. - redshift-support-specialist 1.8.3: aws-samples becomes andrehass. aws-samples is an organization; andrehass opened the skill's pull request and wrote nearly all of its commits. Each skill gets a CHANGELOG line.
ams-thakkar
left a comment
There was a problem hiding this comment.
What needs to change
- Keep the skill's primary service in its catalog tags.
skills/devops-agent-cost-insights/SKILL.md:3is specifically about the cost of AWS DevOps Agent, but line 8 now lists only CloudWatch and CloudTrail. RemovingAIDevOpsis right because it is a metric namespace, not the service name. AddAWS DevOps Agenttovocabulary.jsonand useAWS DevOps Agent, Amazon CloudWatch, AWS CloudTrailhere, so the catalog can find this skill by its actual subject.
Nits (non-blocking)
- Drop the
Amazon Kinesis→Amazon Kinesis Data Streamsalias fromvocabulary.json:67. The changed skill now uses the precise name directly, and the broad alias could give the wrong correction for a family-level or Kinesis Video Streams skill later. - The open-overlap list misses #25, #230 and the stacked draft #246, which also touch
vocabulary.json. Whichever merges second needs a rebase and another self-check.
Thanks
I confirm the aiml-access-diagnostics row you asked me about: author: ams-thakkar, service Amazon SageMaker AI, and domains Security, AI/ML are all correct.
The Redshift change to andrehass is reasonable: aws-samples identifies the source project/organization, not an individual owner.
@miscreantmoogly, the remaining filter cleanups are consistent with the skills' scopes, and the author mappings match the original PR openers. I also confirmed that the catalog renders tipuq, inesttia as two separate profile links.
What I verified on a trial merge with current main:
- All 271 unit tests pass.
validate_skills,validate_skill_evals,scan_aws_identifiers, andmkdocs build --strictpass.- All 13 proposed GitHub author accounts resolve.
- Each of the 20 changed skills advances one patch version and has a matching changelog entry.
This is feedback and confirmation only. Another admin should provide the approval, as the PR description requests.
devops-agent-cost-insights reports on what AWS DevOps Agent itself costs, so its aws-services value now leads with AWS DevOps Agent, which joins the approved vocabulary. Also drop the Amazon Kinesis alias: the one skill that used the broad name now uses Amazon Kinesis Data Streams directly, and the alias could suggest the wrong name for a skill about another Kinesis service.
|
Thanks @ams-thakkar! Added |
Description
Skill catalogs build their filters from each skill's
aws-servicesandtechnical-domainsvalues, exactly as written, and linkmetadata.authorto the GitHub profile of that name. Some of these values are wrong: anAllservice,Databasenext toDatabases, skills with no tags, and authors whose links 404 or open the wrong profile. Parts 2 to 5 below are separate commits, so each can be reviewed on its own. Parts 3, 4 and 4b can each be dropped too (4b with a one-line CHANGELOG fix); part 2 can't, because parts 4, 4b and 5 reuse its version bumps.1. Connectors: already fixed in #234. Skills and connectors on
mainare released together, and one change that breaks the release rules holds back everything else. Two connector changes had done that: Elastic's logos changed at a version that was already released, and Coralogix changed an auth field's label without a major version bump. So the connector fix had to come first; until it merged, nothing here could be released. #234 moved Elastic to 1.0.1 and Coralogix to 2.0.0, so this PR leaves connectors alone, and its changes are released once it merges.2. Filter values (
fix(skills): clean up AWS service and domain values)aws-services: "All". It isn't a service, and a skill that applies to every service leaves the key out.CloudWatch, CloudTrail, AIDevOps→AWS DevOps Agent, Amazon CloudWatch, AWS CloudTrailDatabase→DatabasesSecurity→Security, AI/ML;Amazon SageMaker→Amazon SageMaker AIAmazon EKSandContainers, OperationsContainers, OperationsAmazon EKSandContainers, ObservabilityAmazon RDSandDatabasesAmazon OpenSearch ServiceandAnalyticsAmazon Kinesis→Amazon Kinesis Data Streams, the service it readsOwners, please say if a value is wrong for your skill.
3. Vocabulary (
chore(vocabulary): add Amazon EBS, Amazon FSx for NetApp ONTAP, Amazon Keyspaces and Amazon Kinesis Data Streams). AddsAmazon EBS,Amazon FSx for NetApp ONTAP,Amazon KeyspacesandAmazon Kinesis Data Streams, which skills already use, plus anAmazon Kinesis→Amazon Kinesis Data Streamsalias that part 5 removes again. Part 5 also addsAWS DevOps Agent, so the net change is five values and no alias.Amazon CloudFrontandAWS Elemental MediaTailorare left to #188, which adds them. The file isn't published, so no versions change. It's under/.github/, so it needs an admin's approval.4. Authors (
fix(skills): set each skill's author to their GitHub username). Each new name is the account that opened the skill's original pull request. Please check your row, and say so here if you'd rather be listed differently.vggargav→ @vipulg1993genealpe→ @genedragon, in the README's Maintainers list tooholmalla→ @kaiserschmarmholmalla→ @kaiserschmarmholmalla→ @kaiserschmarmkjjanaki→ @kalyanjanakistharolz→ @shinutharol2exttamrish→ @ams-thakkarhokang→ @howard-m-ksmnixon→ @sean-nixontabhiman→ @abhit-cloud4b. Authors still to confirm (
fix(skills): set four more skills' authors to their GitHub usernames). It's a separate commit, so it can be dropped, or a row changed, if the answer is no. Dropping it needs a one-line fix in devops-agent-cost-insights' 1.0.2 CHANGELOG. A no on the first row meansinesttiaalone, nottqqureshagain, because that account doesn't exist.tqquresh, inesttia→tipuq, inesttiatipuqstay a co-author on these two skills?yuriypr→ @yprikhodkoyprikhodkorather thanYuriyPr?aws-samples→ @andrehassaws-samplesbe replaced with your username?5. Review follow-up (
fix(skills): tag devops-agent-cost-insights with AWS DevOps Agent). From ams-thakkar's review: devops-agent-cost-insights now leads withAWS DevOps Agent, which joins the vocabulary, and part 3'sAmazon Kinesisalias is dropped. It reuses part 2's 1.0.2.Every skill moves one version step from
main, even when two parts change it, and gets a CHANGELOG line. Organizations and teams stay as authors (coralogix,honeycombio/agentic-intelligence), because their links work.Review. @ams-thakkar and @yprikhodko, please confirm your own rows in a comment. Since this writes both of your names into bylines, the approving review should come from another admin, such as @shyamkulkarni, @SimonCMoore or @udid-aws.
Open PRs that overlap. Please rebase after this merges and re-run the checks:
ams-thakkaras the author, addAI/ML, and useAmazon SageMaker AIrather thanAmazon SageMaker.Amazon CloudFront) and Add sustainability-operation-review skill #230 (@rrrazvan, addsAWS Sustainability) also editvocabulary.json: whichever merges second rebases and re-runs the checks.vocabulary.json: it rebases after this merges and re-runs the self-check.Not in this PR.
AI/MLorMachine Learning), and checks that stop these values coming back. Both are separate follow-ups.Type of change
Required before review
/skills, MCP servers in/mcp, custom agents in/custom-agents; skills and MCP servers are never mixed in one directory.validate-skill-evalspasses.Testing
Run on this branch against
main, with the commands the workflows use:python3 .github/scripts/validate_skills.py --base-ref origin/main --head-ref HEAD: 100 skills, 0 errors. On the skills this touches, the warnings left are older ones (missingmetadata.summary, msk-operations' body length, a<script>in a redshift-support-specialist template) andAmazon CloudFrontandAWS Elemental MediaTailor, which Add the AWS Elemental Media Services and CloudFront to the skill vocabulary #188 adds. Each commit passes on its own as well.python3 -I -m unittest discover -s tests -v: 258 tests pass.python3 .github/scripts/validate_skill_evals.py --base-ref origin/main --head-ref HEAD: 0 failures. 13 of the 20 skills are legacy and only warn.python3 .github/scripts/scan_aws_identifiers.py --base-ref origin/main --head-ref HEAD: no findings.mkdocs build --strict: passes.gh api users/<name>finds every new author.structurechecks pass. n/a: metadata-only change;validate-skillschecks the frontmatter, and the instructions the evals test don't change.best-practicesrun across multiple iterations, with consistent results. n/a: no skill's instructions change.functionalrun with and without the change, so the difference in the agent's behavior is visible. n/a: no skill's instructions change, so the agent's behavior doesn't either.evals/evals.jsonincludes positive cases (should trigger) and at least one negative case (should not trigger on unrelated input). n/a: noevals.jsonchanges.Skills — complete if this PR adds or changes a skill
SKILL.mdfrontmatter is valid:namematches the directory,descriptionpresent and within length limits, body at most 500 lines. n/a: the frontmatter is valid in every skill here (validate-skills: 0 errors), but msk-operations' body was already 569 lines, and this PR changes no skill body.descriptionis written from the agent's point of view — scenarios, services, error types, symptoms. n/a: no description changes.metadata.aws-devops-agent-skills.agent-types(INCIDENT_RCA,INCIDENT_TRIAGE,INCIDENT_MITIGATION, …), not leftGENERICwithout reason. n/a: no agent types change.references/orassets/and loads on demand. n/a: no files are added.<script>element (a warning onmain), which this PR doesn't change.Security and safety
License confirmation