Skip to content

Conform to Agent Plugins 1.0.0 and the Agent Skills specification - #4

Closed
jpantsjoha wants to merge 3 commits into
mainfrom
feature/googleplugin-update
Closed

Conform to Agent Plugins 1.0.0 and the Agent Skills specification#4
jpantsjoha wants to merge 3 commits into
mainfrom
feature/googleplugin-update

Conversation

@jpantsjoha

Copy link
Copy Markdown
Owner

Summary

Brings the plugin into conformance with Agent Plugins 1.0.0 — the open, vendor-neutral packaging specification announced by Google on 2026-08-05 and co-maintained by Amazon, Cursor, Google, Microsoft, OpenAI and Vercel — and with the Agent Skills specification it references.

Before this PR, a conformant Agent Plugins client would have loaded nothing from this plugin. Two independent fatal faults, neither visible to the existing gate:

# Fault Spec clause Effect
1 plugin.json omitted $schema §5.2 — closed schema, $schema in required Client rejects the plugin entirely
2 All 17 skills carried disallowed top-level frontmatter keys Agent Skills closed field set; Agent Plugins §6.1 Client skips every skill

make gate passed throughout, because it enforced this repo's own ADR-005 contract rather than the published specifications. That is the real lesson here, and the reason the validators were inverted rather than extended.


Standards this plugin now conforms to

Standard Version Artefact Authority Verified by
Agent Plugins — manifest 1.0.0 plugin.json plugin.schema.json check-jsonschema against the live schema
Agent Plugins — MCP config 1.0.0 mcp.json mcp.schema.json check-jsonschema against the live schema
Agent Skills — skill format current skills/*/SKILL.md ×17 agentskills.io/specification skills-ref, the official reference validator
MCP — transport declaration stdio mcp.json Agent Plugins §7 transport rules make spec

Normative rules now enforced in CI, quoted from the specification:

  • Manifest field set is closed — only $schema, name, version, description, author, homepage, repository, license, keywords, extensions.
  • Skill frontmatter field set is closed — only name, description, license, compatibility, metadata, allowed-tools. metadata is a map from string keys to string values.
  • Skills are discovered non-recursively at skills/*/SKILL.md.
  • Clients expand only ${PLUGIN_ROOT} and ${PLUGIN_DATA}; every other placeholder is passed through literally.
  • mcp.json $schema MUST match the version plugin.json targets.
  • Transports are discriminated on type; a server omitting it is skipped.
  • Paths MUST resolve inside the plugin root after symlink resolution.

What changed

Manifests

  • plugin.json declares the Agent Plugins manifest schema.
  • New root mcp.json — the portable MCP manifest, with an explicit "type": "stdio".
    • Ships the gcloud server only. toolbox needs --config <user-specific tools.yaml>, which no spec placeholder can express; shipping it would mean a server that fails on every fresh install. It stays a documented opt-in in skills/mcp-servers/.
    • Omits env entirely. ${GCP_PROJECT_ID} does not expand — it would have been passed through literally, setting CLOUDSDK_CORE_PROJECT to the 17-character string ${GCP_PROJECT_ID}. Clients may inherit the ambient environment, and gcloud-mcp already reads project and ADC from it.

Skills — all 17 reduced to {name, description, license, metadata}. Routing keywords fold into description, which is what a conformant client actually routes on. The plugin's contract moves to namespaced string values under metadata.

Validatorsvalidate_skills.py inverted to enforce the closed field set it previously violated. New validate_agent_plugins.py (make spec) reports against the specification's own FATAL / MCP / SKILL failure boundaries. Smoke tests 136 → 212.

Governance — ADR-007 records the decision and the alternatives weighed; ADR-005 marked superseded.


Cross-model validation

Per the repo's doctrine that neither model both writes and self-approves, the work was tested by an independent agent against a clean git archive export — what a user actually installs.

Kimi (independent harness, skills loaded via --skills-dir)

  • Discovered 17/17 skills.
  • Routed 5/5 requests correctly on the first pass, then 9/9 on a second pass designed to probe both sides of every contested pair.
  • Flagged four pairs at risk of misrouting now that routing depends on descriptions alone: gcp-ops/logging-monitoring, iam/gcp-security, vertex-ai/agent-architect, solution-designer/gcp-architect. Acted on — each description now states which skill owns the case and names the other.
  • Its two "concrete regressions" did not survive checking: the keywords it reported missing from cloud-storage and bigquery are present in the description prose, just not in the trailing keyword list. Reported here rather than silently dropped, because the underlying concern was sound even though the examples were not — a sweep found four other skills with a trigger recorded only in metadata, where nothing can route on it. make validate now fails on that condition.

Codex — could not run. The CLI returned You've hit your usage limit … try again at Aug 8th, 2026 12:43 PM, so no independent OpenAI-side audit was produced. This PR therefore has one independent reviewer, not two. Worth rerunning codex exec against the sandbox once quota resets if a second opinion is wanted before merge.


Evidence a reviewer can reproduce

make gate          # 212 tests, 101 URLs, 4/4 harnesses installable

# Independent — validates against upstream, not against this repo's transcription
for s in skills/*/; do
  uvx --from 'git+https://github.com/agentskills/agentskills.git#subdirectory=skills-ref' \
      skills-ref validate "$s"
done                                                        # -> 17/17 valid

uvx check-jsonschema --schemafile https://agent-plugins.org/schemas/1.0.0/plugin.schema.json plugin.json
uvx check-jsonschema --schemafile https://agent-plugins.org/schemas/1.0.0/mcp.schema.json mcp.json

The gate was also negative-tested: dropping $schema, removing a transport type, reintroducing a non-expanding ${...} placeholder, and re-adding a legacy top-level triggers field are each caught with an actionable message.


Compatibility

No breaking change for existing users. The specification closes the plugin.json field set and fixes component locations, but says nothing about sibling directories — so .claude-plugin/, .kimi-plugin/, gemini-extension.json, .mcp.json, mcp_config.json and .agents/ are untouched. make manifest confirms Claude Code, Antigravity/Gemini, Codex and Kimi all remain installable.

Agent Plugins 1.0.0 deliberately specifies no install mechanism, distribution protocol, permission model or sandboxing, so marketplace packaging stays harness-specific for now.

Follow-ups (not in this PR)

  • .claude/CLAUDE.md is tracked in git despite the file itself stating it is gitignored and never committed. The internal construction guide currently ships to the public repo and to anyone installing the plugin. Worth a decision: untrack it, or drop the claim.
  • Skill metadata/version (0.1) and plugin version (0.1.0) drift. Harmless, but they read as the same field.

Two independent fatal faults meant a conformant Agent Plugins client would
load nothing from this plugin:

1. plugin.json omitted the required $schema. It sits on a closed schema, so
   §5.2 makes the omission fatal — the client rejects the plugin outright.
2. All 17 skills carried top-level frontmatter keys (version, triggers,
   required_scopes, mcp_servers, persona, tier, gate, owns_eval) outside the
   closed Agent Skills field set. Agent Plugins §6.1 requires clients to skip
   every non-conforming skill.

Changes:
- plugin.json declares the manifest schema.
- New root mcp.json: the portable MCP manifest, with an explicit stdio
  transport discriminator. It ships the gcloud server only — toolbox needs a
  user-specific tools.yaml that no spec placeholder can express, so it stays a
  documented opt-in. env is omitted entirely: only ${PLUGIN_ROOT} and
  ${PLUGIN_DATA} expand, so ${GCP_PROJECT_ID} would have been passed through
  literally and mis-set the project.
- All 17 SKILL.md frontmatters reduced to {name, description, license,
  metadata}. Routing keywords folded into description, which is what a
  conformant client actually routes on; the plugin's own contract moves to
  namespaced string values under metadata.
- validate_skills.py inverted: it now enforces the closed Agent Skills field
  set instead of requiring the fields that broke conformance.
- New validate_agent_plugins.py: dependency-free conformance check wired in as
  `make spec`, reporting against the spec's own FATAL/MCP/SKILL failure
  boundaries.
- Smoke tests rewritten: 136 -> 195, covering both specs.

Verified against the official reference validator (agentskills/agentskills
skills-ref) — 17/17 valid — and both published JSON schemas via
check-jsonschema. All four harnesses remain installable.
Governance:
- ADR-007 records the decision, the alternatives weighed, and the two fatal
  faults that prompted it. ADR-005 marked superseded — its contract survives,
  only its location changed.
- SPEC.md, shared/conventions.md, plugin.yaml, .claude/CLAUDE.md and the
  coverage matrix updated to the new frontmatter shape.
- README gains a standards-conformance section with commands a reader can run
  to reproduce the evidence independently.

Routing, from cross-model testing:

An independent Kimi agent loaded the plugin from a clean sandbox export and
routed five requests against it. All 17 skills were discovered and all five
routed correctly, but it flagged pairs that could misroute now that routing
depends on descriptions alone: gcp-ops/logging-monitoring, iam/gcp-security,
vertex-ai/agent-architect, solution-designer/gcp-architect.

Its two "concrete regressions" did not survive checking — the keywords it
reported missing from cloud-storage and bigquery are present in the description
prose, just not in the trailing keyword list. But four other skills did have a
trigger recorded only in metadata, where nothing can route on it.

So each overlapping pair now states which skill owns the case and names the
other, and `make validate` fails if any metadata trigger is absent from its
description — keeping that index a derived view rather than a rival source of
truth.

Also: author URL now points at jpantsjoha.com. The schema permits one author
URL, and an owned domain outlives any platform profile.

make gate: 212 tests, 101 URLs, all four harnesses installable.
Adds validate_agent_plugins.py as a blocking CI step — it is deterministic and
offline, since it transcribes the published schemas rather than fetching them.

Adds an advisory step that validates against UPSTREAM instead: the official
skills-ref reference validator and the live JSON schemas. That is what catches
spec drift, but it depends on the network, so it warns rather than blocks.
@jpantsjoha

Copy link
Copy Markdown
Owner Author

Closing — fully superseded, all additive content extracted

Reviewed against main @ 5546f67. This branch's unique contributions are all now in main, byte-identical:

Extracted to What
#8 .github/workflows/gate.yml — the conformance CI step, and the upstream reference-validator drift check
#9 scripts/validate_skills.py + tests/… — the routable-triggers guard
#9 skills/mcp-servers/SKILL.md — the "ships vs opt-in" section

Verified: git diff origin/main origin/feature/googleplugin-update reports no difference on any of those four files.

Why merge rather than close is not an option

The remaining delta is −969 lines, and all of it is regression against work merged in #6#11:

  • Deletes CHANGELOG.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md, all three issue templates, the PR template, and skills/mcp-servers/references/agent-plugins-mcp.md
  • Reverts version 0.2.0 → 0.1.0 across the manifests, which would break the v0.2.0 tag and client cache invalidation
  • Restores the make manifest message claiming the plugin is "installable across all 4 harnesses" — corrected in ci: actually enforce Agent Plugins conformance in the pipeline #8, because that check only verifies manifests parse and agree; it installs nothing
  • Reverts the routing disambiguation clauses on eight skills, validated in Antigravity (6/6) and Kimi (7/7)

Credit where it is due

This branch caught something the merged line missed: CI never invoked validate_agent_plugins.py. The workflow calls each script directly rather than running make gate, so adding spec to the Makefile target never reached CI. The README claim that conformance was "enforced in CI" was false until #8 landed this branch's fix.

It also identified that a trigger recorded only in metadata is unroutable — nothing reads metadata, clients match on description alone — which is now a gate check with a negative test (#9).

Two real defects, neither found by the main line of work. Closing as superseded, not as wrong.

@jpantsjoha jpantsjoha closed this Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants