Repository navigation
[Security] iam-access-navigator: IAM risk review, access-key hygiene and drift detection (skill + custom agent + MCP) - #200
Open
jgq-amazon wants to merge 5 commits into
Open
jgq-amazon wants to merge 5 commits into
jgq-amazon wants to merge 5 commits into
Conversation
Add a read-only IAM review bundle for AWS DevOps Agent: - skills/iam-access-navigator: IAM risk triage, blast radius, least-privilege synthesis, access-key hygiene, policy validation, rollback planning and change forensics, with skill-eval results. - custom-agents/iam-access-navigator: interactive agent plus scheduled access-key hygiene and IAM drift jobs. - mcp/iam-drift-diff: read-only MCP server that captures IAM snapshots and computes an exact, deterministic diff between them (SigV4 Lambda Function URL, private KMS-encrypted snapshot bucket, access logging). No component writes to IAM.
This PR is on hold — checklist incompleteRequired items in the PR description are unchecked. A maintainer will not review this PR until each one is checked and actually done. The check's summary lists them. Complete the work, tick the boxes, and this clears automatically. CONTRIBUTING.md explains each item. |
20 of 26 tasks
…ty review - Roles under /service-role/ are fully fingerprinted and diffed; only /aws-service-role/ (service-linked) roles are presence-only. Snapshot schema 2; the first diff after upgrading reports each affected role once as fingerprint_scope_changed (direction info). - Baseline pointer reads fail closed: only "not found" means no baseline; other errors return a retryable error with nothing written, pruned or compared. An invalid pointer, or one naming deleted snapshots, diffs from the oldest retained snapshot. - Trust high_risk tracks the condition keys guarding each principal, flags new unconditioned Federated principals and removed trust Deny statements. - Policy-body diffs render NotAction/NotResource and report denies_added / denies_removed. - Every S3 call passes ExpectedBucketOwner; the execution role's S3 statement requires aws:ResourceAccount. - Callers get AWS error codes only; refused and failed calls write an audit line; a malformed request-context header cannot fail a tool. - Dedicated log group with LogRetentionDays (default 365). - Drift prompt updated for the new fields; README and CHANGELOG updated. - 87 unit tests (was 54), including execution-role scope and account-gate checks.
The invoke role has a fixed RoleName, so CloudFormation requires CAPABILITY_NAMED_IAM. README deploy command and samconfig.toml updated.
… rule From a live drift round on 1.0.1: - A grant that gains a condition is reported in grants_conditioned (direction down) and one that loses a condition in grants_unconditioned (up), instead of as a removed plus an added grant (mixed). The 1.0.1 execution-role update itself was reported as mixed and drew a recommendation for a tightening. - Drift prompt: a single authoritative recommendation rule, and the run must end with the full report even after updating an artifact. - 89 unit tests.
…ey match a change From drift round 2: one run mentioned an unrelated open recommendation and the other did not. The prompt now says to mention an existing open recommendation only when it matches a change in the current diff.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Read-only IAM review for AWS DevOps Agent. The skill triages IAM risk, rates the blast radius of a change, generates least-privilege policies from observed CloudTrail activity, reviews access-key hygiene with an identity-aware migration path, validates policies, and plans a safe rollback, teaching as it works. The custom agent packages the skill for interactive use and runs two scheduled jobs: an access-key hygiene sweep and an IAM drift check. The iam-drift-diff MCP server gives the drift job something the agent cannot do reliably on its own: hold a prior IAM state and compute an exact, complete set-difference across the whole account. In prompt-only testing the agent missed a real policy detach twice and once reported "no drift" with a false coverage claim; the server does the comparison as deterministic set math and returns machine-readable completeness flags.
The MCP reads control-plane data that
use_awscan also reach. The gap it fills is computation and persistence, not data access; see "Why not just call the AWS APIs?" inmcp/iam-drift-diff/README.md. If reviewers rule that out, the skill and agent stand on their own and the drift job can be dropped.Related work. DevOps Agent's built-in proactive prevention produces cross-service operational recommendations; the scheduled jobs here build on that recommendations surface rather than replace it, are IAM-specific, and file a recommendation only on a risk-increasing change. The
drift-detectioncustom agent proposed in #26 is a different kind of drift: it compares resource configuration (encryption, tags, lifecycle) against a fixed baseline and does not examine IAM principals, policies or trust, whileiam-drift-diffdiffs IAM state against its own previous snapshot. Neither reuses the other's checks.Type of change
Required before review
/skills, MCP servers in/mcp, custom agents in/custom-agents; skills and MCP servers are never mixed in one directory.Testing
structurechecks pass. 12/12 (skills/iam-access-navigator/evals/structure/structure-tests-results-v8.json).best-practicesrun across multiple iterations, with consistent results. v8: 100% on all 3 iterations (14 passed, 3 skipped, 0 failed). An earlier run on a previous revision (v4) failed BP-03 once on 1 of 3 iterations at medium confidence; v8 is the committed result.functionalrun with and without the change, so the difference in the agent's behavior is visible. v3, 10 evals x 3 iterations, 54 runs, 0 failed. Assertions: with skill 55/72 (76%), without 43/75 (57%); one with-skill run produced no final response, so 3 of its assertions were not scored. Known weak spots: the "make an IAM change" request does not load the skill (0/3; the base agent declines safely anyway); the forensics eval loads it 2 times in 3; and on the session-name eval the with-skill expected output passed 1/3 vs 3/3 without (one run had no final response, one investigated the account instead of explaining the trap).evals/evals.jsonincludes positive cases (should trigger) and at least one negative case (should not trigger on unrelated input). 8 positive, 2 negative (RDS Performance Insights, EKS node groups).Also validated end to end in a test DevOps Agent space: interactive sessions, both schedule triggers, recommendations and the "IAM Drift Report" artifact, and the MCP over SigV4. Drift tests caught policy attach/detach, inline add/change/remove with exact grants, trust changes with
high_risk, and new principals; stayed quiet on cosmetic reordering and AWS-managed churn; an ad-hoc diff between scheduled runs did not hide a change from the next report; and an instruction planted in a policy condition value was quoted as evidence and flagged, not followed. Least-privilege was also live-validated on one clean invocation per case: a role granted only S3 object actions kept them and flagged them as unverified (S3 object calls are data events, not visible toLookupEvents) instead of calling the role unused, and a zero-activity role without S3 grants was still reported as unused (a one-hour-old fixture). MCP unit tests: 54 passed.sam validate --lint: valid.Skills — complete if this PR adds or changes a skill
SKILL.mdfrontmatter is valid:namematches the directory,descriptionpresent and within length limits, body at most 500 lines.descriptionis written from the agent's point of view — scenarios, services, error types, symptoms.metadata.aws-devops-agent-skills.agent-types(INCIDENT_RCA,INCIDENT_TRIAGE,INCIDENT_MITIGATION, …), not leftGENERICwithout reason.CHAT, PREVENTION.references/orassets/and loads on demand.MCP servers — complete if this PR adds or changes an MCP server
AuthType: AWS_IAM.iam:GetAccountAuthorizationDetailsonly, object actions on its own bucket prefix, and its own KMS key.ALLOWED_ACCOUNTS(explicit, no wildcard) is checked at startup and on every call.template.yaml(SAM) orcdk.*(CDK),src/, andtests/are present and tests pass.Custom agents — complete if this PR adds or changes a custom agent
SYSTEM_PROMPT.mdis present and is what gets pasted into the agent configuration.Security and safety
License confirmation