Add VPC DNS diagnostics MCP - #37
Open
ddericco wants to merge 9 commits into
Open
Conversation
Adds mcp/aws-vpc-dns-diagnostics-mcp, an MCP server for AWS DevOps Agent that diagnoses VPC DNS resolution two ways no describe API can. Mode A (dns_probe_*) runs live, comparative, multi-resolver DNS resolution inside an EC2 instance via SSM Run Command, returning what a name actually resolves to from that subnet, which resolver answered (hostname.bind), and how a custom or hybrid resolver diverges from the VPC resolver. The EC2, VPC, and Route 53 describe APIs return DNS configuration; they never return that ground truth. Mode B (dns_simulate_*) is read-only pre-change validation. It builds the VPC's effective configuration as the union of directly attached and Route 53 Profile-inherited constructs, applies a proposed change symbolically, and reports which currently-resolving names would break. A seven-level precedence engine and six trap detectors (VPCE-shadow-NXDOMAIN, broad-FORWARD-sweep, flag-AND-mismatch, DNS-Firewall-block, Profile-union-shift, resolver-disabled) name the mechanism rather than only the affected names. list_sops and get_sop serve 16 diagnostic runbooks bundled in the deployment package, so the agent fetches interpretation guidance at runtime. Transport and auth follow the documented DevOps Agent path: FastMCP over Streamable HTTP via Lambda Web Adapter, exposed on a Lambda Function URL with AuthType AWS_IAM and RESPONSE_STREAM, registered with the mcpserversigv4 service type. No AgentCore Gateway or OAuth front end is required. Security model. The central Lambda execution role holds only sts:AssumeRole and assumes one of two per-target-account scoped roles per call. Mode B's role never holds ssm:SendCommand. The probe role's sole privileged grant is ssm:SendCommand resource-scoped to one purpose-built SSM document that accepts three allowedPattern-validated parameters and renders a fixed read-only probe set (dig, cat /etc/resolv.conf, resolvectl, getent). The server sends structured parameters, never a command string. Account, region, VPC, and resolver allowlists are enforced in one place, and the server refuses to start on a wildcard when StageName=prod. There are no credentials to store, so Secrets Manager is not used; the function needs no VPC attachment because it contacts only AWS control-plane APIs. Both deviations from the common reference pattern are documented in the README. Cross-account constructs shared via RAM or contributed through a Route 53 Profile may be enumerable but opaque. Denied detail reads become OPAQUE markers rather than failing the model build, and an opaque firewall rule is evaluated first because a hidden block list may cover any name. Reporting says 'cannot determine from this account' rather than 'not affected'. Validation. 77 unit tests, no AWS calls. Deployed to a test account and drove all six tools over a SigV4-signed Function URL: unsigned requests return 403, signed return 200 with an mcp-session-id; Mode A reproduced a custom-resolver divergence (10.42.200.99 via a local resolver vs 10.42.200.10 from the PHZ, with getent following the custom resolver); Mode B read the live effective config and correctly predicted a broad '.' FORWARD rule would sweep seven names. Live testing surfaced three defects since fixed: a dependency layer built for the build host's platform rather than manylinux2014_x86_64, two missing EC2 read grants for DHCP discovery, and add_resolver_rule ignoring the documented target_ips shape. test-infra/ holds CloudFormation fixtures that reproduce each diagnostic scenario end to end, including a two-account provider/consumer pair for the cross-account opacity cases. Licensed under Apache-2.0.
Security review callout: with no AllowedAccounts value the deployment defaulted
to '*', which is an implicit allow-all account scope. For a DevOps Agent
integration the account allowlist is the boundary that stops the server assuming
a role into an arbitrary account, so it must be stated explicitly.
Three separate routes reached allow-all, and the template default was only one of
them:
- deploying without overriding the AllowedAccounts parameter
- passing '*' explicitly
- the env var being absent, because _load_allowlist() fell back to '*'
Changing only the template would have left the other two open, and the existing
_enforce_prod_allowlists() gate covers wildcards solely when STAGE_NAME=prod,
leaving a dev or staging deployment attached to a real Agent Space unprotected.
template.yaml: AllowedAccounts loses its Default and gains
AllowedPattern '^[0-9]{12}$' with a ConstraintDescription, so CloudFormation
rejects a missing or malformed value per list item before the function boots.
Confirmed via validate-template that DefaultValue is now null.
src/server.py: new _enforce_account_allowlist() runs at import, unconditionally
and in every stage. It refuses unset, empty, whitespace-only, any entry equal to
'*', and any entry that is not a 12-digit account ID. Because it runs before the
allowlists load, _load_allowlist()'s '*' fallback is unreachable for accounts and
_validate()'s allow-all-on-empty branch cannot be hit for accounts either. Both
docstrings now record that invariant.
Regions, VPCs, and resolvers deliberately keep their permissive defaults.
Accounts are known in advance; requiring ALLOWED_VPCS up front would force a
redeploy to diagnose a new VPC. Those three stay guarded by the prod gate, and
resolvers additionally by literal-IP-only fail-closed behaviour plus a startup
warning.
Docs: README parameter table, Controls table, and Safety blocked list;
ARCHITECTURE fail-closed table.
Tests: 88 passing, up from 77. Eleven new tests in TestAccountAllowlistRequired
cover unset, empty, whitespace-only, wildcard, wildcard mixed with a real
account, malformed, the prod-stage belt-and-braces case, three valid forms, and a
guard asserting template.yaml never regains a Default. Verified the new tests
fail when the fix is reverted.
Also removes a duplicate _enforce_prod_allowlists() call left at module level by
the edit that inserted the new gate. Harmless but confusing; caught in review.
Verified: security review re-run returns PASS with no findings; all five earlier
findings confirmed still cleared; no bypass found on the account path and no tool
reaches an AWS API without passing _preflight(). Redeployed to a test account and
confirmed the tools still work end to end.
ddericco
force-pushed
the
feature/aws-vpc-dns-diagnostics-mcp
branch
from
July 31, 2026 14:50
a7a76e9 to
f636f56
Compare
…ivation, validate change fields, remove incorrect EICE reference
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
mcp/aws-vpc-dns-diagnostics-mcp, an MCP server for AWS DevOps Agent that diagnoses VPC DNS resolution in beyond that the AWS APIs can determine.dns_probe_*(Mode A): runs comparative, multi-resolver DNS resolution inside an EC2 instance via SSM Run Command. It returns what a name actually resolves to from that subnet, which resolver answered (hostname.bind), and how a custom or hybrid resolver diverges from the VPC resolver. The EC2, VPC, and Route 53 describe APIs return DNS configuration but do not have visibility into what a client actually sees.dns_simulate_*(Mode B): read-only, pre-change validation. It builds the VPC's effective config as the union of directly attached and Route 53 Profile-inherited constructs, applies a proposed change symbolically, and reports which currently-resolving names would break. A seven-level precedence engine and six trap detectors name the mechanism rather than just the affected zones: VPCE-shadow-NXDOMAIN, broad-FORWARD-sweep, flag-AND-mismatch, DNS-Firewall-block, Profile-union-shift, resolver-disabled.list_sops/get_sop: serves 16 diagnostic runbooks bundled in the deployment package, so the agent fetches interpretation guidance at runtime rather than carrying it in context.Companion skill to help the agent decide when to invoke and define a consistent investigation sequence
Transport follows the documented DevOps Agent path: FastMCP over Streamable HTTP via Lambda Web Adapter, on a Lambda Function URL with
AuthType: AWS_IAMandRESPONSE_STREAM, registered with themcpserversigv4service type. No AgentCore Gateway or OAuth front end required.Type of change
This is a new MCP server under
mcp/. There is no checkbox for that, so ticking the closest option.Testing
Unit: 88 tests, no AWS calls. Cross-account denial paths use fake sessions raising real botocore exceptions.
sam buildandsam validate --lintboth clean.Live testing: deployed and drove all six tools over a SigV4-signed Function URL. Unsigned requests return 403; signed return 200 with an
mcp-session-id. Mode A reproduced a custom-resolver divergence (10.42.200.99from a local resolver vs10.42.200.10from the PHZ, withgetentfollowing the custom resolver). Mode B read the live effective config and correctly predicted a broad.FORWARD rule would sweep seven names.Security review: passed the MCP security review skill. Five findings from the first pass were remediated and a re-review returned PASS with no findings and no new issues. Guard tests assert the IAM grants cannot silently re-widen.
Agent Skill Eval:
report.jsonshows"passed": trueat the root, overall score 0.96, with audit 100/100, functional 0.9 (grade A), and trigger passing. Six functional evals and eight trigger queries, six of them negative. Artifacts committed underskills/aws-vpc-dns-investigation/evals/.Notes:
test-infra/holds CloudFormation fixtures that reproduce each diagnostic scenario end to end, including a two-account provider/consumer pair for the cross-account opacity cases. Let me know if these should be removed.License confirmation