feat: add opt-in OAuth authentication for MCP

docs/mcp.md

@@ -4,13 +4,60 @@ The Data API exposes commands as MCP tools over Streamable HTTP. Use `tools/list
 
 ## Endpoint and requests
 
-The default endpoint is `/v1/mcp`. It follows the HTTP and MCP root paths described in the [configuration guide](../CONFIGURATION.md#mcp-logging). Keep authentication paths aligned when changing the endpoint.
+The default endpoint is `/v1/mcp`. It follows the HTTP and MCP root paths described in the [configuration guide](../CONFIGURATION.md#mcp-logging). The server derives the MCP authentication paths from that configuration.
 
 Tool execution is disabled by default. Enable it with `stargate.feature.flags.mcp=true`, or send `Feature-Flag-mcp` with value `true` when the server has no configured override. This flag does not grant database permissions.
 
 Current tests send protocol version `2026-07-28` and call tools without initialization or a session ID. Each request carries its own protocol metadata. [McpHttpTestSupport](../src/test/java/io/stargate/sgv2/jsonapi/api/v1/mcp/McpHttpTestSupport.java) defines the tested headers and JSON-RPC envelope.
 
-The current implementation uses the existing Data API `Token` header. A missing token returns HTTP 401 before tool execution. OAuth connection and migration instructions remain separate work in [#2643](https://github.com/stargate/data-api/issues/2643). These tests do not establish compatibility with every MCP client or protocol revision.
+The existing Data API `Token` and `X-Cassandra-Token` headers remain supported on ordinary REST endpoints when header authentication is enabled. MCP retains that behavior until OAuth mode is enabled. Missing credentials return HTTP 401 before tool execution. These tests do not establish compatibility with every MCP client or protocol revision.
+
+## OAuth authentication
+
+OAuth mode is disabled by default. Enabling it makes the MCP transport accept only Bearer access tokens and leaves ordinary REST authentication unchanged. Configure the public MCP origin, authorization server, and JWT trust settings. For the default endpoint:
+
+```properties
+stargate.auth.bearer.enabled=true
+stargate.feature.flags.mcp=true
+stargate.mcp.auth.public-base-url=https://api.example.com
+stargate.mcp.auth.server-url=https://auth.example.com
+mp.jwt.verify.issuer=https://auth.example.com
+mp.jwt.verify.publickey.location=https://auth.example.com/.well-known/jwks.json
+mp.jwt.verify.audiences=https://api.example.com/v1/mcp
+```
+
+Use the issuer and verification key published by your authorization server. A local public-key file can replace the JWKS URL. Bearer authentication requires an explicit issuer, verification key, and audience at startup. With MCP enabled, the audience must include the public MCP resource URL.
+
+The public base URL contains the scheme, host, and optional port. The server derives the MCP resource path from its HTTP and MCP root configuration. For example, HTTP root `/service` and MCP root `/tools` require audience `https://api.example.com/service/tools`. OAuth mode requires a dedicated MCP path beneath the HTTP root. Root-only configurations are rejected at startup because the current framework does not keep those transport routes within application authentication. Public discovery requires MCP to be enabled in server configuration. Its metadata is available without credentials at `/.well-known/oauth-protected-resource`, under the HTTP root when configured, and is absent when Bearer authentication or MCP is disabled.
+
+Unauthenticated MCP requests receive HTTP 401 with an absolute `WWW-Authenticate: Bearer resource_metadata="..."` URL. Send `Authorization: Bearer <access-token>` on every subsequent HTTP request, including initialization for older protocols and catalog requests. The validator checks the token before tool execution. Legacy MCP headers and invalid Bearer credentials cannot fall back to legacy authentication in OAuth mode. When OAuth mode is disabled, legacy MCP authentication keeps its existing precedence. An MCP access token alone does not authenticate an ordinary REST request.
+
+REST and MCP resolve the same authenticated credential. A JWT's encoded token is retained separately from its subject or client ID. Tool names, arguments, and result formats do not change. Optional `stargate.mcp.auth.scopes-supported` values describe scopes supported by the deployment; advertising them does not grant database permissions.
+
+This endpoint is an OAuth protected resource, not an authorization server. Browser login, client registration, organization/role selection, and token issuance remain outside Data API. The backend must support the agreed JWT credential contract. Successful initialization or tool discovery does not prove authorized database execution.
+
+Local verification used DSE 6.9.21's native OIDC authentication alongside internal password authentication. It required an OIDC issuer, the MCP resource audience, `allow_token_via_password_authentication=true`, and `token_username=token`. DSE mapped an issuer-signed claim to a restricted database role. The issuer derived that claim from validated organization membership and role selection. Internal password authentication alone cannot validate these JWT credentials. Each deployment needs matching issuer, audience, and database permission configuration.
+
+The compatibility tests inspect the HTTP-to-command boundary with a processor test double. Validate real permitted and denied database calls against the deployment's backend as well. Migration work remains tracked in [#2643](https://github.com/stargate/data-api/issues/2643).
+
+### Reusable OAuth smoke test
+
+With OAuth and MCP enabled, use an existing collection and a known document to test the database path. Run from the repository root with Python 3.10 or later.
+
+```sh
+python3 docs/mcp/oauth_smoke.py \
+  --endpoint https://api.example.com/v1/mcp \
+  --keyspace demo_readonly --collection messages \
+  --filter '{"id":"welcome"}' \
+  --expect-document '{"id":"welcome","value":"hello"}' \
+  --denied-keyspace demo_other_org
+```
+
+The [script](mcp/oauth_smoke.py) prompts for an access token without echoing it. Automation can supply `MCP_ACCESS_TOKEN` through the environment. Obtain the token from the deployment's authorization server for the exact MCP resource audience. The script consumes that token and does not perform browser login or token issuance.
+
+It checks the unauthenticated HTTP 401 challenge, public resource metadata, authenticated tool catalog, and exact `findOne` document. It prints the available tool names and exits nonzero on HTTP, protocol, application, or document mismatches. The optional `--denied-keyspace` repeats the read against another existing keyspace and requires `UNAUTHORIZED_ACCESS`. No write tools are called. The script uses the stateless protocol version described above and does not test older session-based clients.
+
+Run the script's focused tests with `python3 -B -m unittest discover -s docs/mcp -p 'test_*.py' -v`.
 
 ## Results and errors
 

docs/mcp/oauth_smoke.py

@@ -0,0 +1,207 @@
+#!/usr/bin/env python3
+"""Check a Data API OAuth resource and a database read with an existing token."""
+
+from __future__ import annotations
+
+import argparse
+from dataclasses import dataclass
+import getpass
+import ipaddress
+import json
+import os
+import re
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+import warnings
+
+PROTOCOL = "2026-07-28"
+MAX_BODY = 2 * 1024 * 1024
+
+
+class SmokeFailure(ValueError):
+    pass
+
+
+def require(condition, message):
+    if not condition:
+        raise SmokeFailure(message)
+
+
+def origin(url):
+    try:
+        parts = urllib.parse.urlsplit(url)
+        host, port = parts.hostname, parts.port
+        try:
+            loopback = ipaddress.ip_address(host or "").is_loopback
+        except ValueError:
+            loopback = host == "localhost"
+        require(host and (parts.scheme == "https" or parts.scheme == "http" and loopback)
+                and parts.username is None and parts.password is None
+                and not parts.query and not parts.fragment,
+                "URLs must use HTTPS or loopback HTTP without credentials, queries, or fragments")
+        return parts.scheme, host, port or (443 if parts.scheme == "https" else 80)
+    except ValueError as error:
+        if isinstance(error, SmokeFailure):
+            raise
+        raise SmokeFailure("Invalid URL") from None
+
+
+def json_object(raw):
+    try:
+        value = json.loads(raw)
+    except (ValueError, UnicodeDecodeError):
+        raise SmokeFailure("Expected a JSON object") from None
+    require(isinstance(value, dict), "Expected a JSON object")
+    return value
+
+
+class NoRedirect(urllib.request.HTTPRedirectHandler):
+    def redirect_request(self, *args):
+        return None
+
+
+@dataclass
+class Response:
+    status: int
+    headers: object
+    body: bytes
+
+    def json(self):
+        return json_object(self.body)
+
+
+class Client:
+    def __init__(self, endpoint):
+        origin(endpoint)
+        self.endpoint = endpoint
+        self.sequence = 0
+        self.opener = urllib.request.build_opener(NoRedirect())
+
+    def request(self, url, data=None, headers=None):
+        origin(url)
+        request = urllib.request.Request(url, data=data, headers=headers or {})
+        try:
+            response = self.opener.open(request, timeout=20)
+        except urllib.error.HTTPError as error:
+            response = error
+        with response:
+            body = response.read(MAX_BODY + 1)
+            require(len(body) <= MAX_BODY, "Response exceeds size limit")
+            return Response(response.code, response.headers, body)
+
+    def rpc(self, method, params, token=None):
+        self.sequence += 1
+        payload = {"jsonrpc": "2.0", "id": self.sequence, "method": method,
+                   "params": {**params, "_meta": {
+                       "io.modelcontextprotocol/protocolVersion": PROTOCOL,
+                       "io.modelcontextprotocol/clientCapabilities": {}}}}
+        headers = {"Content-Type": "application/json", "Accept": "application/json, text/event-stream",
+                   "MCP-Protocol-Version": PROTOCOL, "Mcp-Method": method}
+        if method == "tools/call":
+            headers["Mcp-Name"] = params["name"]
+        if token:
+            headers["Authorization"] = "Bearer " + token
+        return self.request(self.endpoint, json.dumps(payload).encode(), headers)
+
+    def result(self, method, params, token):
+        response = self.rpc(method, params, token)
+        require(response.status == 200, f"{method} returned HTTP {response.status}")
+        body = response.json()
+        require(body.get("jsonrpc") == "2.0" and body.get("id") == self.sequence,
+                "JSON-RPC response did not match its request")
+        require("error" not in body and isinstance(body.get("result"), dict),
+                "MCP returned a protocol error or no result")
+        return body["result"]
+
+
+def smoke_test(endpoint, token, keyspace, collection, filter_value, expected_document,
+               denied_keyspace=None):
+    require(isinstance(token, str) and bool(token) and not any(c.isspace() for c in token),
+            "A nonempty access token without whitespace is required")
+    require(isinstance(filter_value, dict) and isinstance(expected_document, dict),
+            "The filter and expected document must be JSON objects")
+    client = Client(endpoint)
+    challenge = client.rpc("tools/list", {})
+    require(challenge.status == 401, f"Unauthenticated MCP returned HTTP {challenge.status}, expected 401")
+    header = challenge.headers.get("WWW-Authenticate", "")
+    metadata = re.search(r'\bresource_metadata="([^"]+)"', header)
+    require(header.lower().startswith("bearer ") and metadata is not None,
+            "The 401 challenge is missing Bearer resource metadata")
+    metadata_url = metadata.group(1)
+    require(origin(metadata_url) == origin(endpoint), "Resource metadata must use the MCP origin")
+    response = client.request(metadata_url)
+    require(response.status == 200, f"Resource metadata returned HTTP {response.status}")
+    document = response.json()
+    servers = document.get("authorization_servers")
+    require(document.get("resource") == endpoint and isinstance(servers, list) and bool(servers)
+            and all(isinstance(server, str) for server in servers)
+            and isinstance(document.get("bearer_methods_supported"), list)
+            and "header" in document["bearer_methods_supported"], "Resource metadata is inconsistent")
+    for server in servers:
+        origin(server)
+    print("PASS  Unauthenticated challenge and protected-resource metadata")
+
+    catalog = client.result("tools/list", {}, token)
+    tools = catalog.get("tools")
+    require(isinstance(tools, list) and bool(tools)
+            and all(isinstance(tool, dict) and isinstance(tool.get("name"), str)
+                    and bool(tool["name"]) for tool in tools), "Invalid tool catalog")
+    names = [tool["name"] for tool in tools]
+    require(len(names) == len(set(names)) and "findOne" in names and not catalog.get("nextCursor"),
+            "Expected a complete, unique catalog containing findOne")
+    print(f"PASS  Authenticated catalog ({len(names)} tools): "
+          + json.dumps(names).replace(token, "[REDACTED]"))
+
+    arguments = {"keyspace": keyspace, "collection": collection, "filter": filter_value}
+    result = client.result("tools/call", {"name": "findOne", "arguments": arguments}, token)
+    content = result.get("structuredContent")
+    require(result.get("isError", False) is False and isinstance(content, dict)
+            and not content.get("errors") and content.get("document") == expected_document,
+            "Database read did not return the expected document without application errors")
+    print("PASS  Database read returned the exact expected document")
+    if denied_keyspace is not None:
+        result = client.result("tools/call", {"name": "findOne", "arguments": {
+            **arguments, "keyspace": denied_keyspace}}, token)
+        content = result.get("structuredContent")
+        require(result.get("isError") is True and isinstance(content, dict)
+                and isinstance(content.get("errors"), list) and len(content["errors"]) == 1
+                and isinstance(content["errors"][0], dict)
+                and content["errors"][0].get("errorCode") == "UNAUTHORIZED_ACCESS",
+                "Denied read did not return UNAUTHORIZED_ACCESS")
+        print("PASS  Other keyspace read was denied with UNAUTHORIZED_ACCESS")
+    return names
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    for name in ("endpoint", "keyspace", "collection"):
+        parser.add_argument("--" + name, required=True)
+
+    def object_argument(value):
+        try:
+            return json_object(value)
+        except SmokeFailure as error:
+            raise argparse.ArgumentTypeError(str(error)) from None
+
+    parser.add_argument("--filter", required=True, type=object_argument, dest="filter_value")
+    parser.add_argument("--expect-document", required=True, type=object_argument, dest="expected_document")
+    parser.add_argument("--denied-keyspace")
+    args = parser.parse_args()
+    try:
+        with warnings.catch_warnings():
+            warnings.simplefilter("error", getpass.GetPassWarning)
+            token = os.environ.get("MCP_ACCESS_TOKEN") or getpass.getpass("OAuth access token: ")
+        smoke_test(token=token, **vars(args))
+    except SmokeFailure as error:
+        print("FAIL  " + str(error), file=sys.stderr)
+        return 1
+    except (OSError, ValueError, EOFError, getpass.GetPassWarning):
+        print("FAIL  Request or token input failed", file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())