MASQUE Connector for gVPC #970

Goal

Provide IPv6 connectivity from tenant sites and clients to their VPC workloads using CONNECT-IP, served from Galactic edge nodes.

Support HTTP/3 over QUIC on UDP/443 and an HTTP/2 over TLS fallback on TCP/443. The fallback carries packets in capsules and establishes a new CONNECT-IP tunnel. Address retention and session continuity across reconnects are defined during design. Both transports use the same authentication, VPC selection, policy and ownership rules.

Build a reusable VPC attachment that isolates tenant traffic and manages routes through startup, failure and shutdown.

Allocate a /96 to each connector, following the gVPC tenant addressing plan. Clients may subdivide that prefix and manage addresses behind the tunnel; gVPC routes the aggregate /96 to the connector's active attachment.

Done when

1. Independent clients can connect, send, receive, disconnect and reconnect over both transports, carrying 1280-byte IPv6 packets in both directions. HTTP/2 passes with UDP/443 blocked. Path-MTU changes and oversized packets produce correct ICMPv6 Packet Too Big or explicit tunnel failure behavior.
2. Both transports enforce VPC authorization and traffic policy in both directions, including rejection of unauthorized requests. Credential expiry or access revocation stops existing forwarding within a defined bound.
3. Two tenants with identical addresses on the same node remain isolated.
4. Each connector has one active forwarding owner. A failed instance or one cut off from the API stops forwarding and its routes are withdrawn within defined limits; ownership transfer never permits old and new owners to forward concurrently for the same connector.
5. Each connector receives a /96. Addresses in client-managed subdivisions are reachable through its aggregate gVPC route, and traffic using source addresses outside the delegated prefix is rejected.
6. Supported connector counts and throughput are measured for both transports, resource limits are enforced under overload, and operators can observe connection failures, policy drops and route status.

Out of scope

• Public API development and plugin/portal integration.
• Client development, including changes to datum-cloud/connect or datum-cloud/app for transport, automatic fallback, enrollment, local networking or UI. Service authentication contracts and independent-client testing remain in scope.

Phases

1. Design and prove

• Define attachment ownership, per-connector forwarding claims, expiry and replacement fencing
• Define address retention, session continuity and ownership transfer across reconnects, including HTTP/2 fallback
• Define client authentication, VPC authorization, credential expiry and access revocation, including bounds for terminating existing forwarding
• Specify claim lifetime, forwarding-expiry and route-withdrawal bounds, and per-request resource limits
• Define connector-count and throughput targets, resource budgets and overload acceptance criteria for both transports
• Select libraries and independent test clients for both transports
• Prove tenant isolation on Talos/Cilium with overlapping addresses
• Prove CONNECT-IP interoperability, minimum packet size and HTTP/2 fallback against the acceptance criteria above

Done when decisions are documented and reviewed, and the prototypes pass. A failed prototype requires a design revision and a passing retest.

2. VPC attachment and ownership

• Provision an isolated, host-managed VPC attachment for a service pod
• Fence MASQUE route and datapath mutations to its owned state
• Expire forwarding authority and withdraw routes locally, including during API outages

Done when crash, API-partition and replacement tests meet the defined expiry bounds and prevent concurrent forwarding by old and new owners for the same connector.

3. CONNECT-IP service

• Implement client admission and VPC selection
• Enforce credential expiry and access revocation for active sessions within the defined bounds
• Serve both transports with shared policy and ownership enforcement
• Verify TLS certificate validation and HTTP/2 fallback with UDP/443 blocked
• Implement bounded capsule parsing and buffering, backpressure and independent request cancellation
• Each connector receives a /96 consistent with gVPC addressing and may subdivide it locally
• Publish the aggregate /96 route to the active attachment and deliver return traffic to client-managed addresses and subdivisions
• Enforce source-address ownership within each connector's delegated /96
• Enforce traffic policy in both directions
• Verify ICMPv6 Packet Too Big handling and path-MTU changes, including paths that cannot carry the required 1280-byte packets
• Expose connection failures, policy drops, active connector counts, resource usage and route status
• Measure connector capacity and throughput on both transports and verify resource limits under overload

Done when the connectivity, prefix-allocation, routing, authorization, isolation, MTU, capacity and observability acceptance criteria pass on a staging edge node.

Protocol references

• CONNECT-IP: RFC 9484 — Proxying IP in HTTP.
• Datagrams and capsules: RFC 9297 — HTTP Datagrams and the Capsule Protocol, and RFC 9221 — QUIC DATAGRAM extension.
• QUIC and encryption: RFC 9000 — QUIC transport, RFC 9001 — Using TLS to Secure QUIC, and RFC 8446 — TLS 1.3.
• HTTP/3: RFC 9114, with the Extended CONNECT mechanism defined in RFC 9220.
• HTTP/2 fallback: RFC 9113, with the Extended CONNECT mechanism defined in RFC 8441. CONNECT-IP uses this mechanism as specified in RFC 9484.