Repository navigation
Private service routes can fail silently after a roll or on IPv4 #986
Description
Private service routes from #851 have follow-ups that did not block the review. Each can leave a service unreachable with no error, or keep access longer than intended.
- A router binary that rolls before the agent on a node cannot program service routes until the agent reloads.
- Some services stay unreachable with no error and no metric, because a missing next hop falls back to the old behavior.
- A pod with any default route on another link loses its VPC default route.
- Persistent access has no limit on who may set it.
Done when each is fixed, or documented with its roll order or trust boundary. The detail is in the first comment.
Related to #851
Galactic Edge integration (#992)
Implement this milestone as a consumer of the Galactic Edge provider contract. #992 owns provider authorization, host attachment lifecycle, shared-state fencing and maintenance, independently enforced forwarding expiry, prefix conflict arbitration/publication, and gVPC traffic policy. MASQUE owns client authentication, connector /96 assignment and retention, session packet checks, transport behavior and its provider adapter. Use the foundation's versioned contract and existing provider helpers; do not introduce a second host-networking controller, lease store or BGP publisher. Any required foundation change must be tracked and qualified under #992 before this milestone claims integrated acceptance.
Integration disposition and acceptance
This issue concerns the legacy private-service-route path and retains its own remediation scope. MASQUE uses #998 EdgeRoute publication and #997 gates; it must not adopt persistent ServiceRouteAuthorization as an alternate way to publish connector /96s or bypass expiry.
- Define connector admission, addressing and ownership contracts #972/Implement /96 connector addressing and aggregate return routing #978 document which, if any, findings in this issue and its comments are reachable through MASQUE's selected Edge/gVPC workload path. Map each applicable finding to a fix or documented trust/upgrade constraint and a regression check.
- Integrate MASQUE with Galactic Edge provider attachments #975/Add reproducible end-to-end feature acceptance tests #985 verify that unrelated pod churn, stale cleanup and restart preserve active MASQUE attachments/routes. Prove MASQUE packet attachments on the Galactic Edge foundation #974/Implement /96 connector addressing and aggregate return routing #978 cover local/remote traffic in the chosen routing configuration.
- Add reproducible end-to-end feature acceptance tests #985 records evidence for non-applicable findings rather than silently ignoring the issue or requiring unrelated IPv4/private-service work to close MASQUE Connector for gVPC #970. This issue itself stays open until its own acceptance is met.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Activity
ecv commentedon Oct 22, 2026
Detail, as of #851 at d67c997:
serviceroutemap.OpenPinnednow fails hard ifservice_peer_tableis not pinned, and the pin exists only after the new usid program loads. Document the roll order, or treat a missing pin as the old fallback. Nothing tests the missing-pin case.attachmentNextHopreturns nil on a family mismatch or a missingPodSubnet, andSetPeer(nil)falls back todest_addr. IPv4 virtual services and attachments with several subnets stay unreachable, and the new alert counts only programming errors. Return an error or count a metric.cni/netns.go, any default route on another link suppresses the VPC default, including blackhole, unreachable and ECMP defaults withLinkIndex0. Require a unicast default withLinkIndex > 0.ServiceRouteAuthorization.persistentdrops the node-side deadline backstop. Only RBAC on ServiceRoutePolicy limits who sets it. State that only the network control plane writes these.TestPersistentPathSurvivesControllerOutageAndWithdrawscovers only a running router.runbook_urlanchor#galacticservicerouteprogrammingfailingdoes not exist in the infra galactic runbook.ecv commentedon Oct 22, 2026
From #851 at a28836b: when a pod's veth disappears,
recoverableEntriesreturns empty entries,repairDesiredcloses the node's policy gate, and every service route on that node is off until the whole policy is rebuilt. Pod churn triggers this. Withdraw only the affected intent's entries. Separately,clearWithdrawnAdvertisementclears the attachment's node, interfaces and PodSubnet. A cache-lag NotFound after a fast delete and recreate makes that status flap until the next reconcile.