Repository navigation
Protect private DNS ownership and make readiness trustworthy #261
Description
Summary
Resolve the ownership, recovery, and validation findings from the initial Private DNS review.
Why it matters
A customer must not lose a name or address to another network, and a Ready status must mean the service can actually answer safely.
Acceptance criteria
- Releasing or changing a DNS address does not leak its allocation, and the same address cannot be assigned to different customers.
- A newly created network context cannot take over a name owned by an established context.
- DNS recovers after a restart and does not report Ready while it is intentionally refusing queries.
- Isolation tests detect a broken lookup path instead of passing without proving that names are isolated.
- Public or reserved domain suffixes cannot be claimed inappropriately.
- Each review finding is verified as fixed or recorded as an explicitly accepted exception.
Related work
Parent: Restrict private DNS access to approved networks.
Original report and engineering evidence are preserved in the discussion.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
None yet
Activity
ecv commentedon Oct 22, 2026
Detail by source PR:
#233, resolver contexts and access
ClaimDestinationchecks(region, address)only against other destination claims. Generated regional addresses live underdns-address-claim-*, so one context's node address can equal another's regional address. It also accepts loopback, link-local and multicast addresses.platform/allocator.gonever releases destination claims, so each address edit leaks a ConfigMap and blocks that address for every project for good.consumerIDis settled by name order, so a later context with an earlier-sorting name takes ownership, andconfirmAccessWithdrawaltombstones the incumbent's access. Rank bycreationTimestamp, then UID.DNSResolverBindingin the platform namespace is not watched, so serving acknowledgements are seen only on the 15s requeue.#237, serving recovery
restoreActive, anApplyForceerror returns without marking activation needed. If the daemons were down at restart, nothing retries until the agent restarts again.probeLocalDaemonsaccepts any DNS response, REFUSED and SERVFAIL included.failClosedwrites no marker, so after a fail-closed and a restart the node can report Ready while gated. Query a known name from the active snapshot.UnsafeAllowNoDNSVerify: true, so the probe path is never exercised.canRetainActivedrops all serving on any rejected record removal or version bump. The body says invalid updates retain accepted state. Decide which, and say so.#239, qualification tests
cloudflare.comwith DNSSEC, so CI depends on internet egress.#243, domain allocation
domainSuffixis checked only for DNS-1123 shape, socomorcluster.localbecomes a private zone that shadows real names. Add an allowlist such as.internal.domainSuffix.Status.ManagedNamespaceis zeroed on every reconcile and refilled only when ready, so it flaps.ecv commentedon Oct 22, 2026
Three more follow-ups from #230, which approved without them:
internal/agent/reader.goand the CLI zone import still list private zones and their records.ecv commentedon Oct 22, 2026
More follow-ups, from #232 at 29eea5f (approved once unsupported events stopped being terminated):
MaxAckPendingis 64 andRetryusesNakWithDelay, so redelivery can put an activation ahead of its chunks. UseMaxAckPending=1for publication consumers, or test that receivers buffer by dependency.SnapshotStore.Putcompares whole envelopes, so a republish with a freshPublishedAtorEventIDfails. Compare the payload hash.MemberIDto its subject.ZoneAttachment.Pendingis unusable, becauseServingSnapshot.Validaterequires a nonzero epoch and revision on every attachment.listType=maporMaxItems, and azoneApiceschange needs no generation or revision bump..internal-dns-e2e/is not in.gitignore. Rotate the NATS passwords from that directory if they were used anywhere real.ecv commentedon Oct 22, 2026
From #236 at e63c21a (approved once UDP PROXY was removed):
Replacerebuilds every context's cache on any change.ecv commentedon Oct 22, 2026
From #234 at 6111210 and #235 at f2e0a60 (both approved once their blockers were fixed):
fitSnapshotsizes the payload beforepublicationDependenciesaddsRequiredPublications, so a snapshot that fit can fail the later size check on every tick and stall the shard. Count the required publications in the fit.Step, which is safe, butreportCapacityis skipped on that path, so nothing reports it. SetCapacityAvailableor alert.Refreshstill fans out per project with no concurrency bound, and the request timeout applies only when set.ecv commentedon Oct 22, 2026
Status after #233 at ae2a3d6 and #237 at 0f08c61.
Now fixed: non-routable destinations are rejected, destinations inside the regional prefix or with an existing regional claim are refused, duplicate
consumerIDownership goes to the oldest context, a failed restore marks activation needed, and a fail-closed persists its marker across restarts.Still open:
DNSResolverBindingis still not watched.fd54::1%eth0passesIsGlobalUnicastand missesPrefix.Contains, skipping both checks. Strip or reject the zone inClaimDestination.reservedoes not skip a candidate that already holds a destination claim.probeLocalDaemonsstill accepts REFUSED and SERVFAIL, and the restart tests still never run the probe.scotwells commentedon Oct 29, 2026
The description now explains the problem and expected outcome in plain language. The original report is preserved below for the observations, diagnostics, and implementation details behind this work. It describes the situation when the issue was reported; later comments and the issue state record subsequent progress.
Original report and engineering evidence
The internal DNS stack has follow-ups that did not block its review but will bite in production. They come from feat: Manage private DNS contexts and resolver access, feat: Preserve private DNS isolation through serving failures, test: Validate internal DNS on the shared Kubernetes environment and Allocate private DNS domains requested by consumer services.
Done when each item is fixed or recorded as accepted. The detail, with file references, is in the first comment.
Related to feat: Manage private DNS contexts and resolver access, feat: Preserve private DNS isolation through serving failures, test: Validate internal DNS on the shared Kubernetes environment, Allocate private DNS domains requested by consumer services
[-]Internal DNS can leak addresses, hand over names and report Ready falsely[/-][+]Protect private DNS ownership and make readiness trustworthy[/+]scotwells commentedon Oct 29, 2026
Fresh staging checks on 29 October verified two isolated Kata networks resolve the same private name to independent addresses through both advertised resolvers over UDP and TCP. The east matrix passed all 24 queries, covering both its regional-forwarding and local-answer profiles; four initial Kubernetes proxy errors happened before DNS execution and passed on bounded retry.
A second zone became available only after explicit association (first observed answer 7.30 s after creation); removing it stopped answers after 4.14 s. Recreated zones and networks received new identities, and old associations stayed invalid until explicitly recreated. Restoring owner access did not restore the revoked network. Stale zone replacement was rejected by UID precondition. These are observed timings, not agreed service-target passes.
An approved one-time secondary-resolver replacement also preserved denial: the access monitor saw 87 NXDOMAIN replies and 3 transport failures for the revoked network, with zero unauthorized answers, while its separate authorized-network control returned 90/90 correct replies. Post-recovery checks passed both protocols and addresses. Restart measurements record the interruption separately.
Public authoritative checks returned REFUSED/no private answers on both staging IPv4 endpoints over UDP/TCP, with successful public SOA controls. Still incomplete: delayed broker replay, authorization/ownership attack cases, regional withdrawal, dependency outage/lease expiry, and a deliberate software isolation-failure test. Predicate sensitivity against deliberately granted access was checked, but is narrower than that fault test. No issue acceptance criteria are being closed from these samples.
scotwells commentedon Oct 29, 2026
Final cleanup verification adds a remaining access-state gap: the source/edge test resources are gone and all test record names are absent from inspected active configurations, but one deleted network's forwarding route remained on central node-b over three minutes later. No stale private answer was demonstrated; full active-state cleanup is not confirmed. The new deleted-network routing issue, attached to the access epic, records the evidence, limits and acceptance checks. No additional restart or shared checkpoint deletion was performed.