fix: Collect a projected interface before its name is reused - #496
Conversation
An interface is published under a name a replacement reuses on purpose, and the network a published copy names cannot be changed after it is created. So a copy that outlived the interface it was published from held the name against every replacement that followed, and the replacement was never published at all. A consumer kept reading the interface their previous instance had, and a network service resolved its members to it. Nothing collected the copy because nothing was holding the interface open long enough to say where its copy had gone. A published interface that simply vanished took with it the labels naming the project its copy was handed to. Ordered teardown is the whole of the fix. Nothing here detects a copy that outlived its interface, or takes a name back from one, because a copy that reaches that state is a teardown that did not run rather than a case to recover from. Key changes: - Hold a published interface on the hub with a finalizer until the copy handed to its project has been collected, so the project's copy never outlives the interface it describes - Wait for a teardown still in flight rather than writing through the copy it is collecting - Refuse to write over an interface in a project namespace that is not a published copy Fixes #495 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
66c585f to
38131ed
Compare
|
Validated in staging on the branch build A workload was created on network Deleting the workload collected that copy on the first check, which is the ordered teardown this change adds. Recreating the workload under the same name on a different network published the copy on Deleting it again left nothing behind in either plane. No immutability errors and no projector reconcile errors appeared on any of the three operator pods for the whole run. The chainsaw scenario added here covers the cell to hub hop, which is as far as the e2e environment reaches. The projector hop has no automated coverage, because that environment wires the cell and the manager to different apiservers, so the staging run above is what stands behind it for now. One thing worth separating out. Staging could not allocate an address at all until its IPAM quota was raised, and production was about two days from the same wall. That is unrelated to this change and is tracked in milo-os/ipam#131. |
2 participants
Summary
A workload recreated under the same name reuses its interface name, and a published copy of an interface cannot move to a different network once created.
So a copy left behind by a deleted interface held that name, the replacement was never published, and a network service resolved its members to the interface a previous instance had and served no traffic.
A published interface is now held open until the copy handed to its project has been collected, and a name held by a copy that cannot be converged is reported rather than written over.
One orphan in production still needs deleting by hand, since this stops new ones without cleaning up the old.
Test plan
Fixes #495