Reconfiguration Trigger SPI

[SamBarker]

Summary

• Defines a pluggable SPI for triggering KafkaProxy.reconfigure(), discovered via ServiceLoader
• Formalises the trigger responsibilities established during the Proposal 083 review (failure policy, rollback, concurrency handling, debouncing, config persistence)
• Three interfaces: ReconfigurationTrigger, ReconfigurationTriggerFactory, ReconfigurationTriggerContext
• Configuration follows the existing type + config plugin pattern
• Includes a file watcher sketch to demonstrate the SPI is sufficient for the common case

Context

Proposal 083 delivered KafkaProxy.reconfigure(Configuration) but explicitly deferred trigger mechanisms. The PR #83 discussion explored file watchers, HTTP endpoints, and operator callbacks, ultimately agreeing to decouple the trigger from the core reconfiguration machinery. This proposal provides the extension point that makes hot reload usable in the shipped binary and extensible for custom control planes.

🤖 Generated with Claude Code

[robobario]

Something I was trying to understand was why we need a different mechanism for reload vs initial Configuration of the Proxy. I squint at it and think "oh, shouldn't initialization just be a kind of reload going from something to nothing", pointed Claude at it and think its explanation is sane:

The proposal does explain part of this (the create()/start() split), but your question goes deeper — why does initial load go through a separate startup path at all, rather than through reconfigure()?

The key reason is that reconfigure() semantics are fundamentally "converge from a prior running state." The proxy needs to already have virtual clusters bound to ports, initialized subsystems, etc. to know what to change. Without a running state, reconfigure() throws
IllegalStateException per 083. So you can't bootstrap from nothing through reconfigure().

For the file watcher specifically, the code similarity is real — both initial load and change detection read the same file and call parseConfiguration(InputStream). But two things make unification difficult:

1. The trigger doesn't know what to watch until after startup — it gets configFilePath() from the context, which is determined during proxy startup. There's a bootstrapping dependency.
2. Failure semantics differ — startup failure means "don't start the proxy"; reconfigure failure has a policy (rollback, best-effort, shutdown). Conflating them would force an awkward policy at startup time.

There's also a more subtle point: static configuration (the reconfigurationTrigger section itself, listen ports, etc.) is explicitly out of scope for reconfigure(). This means the startup path and the reconfigure path will always handle different sets of config, so they can't be fully unified regardless.

The duplication in the file watcher case (open file → parse → do something with Config) is minor enough that it doesn't seem worth collapsing the abstraction boundary. The two "do something" operations are genuinely different enough in semantics and failure handling that keeping them separate is the right call. Whether that's worth a comment in the proposal is a fair question though — the lifecycle section explains the create()/start() split but doesn't explicitly address why initial load isn't routed through reconfigure().

So yeah, there's a granularity difference where the proxy config has some global config, where reconfigure is really meant to be scoped to the virtual clusters. That and failure semantics are convincing.

Does that match your ideas @SamBarker? It's not really something new in this proposal, it's just been on my mind during the reload implementation.

[SamBarker]

I've had something similar on my mind, but haven't explored too deeply. I'm not entirely buying Claude's thinking. I'm about to push up a big(ish) change around how this interacts with config v2. I think there might be a way to pick apart the fixed config from the virtual cluster config and merge the startup and reload paths.

[robobario]

though maybe there is an implication here, like I'm wondering, should the user be able to plug in something like a Config source? A single place where we can obtain the startup Config, and it can be hooked into the hot reload trigger lifecycle? So there's a single component that knows about the File (or other sources like HTTP etc), how to parse it and integrate with hot reload and startup.

I can see that there's a bootstrapping problem, if we define the config sources or reload triggers in the main config file we have to load this to get at the SPIs 🤔 Maybe it implies we need to have a bootstrap config where these fundamental configuration options can be defined, separate from our existing proxy config files? By default, if the user didn't supply one, it would be implied that they are using a Configuration File source identified by the other command line args

[SamBarker]

well that escalated quickly...

Ok see what you and your Claude make of that update... ;)