Can the number of calls to the k8s API server used for drift detection be reduced? #5792
-
|
Hello, This means the only way to reduce the API server request volume caused by the Flux kustomization controller is The provider team also mentioned that ArgoCD uses the k8s informer API for drift detection, i.e. is getting Are there any plans to change the drift detection mechanism in the Flux GOTK? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
|
We have no plans on changing how drfit detection works. The interval on Flux Kustomization should be long enough to avoid DDOS-ing the API server, see: https://fluxcd.io/flux/components/kustomize/kustomizations/#recommended-settings Flux is mean to run at scale with minimal resource usage, so we can not use informers like Argo, this would result in GB of RAM and huge amount of traffic as we'll end up watching everything that's in etcd. |
Beta Was this translation helpful? Give feedback.
-
|
There's been discussion about this — using watches/informers instead of polling is on the Flux roadmap but hasn't landed yet. The kustomize-controller currently does a server-side dry-run apply on each reconciliation to detect drift, which generates a full read+compare cycle per managed resource per interval. A few things you can do right now to reduce load: 1. Stagger intervals across Kustomizations — don't set them all to the same interval. Spread them out so they don't all hit the API server at the same time: # High-priority (infra, networking)
spec:
interval: 5m
# Medium (apps)
spec:
interval: 10m
# Low-priority (monitoring dashboards, etc.)
spec:
interval: 30m2. Disable drift detection where you don't need it — for resources that are only managed by Flux and never manually changed, you can reduce the reconciliation frequency significantly: spec:
interval: 1h
# Only reconcile on git changes, not on a timerCombined with webhooks (GitHub/GitLab → Flux receiver), you get near-instant deployment on push without constant polling. 3. Use spec:
wait: false # don't poll for resource health after apply4. Set up a Flux webhook receiver for source reconciliation so you can use longer source intervals without sacrificing deploy speed: # GitRepository
spec:
interval: 1h # long poll interval since webhook handles itThe ArgoCD comparison is valid — ArgoCD's informer-based approach generates fewer API calls for drift detection. But it comes with its own trade-offs (higher memory usage, watch connection overhead). The Flux team has discussed adding an opt-in informer mode in future releases. For now, the combination of staggered intervals + webhook receivers + disabling unnecessary health checks can reduce API server load by 80%+ compared to the default configuration. |
Beta Was this translation helpful? Give feedback.
There's been discussion about this — using watches/informers instead of polling is on the Flux roadmap but hasn't landed yet. The kustomize-controller currently does a server-side dry-run apply on each reconciliation to detect drift, which generates a full read+compare cycle per managed resource per interval.
A few things you can do right now to reduce load:
1. Stagger intervals across Kustomizations — don't set them all to the same interval. Spread them out so they don't all hit the API server at the same time:
2. Disable drif…