You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Expose controller fleet-health metrics in Prometheus format #183
The controller's fleet state is only visible through the management UI. adminListen must be a loopback address and has no authentication of its own, so a metrics scraper (Prometheus, Grafana Alloy, an OpenTelemetry Collector) cannot reach it.
That makes it hard to alert on the things an operator of a managed fleet cares about:
devices that stopped checking in
devices whose last configuration apply failed
devices that are on an older configuration revision than the active one
the spread of agent versions during a rollout
whether telemetry events are still arriving
Usage reporting from #160 covers tokens and cost. It does not cover fleet health.
Proposal
An opt-in metricsListen controller setting. When set, the controller serves GET /metrics in the Prometheus text format on that address. Unset keeps today's behavior.
Labels would come from closed sets or from the OS and agent version of enrolled devices. Hostnames, users, device IDs, MCP server names and tool inputs would never be exported.
Open questions
Are the metric names and the metricsListen setting the right shape, or would you rather expose this through an OpenTelemetry exporter?
Is a separate unauthenticated listener acceptable, or should the endpoint require a token like the usage ingest does?
With more than one controller replica, the connected gauge and the counters are per replica. Is that acceptable, or should connection state live in the database?
Implementation
A draft is up in #180. I'm opening this issue so the design can be discussed before the PR is reviewed in detail.
Activity