From c1e3091bbf0ec14b412f1299ce7ac78cf2e49d13 Mon Sep 17 00:00:00 2001 From: Douglas Adolph <116844282+dcadolph@users.noreply.github.com> Date: Mon, 2 Nov 2026 00:25:23 -0600 Subject: [PATCH 1/3] Require a credential for reads when the server holds one --- UPGRADING.md | 13 ++ docs/getting-started.md | 2 +- docs/index.md | 2 +- docs/operator/helm.md | 2 +- docs/operator/rbac.md | 7 +- docs/operator/server-mode.md | 2 +- docs/production-readiness.md | 4 +- docs/reference/api.md | 20 +-- internal/browser/browser_test.go | 52 ++++++++ internal/server/auth.go | 55 ++++++++- internal/server/auth_test.go | 114 ++++++++++++++++++ internal/server/distributions_handler_test.go | 4 +- internal/server/errors.go | 2 + internal/server/oidc_test.go | 6 +- internal/server/openapi.yaml | 6 +- internal/server/ratelimit_test.go | 2 + internal/server/server.go | 7 ++ internal/server/server_test.go | 6 +- internal/server/static/js/api.js | 10 +- internal/server/static/js/app.js | 9 +- internal/server/static/js/auth.js | 4 +- internal/server/static/js/live.js | 100 +++++++++++---- internal/server/static/js/router.js | 10 +- internal/server/static/js/shell.js | 2 +- internal/server/system_handler_test.go | 6 +- 25 files changed, 380 insertions(+), 67 deletions(-) diff --git a/UPGRADING.md b/UPGRADING.md index 871e6b7..a184693 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -14,6 +14,19 @@ stability contract and the GitHub release notes for the full feature list. 5. Verify `kubectl logs deploy/fleetsweeper | grep migrate` shows no errors and `/readyz` returns 200. +## v0.15.0 + +No schema migration. + +- **Reads need a credential.** A server that holds any credential (a bootstrap + `--auth-token`, single sign-on, or at least one API key) now answers 401 to + an `/api` read that carries neither a bearer token nor a session cookie. + Before, such a read was served as an anonymous viewer. Give every API client + and scraper of `/api` a viewer key. `/healthz`, `/readyz`, `/openapi.yaml`, + `/oidc/*`, signed `/api/webhooks/*` calls, and the dashboard's static files + stay open. The dashboard asks for a token on first load. `--insecure`, + `--demo`, and a server with no credential at all keep anonymous reads. + ## v0.14.0 No schema migration. Reports, scores, and a few API shapes change on the diff --git a/docs/getting-started.md b/docs/getting-started.md index 7c2db7a..a5839d3 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -163,6 +163,6 @@ See [outliers](concepts/outliers.md) for the full statistical treatment. behind norm-based detection. - [Server mode](operator/server-mode.md). Run as a service with dashboard and API. - [ClusterScan CRD](operator/clusterscan.md). Full spec reference. -- [RBAC and API keys](operator/rbac.md). Multi-tenant access control. +- [RBAC and API keys](operator/rbac.md). Role and cluster-scoped API keys for writes. - Go SDK. A standard-library client for the HTTP API lives in the `client` package, with contract tests that run against a live server. diff --git a/docs/index.md b/docs/index.md index 1781de5..922eeb1 100644 --- a/docs/index.md +++ b/docs/index.md @@ -50,6 +50,6 @@ Then visit the dashboard at port 8080 of the fleetsweeper service. - [Getting Started](getting-started.md). First scan in five minutes. - [Operator overview](operator/overview.md). Declarative scans with ClusterScan CRs and a reconciler. -- [RBAC and API keys](operator/rbac.md). Multi-tenant access control. +- [RBAC and API keys](operator/rbac.md). Role and cluster-scoped API keys for writes. - [The fleet is the policy](concepts/fleet-is-policy.md). Why norm-based detection beats rule authoring at scale. diff --git a/docs/operator/helm.md b/docs/operator/helm.md index 2c0856a..e1b6b3f 100644 --- a/docs/operator/helm.md +++ b/docs/operator/helm.md @@ -30,7 +30,7 @@ helm install fleetsweeper deploy/helm/fleetsweeper \ ## CRD lifecycle `crds.install` defaults to `true` so an out-of-the-box `helm install` is -self-contained. In multi-tenant or GitOps environments where CRDs are +self-contained. In shared or GitOps clusters where CRDs are managed separately, set it to `false` and apply the CRDs from `deploy/crds/` independently. diff --git a/docs/operator/rbac.md b/docs/operator/rbac.md index 88cce18..a61ca9a 100644 --- a/docs/operator/rbac.md +++ b/docs/operator/rbac.md @@ -17,8 +17,11 @@ Fleetsweeper has two authentication paths, designed to coexist: | `operator` | Trigger scans, acknowledge findings, and manage tags and locations within scope. Cannot manage groups or keys. | | `viewer` | Read-only. Cannot reach `/admin/*`. | -Read-only endpoints (GET) are open to viewers and above (or to anyone when -`--insecure` is set). Mutating endpoints require `operator` or `admin`. +Read-only endpoints (GET) need a viewer key or above, the bootstrap token, or +a single sign-on session. A request with no credential gets 401. Only +`--insecure`, `--demo`, or a server with no token, no single sign-on, and no +keys at all serves reads to anyone. Mutating endpoints require `operator` or +`admin`. ## Cluster scope diff --git a/docs/operator/server-mode.md b/docs/operator/server-mode.md index 2534f73..be1afc5 100644 --- a/docs/operator/server-mode.md +++ b/docs/operator/server-mode.md @@ -46,7 +46,7 @@ By default, mutating endpoints (`POST /api/scans`, `POST /api/groups`, `DELETE /api/groups/...`) return 403 because `serve` will scan whatever kubeconfig the process holds. Two ways to allow mutations: -1. **Recommended**: set `--auth-token` to a long random string. Clients must send `Authorization: Bearer ` on mutating requests. Comparison is constant-time. +1. **Recommended**: set `--auth-token` to a long random string. Clients must then send `Authorization: Bearer ` on every API request, reads included. Comparison is constant-time. The dashboard asks for the token on first load. 2. **Explicit opt-out**: pass `--insecure` to disable authentication entirely. A loud warning is logged at startup. `--demo` turns authentication off on its own. The demo serves a synthetic fleet from an diff --git a/docs/production-readiness.md b/docs/production-readiness.md index fb34666..42b0b1c 100644 --- a/docs/production-readiness.md +++ b/docs/production-readiness.md @@ -6,7 +6,7 @@ What Fleetsweeper ships ready for the box. | ---------------------- | -------------- | | HA backend | SQLite default, PostgreSQL via `--db-driver=postgres` | | Multi-replica safe | Kubernetes Lease-based leader election (`coordination.k8s.io/v1`) | -| Multi-tenant RBAC | Scoped API keys (`admin`, `operator`, `viewer`) with cluster scope (`*`, names, or `group:`) on writes. Reads are not scoped | +| Access control | Every API read and write needs a credential. Role and cluster-scoped API keys (`admin`, `operator`, `viewer`; `*`, names, or `group:`) for writes. Reads are not scoped by cluster | | Audit log | Every mutating request captured. Queryable at `GET /api/admin/audit`. Retention via `--audit-retention` | | Declarative ops | `ClusterScan` CRD with an in-process reconciler | | GitOps integrations | `FleetDriftReport` CR plus `PolicyReport` (wgpolicyk8s.io) | @@ -27,4 +27,4 @@ What Fleetsweeper ships ready for the box. - [Backends](operator/backends.md). SQLite vs Postgres trade-offs. - [Leader election](operator/leader-election.md). Multi-replica safety details. - [Audit log](operator/audit.md). What is captured and how to query it. -- [RBAC and API keys](operator/rbac.md). Scoped multi-tenant access. +- [RBAC and API keys](operator/rbac.md). Role and cluster-scoped API keys for writes. diff --git a/docs/reference/api.md b/docs/reference/api.md index 1b37ad6..aa1f022 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -7,21 +7,27 @@ disagree. ## Authentication -All mutating endpoints require one of: +Every `/api` request needs `Authorization: Bearer `, where `` is +the bootstrap `--auth-token` value or a stored API key, or a single sign-on +session cookie. A request without one gets 401. Two modes need no credential: -- `--insecure` (development only), -- `--demo`, whose synthetic in-memory world runs open as anonymous admin, or -- `Authorization: Bearer ` where `` is either the bootstrap - `--auth-token` value or a stored API key. +- `--insecure` (development only), which runs as anonymous admin, and +- `--demo`, whose synthetic in-memory world runs open as anonymous admin. + +A server started with no token, no single sign-on, and no API keys serves +reads anonymously and refuses writes. The first key created against its store +closes anonymous reads within ten seconds, without a restart. GET endpoints are open to viewers, operators, and admins. `POST /api/whatif` is a read too: it projects a fix without writing anything, so viewers may call -it within their cluster scope. +it within their cluster scope. `/healthz`, `/readyz`, `/openapi.yaml`, the +`/oidc/*` login paths, inbound `/api/webhooks/*` calls (which carry their own +signature), and the dashboard's static files need no credential. ## Common headers - `Content-Type: application/json; charset=utf-8` on responses. -- `Authorization: Bearer ` on requests (omit only when `--insecure` or `--demo`). +- `Authorization: Bearer ` on requests (omit only when `--insecure` or `--demo`, or on a server that holds no credential). - `X-Fleetsweeper-Signature: sha256=` on inbound webhook calls. ## Endpoints diff --git a/internal/browser/browser_test.go b/internal/browser/browser_test.go index d1de0cd..6d2ed48 100644 --- a/internal/browser/browser_test.go +++ b/internal/browser/browser_test.go @@ -751,3 +751,55 @@ func TestDashboardUngraded(t *testing.T) { t.Errorf("console errors:\n%s", strings.Join(errs, "\n")) } } + +// TestDashboardSignIn serves real stored data behind a bootstrap token and checks the +// dashboard shows a sign-in panel instead of data, that a token pasted into the drawer loads +// the dashboard, and that the live stream connects with that token. +func TestDashboardSignIn(t *testing.T) { + st, err := store.NewSQLite(filepath.Join(t.TempDir(), "signin.db")) + if err != nil { + t.Fatalf("open store: %v", err) + } + if err := server.SeedDemo(t.Context(), st, time.Now().UTC().Truncate(time.Minute)); err != nil { + t.Fatalf("seed: %v", err) + } + srv := server.New(server.Config{ + Store: st, Registry: scanner.NewRegistry(), Log: zap.NewNop(), Workers: 2, AuthToken: "sign-in-token", + }) + ts := httptest.NewServer(srv.Handler()) + t.Cleanup(func() { + ts.Close() + srv.Close() + st.Close() + }) + ctx := newBrowser(t) + log := &consoleLog{route: "boot"} + log.listen(ctx) + err = run(ctx, 30*time.Second, + chromedp.Navigate(ts.URL+"/"), + chromedp.Evaluate(`localStorage.setItem('fleetsweeper-welcome-dismissed', '1')`, nil), + chromedp.Reload(), + chromedp.WaitVisible(`#sign-in`, chromedp.ByQuery), + ) + if err != nil { + var html string + _ = run(ctx, 5*time.Second, chromedp.Evaluate(`document.getElementById('ct') ? document.getElementById('ct').innerHTML.slice(0, 600) : 'no ct'`, &html)) + t.Fatalf("sign-in panel never showed: %v\n%s\n%s", err, html, strings.Join(log.take(), "\n")) + } + err = run(ctx, 30*time.Second, + clickWhenPresent(`#sign-in [data-action="token"]`), + chromedp.WaitVisible(`#tokenInput`, chromedp.ByQuery), + chromedp.SendKeys(`#tokenInput`, "sign-in-token", chromedp.ByQuery), + clickWhenPresent(`.drawer-actions button.btn-ac`), + chromedp.Poll(`!document.getElementById('sign-in') && `+settledJS, nil, chromedp.WithPollingTimeout(settleTimeout)), + ) + if err != nil { + t.Fatalf("dashboard did not load after the token: %v", err) + } + var live bool + err = run(ctx, 15*time.Second, chromedp.Poll(`document.querySelector('#live-toggle.on') !== null`, &live, + chromedp.WithPollingTimeout(10*time.Second))) + if err != nil || !live { + t.Fatalf("live stream did not connect with the token: %v", err) + } +} diff --git a/internal/server/auth.go b/internal/server/auth.go index de0a8ec..01a917a 100644 --- a/internal/server/auth.go +++ b/internal/server/auth.go @@ -3,6 +3,7 @@ package server import ( "context" "errors" + "fmt" "net/http" "slices" "strings" @@ -107,7 +108,8 @@ func roleAllowsWrite(role string) bool { // 1. --insecure mode → anonymous admin // 2. Bearer matches bootstrap --auth-token → built-in admin // 3. Bearer matches an api_keys row → that key -// 4. Otherwise: 401 on mutating requests, anonymous viewer on reads +// 4. No credential, and the server holds one (token, OIDC, or any API key) → 401 +// 5. No credential, and the server holds none → viewer on reads, 401 on writes func (s *Server) authMiddleware(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // /webhooks/* endpoints carry their own request authentication @@ -121,6 +123,7 @@ func (s *Server) authMiddleware(next http.Handler) http.Handler { actor, err := s.resolveActor(r) if err != nil { + w.Header().Set("WWW-Authenticate", `Bearer realm="fleetsweeper"`) writeError(w, http.StatusUnauthorized, err.Error()) return } @@ -153,9 +156,9 @@ func pathIsWebhook(path string) bool { return strings.HasPrefix(path, "/webhooks/") } -// resolveActor inspects the request and the server config to produce an Actor. -// A nil Actor with a nil error means an anonymous viewer (used for read-only -// requests in non-insecure mode without a token). +// resolveActor inspects the request and the server config to produce an Actor. A request with +// no credential gets an anonymous viewer only when the server holds no credential of its own; +// otherwise it gets ErrMissingCredential. func (s *Server) resolveActor(r *http.Request) (*Actor, error) { if s.insecure { return &Actor{ @@ -171,6 +174,9 @@ func (s *Server) resolveActor(r *http.Request) (*Actor, error) { if actor := s.actorFromSessionCookie(r); actor != nil { return actor, nil } + if s.holdsCredential(r.Context()) { + return nil, s.missingCredential() + } return &Actor{Role: store.RoleViewer}, nil } @@ -208,6 +214,47 @@ func (s *Server) resolveActor(r *http.Request) (*Actor, error) { }, nil } +// keyRecheckInterval is how long a server that found no API keys waits before it looks again, so +// a key created with the CLI against the same store ends anonymous reads without a restart. +const keyRecheckInterval = 10 * time.Second + +// holdsCredential reports whether the server has a credential a caller could present: the +// bootstrap token, an OIDC login, or at least one API key row. A failed key lookup counts as +// holding one, so the server refuses the anonymous read rather than serve it. +func (s *Server) holdsCredential(ctx context.Context) bool { + if s.authToken != "" || s.oidc != nil || s.keysSeen.Load() { + return true + } + s.keyCheckMu.Lock() + defer s.keyCheckMu.Unlock() + if s.keysSeen.Load() { + return true + } + if !s.keyCheckedAt.IsZero() && time.Since(s.keyCheckedAt) < keyRecheckInterval { + return false + } + keys, err := s.store.ListAPIKeys(ctx) + if err != nil { + s.log.Warn("auth: api key lookup failed: refusing anonymous read", zap.Error(err)) + return true + } + if len(keys) > 0 { + s.keysSeen.Store(true) + return true + } + s.keyCheckedAt = time.Now() + return false +} + +// missingCredential returns the error for a request that carries no credential, naming the +// single sign-on path when the server has one. +func (s *Server) missingCredential() error { + if s.oidc != nil { + return fmt.Errorf("%w: send a bearer token or sign in at /oidc/login", ErrMissingCredential) + } + return fmt.Errorf("%w: send a bearer token", ErrMissingCredential) +} + // keyTouchInterval is the minimum time between last-used writes for one API key. const keyTouchInterval = time.Minute diff --git a/internal/server/auth_test.go b/internal/server/auth_test.go index 09a02a5..2b769db 100644 --- a/internal/server/auth_test.go +++ b/internal/server/auth_test.go @@ -499,3 +499,117 @@ func TestActorFromContext_Fallback(t *testing.T) { t.Errorf("expected deny-all default, got %+v", a) } } + +// newCredentialTestServer builds a server over a fresh store with the given bootstrap token, which +// may be empty. +func newCredentialTestServer(t *testing.T, token string) (*Server, *store.SQLite) { + t.Helper() + s, err := store.NewSQLite(filepath.Join(t.TempDir(), "cred.db")) + if err != nil { + t.Fatalf("open store: %v", err) + } + srv := New(Config{Store: s, Registry: scanner.NewRegistry(), Log: zap.NewNop(), Workers: 2, AuthToken: token}) + t.Cleanup(func() { + srv.Close() + s.Close() + }) + return srv, s +} + +// TestAnonymousReadsOnlyWithoutCredentials verifies a request with no credential reads only from +// a server that holds none: a bootstrap token, an OIDC login, or one API key row each close the +// API to anonymous callers, while health, the spec, and the dashboard's static files stay open. +func TestAnonymousReadsOnlyWithoutCredentials(t *testing.T) { + t.Parallel() + tests := []struct { + Name string + Token string + WithKey bool + WithOIDC bool + Path string + Bearer string + WantCode int + WantMessage string + }{{ + Name: "Test 0: Token server refuses an anonymous report read.", Token: "boot", + Path: "/api/scans/latest/report", WantCode: http.StatusUnauthorized, + WantMessage: "credential required: send a bearer token", + }, { + Name: "Test 1: Token server refuses an anonymous scan list.", Token: "boot", + Path: "/api/scans", WantCode: http.StatusUnauthorized, + WantMessage: "credential required: send a bearer token", + }, { + Name: "Test 2: Token server serves the bootstrap token.", Token: "boot", Bearer: "boot", + Path: "/api/scans", WantCode: http.StatusOK, + }, { + Name: "Test 3: Token server keeps health open.", Token: "boot", Path: "/healthz", WantCode: http.StatusOK, + }, { + Name: "Test 4: Token server keeps the dashboard index open.", Token: "boot", Path: "/", WantCode: http.StatusOK, + }, { + Name: "Test 5: Token server keeps the spec open.", Token: "boot", Path: "/openapi.yaml", WantCode: http.StatusOK, + }, { + Name: "Test 6: Server with no credential keeps anonymous reads.", Path: "/api/scans", WantCode: http.StatusOK, + }, { + Name: "Test 7: One API key row closes anonymous reads.", WithKey: true, + Path: "/api/scans", WantCode: http.StatusUnauthorized, + WantMessage: "credential required: send a bearer token", + }, { + Name: "Test 8: OIDC closes anonymous reads and names the login path.", WithOIDC: true, + Path: "/api/scans", WantCode: http.StatusUnauthorized, + WantMessage: "credential required: send a bearer token or sign in at /oidc/login", + }} + for testNum, test := range tests { + t.Run(fmt.Sprintf("test %d", testNum), func(t *testing.T) { + t.Parallel() + srv, st := newCredentialTestServer(t, test.Token) + if test.WithKey { + saveTestKey(t, st, "key_cred", store.RoleViewer, store.ScopeWildcard) + } + if test.WithOIDC { + srv.oidc = &oidcRuntime{} + } + req := httptest.NewRequest(http.MethodGet, test.Path, nil) + if test.Bearer != "" { + req.Header.Set("Authorization", "Bearer "+test.Bearer) + } + w := httptest.NewRecorder() + srv.Handler().ServeHTTP(w, req) + if w.Code != test.WantCode { + t.Fatalf("%s: want %d, got %d body %s", test.Name, test.WantCode, w.Code, w.Body.String()) + } + if test.WantCode != http.StatusUnauthorized { + return + } + var body errorResponse + if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil { + t.Fatalf("%s: decode: %v", test.Name, err) + } + if diff := cmp.Diff(test.WantMessage, body.Error); diff != "" { + t.Errorf("%s: message mismatch (-want +got):\n%s", test.Name, diff) + } + if got := w.Header().Get("WWW-Authenticate"); got != `Bearer realm="fleetsweeper"` { + t.Errorf("%s: WWW-Authenticate = %q", test.Name, got) + } + }) + } +} + +// TestHoldsCredentialSeesLateKey verifies a server that started with no API keys stops serving +// anonymous reads once a key appears in its store and the recheck interval has passed. +func TestHoldsCredentialSeesLateKey(t *testing.T) { + t.Parallel() + srv, st := newCredentialTestServer(t, "") + if srv.holdsCredential(t.Context()) { + t.Fatal("empty store: want no credential") + } + saveTestKey(t, st, "key_late", store.RoleViewer, store.ScopeWildcard) + if srv.holdsCredential(t.Context()) { + t.Fatal("inside the recheck interval: want the cached answer") + } + srv.keyCheckMu.Lock() + srv.keyCheckedAt = time.Now().Add(-keyRecheckInterval) + srv.keyCheckMu.Unlock() + if !srv.holdsCredential(t.Context()) { + t.Fatal("after the recheck interval: want the late key seen") + } +} diff --git a/internal/server/distributions_handler_test.go b/internal/server/distributions_handler_test.go index b09310d..aefa652 100644 --- a/internal/server/distributions_handler_test.go +++ b/internal/server/distributions_handler_test.go @@ -296,8 +296,8 @@ func TestWhatIfAuth(t *testing.T) { if code := serve(t, h, http.MethodPost, "/api/whatif", outside, viewer, nil); code != http.StatusForbidden { t.Errorf("viewer outside scope: want 403, got %d", code) } - if code := serve(t, h, http.MethodPost, "/api/whatif", body, "", nil); code != http.StatusForbidden { - t.Errorf("anonymous: want 403, got %d", code) + if code := serve(t, h, http.MethodPost, "/api/whatif", body, "", nil); code != http.StatusUnauthorized { + t.Errorf("anonymous: want 401, got %d", code) } entries, err := st.ListAuditEntries(context.Background(), store.AuditListOptions{Limit: 100}) if err != nil { diff --git a/internal/server/errors.go b/internal/server/errors.go index c431347..fedfbc5 100644 --- a/internal/server/errors.go +++ b/internal/server/errors.go @@ -7,4 +7,6 @@ var ( ErrServer = errors.New("server") // ErrBadRequest indicates invalid client input. ErrBadRequest = errors.New("bad request") + // ErrMissingCredential indicates a request with no credential to a server that holds one. + ErrMissingCredential = errors.New("credential required") ) diff --git a/internal/server/oidc_test.go b/internal/server/oidc_test.go index fc55cba..0430264 100644 --- a/internal/server/oidc_test.go +++ b/internal/server/oidc_test.go @@ -123,7 +123,7 @@ func TestActorFromSessionCookie(t *testing.T) { } // TestSessionCookieRejectedWhenInvalid verifies a junk cookie does not -// promote the caller; the response falls back to viewer (403 on admin GET). +// authenticate the caller: a server with single sign-on answers 401. func TestSessionCookieRejectedWhenInvalid(t *testing.T) { t.Parallel() srv := newOIDCTestServer(t) @@ -132,8 +132,8 @@ func TestSessionCookieRejectedWhenInvalid(t *testing.T) { req.AddCookie(&http.Cookie{Name: sessionCookieName, Value: "garbage.signature"}) w := httptest.NewRecorder() srv.mux.ServeHTTP(w, req) - if w.Code != http.StatusForbidden { - t.Errorf("junk cookie: want 403, got %d", w.Code) + if w.Code != http.StatusUnauthorized { + t.Errorf("junk cookie: want 401, got %d", w.Code) } } diff --git a/internal/server/openapi.yaml b/internal/server/openapi.yaml index c6b9ea2..0f15b74 100644 --- a/internal/server/openapi.yaml +++ b/internal/server/openapi.yaml @@ -24,14 +24,16 @@ servers: security: - bearerAuth: [] - - {} components: securitySchemes: bearerAuth: type: http scheme: bearer - description: Token configured via --auth-token. Required for POST/DELETE. + description: >- + The bootstrap --auth-token or a stored API key. Every /api path needs it once the + server holds a credential (a bootstrap token, single sign-on, or any API key). A + server that holds none serves reads anonymously and refuses writes. parameters: Scan: diff --git a/internal/server/ratelimit_test.go b/internal/server/ratelimit_test.go index 77f1f4b..2f0432b 100644 --- a/internal/server/ratelimit_test.go +++ b/internal/server/ratelimit_test.go @@ -196,6 +196,7 @@ func TestRateLimitTrustedProxyHeader(t *testing.T) { req := httptest.NewRequest(http.MethodGet, "/api/scans", nil) req.RemoteAddr = "10.0.0.1:443" req.Header.Set("X-Forwarded-For", client) + req.Header.Set("Authorization", "Bearer bootstrap-secret") w := httptest.NewRecorder() srv.mux.ServeHTTP(w, req) got = append(got, w.Code) @@ -284,6 +285,7 @@ func TestRateLimitMiddleware_RemainingHeader(t *testing.T) { t.Parallel() srv := newRateLimitedTestServer(t, 60, 60) req := httptest.NewRequest(http.MethodGet, "/api/scans", nil) + req.Header.Set("Authorization", "Bearer bootstrap-secret") w := httptest.NewRecorder() srv.mux.ServeHTTP(w, req) if w.Code != http.StatusOK { diff --git a/internal/server/server.go b/internal/server/server.go index af8fefb..1ea7d3b 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -143,6 +143,13 @@ type Server struct { // keyTouches records when each API key's last-used time was last written, so a busy key // writes at most once per keyTouchInterval. keyTouches map[string]time.Time + // keysSeen is true once the store has held at least one API key row. It never goes back to + // false, so a server that ever had a key never serves an anonymous read again. + keysSeen atomic.Bool + // keyCheckMu guards keyCheckedAt and serializes the API key lookup. + keyCheckMu sync.Mutex + // keyCheckedAt is when the store was last found to hold no API keys. + keyCheckedAt time.Time } // Config holds server configuration. diff --git a/internal/server/server_test.go b/internal/server/server_test.go index 9be8831..4a40479 100644 --- a/internal/server/server_test.go +++ b/internal/server/server_test.go @@ -94,12 +94,12 @@ func TestBearerAuth(t *testing.T) { t.Fatalf("right token: want 201, got %d body %s", w.Code, w.Body.String()) } - // GET should always pass when no Authorization header is supplied. + // A read with no Authorization header is refused once the server holds a token. read := httptest.NewRequest(http.MethodGet, "/api/groups", nil) w = httptest.NewRecorder() srv.mux.ServeHTTP(w, read) - if w.Code != http.StatusOK { - t.Fatalf("read without token: want 200, got %d", w.Code) + if w.Code != http.StatusUnauthorized { + t.Fatalf("read without token: want 401, got %d", w.Code) } } diff --git a/internal/server/static/js/api.js b/internal/server/static/js/api.js index 685965c..e1fdb2e 100644 --- a/internal/server/static/js/api.js +++ b/internal/server/static/js/api.js @@ -52,15 +52,17 @@ export function apiSend(method, path, body) { let catalog = null; // loadScannerCatalog fetches the scanner list once, keyed by id. A failure -// leaves the catalog empty so labels fall back to ids. +// returns an empty catalog so labels fall back to ids, and is not kept, so the +// next call tries again. export async function loadScannerCatalog() { if (catalog) return catalog; - catalog = {}; + const next = {}; try { - (await api('/scanners')).forEach((s) => { catalog[s.id] = s; }); + (await api('/scanners')).forEach((s) => { next[s.id] = s; }); } catch (_) { - catalog = {}; + return {}; } + catalog = next; return catalog; } diff --git a/internal/server/static/js/app.js b/internal/server/static/js/app.js index 5ee1735..03a4a13 100644 --- a/internal/server/static/js/app.js +++ b/internal/server/static/js/app.js @@ -2,11 +2,11 @@ // theme, and the global actions, then starts the router. import { installActions, on } from './actions.js'; -import { startRouter } from './router.js'; +import { startRouter, render } from './router.js'; import { applyTheme, cycleTheme, themeLabel } from './theme.js'; import { loadScannerCatalog } from './api.js'; import { copyText, byId } from './dom.js'; -import { toggleLive, startLive } from './live.js'; +import { toggleLive, startLive, reconnectLive } from './live.js'; import { openTokenDrawer, onTokenChange } from './auth.js'; import { paintToken } from './shell.js'; import { startTour } from './tour.js'; @@ -61,6 +61,11 @@ on('skip', () => byId('ct').focus()); on('live-toggle', () => toggleLive()); on('token', () => openTokenDrawer()); onTokenChange(paintToken); +onTokenChange(async () => { + await loadScannerCatalog(); + render(); + reconnectLive(); +}); on('tour-start', () => startTour()); on('shortcuts', () => toggleShortcuts()); on('help-menu', () => openHelpMenu()); diff --git a/internal/server/static/js/auth.js b/internal/server/static/js/auth.js index 82e3ddf..f5b9efa 100644 --- a/internal/server/static/js/auth.js +++ b/internal/server/static/js/auth.js @@ -44,8 +44,8 @@ export function openTokenDrawer() { const set = hasToken(); openDrawer({ title: 'API token', - body: '

Acknowledging findings and starting scans need a token when the server runs with ' + - '--auth-token or API keys. It stays in this tab only and is never written to disk. ' + + body: '

Reading this server needs a token when it runs with --auth-token, ' + + 'API keys, or single sign-on. It stays in this tab only and is never written to disk. ' + 'A single sign-on session needs no token.

' + ``, actions: [ diff --git a/internal/server/static/js/live.js b/internal/server/static/js/live.js index 0e484b6..7713443 100644 --- a/internal/server/static/js/live.js +++ b/internal/server/static/js/live.js @@ -5,55 +5,105 @@ import { refresh, P } from './router.js'; import { toast } from './components/toast.js'; +import { authHeaders } from './api.js'; // STORE_KEY is the localStorage key of the viewer's choice. const STORE_KEY = 'fleetsweeper-live'; +// state.source is the AbortController of the open stream, or null. const state = { on: true, source: null, connected: false }; // payloadOf returns the payload of a server event, or an empty object. -function payloadOf(ev) { +function payloadOf(data) { try { - return JSON.parse(ev.data).payload || {}; + return JSON.parse(data).payload || {}; } catch (_) { return {}; } } -// connect opens the event stream. EventSource reconnects on its own after a -// dropped connection. -function connect() { - if (state.source || typeof EventSource === 'undefined') return; - const es = new EventSource('/api/events'); - state.source = es; - es.onopen = () => { - state.connected = true; - paintLive(); - }; - es.onerror = () => { - state.connected = false; - paintLive(); - }; - es.addEventListener('scan.complete', (ev) => { - const p = payloadOf(ev); +// RETRY_MS is how long the stream waits before it reconnects after a drop. +const RETRY_MS = 3000; + +// handleEvent acts on one server event. +function handleEvent(type, data) { + if (type === 'scan.complete') { + const p = payloadOf(data); toast(`Scan complete${typeof p.score === 'number' ? (p.grade ? `: fleet score ${p.score} (${p.grade})` : ': not graded, too little of the fleet was observed') : ''}.`, 'ok'); refresh(); - }); - es.addEventListener('scan.failed', (ev) => { - toast('Scan failed: ' + (payloadOf(ev).error || 'no reason given') + '.', 'crit'); - }); - es.addEventListener('alert.received', () => { + } else if (type === 'scan.failed') { + toast('Scan failed: ' + (payloadOf(data).error || 'no reason given') + '.', 'crit'); + } else if (type === 'alert.received') { if (P.page === 'alerts') refresh(); - }); + } +} + +// readStream parses the server-sent events in body and hands each to +// handleEvent until the stream ends. +async function readStream(body) { + const reader = body.pipeThrough(new TextDecoderStream()).getReader(); + let buf = ''; + for (;;) { + const { value, done } = await reader.read(); + if (done) return; + buf += value; + let at; + while ((at = buf.indexOf('\n\n')) >= 0) { + const block = buf.slice(0, at); + buf = buf.slice(at + 2); + let type = 'message'; + const data = []; + block.split('\n').forEach((line) => { + if (line.startsWith('event:')) type = line.slice(6).trim(); + else if (line.startsWith('data:')) data.push(line.slice(5).trimStart()); + }); + if (data.length) handleEvent(type, data.join('\n')); + } + } +} + +// connect opens the event stream with the tab's credential. The stream is read +// with fetch rather than EventSource so it can carry the bearer token. A drop +// reconnects after RETRY_MS. A 401 stops until the token changes. +async function connect() { + if (state.source) return; + const ctl = new AbortController(); + state.source = ctl; + while (state.source === ctl) { + let stop = false; + try { + const r = await fetch('/api/events', { headers: { Accept: 'text/event-stream', ...authHeaders() }, signal: ctl.signal }); + if (r.ok && r.body) { + state.connected = true; + paintLive(); + await readStream(r.body); + } else if (r.status === 401 || r.status === 403) { + stop = true; + } + } catch (_) { /* dropped or aborted */ } + state.connected = false; + paintLive(); + if (stop || state.source !== ctl) break; + await new Promise((done) => setTimeout(done, RETRY_MS)); + } + if (state.source === ctl) state.source = null; } // disconnect closes the event stream. function disconnect() { - if (state.source) state.source.close(); + if (state.source) state.source.abort(); state.source = null; state.connected = false; } +// reconnectLive reopens the stream with the current credential when live +// updates are on. +export function reconnectLive() { + disconnect(); + if (state.on) connect(); + paintLive(); +} + // isLive reports whether live updates are on. export function isLive() { return state.on; diff --git a/internal/server/static/js/router.js b/internal/server/static/js/router.js index 63068f8..56fec09 100644 --- a/internal/server/static/js/router.js +++ b/internal/server/static/js/router.js @@ -116,10 +116,18 @@ export async function render() { if (result.post) await result.post(); } } catch (e) { - el.innerHTML = '
Error: ' + esc(e.message) + '
'; + el.innerHTML = e.status === 401 ? signInPanel(e.message) : '
Error: ' + esc(e.message) + '
'; } } +// signInPanel explains that the server needs a credential to read and offers +// the token drawer, plus single sign-on when the server's message names it. +function signInPanel(message) { + const sso = message.includes('/oidc/login') ? ' Sign in' : ''; + return '

This server needs a credential before it shows anything.

' + + '

' + sso + '

'; +} + // refresh re-renders the current page in place: the view fetches its data // again and the result is morphed into the page, so scroll, filters, open // sections, and focus survive. A failed refresh leaves the page as it was. diff --git a/internal/server/static/js/shell.js b/internal/server/static/js/shell.js index d196890..968a48a 100644 --- a/internal/server/static/js/shell.js +++ b/internal/server/static/js/shell.js @@ -78,7 +78,7 @@ export function renderBar(extra) { '' + '' + '' + - `` + + `` + `` + ''; morph(document.getElementById('bar'), h); diff --git a/internal/server/system_handler_test.go b/internal/server/system_handler_test.go index 2c0846b..c71485b 100644 --- a/internal/server/system_handler_test.go +++ b/internal/server/system_handler_test.go @@ -39,14 +39,14 @@ func TestAdminSystemReturnsFeatures(t *testing.T) { } } -// TestAdminSystemGuardedByAdmin verifies non-admin actors are rejected. +// TestAdminSystemGuardedByAdmin verifies a caller with no credential is rejected. func TestAdminSystemGuardedByAdmin(t *testing.T) { t.Parallel() srv, _ := newAuthTestServer(t) req := httptest.NewRequest(http.MethodGet, "/api/admin/system", nil) // no auth header w := httptest.NewRecorder() srv.mux.ServeHTTP(w, req) - if w.Code != http.StatusForbidden { - t.Errorf("want 403 without admin token, got %d", w.Code) + if w.Code != http.StatusUnauthorized { + t.Errorf("want 401 without a token, got %d", w.Code) } } From 271fba15408da95c945d714ffc7eb0061b6760a2 Mon Sep 17 00:00:00 2001 From: Douglas Adolph <116844282+dcadolph@users.noreply.github.com> Date: Mon, 2 Nov 2026 00:31:19 -0600 Subject: [PATCH 2/3] Let the chart scan a fleet from a kubeconfig Secret --- cmd/cmd_serve.go | 14 ++++- cmd/cmd_serve_test.go | 29 +++++++++ deploy/helm/fleetsweeper/templates/NOTES.txt | 12 ++++ .../fleetsweeper/templates/deployment.yaml | 16 +++++ deploy/helm/fleetsweeper/values.yaml | 9 +++ docs/operator/helm.md | 25 ++++++++ internal/kube/chart_test.go | 63 +++++++++++++++++++ internal/kube/client.go | 18 +++++- internal/kube/client_test.go | 30 +++++++++ 9 files changed, 212 insertions(+), 4 deletions(-) create mode 100644 internal/kube/chart_test.go diff --git a/cmd/cmd_serve.go b/cmd/cmd_serve.go index c87e5db..9174505 100644 --- a/cmd/cmd_serve.go +++ b/cmd/cmd_serve.go @@ -50,7 +50,7 @@ func init() { serveCmd.Flags().String("seal-key", "", "HMAC-SHA256 key used to sign saved scan reports. Empty disables sealing.") serveCmd.Flags().Bool("controller", false, "Enable the ClusterScan controller: watch ClusterScan CRs in the home cluster and reconcile them by triggering scans on their declared interval.") serveCmd.Flags().String("controller-namespace", "", "Namespace the controller watches. Empty watches all namespaces (requires cluster-wide RBAC).") - serveCmd.Flags().String("controller-context", "", "Kubeconfig context the controller uses to reach the home cluster. Empty uses the default context or in-cluster config.") + serveCmd.Flags().String("controller-context", "", "Kubeconfig context the controller uses to reach the home cluster. Empty uses the in-cluster config inside a pod, else the kubeconfig's current context.") serveCmd.Flags().Duration("controller-poll", 15*time.Second, "How often the controller re-evaluates ClusterScan resources.") serveCmd.Flags().Bool("leader-election", true, "Enable leader election for the scheduler and controller. Honored only when running inside Kubernetes. Multiple replicas without leader election will double-fire scheduled work.") serveCmd.Flags().String("leader-namespace", "", "Namespace for the leader-election Lease object. Defaults to $POD_NAMESPACE.") @@ -218,7 +218,7 @@ func runServe(cmd *cobra.Command, _ []string) error { ctrlNS, _ := cmd.Flags().GetString("controller-namespace") ctrlContext, _ := cmd.Flags().GetString("controller-context") ctrlPoll, _ := cmd.Flags().GetDuration("controller-poll") - dyn, events, err := controller.Clients(kubeconfigPath, ctrlContext) + dyn, events, err := controller.Clients(controllerKubeconfig(kubeconfigPath, ctrlContext, kube.InCluster()), ctrlContext) if err != nil { log.Warn("controller: client setup failed", zap.Error(err)) return @@ -391,6 +391,16 @@ func resolveScheduleContexts(kubeconfigPath string, explicit []string, all bool) return explicit, nil } +// controllerKubeconfig returns the kubeconfig path the controller reaches its home cluster with. A +// pod with no --controller-context uses its ServiceAccount, so a mounted fleet kubeconfig never +// moves the controller off the cluster that holds its ClusterScan objects. +func controllerKubeconfig(kubeconfigPath, controllerContext string, inCluster bool) string { + if controllerContext == "" && inCluster { + return "" + } + return kubeconfigPath +} + // openServeStore opens whichever backend --db / --db-driver point at. --demo // runs on an in-memory SQLite database the server seeds with the demo fleet, // so it refuses a --db rather than write the demo into a real database. diff --git a/cmd/cmd_serve_test.go b/cmd/cmd_serve_test.go index 3b58655..15546a0 100644 --- a/cmd/cmd_serve_test.go +++ b/cmd/cmd_serve_test.go @@ -245,3 +245,32 @@ func TestLeaderElectionPlan(t *testing.T) { }) } } + +// TestControllerKubeconfig verifies the controller stays on the pod's own cluster when a fleet +// kubeconfig is mounted, and follows the kubeconfig when a context is named or it runs outside a +// pod. +func TestControllerKubeconfig(t *testing.T) { + t.Parallel() + tests := []struct { + Path string + Context string + InCluster bool + WantResult string + }{{ // Test 0: Pod with a fleet kubeconfig and no context uses its ServiceAccount. + Path: "/var/run/fleetsweeper/kubeconfig", InCluster: true, WantResult: "", + }, { // Test 1: Pod with a named context reads the kubeconfig. + Path: "/var/run/fleetsweeper/kubeconfig", Context: "home", InCluster: true, + WantResult: "/var/run/fleetsweeper/kubeconfig", + }, { // Test 2: Outside a pod the kubeconfig is used. + Path: "/home/u/.kube/config", WantResult: "/home/u/.kube/config", + }} + for testNum, test := range tests { + t.Run(fmt.Sprintf("test %d", testNum), func(t *testing.T) { + t.Parallel() + got := controllerKubeconfig(test.Path, test.Context, test.InCluster) + if diff := cmp.Diff(test.WantResult, got); diff != "" { + t.Errorf("mismatch (-want +got):\n%s", diff) + } + }) + } +} diff --git a/deploy/helm/fleetsweeper/templates/NOTES.txt b/deploy/helm/fleetsweeper/templates/NOTES.txt index a6585ba..423c9e3 100644 --- a/deploy/helm/fleetsweeper/templates/NOTES.txt +++ b/deploy/helm/fleetsweeper/templates/NOTES.txt @@ -5,6 +5,18 @@ Fleetsweeper {{ .Chart.AppVersion }} is installed. kubectl --namespace {{ .Release.Namespace }} port-forward svc/{{ .Release.Name }} 8080:{{ .Values.service.port }} Then open http://localhost:8080. +{{- if .Values.kubeconfig.existingSecret }} + + Scans read every context in the kubeconfig stored in Secret + {{ .Values.kubeconfig.existingSecret }}, key {{ .Values.kubeconfig.key }}. +{{- else }} + + Scans see only this cluster, through the release's ServiceAccount. To scan + a fleet, store a kubeconfig in a Secret and set kubeconfig.existingSecret: + + kubectl --namespace {{ .Release.Namespace }} create secret generic fleet-kubeconfig --from-file=config=$HOME/.kube/config + helm upgrade {{ .Release.Name }} --reuse-values --set kubeconfig.existingSecret=fleet-kubeconfig +{{- end }} {{- if .Values.auth.token }} diff --git a/deploy/helm/fleetsweeper/templates/deployment.yaml b/deploy/helm/fleetsweeper/templates/deployment.yaml index 2ff3f63..a9ab599 100644 --- a/deploy/helm/fleetsweeper/templates/deployment.yaml +++ b/deploy/helm/fleetsweeper/templates/deployment.yaml @@ -42,6 +42,9 @@ spec: {{- if .Values.scheduler.allContexts }} - --all-contexts {{- end }} + {{- if .Values.kubeconfig.existingSecret }} + - --kubeconfig=/var/run/fleetsweeper/kubeconfig + {{- end }} {{- range .Values.cors.origins }} - --cors-origin={{ . }} {{- end }} @@ -163,6 +166,11 @@ spec: mountPath: /data - name: tmp mountPath: /tmp + {{- if .Values.kubeconfig.existingSecret }} + - name: kubeconfig + mountPath: /var/run/fleetsweeper + readOnly: true + {{- end }} volumes: - name: data {{- if .Values.persistence.enabled }} @@ -173,3 +181,11 @@ spec: {{- end }} - name: tmp emptyDir: {} + {{- if .Values.kubeconfig.existingSecret }} + - name: kubeconfig + secret: + secretName: {{ .Values.kubeconfig.existingSecret }} + items: + - key: {{ .Values.kubeconfig.key }} + path: kubeconfig + {{- end }} diff --git a/deploy/helm/fleetsweeper/values.yaml b/deploy/helm/fleetsweeper/values.yaml index c418bb0..63acaa1 100644 --- a/deploy/helm/fleetsweeper/values.yaml +++ b/deploy/helm/fleetsweeper/values.yaml @@ -25,6 +25,15 @@ scheduler: allContexts: true contexts: [] +# Kubeconfig the server scans with. Without one the server sees only the cluster it runs in, +# through its ServiceAccount. To scan a fleet, store a kubeconfig in a Secret and name it here: +# kubectl create secret generic fleet-kubeconfig --from-file=config=$HOME/.kube/config +kubeconfig: + # Name of an existing Secret holding the kubeconfig. Empty scans the host cluster only. + existingSecret: "" + # Key in that Secret whose value is the kubeconfig file. + key: config + persistence: enabled: true size: 5Gi diff --git a/docs/operator/helm.md b/docs/operator/helm.md index e1b6b3f..d2eba32 100644 --- a/docs/operator/helm.md +++ b/docs/operator/helm.md @@ -19,6 +19,8 @@ helm install fleetsweeper deploy/helm/fleetsweeper \ | `cors.origins` | `[]` | CORS allowlist. Empty refuses cross-origin requests. | | `scheduler.interval` | `""` | Auto-scan interval (e.g. `30m`). Empty disables. | | `scheduler.allContexts` | `true` | Use every kubeconfig context for scheduled scans. | +| `kubeconfig.existingSecret` | `""` | Secret holding the kubeconfig to scan with. Empty scans the host cluster only. | +| `kubeconfig.key` | `config` | Key in that Secret whose value is the kubeconfig file. | | `persistence.enabled` | `true` | PVC for `/data/fleet.db`. | | `persistence.size` | `5Gi` | Disk size. | | `rbac.create` | `true` | Install least-privilege ClusterRole + binding. | @@ -27,6 +29,29 @@ helm install fleetsweeper deploy/helm/fleetsweeper \ | `controller.namespace` | `""` | Restrict the controller to one namespace. | | `controller.pollInterval` | `15s` | How often the reconciler re-evaluates. | +## Scanning a fleet + +Without a kubeconfig the server sees one cluster: the one it runs in, reached +through the release's ServiceAccount and named `in-cluster` in reports. To +scan a fleet, store a kubeconfig in a Secret and point the chart at it: + +```shell +kubectl --namespace fleetsweeper create secret generic fleet-kubeconfig \ + --from-file=config=$HOME/.kube/config +helm upgrade fleetsweeper deploy/helm/fleetsweeper --namespace fleetsweeper \ + --reuse-values --set kubeconfig.existingSecret=fleet-kubeconfig +``` + +The Secret is mounted read-only at `/var/run/fleetsweeper/kubeconfig` and +passed as `--kubeconfig`. Scheduled and triggered scans then read every context +in it. Each context needs read access on its own cluster, the same access the +chart's ClusterRole grants on the host. A kubeconfig whose users run an exec +plugin (`aws`, `gcloud`, `kubelogin`) needs that binary in the image, which the +distroless image does not carry, so use static tokens or client certificates +for in-cluster scans. The ClusterScan controller keeps using the +ServiceAccount for its own cluster unless `--controller-context` names a +context. + ## CRD lifecycle `crds.install` defaults to `true` so an out-of-the-box `helm install` is diff --git a/internal/kube/chart_test.go b/internal/kube/chart_test.go new file mode 100644 index 0000000..4feb3ee --- /dev/null +++ b/internal/kube/chart_test.go @@ -0,0 +1,63 @@ +package kube + +import ( + "fmt" + "os/exec" + "strings" + "testing" +) + +// TestChartKubeconfigSecret renders the Helm chart and verifies kubeconfig.existingSecret mounts +// the Secret read-only and points --kubeconfig at it, and that without it the chart passes no +// kubeconfig and its notes say scans see only the host cluster. +func TestChartKubeconfigSecret(t *testing.T) { + t.Parallel() + helm, err := exec.LookPath("helm") + if err != nil { + t.Skip("helm not on PATH") + } + tests := []struct { + Set []string + WantIn []string + WantNot []string + }{{ // Test 0: A Secret mounts read-only and feeds --kubeconfig. + Set: []string{"kubeconfig.existingSecret=fleet"}, + WantIn: []string{ + "- --kubeconfig=/var/run/fleetsweeper/kubeconfig", + "mountPath: /var/run/fleetsweeper\n readOnly: true", + "secretName: fleet\n items:\n - key: config\n path: kubeconfig", + "Scans read every context in the kubeconfig stored in Secret\n fleet, key config.", + }, + WantNot: []string{"Scans see only this cluster"}, + }, { // Test 1: A custom key is the one projected. + Set: []string{"kubeconfig.existingSecret=fleet", "kubeconfig.key=kubeconfig.yaml"}, + WantIn: []string{"- key: kubeconfig.yaml\n path: kubeconfig"}, + }, { // Test 2: No Secret, no kubeconfig, and notes say the host cluster only. + WantIn: []string{"Scans see only this cluster, through the release's ServiceAccount."}, + WantNot: []string{"--kubeconfig", "/var/run/fleetsweeper"}, + }} + for testNum, test := range tests { + t.Run(fmt.Sprintf("test %d", testNum), func(t *testing.T) { + t.Parallel() + args := []string{"install", "fs", "../../deploy/helm/fleetsweeper", "--dry-run=client"} + for _, s := range test.Set { + args = append(args, "--set", s) + } + out, err := exec.Command(helm, args...).CombinedOutput() + if err != nil { + t.Fatalf("helm install --dry-run: %v\n%s", err, out) + } + got := string(out) + for _, want := range test.WantIn { + if !strings.Contains(got, want) { + t.Errorf("rendered chart missing %q", want) + } + } + for _, not := range test.WantNot { + if strings.Contains(got, not) { + t.Errorf("rendered chart holds %q", not) + } + } + }) + } +} diff --git a/internal/kube/client.go b/internal/kube/client.go index 8657f98..a70544e 100644 --- a/internal/kube/client.go +++ b/internal/kube/client.go @@ -167,7 +167,12 @@ func ConnectAll(ctx context.Context, kubeconfigPath string, contexts []string, w return clients } -// AvailableContexts returns all context names defined in the kubeconfig file. +// InClusterContext is the context name of the cluster a pod runs in, reached through its +// ServiceAccount when no kubeconfig is given. LoadConfig resolves it to the in-cluster config. +const InClusterContext = "in-cluster" + +// AvailableContexts returns all context names defined in the kubeconfig file. A pod given no +// kubeconfig gets InClusterContext alone, so it scans the cluster it runs in. func AvailableContexts(kubeconfigPath string) ([]string, error) { loader := &clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfigPath} cfg, err := loader.Load() @@ -175,7 +180,16 @@ func AvailableContexts(kubeconfigPath string) ([]string, error) { return nil, fmt.Errorf("%w: %w", ErrLoadConfig, err) } - return contextNames(cfg), nil + return contextsOrInCluster(contextNames(cfg), kubeconfigPath, InCluster()), nil +} + +// contextsOrInCluster returns names, or InClusterContext alone when names is empty, no +// kubeconfig path was given, and the process runs in a pod. +func contextsOrInCluster(names []string, kubeconfigPath string, inCluster bool) []string { + if len(names) == 0 && kubeconfigPath == "" && inCluster { + return []string{InClusterContext} + } + return names } // contextNames extracts sorted context names from a kubeconfig. diff --git a/internal/kube/client_test.go b/internal/kube/client_test.go index 3c15ee5..4ff7b85 100644 --- a/internal/kube/client_test.go +++ b/internal/kube/client_test.go @@ -193,3 +193,33 @@ func TestLoadConfigSelectsContext(t *testing.T) { }) } } + +// TestContextsOrInCluster verifies a pod with no kubeconfig scans its own cluster and that any +// kubeconfig, given or holding contexts, wins over that fallback. +func TestContextsOrInCluster(t *testing.T) { + t.Parallel() + tests := []struct { + Names []string + Path string + InCluster bool + WantResult []string + }{{ // Test 0: Pod with no kubeconfig scans its own cluster. + InCluster: true, WantResult: []string{InClusterContext}, + }, { // Test 1: Laptop with no kubeconfig finds nothing. + WantResult: nil, + }, { // Test 2: Pod with a mounted kubeconfig uses its contexts. + Names: []string{"east", "west"}, Path: "/var/run/fleetsweeper/kubeconfig", InCluster: true, + WantResult: []string{"east", "west"}, + }, { // Test 3: Pod with an empty mounted kubeconfig does not fall back. + Path: "/var/run/fleetsweeper/kubeconfig", InCluster: true, WantResult: nil, + }} + for testNum, test := range tests { + t.Run(fmt.Sprintf("test %d", testNum), func(t *testing.T) { + t.Parallel() + got := contextsOrInCluster(test.Names, test.Path, test.InCluster) + if diff := cmp.Diff(test.WantResult, got); diff != "" { + t.Errorf("mismatch (-want +got):\n%s", diff) + } + }) + } +} From 7fe1bb201c457727f2675f0689c8d8d59a59e9e1 Mon Sep 17 00:00:00 2001 From: Douglas Adolph <116844282+dcadolph@users.noreply.github.com> Date: Mon, 2 Nov 2026 00:31:42 -0600 Subject: [PATCH 3/3] Publish the signed container image with each release --- .github/workflows/release.yaml | 19 +++++++++++++++---- .goreleaser.yaml | 2 ++ README.md | 12 +++++++++--- UPGRADING.md | 11 +++++++++++ deploy/helm/fleetsweeper/Chart.yaml | 4 ++-- docs/index.md | 10 +++++++--- 6 files changed, 46 insertions(+), 12 deletions(-) diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index 4ae2e0f..458446c 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -6,6 +6,8 @@ on: permissions: contents: write + # packages lets the job push the release images to ghcr.io with GITHUB_TOKEN. + packages: write # id-token lets cosign mint a short-lived Fulcio certificate from the # workflow's OIDC identity for keyless signing; attestations lets the build # publish SLSA provenance for the released artifacts. @@ -22,15 +24,24 @@ jobs: - uses: actions/setup-go@v6 with: go-version-file: go.mod + # QEMU and buildx build the arm64 image on the amd64 runner. + - uses: docker/setup-qemu-action@v4 + - uses: docker/setup-buildx-action@v4 + - name: Log in to ghcr.io + uses: docker/login-action@v4 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} - uses: anchore/sbom-action/download-syft@v0 - uses: sigstore/cosign-installer@v3 - uses: goreleaser/goreleaser-action@v7 with: version: "~> v2" - # Container images are deferred (ghcr publishing not set up yet), so - # docker is skipped; the checksums are cosign-signed keyless via the - # workflow OIDC identity. - args: release --clean --skip=docker + # Builds and pushes the amd64 and arm64 images and their manifest to + # ghcr.io. The checksums and the image manifests are cosign-signed + # keyless via the workflow OIDC identity. + args: release --clean env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - name: Attest build provenance diff --git a/.goreleaser.yaml b/.goreleaser.yaml index 574e0e4..ba0812b 100644 --- a/.goreleaser.yaml +++ b/.goreleaser.yaml @@ -57,6 +57,8 @@ dockers: goarch: arm64 build_flag_templates: - "--platform=linux/arm64" + - "--label=org.opencontainers.image.source=https://github.com/dcadolph/fleetsweeper" + - "--label=org.opencontainers.image.licenses=MIT" docker_manifests: - name_template: "ghcr.io/dcadolph/fleetsweeper:{{ .Version }}" diff --git a/README.md b/README.md index df7ae07..75c4a92 100644 --- a/README.md +++ b/README.md @@ -149,14 +149,20 @@ part. The outlier detection under it is the real part. ## Install for real ``` -helm install fleetsweeper deploy/helm/fleetsweeper \ +kubectl create namespace fleetsweeper +kubectl --namespace fleetsweeper create secret generic fleet-kubeconfig \ + --from-file=config=$HOME/.kube/config +helm install fleetsweeper deploy/helm/fleetsweeper --namespace fleetsweeper \ --set auth.token=$(openssl rand -hex 32) \ + --set kubeconfig.existingSecret=fleet-kubeconfig \ --set controller.enabled=true kubectl apply -f deploy/examples/clusterscan-prod.yaml ``` -The controller reconciles `ClusterScan` resources and writes outcomes back -to `.status`. Full installation paths in +The chart pulls the signed `ghcr.io/dcadolph/fleetsweeper` image and scans +every context in the kubeconfig Secret. Without the Secret it scans only the +cluster it runs in. The controller reconciles `ClusterScan` resources and +writes outcomes back to `.status`. Full installation paths in [`docs/operator/helm.md`](docs/operator/helm.md). Scoped API keys for pipelines in [`docs/operator/rbac.md`](docs/operator/rbac.md). diff --git a/UPGRADING.md b/UPGRADING.md index a184693..fc2cda1 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -27,6 +27,17 @@ No schema migration. stay open. The dashboard asks for a token on first load. `--insecure`, `--demo`, and a server with no credential at all keep anonymous reads. +- **The container image is published.** Each release pushes + `ghcr.io/dcadolph/fleetsweeper:` and `:latest` for amd64 and + arm64, signed keyless with cosign. The chart's default image now pulls. + +- **The chart can scan a fleet.** Set `kubeconfig.existingSecret` to a Secret + holding a kubeconfig and the server scans every context in it. Without one, + scheduled scans see the host cluster through the ServiceAccount, reported as + the context `in-cluster`. Before, a pod with no kubeconfig found no contexts + and scheduled scans did nothing. The ClusterScan controller stays on the + host cluster unless `--controller-context` names a context. + ## v0.14.0 No schema migration. Reports, scores, and a few API shapes change on the diff --git a/deploy/helm/fleetsweeper/Chart.yaml b/deploy/helm/fleetsweeper/Chart.yaml index ccbf600..5170ec4 100644 --- a/deploy/helm/fleetsweeper/Chart.yaml +++ b/deploy/helm/fleetsweeper/Chart.yaml @@ -2,8 +2,8 @@ apiVersion: v2 name: fleetsweeper description: Multi-cluster Kubernetes fleet comparison and audit type: application -version: 0.14.0 -appVersion: "0.14.0" +version: 0.15.0 +appVersion: "0.15.0" home: https://github.com/dcadolph/fleetsweeper sources: - https://github.com/dcadolph/fleetsweeper diff --git a/docs/index.md b/docs/index.md index 922eeb1..0a3ff38 100644 --- a/docs/index.md +++ b/docs/index.md @@ -36,14 +36,18 @@ Run both together. ## Three-line install ```shell -kubectl apply -f https://raw.githubusercontent.com/dcadolph/fleetsweeper/main/deploy/crds/clusterscan.yaml -helm install fleetsweeper deploy/helm/fleetsweeper \ +kubectl --namespace fleetsweeper create secret generic fleet-kubeconfig --from-file=config=$HOME/.kube/config +helm install fleetsweeper deploy/helm/fleetsweeper --namespace fleetsweeper \ --set auth.token="$(openssl rand -hex 32)" \ + --set kubeconfig.existingSecret=fleet-kubeconfig \ --set controller.enabled=true kubectl apply -f deploy/examples/clusterscan-prod.yaml ``` -Then visit the dashboard at port 8080 of the fleetsweeper service. +Create the `fleetsweeper` namespace first. The chart installs the CRDs and +pulls the signed `ghcr.io/dcadolph/fleetsweeper` image. Without the +kubeconfig Secret it scans only the cluster it runs in. Then visit the +dashboard at port 8080 of the fleetsweeper service and paste the token. ## Where to go next