Repository navigation
workloadmanager: emit Kubernetes audit events for GC-triggered sandbox deletions - #515
SoumaCloud wants to merge 1 commit into
Conversation
The garbage collector deleted sandboxes (both idle-timeout and expired) without emitting any Kubernetes Event. Operators using `kubectl get events`, Loki, Falco, or the K8s audit sink had no way to tell whether a sandbox disappeared due to a GC sweep or an unexpected error. This commit introduces pkg/workloadmanager/audit.go, which provides a single sandboxAuditEvent helper that creates a corev1.Event in the sandbox's namespace. The helper is nil-safe (returns early when the clientset is nil, as in unit tests that build K8sClient without one) and recovers from panics so it can never abort a caller's control path. garbage_collection.go now calls sandboxAuditEvent(AuditReasonDeleted) immediately after each successful sandbox/sandboxClaim deletion, before the store record is removed. The Event carries: - Reason: "SandboxDeleted" - Type: Normal - Message: human-readable description including namespace, name, session ID - Annotation agentcube.io/owner-hash: SHA-256-shortened owner ID (not the raw sub claim) for correlating events without storing PII. Five unit tests in audit_test.go cover the Normal/Warning type split, per-reason annotation behaviour, empty owner-hash, and nil-clientset safety. Fixes: volcano-sh#513 Signed-off-by: Soumadip Patra <soum23dip@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #515 +/- ##
==========================================
+ Coverage 60.45% 64.18% +3.72%
==========================================
Files 35 36 +1
Lines 3616 3691 +75
==========================================
+ Hits 2186 2369 +183
+ Misses 1211 1089 -122
- Partials 219 233 +14
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Problem
The garbage collector (
gc.once()) deletes sandboxes — both idle-timed-out andTTL-expired — without emitting any Kubernetes Event. Operators relying on
kubectl get events, Loki, Falco, or the K8s audit sink had no observablesignal to distinguish a GC sweep from an unexpected disappearance.
This was confirmed by grepping the codebase:
Closes #513
Solution
Introduces
pkg/workloadmanager/audit.gowith asandboxAuditEventhelperthat emits a
corev1.Eventin the sandbox's namespace. The helper is:tests that construct
K8sClientwithout a full API server)recover()so a failing eventemission can never abort the caller's deletion path
V(2)and do not block the GC loopgarbage_collection.gonow callssandboxAuditEvent(AuditReasonDeleted)immediately after each successful Kubernetes deletion (before the store record
is removed). Each Event carries:
ReasonSandboxDeletedTypeNormalMessagesandbox <ns>/<name> GC-deleted (session <id>)Source.Componentagentcube-workloadmanageragentcube.io/owner-hashTests
Five unit tests in
audit_test.gocover:SandboxCreated→EventTypeNormalSandboxCreateFailed→EventTypeWarningSandboxDeletedwith correct annotationownerHash→ no annotation writtenAll existing GC tests continue to pass (
go test ./pkg/workloadmanager/...).Signed-off-by: Soumadip Patra soum23dip@gmail.com