Repository navigation
test(aifirewall): property tests for Plan and ValidateGuardrail; plan server maps in key order - #326
Conversation
Vulnetix config planManager
0 to create, 1 to update, 0 to delete, 0 unchanged |
0x73746F66
left a comment
0x73746F66
left a comment
There was a problem hiding this comment.
Reuse review: the plan fix generalises the existing sortedKeys helper to sortedKeys[V any] and routes every server-map loop in Plan through it, instead of adding a second sorter. (Optional: its body could be slices.Sorted(maps.Keys(m)) from the stdlib; not required.) The property tests use math/rand with fixed seeds and the real Plan/ValidateGuardrail. No changes needed.
0x73746F66
left a comment
0x73746F66
left a comment
There was a problem hiding this comment.
Reuse review: no reinvented Vulnetix patterns found. The change stays inside the package it covers, uses stdlib/testify helpers already used by the repo's tests, and adds no new client, auth, config-schema or shared-package duplicate (checked internal/testutils and sibling repos). Rebased onto current main locally: go vet and go test pass for every touched package. The earlier red checks (CodeQL Analyze, Install a release with verified provenance, Lockfile) were GitHub API rate-limit failures, not code failures; this will be re-run on a rebased head before merge.
… server maps in key order Adds testing/quick properties over generated policy files and server states: Plan runs in kind order, is deterministic, and converges (applying its mutating changes leaves nothing to change); ValidateGuardrail accepts exactly the enforceable rule type and action pairs and answers for any input. The determinism property failed: Plan ranged over server.Guardrails and server.Providers directly, so guardrail drift/deletes and provider drift came out in map order, different on every apply. Those loops now go through sortedKeys, made generic, as the model loop already did. Closes #222
ae43f1d to
ad8a89d
Compare
Vulnetix config planManager
0 to create, 1 to update, 0 to delete, 0 unchanged |
Closes #222
What
New
pkg/aifirewall/plan_property_test.go(stdlibtesting/quick, likeinternal/versionsandpkg/tea):TestPlanRunsInKindOrder: guardrail < model < provider < key < settings for any generated file/server pair.TestPlanIsDeterministic: equal inputs give the same plan in the same order.TestPlanConverges: applyingMutating(Plan(...))to the server and planning again yields no mutating change. Provider keys and anyProvider models are not generated: both are re-sent on every apply by design and stay covered by the example tests.TestValidateGuardrailAcceptsExactlyTheEnforceablePairs: all 12 rule types x 4 actions against an explicit table of the pairs the gateway enforces (delimiter_integrity block/flag; redact only on content rules; strip only on tool/mcp).TestValidateGuardrailIsTotal: arbitrary strings never panic and an unknown rule type or action is always refused.Bug found and fixed
TestPlanIsDeterministicfailed on main:Planranged overserver.Guardrails(prune deletes and drift) andserver.Providers(drift) directly, so those changes came out in Go map order, which differs between runs;applyprinted and executed them in a different order each time. Both loops now usesortedKeys(made generic), which the model loop already used for the same reason. No behaviour change beyond the order.Verified
go test ./pkg/aifirewall/passes (existing tests unchanged, includinghelpers_test.go'ssortedKeyscase);go vet,gofmtclean.