test: add policy SQL tests for rightsizing, snapshots, container registries and app service plans - #10
Conversation
…stries and app service plans Adds a shared run_policy fixture in tests/policies/conftest.py and unit tests for four policies: rightsizing_utilization (idle vs underutilized grading, metric averaging, the 5% and 20% CPU limits, cost summing, case-insensitive join), old_snapshots (the 30 day boundary and cost), idle_container_registries and idle_app_service_plans (pricing tiers). Refs raphgm#6.
raphgm
left a comment
raphgm
left a comment
There was a problem hiding this comment.
Verified: pulled the branch, ran all 42 tests locally (35 new + 7 pre-existing), all pass. Spot-checked test_old_snapshots.py against the real policy SQL — the 30-day boundary and $0.05/GB cost formula both match exactly. Good methodology described in the PR body (mutation-tested each policy 17 ways to confirm the tests actually catch regressions). Author has a real, established account (10 months old, 36 repos, 17 followers). No overlap with the pricing fixes I just pushed to redis_idle/stopped_not_deallocated_vms/vmss/aks/batch — different policies entirely. Approving.
|
Thanks for the thorough review and the quick merge. Glad the mutation checks were useful. If it would help, I can add tests in the same style for the policies you just changed the pricing on (redis_idle, stopped_not_deallocated_vms, vmss, aks and batch), so those fixes stay fixed. Or tell me which policies you'd like covered next and I'll start there. |
Adds the 5 checks built since the last README update (idle_batch_pool, idle_premiumv2_disk_overage, cosmosdb_mongo_idle_ru, cosmosdb_cassandra_idle_ru, idle_container_apps_dedicated), notes the live-pricing fixes to stopped_not_deallocated_vms and redis_idle, and adds a Testing section documenting 'pytest tests/ -v' (verified: 45/45 pass) now that #10 merged real unit test coverage. Also removes the stale 'needs a proper test suite' line from the contributing pointer since that's no longer true.
2 participants
Adds unit tests for four more policies, following the pattern from the aks_idle_nodepool tests. They run each policy's SQL through PolicyRunner against small in-memory DuckDB tables, so no cloud access is needed.
tests/policies/conftest.py: a sharedrun_policyfixture, so each test file only builds its input tables.rightsizing_utilization: idle versus underutilized grading, each network and disk axis on its own, the 5% and 20% CPU limits, metric averaging, cost summing, the case-insensitive resource ID join, and the exclusions (non-VM services, zero cost, no metrics).old_snapshots: the strict 30 day boundary and the cost formula.idle_container_registriesandidle_app_service_plans: the pricing tiers (Standard and Premium, Free versus paid).To check that the tests can actually fail, I temporarily broke a copy of each policy in 17 ways (changed a boundary, a price, a join, an aggregate, a filter), and every one made at least one test fail. The policy files are unchanged in this PR. All 45 tests pass locally with
pytest.Refs #6. More policies can follow the same pattern if you would like them.