Repository navigation
feat(inmemory): optional max_entries LRU capacity without changing defaults (#54) - #85
Conversation
757a216 to
dbd66b8
Compare
…expired reclaim (#54)
Exact-head CI complete — ready for independent counteraudit, NOT ready to merge (2026-11-21)Audited candidate SHA: All latest-HEAD workflows completed SUCCESS:
Reproducible scaling from the current exact-head CI:
100k writes on Python 3.8: 0.493 s fill; 0.686 s churn; 1.715 s expiry reclaim. Python 3.12: 0.331 s / 0.451 s / 1.099 s. Microbenchmarks only: do not infer production request latency or payload-byte memory limits. No claim of proven asymptotic complexity from one timing run. Specific counteraudit questions: independently validate Decision: technical CI preflight PASS; independent external code review and risk sign-off still OPEN. Do not ready/merge #85, amend 0.5.0, release 0.6.0, or count this test work as an independent consumer. Refs #54 #86 #88 #91. |
Counteraudit request — fix for mutable
|
Final counteraudit handoff — exact HEAD is CI-clean (2026-11-21)Exact HEAD: New exact-head verified workflows — ALL SUCCESS:
10k → 100k capacity mean per-write cost ratio from exact-head CI (backend-only microbenchmark, not request latency):
Independent reviewer decision needed: Verify (1) that all public post-construction attempts to mutate Proposed disposition: CI preflight GO; await independent counteraudit GO/NO-GO before ready/merge. This success does not authorize a 0.6.0 tag/RC/PyPI, external marketing or 0.6.0 launch. Release checklist #92, security review #53, external consumers #91 (0/2) and Pages #48 remain separate gates. |
Purpose
Implement #54 as an opt-in feature for a future minor release (candidate 0.6.0). Published
0.5.0remains unchanged.Contract
InMemoryBackend(max_entries=None): historical per-instance, unbounded and lazy-expiration behavior unchanged.InMemoryBackend(max_entries=N), with positive integer N: keep at most N entries in each backend instance. Track recency on successfulget/get_with_ttlandset, evict the least recently used when full, and use an indexed min-heap of expirations to reclaim one expired key only when a new insertion needs space; updates do not scan the store.expire=None, TTL andclear(namespace/key)semantics, private/auth safety and all legacy default-key vectors. Costs: O(log N) expiry-index insert/update/discard with an exact per-key heap (no stale tombstones, no periodic O(N) compaction); LRU is O(1). Explicit namespace clear scans N keys and removes k matching keys from the indexed heap: O(N + k log N), O(N log N) in the worst case, under the lock. This bulk operation is not part of ordinary writes.Verification
Reviewer questions
Audit the indexed heap for reordering/removal and tie/expiry boundaries, verify Python 3.8/3.12 10k/100k full-capacity churn and TTL reclaim scaling, fairness under concurrent access, Pydantic/FastAPI integration, and the key/token privacy impact. This bounds entry count only, not object size. Refs #54 and future 0.6.0 tracking.
Performance blocker resolution (review required)
A reviewer reproduced O(N) per-write costs at capacities 1k–10k in the original implementation. The current code replaces that design, using a bounded indexed min-heap for expiration and evicting at most one entry per new full-capacity write. No stale expiry generations or growing heap tombstones. The scripted 10k/100k benchmark is a new Python 3.8/3.12 CI gate. Do not merge until the exact final HEAD benchmark, type/lint/security and migration checks pass and independent code review confirms the heap's index invariants.
Base synchronization and benchmark update (2026-11-21)
stablecommit8c079c8dbca645f3a2c87cf971c84a6192cbed41. All branch changes were preserved as one new commit using a guarded ref update; historical individual commit hashes were replaced. GitHub compare confirmed behind 0 / ahead 1, originally only the five functional/CI files, with no deleted documentation. Subsequent benchmark and fuzz additions remain scoped to feat(inmemory): optional bounded cache capacity and eviction #54.fill,full_lru_churn(one entire capacity of new keys beyond a full cache), andfull_expiry_reclaim(full cache of expired entries, then one new key per old entry), each at 10k and 100k on Python 3.8 and 3.12. Scaling fails closed if per-write cost grows more than 4.5x.Counteraudit handoff — synced exact-head candidate
c33aa6f571330299cf5606802e0db811836404b7(production implementation unchanged after merge-sync; only adversarial tests added).stableatd36a21388f25ed9d9528d4ac8ed832a76b571eed(docs(pages): prepare manual hosting, public API and alternatives for 0.6.0 (#48) #89 + docs(migration): runnable published before-and-after demo for adoption #90); GitHub compare verified behind 0. The six existing feat(inmemory): optional bounded cache capacity and eviction #54 files are retained, plustests/test_bounded_inmemory_adversarial.py.get_with_ttlrefreshes LRU; a canceled waiter waiting for the async lock cannot alter the store/index; strictexpire=0boundary matches pre-existing semantics.max_entriesafter construction (attribute is writable; not supported as runtime reconfiguration unless explicitly implemented/documented). Validate single-event-loop/per-process assumption; asyncio locks do not establish thread-safe cross-loop sharing. Confirm performance threshold measures per-write cost, not payload-byte memory or request latency.