Skip to content

feat(inmemory): optional bounded cache capacity and eviction #54

Closed

Description

@agustin18

2026-11-21 IMPLEMENTATION COMPLETED — source merged, package unpublished: PR #85 has been MERGED into stable as 190d105915972d436ddbf4ad152ad654a4db392a, following independent review GO on exact HEAD 098e37cb..., all 29/29 CI and 13/13 migration checks. The max_entries property is read-only after validated construction; bounded LRU/expiry heap semantics remain opt-in and per-instance. Closing #54 as source-implementation complete, not as shipped/released: published 0.5.0 is still unbounded, and #92 governs 0.6.0 packaging, external consumers, security and release. Documentation reconciliation draft #93 keeps 0.5.0 and the unreleased new API separate. Earlier pre-merge notes below are historical.


2026-11-21 final mutability-fix HEAD (counteraudit-ready): Draft #85 is now at 098e37cb4530a3d4fd282080e2d0c3251e089f4a, behind 0, with the public max_entries constructor cap read-only and direct regression for the reproduced 2→1 mutation exploit. Latest-head CI 29/29, real Redis Phase B 13/13, CodeQL/security/Redis/changelog all SUCCESS. Python 3.8/3.12 10k/100k capacity fill, LRU churn, expiry reclaim both pass. Final evidence/comment. Independent signoff remains REQUIRED, PR stays DRAFT, no merge/release.


2026-11-21 exact-head audit readiness: #85 at c33aa6f571330299cf5606802e0db811836404b7 is synchronized (behind 0), remains DRAFT, with all current-HEAD workflows SUCCESS: CI 29/29, migration Phase B 13/13, Redis, CodeQL and installed security. Benchmarks Python 3.8/3.12 both PASS for 10k/100k fill, live churn and TTL reclaim. Exact reproducible evidence and counteraudit targets. NO-GO for merge until independent heap/concurrency contract review and explicit approval, not until more tests merely authored by this assistant.


2026-11-21 counteraudit candidate after #89/#90 merges: Draft #85 merged both documentation commits into its branch via two-parent commit afc05de and GitHub compare confirms behind 0 relative to stable. Added independent adversarial heap/LRU/cancellation/TTL-boundary regression suite (12 seeds × 1,800 index mutations) without changing production implementation. Exact audit candidate HEAD c33aa6f571330299cf5606802e0db811836404b7. In fresh CI run 44873650357, Python 3.8/3.12 10k/100k fill, full-LRU churn and expired-entry reclaim benchmark lanes both passed: Python 3.8 per-write growth 0.986x / 1.127x / 1.007x; Python 3.12 0.877x / 1.210x / 1.311x respectively. These are single-run CI backend microbenchmarks, not HTTP throughput or memory-byte bounds. Remaining exact-head CI/Redis/migration lanes and independent reviewer sign-off must complete before any merge. #85 is still DRAFT. Review the writable max_entries constructor-only contract and single-event-loop lock assumptions. Older benchmark evidence below is historical.


2026-11-21, updated PR #85 scope: Squash-rebased after #87 onto stable commit 8c079c8 with behind 0 / ahead 1 and no deleted docs. CI now benchmarks 10k/100k fill, full-capacity new-key LRU churn, and expired-key reclaim on Python 3.8/3.12. Added seeded state-machine regressions (20 × 2,000 operations) checking store, LRU recency, heap positions and expiry. clear(namespace) complexity is O(N + k log N), worst O(N log N) for N stored/k matching keys; explicit bulk clear holds the async lock, unlike ordinary O(log N) writes. The previous fill-only benchmarks do not sign off this new HEAD. No merge or release; await complete CI and independent invariant review. See #85 and #86.


Benchmarks independently executed in CI (2026-11-21, earlier reviewed head 6c4873614a6827652a7a4f141b356a2b6f2d104f): Python 3.8: 10k inserts 0.03630 s, 100k inserts 0.36470 s (0.00363 vs 0.00365 ms/write, ~1.00x per-write growth). Python 3.12: 10k 0.02764 s, 100k 0.23110 s (0.00276 vs 0.00231 ms/write). CI benchmark run 44857706376. Both benchmark jobs passed. Prior job's 384 passed / 18 skipped still had one missing changed-line coverage branch, specifically heap upward repair after middle key deletion. A targeted regression was added in newer HEAD 7c74a093; its final checks still need to pass. Timings are isolated single-run CI microbenchmarks, not production latency guarantees or direct machine-matched comparisons with the earlier local audit.

Performance blocker addressed in draft (not yet signed off): PR #85 replaced O(N) per-write _prune_expired() scans with an exact indexed expiry min-heap (O(log N) updates/eviction, no stale records) and runs a 10k/100k capacity scalability gate on Python 3.8 and 3.12. Existing unbounded default unchanged. No merged code, no PyPI publication. Before merge, require final SHA CI/benchmark results, independent heap index/TTL/clear review, per-process bound and bytes-limit documentation. The earlier issue text below is historical.



Target: 0.5.x, with a targeted 0.4.x follow-up only for verified correctness issues

Observed behavior in 0.3.0

InMemoryBackend owns a per-instance dict, stores values on set, and lazily removes expired values only when those keys are read. No size or item count cap exists; in a long-lived process with many distinct keys, memory can grow with the number of entries even when old entries have expired.

Scope

  • First benchmark memory and count growth over repeated unique keys and expired entries, including large payloads and concurrent async callers.
  • Design opt-in max entries and possibly max bytes (bytes accounting caveats explained). Choose bounded eviction (LRU or simpler consistent policy) with clear observability, expiration-on-write/read or incremental sweeping, fairness and lock contention constraints.
  • Avoid a default behavior change in 0.3.x. Preserve per-instance isolation, expire=None semantics, clear(namespace/key) behavior and the existing backend protocol.
  • Ensure no unbounded sweeper task, leaked background task or event-loop coupling; use deterministic fake-clock tests and stress tests.

Done when

Documented opt-in bounds/eviction policy with consistent TTL semantics, tests proving bounded growth under unique keys, sensible performance measurements, and compatibility with existing applications. Don't conflate this with Redis memory policies.

Source: InMemoryBackend.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions