Skip to content

Improve pChronicle cache reuse and tolerant batch imports - #142

Merged
reiase merged 2 commits into
mainfrom
feature/pchronicle-cache-import
Oct 2, 2026
Merged

reiase merged 2 commits into
mainfrom
feature/pchronicle-cache-import

Conversation

@reiase

@reiase reiase commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Repeated remote Lance reads can now reuse cached blocks and metadata across opens, with shared download coalescing and observable cache statistics. Imports also tolerate incomplete ACTF metadata and skip invalid files in directory batches even when an explicit input format is selected.

  • Share block-cache state and Lance wrapper identities, cache full reads through aligned blocks, and invalidate remembered metadata after successful mutations.
  • Persist access times for eviction, leave capacity headroom, and keep reads working when local cache writes fail; support configurable 1 MiB blocks and propagate cache settings to catalog workers.
  • Normalize optional ACTF metadata consistently across decoding paths and fall back to full decoding when projected reads cannot handle an incomplete trajectory.
  • Preserve strict single-file import errors, add regression tests, and document cache behavior and troubleshooting in English and Chinese.

Validation

  • Rust/Python formatting and whitespace checks passed.
  • Strict bilingual documentation build and checks passed for 104 HTML pages.
  • just test on the initial PR commit: 872 Rust tests passed (2 skipped), followed by 29 Python tests passed.
  • CI reported two collapsible_if errors in the cache registries. Commit f210c33 collapses those conditions without changing behavior; just fmt-check-rust and both commands in just clippy-deny passed locally after the fix.
  • GitHub Actions passed on f210c3346def7c9f1c5e64a0a39ef1705ec6db39: formatting, Clippy (including storage features), Linux/macOS Rust tests, property tests, Web, real S3/MinIO, Python, examples, documentation, and baseline/candidate benchmarks. The aggregate CI gate is green. Documentation deployment is intentionally skipped for pull requests.

@reiase
reiase merged commit a9c6438 into main Oct 2, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant