Zvec 0.7.0: quantization, RaBitQ, per-index search params, group-by, export

[TheAli711]

Upgrades the engine to Zvec 0.7.0 and exposes its new capabilities, plus concurrency fixes found along the way. Intended for v0.2.0 (contains breaking changes — see below).

Upgrade

• Require zvec>=0.7.0. Fixes the mmap empty-id bug (Failed to find target chunk): 9,279 errors → 0 on the smoke benchmark, recall back to no-mmap parity.
• Shutdown explicitly close()s every collection (releases Zvec's on-disk lock immediately — helps rolling restarts).
• optimize holds the shared lock, so searches keep flowing: 8 queries in 2.1 s (max 2,112 ms) before → 2,609 queries (max 19 ms) after.

Features

• Quantization params (quantize_type fp16/int8/int4, enable_rotate) on hnsw/flat/ivf — opt-in; measured on SIFT1M it costs disk, memory and recall without a speed gain, documented as such.
• hnsw_rabitq / ivf_rabitq index types (Linux x86_64 servers; clean 422 elsewhere).
• use_soar for IVF.
• Search params for every index type (ef, nprobe, radius, is_linear, is_using_refiner, scale_factor).
• POST /collections/{name}/search/group-by.
• GET /collections/{name}/export — NDJSON snapshot stream, re-importable, doesn't block writes, in-band error line if cut short.

Fixes

• Collection names validated against Zvec's 3–64 char limit (were misreported as 409).
• drop() waits for in-flight requests; queued requests never run on a destroyed/closed handle.
• Background recovery can no longer resurrect a dropped collection.
• Export cursor released immediately on client disconnect.

Breaking changes

• Unknown/mistyped keys in index params → 422; in search params → 400 (previously ignored — e.g. ef on a flat/ivf field).
• Collection names must be 3–64 chars (names outside that never worked in the engine).

Tests & benchmarks

• New tests for every feature/fix incl. concurrency races (drop/close vs optimize, export vs drop/close/disconnect, recovery vs drop). Every commit passes ruff/mypy/pytest individually; full suite also passes on linux/amd64 (RaBitQ path).
• Benchmarks: --quantize/--rotate, quant sweep, optimize-load; fixed SIFT1M download (403) and inproc runner locking.

🤖 Generated with Claude Code