Repository navigation
feat(scaling): Qdrant and pgvector stores, shared cache, OpenAI-compatible LLM, all off by default (Phase 8) - #9
Merged
Conversation
…n the base class Retrieval, ingest and workspace management no longer talk to Chroma directly: they go through VectorStore (list, ensure, exists, count, metadata, read_vectors, publish, delete, query). Chroma is the default and behaves exactly as before (same collection names, metadata, atomic swap by renaming, the unfiltered fallback for old collections), which the unchanged suite confirms. VectorStore.query() applies guard_hits() after the adapter's server-side filter: a chunk without a valid clearance, with one the caller may not read, or from another workspace is dropped. A broken filter in an adapter cannot leak, and a new adapter cannot forget it; a test with a deliberately leaky adapter pins that. Isolation: the new scaling settings are pinned off in tests and in the eval redirect, and an explicitly empty PGVECTOR_URI means off (the guard caught the fallback that built one from POSTGRES_PASSWORD).
…compose profiles VECTOR_STORE=qdrant|pgvector (default chroma, unchanged). Qdrant: a versioned collection behind an alias that moves in one atomic request; the model stamp is the collection's metadata. pgvector: one transaction per publish over rag_chunks/rag_workspaces in a configurable schema; no new Python package. Neither has an unfiltered fallback: a failing filter makes dense retrieval unavailable, labelled. The clearance guard stays in VectorStore.query() for both. The shared suite runs every adapter (Chroma, embedded Qdrant, a real Qdrant 1.20 server, a real pgvector server): similarity values within 1e-4 of Chroma's scale and of numpy's cosine, identical ranking against Chroma on the same corpus, server-side clearance filtering, workspace isolation in pgvector's shared table, and a failed publish leaving the old index serving. make eval passes through each real adapter with metrics identical to Chroma's (answer rate and recall 0.9773, abstention 1.0). Mutations (no server-side filter; the wrong score scale) fail the suite. qdrant-client is an optional extra (five small packages). Compose profiles qdrant, pgvector and vllm (loopback; make scale-up SERVICE=...); make up still starts only Postgres. The pgvector pool is closed at exit (Python 3.14 raises otherwise).
…E_BACKEND=postgres) Off by default; SQLite is unchanged. With CACHE_DB_URI several UI processes read and write one cache. The rows are sealed exactly as in SQLite (AES-GCM, row identity as AAD), so the database operator sees no question, answer, citation or vector, and a row moved between clearances still refuses to open. The two statements that differ between the databases (schema and upsert) exist once per backend; everything else is shared with only the placeholders swapped, and a literal % is refused. A replaced row gets a new serial rowid, which the in-memory index uses to notice it. BYTEA comes back as bytes. A connection pool replaces a connection per call. L1 is off in shared mode because another process's prune or delete could not evict it. The existing cache suites (oracle comparison, tamper, review-flag, retention, proxies) now run against both backends through a cache_backend fixture; the dict-style proxy had its own INSERT OR REPLACE, which a new test now guards.
…key, model, thinking, status LLM_PROVIDER=openai already existed but could not serve vLLM properly. Now: LOCAL_LLM_API_KEY (a secret, out of repr) instead of a hard-coded 'local'; LOCAL_LLM_MODEL instead of borrowing OLLAMA_MODEL; Qwen3's thinking, which is on by default in vLLM, is turned off with chat_template_kwargs (LOCAL_LLM_TEMPLATE_KWARGS, empty for servers that reject unknown fields), and any <think> block left in the text, or a cut-off one, is removed so it never reaches the graders; reasoning fields are ignored and a null answer is empty rather than the string 'None'. make status probes GET /models and says unreachable / key refused / model not served. Tested against a stub server speaking the vLLM API over a Unix domain socket (runs in both the online and the offline suite): request shape, bearer auth, thinking in every form, error paths, a real generate_node answer, and the labelled extractive fallback when the server fails.
… tests that pin the profiles docs/scaling.md covers each option (what it gives, what it costs, how to turn it on and off), the measurements (4,000 and 20,000 chunks: the stores differ by single milliseconds at the sizes this project targets), and what does not scale: the keyword index, uploads, the audit log, the policy and the keys stay local files, so several UI processes on one host work and several hosts need shared storage. CI starts the two services and fails (REQUIRE_QDRANT / REQUIRE_PGVECTOR) rather than skips if either is missing; the offline run skips only the Qdrant server tests, which need Python TCP sockets. A new test pins the compose shape (one profile each, loopback only, no secrets, vLLM asks for a GPU), the Makefile targets, and that every scaling variable is read by the code and documented.
… key The shared-cache pool registry and the vector-store instance cache were keyed by the connection string / API key. A dict key is printed by tracebacks and by pytest's assertion output, and an order-dependent test did exactly that. Both are keyed by a digest now, with tests that no key contains the secret, and the order-dependent test checks behaviour (the SQLite backend never opens a pool) instead of global state.
A fresh clone without qdrant-client failed it. Optional modules may fail to import only because exactly that package is missing; anything else still fails the test.
…reset purges them; eval namespaces are forced make status no longer reports healthy for a dead backend: a Vector store line (name and location, never a credential) and a Semantic cache line that actually asks the shared PostgreSQL, both [!!] and exit 1 when they do not answer. make factory-reset purges the workspaces of this install's Qdrant prefix / pgvector schema and the shared cache table before deleting the local folders, and prints NOT PURGED (exit 1) for anything it could not reach or could not empty, so its 'zero-trace' label is true. The eval redirect forces QDRANT_PREFIX=eval_ and PGVECTOR_SCHEMA=eval, so an exported real prefix or schema cannot receive the eval corpus. A test no longer passes the pgvector URI on argv. docs/scaling.md now states the trust boundary (Qdrant and pgvector hold every chunk in the clear; RBAC is this application's, not the store's) and what was checked live: Ollama's /v1 accepts chat_template_kwargs and ignores it (Qwen3 still thinks and returns it in a separate reasoning field, which the provider ignores), so the answer stays clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Everything is opt-in;
make upstill starts only the checkpoint database. Details and measurements:docs/scaling.md.VectorStoreinterface with Chroma still the default. The clearance guard is in the base class, so an adapter with a broken filter cannot leak a chunk (a deliberately leaky test adapter proves it). Qdrant (alias swap) and pgvector (one transaction) publish atomically, have no unfiltered fallback, and match Chroma's similarity scale.CACHE_BACKEND=postgres, sealed exactly as before; the in-process L1 cache is off in this mode.LLM_PROVIDER=openaifor vLLM, LM Studio or llama.cpp, with key, model id,chat_template_kwargsand<think>stripping;make statusprobes it.qdrant,pgvector,vllm, started bymake scale-up SERVICE=..., all on loopback.make statusasks the vector store and the shared cache (exit 1 if dead);make factory-resetpurges the configured external stores and printsNOT PURGEDwhen it cannot;make evalforces its own namespaces.Read before merging
docs/scaling.md, trust boundary).qdrant-client.VECTOR_STORE,QDRANT_URL/API_KEY/PREFIX,PGVECTOR_URI/SCHEMA,CACHE_BACKEND,CACHE_DB_URI,LOCAL_LLM_API_KEY/MODEL/TEMPLATE_KWARGS(see.env.example).Verification
Full gate on this tree (Postgres, Qdrant, pgvector up;
REQUIRE_*set;RAG_ACCEL=required): 898 passed, 6 skipped online; 885 passed, 19 skipped offline; lint and typecheck clean.make evalgives identical numbers through Chroma, a real Qdrant 1.20 server and a real pgvector server (answer rate and retrieval recall 0.9773, abstention accuracy 1.0). The later rebase kept this tree byte-identical. Clean checkout, offline, no services: 833 passed, 70 skipped;test_evalpasses oncemodels/is present.