Repository navigation
feat: first working engine with the RAG course techniques - #4
Merged
Merged
Conversation
A CLI engine on LangChain (ADR-0006) that runs locally on a laptop CPU: - ingest Markdown, text and PDF with recursive, Markdown-aware or semantic chunking; idempotent per content hash - hybrid retrieval: Chroma vectors (similarity, threshold, MMR, metadata filters) + BM25, fused with reciprocal rank fusion - multi-query expansion fused with RRF, cross-encoder reranking, history-aware follow-ups - cited answers from a local Qwen3 4B through Ollama; citations parsed and checked, abstains without evidence - per-stage trace, retrieval evaluation (hit rate, MRR, recall@k) with a config fingerprint - example corpus (English and German) with an evaluation set; CI with lint and an 80% coverage gate
…fline mode - workspace: categories and sections, each section a set of folders with its own index - library: background indexing with progress (one job at a time), pruning of deleted files while keeping documents of unplugged folders, answers scoped to a section or fused across a category (RRF, one rerank) - ore serve: FastAPI on 127.0.0.1 with a per-launch bearer token, loopback Host check and CORS for the app origin; answers stream as NDJSON; errors are RFC 9457 problem details - offline mode: telemetry off, no progress bars, HF_HUB_OFFLINE when every model is on disk - llm provider 'openai' for OpenAI-compatible local servers such as llama.cpp's llama-server - ADR-0007: WPF + WebView2 + React desktop app over the local engine API
3 tasks done
- WPF shell (.NET 10) hosting a React 19 + Tailwind 4 interface in WebView2, in the AI Engineering Arena's sticker-arcade theme (light and dark, fonts bundled for offline use) - sidebar of categories and sections; per section: folders, background indexing with progress, documents and a chat; a category chat searches all its sections; citations as pills and source cards that open the file - the app starts ore serve itself (bundled Python when installed, uv in a checkout), ties it to a job object and routes every API call through an EngineProxy that adds the token and allows only the engine's API paths, so the page never sees the token or talks to a local port - conversations kept per section on this computer; follow-up questions use the recent history - ore serve: --data-dir and --warm; python -m open_rag_engine - tests: 36 .NET (engine start, ready line, locator, proxy, safe file opening), 13 web, engine suite; Windows CI job
3 tasks done
The standard Windows title bar is replaced by one drawn in the Arena's top-bar style: the RAG sticker, the app name, where you are (category > section) and Arena window buttons. It is a WebView2 drag region, so dragging, double-click to maximise and the window menu still work; WindowChrome keeps resizing from a thin border, rounded corners on Windows 11, and the frame follows light or dark mode.
The global keyboard-focus outline was unlayered CSS, so it beat the text area's outline-none and drew a rectangle inside the rounded box. It now lives in the base layer, and the whole box shows focus with a cobalt border and shadow.
A bundled engine loads its models from models_dir/<org>--<name> under the same model names, so the index fingerprint and existing indexes stay valid. --llm-url points the engine at the llama-server the app starts.
- chat: a late reply after switching section can no longer land in, or be saved under, another conversation; saving is debounced and done outside the state update; the chat resets if the engine restarts (+ regression tests) - a connection lost mid-answer becomes a 503 instead of an unhandled exception; every forwarded call and question now always gets an answer for its id; library calls time out instead of hanging - opening sources: only local document paths (no UNC, device or alternate-stream paths), checked off the UI thread; explorer.exe by full path - engine lifecycle: start and restart are serialised, deliberate stops aren't reported as crashes, a start that times out stops the process, only the current engine can report a crash, and unexpected start errors show a message - WebView2: content-security policy on the built page (connect-src 'none'), DenyCors mapping, all permissions denied, no host objects; HttpClient bypasses system proxies - engine: Stop now really stops generation (the next token raises once the client is gone); models load once under a lock, so the warm-up and a first question can't load them twice - proxy: exact allow-list match (\z), questions only via the stream, problem bodies built with a JSON serialiser - deleting a section or category also deletes its saved conversation; checkout discovery is Debug-only
The Python .gitignore template ignored every lib/ folder, so desktop/web/src/lib (the engine client, the WebView2 bridge, chat storage, the mock engine and their tests) never reached the repository and a fresh checkout couldn't build the interface. The rule now covers only the root lib/ and lib64/.
installer/build.py stages a self-contained Open RAG and runs Inno Setup: the published app, a uv-managed CPython 3.12 with the engine and its locked dependencies (CPU torch), the embedding and reranker models from the Hugging Face cache, the Qwen3 GGUF from Ollama (digest-checked) and llama.cpp's llama-server (pinned b12052, sha256-checked). The setup installs per user, keeps the library on uninstall and comes in parts below GitHub's 2 GiB limit. The app starts the bundled llama-server on a free loopback port with its own key and points the engine at it with --llm-url. ChildHost replaces EngineHost and runs both processes, waiting on ORE_READY or a /health probe. Logs rotate at 5 MB. installer/smoke_test.py checks a stage or an installed app end to end. The window also opens the credits' GitHub and LinkedIn links, and only those (ExternalLinks).
The sidebar says who developed Open RAG and opens a Credits sheet with both authors' GitHub and LinkedIn and the source code. Links open in the default browser through the window, which accepts only the links on its own list; a test checks the sheet's links against it.
feat: Windows installer with a bundled language model, plus credits
feat: Open RAG, a Windows desktop app with categories and sections
feat: local API for the desktop app, with sections, categories and offline mode
reset() deleted the chroma folder after delete_collection(). Chroma caches its client per folder and keeps the database open, so on Linux, where the delete succeeds, the next write hit a deleted file (SQLite 1032, readonly database) and CI's rebuild test failed on every run. Windows refused the delete, which hid it. delete_collection() already removes the vectors. Also ignores Vite's cache folder.
Leads with what the app does and how to install it from the release, shows the interface in dark and light mode and the credits, explains the answer pipeline and the desktop processes with two diagrams, and updates the roadmap. The screenshots use demo folders (our RAG lesson scripts and the example pump manual) with the real engine and the bundled model.
GhassenJamoussi99
approved these changes
Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The first working engine: a command-line RAG built with the techniques from the course we just finished, on LangChain, running fully locally on a laptop CPU.
chunking.py[chunking] strategystore.py,factory.py[embeddings] modelretrieval.py[retrieval] search_type,--sourceretrieval.py,fusion.pyvector_weight,keyword_weight,rrf_kfusion.py,pipeline.py--multi-queryrerank.py[rerank],--no-rerankconversation.pyore chatanswer.py[answer] min_rerank_scoreevaluation.pyore evalAlso:
--rebuild, so an index is never mixed.--trace).ore.toml.Default models. All are free, local and permissively licensed (C-02):
intfloat/multilingual-e5-small(MIT) for embeddings.cross-encoder/mmarco-mMiniLMv2-L12-H384-v1(Apache-2.0) for reranking.qwen3:4b-instruct-2507(Apache-2.0) through Ollama for answers.Two notes on the defaults:
qwen3:4btag is now the 2507 thinking release. It can't switch its reasoning off, and on our CPU that cost about 3 minutes per answer, so the default is the Instruct release.[llm] think), and any<think>text is hidden from the output and never parsed for citations.ADR-0006 records the choice of Python and LangChain, and how it fits ADR-0002 and ADR-0003:
RetrievalPipelinekeeps the fixed slots.langchain-communityandlangchain-experimentalpackages are confined to four adapter modules, and a test enforces it.Builds on the design docs merged in #3.
Why
It is step 2 of the roadmap: ingest → hybrid retrieval → reranking → cited answer, plus the evaluation harness.
How it was tested
ruff checkandruff format --checkare clean.Reranking moves the right file up the list. The recall dip is the French question with two expected files, where one fell to 6th.
protection.md. The reranker lifted that file from 4th to 1st.service.md.Known limits (next PRs)
Checklist
docs/, or a new ADR for architectural decisions)