Skip to content

feat: local API for the desktop app, with sections, categories and offline mode - #5

Merged
TWadi merged 13 commits into
twadi/rag-enginefrom
twadi/local-api
Nov 2, 2026
Merged

TWadi merged 13 commits into
twadi/rag-enginefrom
twadi/local-api

Conversation

@TWadi

@TWadi TWadi commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

What

The engine side of the desktop app (step 1 of 3). The app itself and the self-contained installer follow in their own PRs.

Users and their documents:

  • Workspace: categories (a subject or research area) and sections (a course, session, project or reading list). Each section is a set of folders with its own index.
  • Library: the service behind the app.
    • Indexing runs in the background with progress, one job at a time on a laptop CPU.
    • Re-indexing drops files that were deleted.
    • Documents in a folder that is temporarily missing, such as an unplugged USB drive, are kept.
    • Answers come from one section, or from a whole category: each section is searched, the lists are fused with RRF, and the fused candidates are reranked once.
    • Asking a section while it is indexing is a clear 409, not a half-built index.

The local API (ore serve):

  • FastAPI, reachable only from this machine:
    • it binds to 127.0.0.1 and rejects non-loopback Host headers (DNS rebinding);
    • every request needs a per-launch bearer token (ORE_API_TOKEN, at least 32 characters);
    • CORS allows only https://app.ore, the app's WebView2 origin.
  • It prints ORE_READY port=<n> once it listens, and --parent-pid makes it exit with the app.
  • Answers stream as NDJSON: tokens, then the cited answer with each source's path, page, section and a snippet.
  • Errors are RFC 9457 problem details, with no stack traces and no document content.

Offline and quiet:

  • Telemetry is off.
  • No progress bars.
  • When the embedding and reranker models are already on disk, HF_HUB_OFFLINE is set, so nothing calls Hugging Face. That also removes the "unauthenticated requests" warning.

LLM provider openai: for OpenAI-compatible local servers. This is what lets the installer bundle llama.cpp's llama-server instead of Ollama, which is too big to ship.

ADR-0007 records the desktop-app design: a WPF shell, a React interface in WebView2, and the engine as a hidden child process over this API.

Stacked on #4. Merge #4 first. GitHub then retargets this PR to main.

Why

Wadi asked for an offline Windows app for teachers, researchers and students. They organise their documents into categories and sections, and the RAG answers only from the section (or category) they pick.

How it was tested

  • Tests: 112 in total, 96% coverage, ruff clean. The new ones cover:
    • workspace rules;
    • background indexing, pruning and missing folders;
    • section isolation ("a section answers only from its own documents");
    • category fusion;
    • the API's token, loopback-host and CORS guards;
    • NDJSON streaming;
    • problem details;
    • the 409 while indexing.
  • Smoke test with the real models on CPU, over HTTP against ore serve:
    • no token gives 401;
    • created a category and a section, added examples/docs, indexed 10 files;
    • section question: "How much electricity does the pump use?" gave "The pump draws up to 1.2 kW at full speed [2]" (power.md), streamed in 12.5 s;
    • category question: "Wie viel kostet der Drucksensor?" gave "Der Drucksensor PS-12 kostet 39 Euro [1]" (service.md).

Checklist

  • Tests cover the change
  • Docs updated (README, docs/, or a new ADR for architectural decisions)
  • No secrets, keys or private data in the diff

…fline mode

- workspace: categories and sections, each section a set of folders with its own index
- library: background indexing with progress (one job at a time), pruning of deleted files while keeping
  documents of unplugged folders, answers scoped to a section or fused across a category (RRF, one rerank)
- ore serve: FastAPI on 127.0.0.1 with a per-launch bearer token, loopback Host check and CORS for the app
  origin; answers stream as NDJSON; errors are RFC 9457 problem details
- offline mode: telemetry off, no progress bars, HF_HUB_OFFLINE when every model is on disk
- llm provider 'openai' for OpenAI-compatible local servers such as llama.cpp's llama-server
- ADR-0007: WPF + WebView2 + React desktop app over the local engine API
- WPF shell (.NET 10) hosting a React 19 + Tailwind 4 interface in WebView2, in the AI Engineering Arena's
  sticker-arcade theme (light and dark, fonts bundled for offline use)
- sidebar of categories and sections; per section: folders, background indexing with progress, documents and a chat;
  a category chat searches all its sections; citations as pills and source cards that open the file
- the app starts ore serve itself (bundled Python when installed, uv in a checkout), ties it to a job object and
  routes every API call through an EngineProxy that adds the token and allows only the engine's API paths, so the
  page never sees the token or talks to a local port
- conversations kept per section on this computer; follow-up questions use the recent history
- ore serve: --data-dir and --warm; python -m open_rag_engine
- tests: 36 .NET (engine start, ready line, locator, proxy, safe file opening), 13 web, engine suite; Windows CI job
TWadi and others added 11 commits November 2, 2026 19:04
The standard Windows title bar is replaced by one drawn in the Arena's top-bar style: the RAG sticker, the app name,
where you are (category > section) and Arena window buttons. It is a WebView2 drag region, so dragging, double-click
to maximise and the window menu still work; WindowChrome keeps resizing from a thin border, rounded corners on
Windows 11, and the frame follows light or dark mode.
The global keyboard-focus outline was unlayered CSS, so it beat the text area's outline-none and drew a rectangle
inside the rounded box. It now lives in the base layer, and the whole box shows focus with a cobalt border and shadow.
A bundled engine loads its models from models_dir/<org>--<name> under the same model names, so the index
fingerprint and existing indexes stay valid. --llm-url points the engine at the llama-server the app starts.
- chat: a late reply after switching section can no longer land in, or be saved under, another conversation; saving is
  debounced and done outside the state update; the chat resets if the engine restarts (+ regression tests)
- a connection lost mid-answer becomes a 503 instead of an unhandled exception; every forwarded call and question now
  always gets an answer for its id; library calls time out instead of hanging
- opening sources: only local document paths (no UNC, device or alternate-stream paths), checked off the UI thread;
  explorer.exe by full path
- engine lifecycle: start and restart are serialised, deliberate stops aren't reported as crashes, a start that times
  out stops the process, only the current engine can report a crash, and unexpected start errors show a message
- WebView2: content-security policy on the built page (connect-src 'none'), DenyCors mapping, all permissions denied,
  no host objects; HttpClient bypasses system proxies
- engine: Stop now really stops generation (the next token raises once the client is gone); models load once under a
  lock, so the warm-up and a first question can't load them twice
- proxy: exact allow-list match (\z), questions only via the stream, problem bodies built with a JSON serialiser
- deleting a section or category also deletes its saved conversation; checkout discovery is Debug-only
The Python .gitignore template ignored every lib/ folder, so desktop/web/src/lib
(the engine client, the WebView2 bridge, chat storage, the mock engine and
their tests) never reached the repository and a fresh checkout couldn't build
the interface. The rule now covers only the root lib/ and lib64/.
installer/build.py stages a self-contained Open RAG and runs Inno Setup:
the published app, a uv-managed CPython 3.12 with the engine and its locked
dependencies (CPU torch), the embedding and reranker models from the Hugging
Face cache, the Qwen3 GGUF from Ollama (digest-checked) and llama.cpp's
llama-server (pinned b12052, sha256-checked). The setup installs per user,
keeps the library on uninstall and comes in parts below GitHub's 2 GiB limit.

The app starts the bundled llama-server on a free loopback port with its own
key and points the engine at it with --llm-url. ChildHost replaces EngineHost
and runs both processes, waiting on ORE_READY or a /health probe. Logs rotate
at 5 MB. installer/smoke_test.py checks a stage or an installed app end to end.

The window also opens the credits' GitHub and LinkedIn links, and only those
(ExternalLinks).
The sidebar says who developed Open RAG and opens a Credits sheet with both
authors' GitHub and LinkedIn and the source code. Links open in the default
browser through the window, which accepts only the links on its own list; a
test checks the sheet's links against it.
feat: Windows installer with a bundled language model, plus credits
feat: Open RAG, a Windows desktop app with categories and sections
@TWadi
TWadi merged commit 7a82736 into twadi/rag-engine Nov 2, 2026
1 of 2 checks passed
@TWadi
TWadi deleted the twadi/local-api branch November 2, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant