Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: CI

on:
pull_request:
push:
branches: [main]

permissions:
contents: read

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
python-version: "3.12"
- name: Install
run: uv sync --locked
- name: Lint
run: |
uv run ruff check src tests
uv run ruff format --check src tests
- name: Test
run: uv run pytest --cov --cov-report=term --cov-fail-under=80

desktop:
runs-on: windows-latest
defaults:
run:
working-directory: desktop
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: "10.0.x"
- uses: actions/setup-node@v4
with:
node-version: "20"
cache: npm
cache-dependency-path: desktop/web/package-lock.json
- name: Interface
working-directory: desktop/web
run: |
npm ci
npx tsc -b
npx eslint .
npx vitest run
npm run build
- name: App and tests
run: dotnet test OpenRag.slnx -c Release
14 changes: 12 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ dist/
downloads/
eggs/
.eggs/
lib/
lib64/
/lib/
/lib64/
parts/
sdist/
var/
Expand Down Expand Up @@ -218,3 +218,13 @@ __marimo__/

# Streamlit
.streamlit/secrets.toml

# open-rag-engine local data
.ore/

# Desktop app build output
desktop/**/bin/
desktop/**/obj/
desktop/web/node_modules/
desktop/web/dist/
desktop/web/.vite/
12 changes: 11 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,14 @@ Types: `feat`, `fix`, `refactor`, `perf`, `test`, `docs`, `ci`, `chore`.
- **Evaluation numbers** come with every change to retrieval or generation: before and after, on the same dataset.
- **Architectural decisions** get a short decision record in `docs/adr/`.

Development setup instructions will be added once the architecture is settled.
## Development setup

You need Python 3.12+, [uv](https://docs.astral.sh/uv/) and, for answers, [Ollama](https://ollama.com/).

```bash
uv sync # installs everything, with CPU-only PyTorch
uv run pytest --cov # tests use small fake models, so they need no downloads
uv run ruff check src tests && uv run ruff format --check src tests
```

The first real `ore` command downloads the embedding and reranker models from Hugging Face (about 1 GB). Answers need `ollama pull qwen3:4b-instruct-2507-q4_K_M`.
157 changes: 136 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,129 @@
# open-rag-engine
# Open RAG

**An open-source RAG engine, built in the open by two engineers learning AI engineering.**
**Ask questions about your own notes, PDFs and manuals, and get answers that cite the page. Everything runs on your computer.**

[![CI](https://github.com/Carthavate/open-rag-engine/actions/workflows/ci.yml/badge.svg)](https://github.com/Carthavate/open-rag-engine/actions/workflows/ci.yml)
[![Latest release](https://img.shields.io/github/v/release/Carthavate/open-rag-engine)](https://github.com/Carthavate/open-rag-engine/releases/latest)
[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)

> 🏗️ **Status: designing the architecture.** There's no code yet. Watch or star the repo to follow along, and open a discussion if you have ideas.
![Open RAG answering a question about reciprocal rank fusion from the RAG course lessons, with cited passages](docs/images/open-rag-dark.png)

## Why we're building it
Open RAG is a free, open-source RAG engine with a desktop app for Windows. We built it from the RAG course we made for
the [AI Engineering Arena](https://carthavate.github.io/ai-engineering-arena/), using every technique the course covers:

We just finished a full course on retrieval-augmented generation and built a small RAG that runs in the browser for our learning platform, [AI Engineering Arena](https://github.com/TWadi/ai-engineering-arena). Now we want to build the real thing: a RAG engine you can run yourself, read end to end, and trust, because every answer shows exactly where it came from.
- hybrid search
- reciprocal rank fusion
- multi-query
- reranking
- cited answers

It's for anyone who studies or teaches from a pile of documents: students, teachers and researchers.

## Download for Windows

1. From the [latest release](https://github.com/Carthavate/open-rag-engine/releases/latest), download all three files
into the same folder: `OpenRag-Setup.exe`, `OpenRag-Setup-1.bin` and `OpenRag-Setup-2.bin`.
2. Run `OpenRag-Setup.exe`. The setup isn't code-signed yet, so Windows may say the publisher is unknown: choose
**More info**, then **Run anyway**.

## What it will do
The setup brings its own Python, models and language model, so you don't need to install anything else. After
installing, it works without internet. It needs Windows 10 or 11 (64-bit), 8 GB of RAM and about 5 GB of disk.

Planned, and subject to the architecture work:
## What it does

- **Ingestion** of PDFs, Markdown and web pages, including tables and images, with structure-aware chunking.
- **Hybrid retrieval:** keyword search (BM25) and dense vectors, fused with reciprocal rank fusion, plus query rewriting, multi-query and MMR for diverse results.
- **Reranking** with a cross-encoder before anything reaches the model.
- **Grounded answers with citations.** When the sources don't support an answer, it says so instead of guessing.
- **Evaluation built in:** retrieval metrics (recall@k, MRR) and answer quality (faithfulness, relevance), run on every change.
- **Tracing:** every step of a query can be inspected.
- **Free to run:** fully local with open models, or with a hosted LLM using your own key.
- **API first**, with a small web UI on top.
![The AquaFlow P200 manual in light mode: a cited answer explains why the pump keeps turning itself off](docs/images/open-rag-light.png)

- **Your documents, organised.** Make categories (a subject, a research area) and sections (a course, a session, a
reading list). Each section has its own folders.
- **Ask one section or a whole category.** Answers come only from the documents you chose.
- **Every answer shows its sources.** Each numbered citation comes with the passage it's based on. One click opens the
document, or shows it in its folder. When the documents don't contain the answer, it says so instead of guessing.
- **Private by design.** The engine listens only on your own computer, behind a token. Nothing leaves the computer: no
account, no telemetry, no cloud.
- **Multilingual.** The embedding, reranking and language models handle English, German, French, Arabic and more, so
you can ask in the language of your documents.
- **Always current.** Click Update after you change a folder: new and changed files are indexed, and deleted ones
drop out.

## How it works

```mermaid
flowchart LR
F[Your folders] --> C[Load and chunk]
C --> V[(Vectors in Chroma)]
C --> K[(BM25 keywords)]
Q[Your question] --> M[Multi-query, optional]
M --> V & K
V & K --> R[Reciprocal rank fusion]
R --> X[Cross-encoder rerank]
X --> L[Qwen3 4B]
L --> A[Answer with checked citations]
```

The desktop app is a WPF window that shows a React interface in WebView2. The interface reaches the engine only through
the window, which adds the engine's token, so the page never holds it. The window starts two helper processes:

- the engine (`ore serve`, Python and LangChain)
- the language model (llama.cpp's `llama-server`)

Both run on 127.0.0.1 and close with the app.

```mermaid
flowchart LR
UI[React interface<br/>WebView2] <-- messages --> W[Open RAG window<br/>WPF, .NET]
W -- token --> E[Engine<br/>ore serve]
E -- key --> LL[llama-server<br/>Qwen3 4B GGUF]
```

The design decisions are recorded in [docs/adr](docs/adr), for example [ADR-0006](docs/adr/0006-python-and-langchain.md)
(Python and LangChain) and [ADR-0007](docs/adr/0007-desktop-app.md) (the desktop app).

## Run the engine from source

You need Python 3.12+, [uv](https://docs.astral.sh/uv/) and [Ollama](https://ollama.com/).

```bash
uv sync
ollama pull qwen3:4b-instruct-2507-q4_K_M
cd examples # a small pump manual, in English and German
uv run ore ingest docs
uv run ore ask "Why does my pump keep turning itself off?"
uv run ore search "error E1042" --trace
uv run ore eval eval.jsonl
uv run ore chat
```

The first run downloads the embedding and reranker models (about 1 GB). Everything then runs offline.

A real answer from the example:

```text
$ ore ask "Wie viel kostet der Drucksensor?"
Der Drucksensor PS-12 kostet 39 Euro [1].
[1] service.md
```

To work on the desktop app, see [desktop/README.md](desktop/README.md). To build the installer, see
[installer/README.md](installer/README.md).

### What's inside

| Course technique | Where | Switch |
|---|---|---|
| Recursive, Markdown and semantic chunking | `chunking.py` | `[chunking] strategy` |
| Multilingual embeddings in Chroma | `store.py`, `factory.py` | `[embeddings] model` |
| Similarity, score threshold, MMR, metadata filters | `retrieval.py` | `[retrieval] search_type`, `--source` |
| BM25 keyword search + vectors, fused with reciprocal rank fusion | `retrieval.py`, `fusion.py` | `vector_weight`, `keyword_weight`, `rrf_k` |
| Multi-query (RAG-Fusion) | `fusion.py`, `pipeline.py` | `--multi-query` |
| Cross-encoder reranking | `rerank.py` | `[rerank]`, `--no-rerank` |
| Conversational follow-ups | `conversation.py` | `ore chat` |
| Cited answers that are checked, and abstention | `answer.py` | `[answer] min_rerank_score` |
| Retrieval evaluation (hit rate, MRR, recall) | `evaluation.py` | `ore eval` |

`ore serve` runs the same engine as a local API for the desktop app: categories, sections, background indexing and
streamed answers, reachable only from this computer.

Every setting lives in [`ore.toml`](examples/ore.toml). `--trace` shows what each stage returned and how long it took.

## Principles

Expand All @@ -30,10 +134,14 @@ Planned, and subject to the architecture work:

## Roadmap

1. **Architecture** (now): requirements, design, first decision records.
2. **MVP:** ingest → hybrid retrieval → answer with citations, plus the evaluation harness.
3. **Quality:** reranking, query transformations, multimodal documents.
4. **Product:** API, web UI, deployment guide.
1. **Architecture:** requirements, design, first decision records. ✅
2. **Engine:** ingest, hybrid retrieval, reranking, answers with citations, and the evaluation harness. ✅
3. **Desktop app:** a Windows app with a self-contained installer. ✅ (0.1)
4. **Next:**
- Check answers against their sources, sentence by sentence ([ADR-0005](docs/adr/0005-verified-citations.md)).
- Read scanned pages and images (OCR).
- Tune BM25 for German and other languages.
- Retrieval over knowledge graphs.

## Contributing

Expand All @@ -43,7 +151,14 @@ We review each other's work through pull requests. See [CONTRIBUTING.md](CONTRIB

[AGPL-3.0](LICENSE). You're free to use, modify and self-host it. If you run a modified version as a network service, you must share your changes under the same license.

The installer bundles models and libraries under their own licences, listed in
[installer/THIRD_PARTY_NOTICES.md](installer/THIRD_PARTY_NOTICES.md).

## Team

- [Wadi](https://github.com/TWadi)
- [Ghassen](https://github.com/GhassenJamoussi99)
![The Credits sheet in the app, with GitHub and LinkedIn links for Wadi and Ghassen](docs/images/open-rag-credits.png)

Open RAG is developed by:

- **Wadi Touil:** [GitHub](https://github.com/TWadi) · [LinkedIn](https://www.linkedin.com/in/wadi-touil-117472319/)
- **Ghassen Jamoussi:** [GitHub](https://github.com/GhassenJamoussi99) · [LinkedIn](https://www.linkedin.com/in/ghassen-jamoussi/)
9 changes: 9 additions & 0 deletions desktop/OpenRag.slnx
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<Solution>
<Folder Name="/src/">
<Project Path="src/OpenRag.App/OpenRag.App.csproj" />
<Project Path="src/OpenRag.Core/OpenRag.Core.csproj" />
</Folder>
<Folder Name="/tests/">
<Project Path="tests/OpenRag.Core.Tests/OpenRag.Core.Tests.csproj" />
</Folder>
</Solution>
60 changes: 60 additions & 0 deletions desktop/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Open RAG (desktop app)

A Windows app for teachers, researchers and students. Organise documents into **categories** (a subject or research
area) and **sections** (a course, session, project or reading list). Ask questions and get cited answers from one section, or
from a whole category. Everything runs on this computer: see [ADR-0007](../docs/adr/0007-desktop-app.md).

It looks like the [AI Engineering Arena](https://carthavate.github.io/ai-engineering-arena): the same "sticker arcade" theme,
in light or dark to match Windows, with the fonts bundled so it looks the same offline.

## How it fits together

```text
OpenRag.exe (WPF)
├─ WebView2 ── the React interface (web/), served from the app's folder as https://app.ore
│ │ messages: api, ask, cancel, pickFolder, openSource, …
│ ▼
├─ EngineProxy ── adds the per-launch token, allows only the engine's API paths
│ │ HTTP on 127.0.0.1
│ ▼
└─ engine ─────── `ore serve` (Python), started and stopped by the app (job object + --parent-pid)
```

The page never sees the token and never talks to a local port. Answers stream back line by line through the window.

| Folder | What |
|---|---|
| `src/OpenRag.App` | WPF shell: window, WebView2, folder dialog, opening cited files, engine lifetime |
| `src/OpenRag.Core` | Finding and starting the engine, the API proxy, safe file opening (unit-tested) |
| `web/` | React 19 + Tailwind 4 interface, with a mock engine for working in a browser |
| `tools/make_icon.py` | Draws the app icon |

## Develop

You need the .NET 10 SDK, Node 20 and, for the engine, `uv` with the repo's Python setup (see the main README).

```bash
cd desktop
dotnet build OpenRag.slnx # builds web/ the first time, then the app
dotnet test OpenRag.slnx
```

Run `src/OpenRag.App/bin/Debug/net10.0-windows/OpenRag.exe`. In a checkout it finds the engine by walking up to
`pyproject.toml` and starts it with `uv run ore serve`. `OPENRAG_ENGINE_DIR` points it at another checkout. Its data lives
in `%LOCALAPPDATA%\OpenRag` and the engine's log is in `%LOCALAPPDATA%\OpenRag\logs\engine.log`.

To work on the interface:

```bash
cd desktop/web
npm run dev # http://localhost:5176 with a mock engine, no Python needed
npm test
```

- **Against a real engine in the browser:** start `ORE_API_TOKEN=<32+ chars> uv run ore serve --origin http://localhost:5176`, then open `http://localhost:5176/?api=http://127.0.0.1:<port>&token=<token>`.
- **Live reload inside the app (Debug builds only):** set `OPENRAG_DEV_URL=http://localhost:5176` before starting `OpenRag.exe`.

## Installer

`installer/build.py` builds `OpenRag-Setup.exe`, a self-contained setup that brings its own Python, models and
llama.cpp server, so it needs no internet after installing. See [installer/README.md](../installer/README.md).
3 changes: 3 additions & 0 deletions desktop/src/OpenRag.App/App.xaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
<Application x:Class="OpenRag.App.App"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" />
39 changes: 39 additions & 0 deletions desktop/src/OpenRag.App/App.xaml.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
using System.IO;
using System.Windows;
using System.Windows.Threading;
using OpenRag.Core;

namespace OpenRag.App;

public partial class App : Application
{
/// <summary>Opens the window, unless Open RAG is already running. Unexpected errors are written down and
/// the window kept alive, instead of closing without a trace.</summary>
protected override void OnStartup(StartupEventArgs e)
{
if (!SingleInstance.Claim())
{
Shutdown();
return;
}
DispatcherUnhandledException += OnUnhandled;
base.OnStartup(e);
new MainWindow().Show();
}

private static void OnUnhandled(object sender, DispatcherUnhandledExceptionEventArgs e)
{
var log = AppPaths.Default.ErrorLog;
try
{
Directory.CreateDirectory(Path.GetDirectoryName(log)!);
File.AppendAllText(log, $"{DateTime.Now:u} {e.Exception}\n\n");
}
catch (Exception logError) when (logError is IOException or UnauthorizedAccessException)
{
// Nowhere to write it; the message box below still tells the user.
}
MessageBox.Show($"Something went wrong, but Open RAG kept running.\n\n{e.Exception.Message}\n\nDetails: {log}", "Open RAG");
e.Handled = true;
}
}
Loading
Loading