███╗   ██╗ ██████╗ ████████╗ █████╗ ██████╗ ███████╗███╗   ██╗███████╗
████╗  ██║██╔═══██╗╚══██╔══╝██╔══██╗██╔══██╗██╔════╝████╗  ██║██╔════╝
██╔██╗ ██║██║   ██║   ██║   ███████║██████╔╝█████╗  ██╔██╗ ██║█████╗  
██║╚██╗██║██║   ██║   ██║   ██╔══██║██╔══██╗██╔══╝  ██║╚██╗██║██╔══╝  
██║ ╚████║╚██████╔╝   ██║   ██║  ██║██████╔╝███████╗██║ ╚████║███████╗
╚═╝  ╚═══╝ ╚═════╝    ╚═╝   ╚═╝  ╚═╝╚═════╝ ╚══════╝╚═╝  ╚═══╝╚══════╝
                                                                     
══════════════════════════════════════════════════════════════════════
 Research Knowledge Base                    by  The Performer  ◆
══════════════════════════════════════════════════════════════════════

Smart academic research assistant — local-first by default, 100% open-source.

[English] | 🇫🇷 Version Française

🎯 Live Demo Available: Try NotaBene instantly without installing anything at notabene-by-theperformer.streamlit.app — pre-indexed on foundational RAG & NLP papers, powered by Google Gemini 2.5 Flash.

Note: On Streamlit Community Cloud (free tier), the app may take 1–2 minutes to wake up from cold start on the first visit.

NotaBene turns your chaotic pile of research papers (PDFs) and web articles into a personal, searchable vector knowledge base. Ask questions with grounded source citations (RAG), generate structured academic summaries, and search by concept rather than just keywords — all running 100% locally and for free, with an optional dual-backend to plug in your favorite cloud LLM when you need extra horsepower.

---

💡 The Story (Why I Built This)

While writing my master's research thesis on concurrency bug analysis in distributed systems, I ran into a classic academic nightmare: dozens of PDFs, preprints, and blog posts were piling up in my downloads folder. Remembering which paper introduced which algorithm, or which benchmark revealed which limitation, became completely unmanageable.

I didn't want to upload sensitive research drafts to paid proprietary cloud tools, so I built NotaBene: a lightweight, modular research assistant that runs locally on your machine, respects your privacy, and actually cites exact document passages instead of hallucinating answers.

---

✨ Features

• 📑 Smart Ingestion: Extracts text and metadata cleanly from PDFs (via PyMuPDF) and web articles (via Trafilatura).
• 🧩 Context-Aware Chunking: Hierarchical text splitting respecting paragraphs, sentences, and window boundaries.
• 🔍 Semantic Search: Search by meaning across your library using multilingual dense embeddings (intfloat/multilingual-e5-base), with source-type filtering.
• 🎯 Structured Research Summaries: Instant 4-quadrant academic breakdown: Problem, Method, Key Result, and Limitations.
• 💬 Grounded Q&A (RAG): Answers questions strictly from your indexed documents, with explicit citations ([Source 1], [Source 2]) and zero fabricated claims.
• ⚡ Content-Hash Deduplication: Automatic SHA-256 fingerprinting prevents re-indexing documents you've already processed.
• 🔄 Dual-Engine LLM Support: Runs 100% locally with Ollama (free & private), or connects to any OpenAI-compatible API (OpenAI, Gemini, Groq, Mistral, DeepSeek, OpenRouter).
• 🖥️ Two Interfaces: A blazing-fast rich terminal CLI (notab) and a sleek, interactive web dashboard (Streamlit).

---

🏗️ Architecture

┌─────────────┐     ┌──────────────┐     ┌─────────────────┐
│  Ingestion  │ --> │  Processing  │ --> │ Local Vector DB  │
│  PDF / Web  │     │ chunking,    │     │   (ChromaDB)    │
│             │     │ embeddings   │     │                 │
└─────────────┘     └──────────────┘     └─────────────────┘
                                                   │
                     ┌──────────────┐              │
                     │  Generation  │ <────────────┘
                     │ (Ollama or   │
                     │ external API)│
                     │ summary, RAG │
                     └──────────────┘
                              │
                    ┌─────────┴─────────┐
                    │                   │
             ┌─────────────┐     ┌─────────────┐
             │  CLI notab  │     │ Streamlit UI│
             └─────────────┘     └─────────────┘

Each module (ingestion, processing, storage, generation) is completely decoupled, heavily tested in isolation with mocks, and verified end-to-end through integration pipelines. Both the CLI and Web App are thin presentation layers on top of the exact same core business logic — zero duplicated code.

---

🛠️ Tech Stack

Component	Tool / Library	Role
PDF Extraction	PyMuPDF (fitz)	High-speed text & metadata parsing
Web Extraction	Trafilatura	Boilerplate removal & main article extraction
Chunking	Custom implementation	Paragraph → sentence → sliding-window splitting
Embeddings	sentence-transformers (multilingual-e5-base)	Normalized dense semantic vectors (768d)
Vector Database	ChromaDB	Cosine-similarity vector storage & retrieval
LLM Generation	Ollama (local) or OpenAI-compatible API	Structured synthesis, grounded RAG
Terminal CLI	Typer + Rich	Colorized terminal UI with tables & panels
Web Interface	Streamlit	Responsive tabbed interface with Material symbols

---

🚀 Quickstart & Installation

Prerequisites

• Python ≥ 3.10
• (Optional for local LLM) Ollama installed and running.

# 1. Clone the repository
git clone https://github.com/ThePerformer0/notabene.git
cd notabene

# 2. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate       # On Windows: venv\Scripts\activate

# 3. Install in editable mode with dev dependencies
pip install -e ".[dev]"

---

Choosing your LLM Backend

Option A — 100% Local & Private (Ollama)

Install Ollama, start the service, and pull your preferred model:

ollama serve
ollama pull llama3.1:8b        # Or llama3.2:3b for lower RAM

Option B — External API (Gemini, OpenAI, Groq, Mistral, OpenRouter)

No heavy local setup required. Grab an API key from your provider and set it as an environment variable:

export NOTABENE_API_KEY="your-api-key"   # Windows PowerShell: $env:NOTABENE_API_KEY="your-api-key"

💡 Tip for Gemini users: Google AI Studio provides a free OpenAI-compatible endpoint! Set base_url to https://generativelanguage.googleapis.com/v1beta/openai/ and model to gemini-2.5-flash.

---

💻 Usage — Terminal CLI (notab)

NotaBene provides a clean, Unix-friendly command-line interface with formatted Rich output:

# Index a PDF document
notab add pdf path/to/research_paper.pdf

# Index a web article
notab add web https://arxiv.org/html/2301.00001

# Semantic search across your entire library
notab search "concurrency bug detection static analysis" --top-k 5

# Structured academic summary (Problem / Method / Result / Limitations)
notab summarize path/to/research_paper.pdf

# Structured summary with an external API model
notab summarize path/to/paper.pdf --backend api --model gpt-4o-mini

# Grounded Q&A (RAG) with strict citations
notab ask "What are the key differences between dynamic and static analysis in this paper?"

# List all indexed documents & chunk counts
notab list

# Remove a document from the vector store
notab remove <document_id>

---

🌐 Usage — Interactive Web Dashboard

If you prefer a graphical interface, launch the Streamlit app:

streamlit run notabene/webapp/app.py

The web dashboard keeps the embedding model warm in memory for instantaneous multi-turn searches across 5 dedicated tabs:

1. Grounded Question-Answering (RAG)

Get syntheses that cite exact chunk references and paper titles:

2. Structured Academic Summary

Extract the 4 core pillars of any research paper in seconds:

---

🧪 Testing & Code Quality

NotaBene enforces strict software engineering standards:

• 100% hermetic unit tests: Zero network calls, zero unmocked LLM requests, and zero real disk writes during testing.
• Strong typing: Full type annotations and frozen=True domain dataclasses.
• Typed exception hierarchy: No bare except Exception blocks anywhere in the codebase.

Run the test suite with coverage:

pytest --cov=notabene -v

============================= 170 passed, 2 skipped in 99s =============================
TOTAL COVERAGE: 89%

---

📌 Known Limitations

• Scanned PDFs: Requires a selectable text layer (OCR pipeline is planned for a future release).
• Context Size for Summaries: notab summarize sends the full extracted text in a single prompt; papers exceeding the model's context window should be chunk-summarized.
• Single-User Local Design: Built for personal academic workflows; not intended for multi-tenant multi-user deployments without an auth layer.

---

🗺️ Roadmap

• Token-Aware Chunking: Tokenizer-aligned boundary splitting.
• Hybrid Search: BM25 keyword matching fused with dense vector cosine similarity.
• OCR Engine: Tesseract integration for scanned historical documents.
• Synthesis Draft Generator: State-of-the-art draft synthesis across user-selected document subsets.

---

📄 License

Distributed under the MIT License. Feel free to use, modify, and build upon it!