Agentic RAG

This is a demo showing how to build an agentic RAG app for chatting with PDFs.

Upload a PDF, ask questions about it, and get answers grounded in the document along with the page numbers the answers came from.

How this app works

This is how the agentic RAG pipeline works in this app:

• The app extracts text from the document.
• It breaks the text into smaller chunks.
• It converts each chunk into a vector and stores the vectors in a vector database.
• When a user asks a question, the app sends the question and recent conversation history to the language model. It also gives the model access to a tool for searching the document.
• The language model decides what information it needs and requests a document search using a focused query.
• The app converts the search query into a vector.
• It searches the vector database for the chunks closest in meaning to the query.
• The retrieved chunks are returned to the language model as the search result.
• The language model reads the results and decides whether it has enough information to answer.
• If it needs more information, it can request another search using a different query.
• Each search request and its result become part of the context for the next model turn.
• The process stops when the language model generates an answer or reaches the app's model-turn limit.
• The final answer is returned to the user, along with the pages from which supporting information was retrieved.

Tech stack

This is a TanStack Start app deployed as a Cloudflare Worker using Alchemy. The app uses the following technologies:

• Cloudflare D1 (database)
• Cloudflare R2 (object storage)
• Cloudflare Queues (sending and receiving messages)
• Cloudflare Vectorize (vector database)
• Alchemy (infrastructure as TypeScript)
• Effect Atom (state management)
• Effect HTTP API (backend API)
• Effect OpenAI (@effect/ai-openai) (talking to OpenAI)
• LangChain (splitting text into chunks)
• pdf.js (reading text out of the PDF in the browser)
• Drizzle ORM (queries against D1)
• shadcn/ui and Tailwind CSS (interface)

What you need

1. A free Cloudflare account.
2. An OpenAI API key.

The app uses OpenAI's embedding model to turn text into vectors and a text generation model to write answers. This is the only part that costs money, and the cost is minimal. Embedding a short PDF and asking 20 to 30 questions will cost you a few cents.

Getting started

Clone the repo and install the packages:

git clone https://github.com/effective-software/effect-agentic-rag.git
cd effect-agentic-rag
pnpm install

Copy the example environment file to .env:

cp .env.example .env

Then fill in the three values:

• OPENAI_API_KEY
• CLOUDFLARE_ACCOUNT_ID
• CLOUDFLARE_API_TOKEN

When you create the Cloudflare API token, give it these permissions (all account scoped, all Edit):

• Workers Scripts
• Workers R2 Storage
• D1
• Queues
• Vectorize

Finally, deploy the app:

pnpm run deploy

Once the deployment is complete, you'll get a URL. Open it.

On the home page, click Upload PDF. This takes you to the /embed page, where you can select a PDF and embed it. Once embedding is complete, you'll be taken to the chat page for that PDF, where you can start asking questions.

Note

You can destroy the deployment by running:

pnpm run destroy

This deletes the Worker, the D1 database, the R2 bucket, the queues, and the Vectorize index, so nothing is left behind in your Cloudflare account. Run pnpm run deploy again whenever you want the app back.