Skip to content

About

Build autonomous AI workflows with multi-agent orchestration, intelligent routing, hybrid RAG, knowledge graphs, MCP tools, persistent memory, and human-in-the-loop execution.

Topics

Resources

Stars

9 stars

Watchers

0 watching

Forks

Repository files navigation

⚡ AgentForge

Build AI Systems That
Think Beyond One Prompt.

Plan. Retrieve. Reason. Collaborate. Execute. Verify.

AgentForge is an extensible multi-agent AI platform built around
LangGraph, hybrid RAG, knowledge graphs, MCP tools,
human-in-the-loop workflows, persistent state, and real-time streaming.

Why AgentForge • Architecture • Capabilities • Quick Start • Roadmap


✦ The Idea

Most AI applications still look like this:

┌──────────┐      ┌─────────┐      ┌──────────┐
│   USER   │ ───► │   LLM   │ ───► │  ANSWER  │
└──────────┘      └─────────┘      └──────────┘

AgentForge is built around a different execution model:

                         ┌──────────────┐
                         │     USER     │
                         └──────┬───────┘
                                │
                                ▼
                       ┌─────────────────┐
                       │ INTENT ROUTER   │
                       └───────┬─────────┘
                               │
                               ▼
                       ┌─────────────────┐
                       │   SUPERVISOR    │
                       └───────┬─────────┘
                               │
              ┌────────────────┼────────────────┐
              │                │                │
              ▼                ▼                ▼
        ┌──────────┐     ┌──────────┐     ┌──────────┐
        │ WEB/RAG  │     │  GRAPH   │     │   MATH   │
        │  AGENT   │     │  AGENT   │     │  AGENT   │
        └────┬─────┘     └────┬─────┘     └────┬─────┘
              │                │                │
              └────────────────┼────────────────┘
                               ▼
                     ┌──────────────────┐
                     │ TOOLS / MCP      │
                     └────────┬─────────┘
                              ▼
                     ┌──────────────────┐
                     │ HUMAN APPROVAL   │
                     └────────┬─────────┘
                              ▼
                     ┌──────────────────┐
                     │  EVALUATION      │
                     └────────┬─────────┘
                              ▼
                         FINAL ANSWER

The current implementation already includes a 12-agent LangGraph workflow, routing, RAG, knowledge-graph reasoning, web retrieval, HITL, persistence, and evaluation.


🚀 Why AgentForge?

AgentForge is built for people who want to move from:

"LLM wrapper"

to:

"AI system."

It combines multiple AI engineering patterns in one extensible foundation:

🧠

Agent Orchestration

Supervisor-driven LangGraph workflows with specialized reasoning stages.

🔎

Hybrid RAG

Vector retrieval + lexical retrieval + fusion + reranking.

🕸️

Knowledge Graph

Relationship-aware reasoning over local knowledge.

🔌

MCP

External tool integration with local fallbacks.

🧑‍⚖️

HITL

Pause workflows and require explicit approval.

💾

Persistent State

Checkpointed graph state + durable conversations.

⚡

Streaming

Real-time FastAPI streaming endpoints.

📊

Observability

Prometheus + Grafana + health/readiness probes.

🔐

Authentication

User-scoped authenticated workflows.

These capabilities are already represented in the repository's current implementation and documentation.


🧩 12-Stage Intelligence Pipeline

AgentForge currently orchestrates:

01  Safety
02  Intent Routing
03  Clarification
04  Query Rewriting
05  Recency Guard
06  Human Approval Gate
07  Web Retrieval
08  Knowledge Graph Reasoning
09  RAG
10  Mathematical Reasoning
11  Response Generation
12  Evaluation

Why this matters

A user does not need to know which AI mechanism is appropriate.

AgentForge decides:

"What kind of reasoning does this task require?"
                ↓
"Which information sources are relevant?"
                ↓
"Which tools should be used?"
                ↓
"Does this require approval?"
                ↓
"How should the answer be verified?"

🔍 Retrieval That Goes Beyond Vector Search

AgentForge's local RAG pipeline combines multiple retrieval strategies:

                   QUERY
                     │
          ┌──────────┴──────────┐
          ▼                     ▼
   Semantic Retrieval      Lexical Retrieval
      (Vector)                  (BM25)
          │                     │
          └──────────┬──────────┘
                     ▼
              Rank Fusion
                     │
                     ▼
                Reranking
                     │
                     ▼
             Grounded Context

The current Chroma-based pipeline supports vector retrieval, lexical ranking, Reciprocal Rank Fusion, and optional LLM reranking.


🕸️ Knowledge Graph Reasoning

Not every question is semantic.

Some questions are about relationships.

AgentForge provides a dedicated graph-oriented retrieval path:

Documents
    ↓
Graph-RAG Store
    ↓
Relationship Retrieval
    ↓
NetworkX Reasoning
    ↓
RAG Grounding
    ↓
Answer

This is designed for relationship-style local queries and uses a separate graph-RAG ingestion path.


🔌 MCP — Tool Connectivity

AgentForge supports MCP-backed tools through a dedicated bridge architecture.

             Agent
               │
               ▼
          MCP Bridge
               │
               ▼
         MCP Tool Server
           ╱         ╲
          ▼           ▼
      Web Search   Calculator

The current implementation also provides local fallbacks when MCP is unavailable.


🛡️ Human-in-the-Loop

AI does not have to blindly continue every workflow.

For supported tasks:

        Agent
          │
          ▼
      Proposed
       Action
          │
     ┌────┴────┐
     ▼         ▼
  APPROVE    REJECT
     │         │
     ▼         ▼
 CONTINUE     STOP

AgentForge persists HITL decisions with user/thread context for auditing.

💾 Persistent Intelligence

AgentForge isn't designed around a disposable request.

LangGraph Checkpointing

Graph State
     ↓
Checkpoint
     ↓
Resume

Conversation Store

User
 ↓
Thread
 ↓
Messages
 ↓
Metadata
 ↓
Audit

The documented persistence layer includes PostgreSQL checkpointer tables, conversation storage, users, and HITL events.


⚡ Real-Time by Design

API

POST /invoke
POST /stream

Operational Endpoints

GET /healthz
GET /readyz
GET /metrics

Persistence

GET /store/threads
GET /store/{thread_id}
POST /feedback

The project also exposes authenticated registration/login and HITL audit endpoints.


📊 Built-In Observability

AgentForge ships with:

FastAPI
   │
   ▼
/metrics
   │
   ▼
Prometheus
   │
   ▼
Grafana

You can monitor:

  • request rate
  • 5xx error rate
  • P95 latency
  • traffic by endpoint
  • status-code distribution

🏗️ Architecture at a Glance

                         ┌───────────────────┐
                         │       USER        │
                         └─────────┬─────────┘
                                   │
                                   ▼
                         ┌───────────────────┐
                         │      FASTAPI      │
                         └─────────┬─────────┘
                                   │
                                   ▼
                         ┌───────────────────┐
                         │   INTENT ROUTER   │
                         └─────────┬─────────┘
                                   │
                                   ▼
                         ┌───────────────────┐
                         │     SUPERVISOR    │
                         └─────────┬─────────┘
                                   │
                  ┌────────────────┼────────────────┐
                  │                │                │
                  ▼                ▼                ▼
             ┌─────────┐      ┌─────────┐      ┌─────────┐
             │   WEB   │      │   RAG   │      │  GRAPH  │
             │  AGENT  │      │  AGENT  │      │  AGENT  │
             └────┬────┘      └────┬────┘      └────┬────┘
                  │                │                │
                  └────────────────┼────────────────┘
                                   │
                                   ▼
                         ┌───────────────────┐
                         │    MCP / TOOLS    │
                         └─────────┬─────────┘
                                   │
                                   ▼
                         ┌───────────────────┐
                         │    HITL GATE      │
                         └─────────┬─────────┘
                                   │
                                   ▼
                         ┌───────────────────┐
                         │    EVALUATION     │
                         └─────────┬─────────┘
                                   │
                                   ▼
                         ┌───────────────────┐
                         │  FINAL RESPONSE   │
                         └───────────────────┘

🧰 Stack

Layer Technology
🐍 Backend Python
⚡ API FastAPI
🧠 Orchestration LangGraph
🖥️ UI Streamlit
🐘 Persistence PostgreSQL
🔎 RAG ChromaDB
🕸️ Graph NetworkX
🔌 Tools MCP
⚡ Cache Redis / in-memory fallback
📈 Monitoring Prometheus + Grafana
🐳 Containers Docker
☸️ Deployment Kubernetes
🔁 CI/CD GitHub Actions

📁 Repository Structure

AgentForge/
│
├── frontend/              # Next.js 16 Multi-Agent Workbench (React 19, Turbopack, Tailwind v4)
├── agent/                 # AI orchestration + agents + tools
├── client/                # Python client
├── service/               # FastAPI service + persistence
├── schema/                # Request / response schemas
├── scripts/               # RAG ingestion & tests
├── docs/                  # Architecture & integration documentation
├── docker/                # Container definitions
├── k8s/                   # Kubernetes manifests
├── monitoring/            # Prometheus configuration
├── media/                 # Architecture assets
│
├── compose.yaml
├── run_service.py
├── run_client.py
├── streamlit_app.py
├── requirements.txt
└── README.md

The current repository separates orchestration, tools, local RAG, graph RAG, persistence, service APIs, monitoring, deployment, and the modern Next.js 16 workbench.


🧪 CI/CD

The repository currently includes:

Continuous Integration

  • Python tests
  • Docker build verification
  • push/PR triggers

Continuous Delivery

  • version-tag releases
  • Docker image publishing
  • manual dispatch

These workflows are already documented in the repository.


🖥️ Next.js 16 Multi-Agent Workbench

AgentForge includes an enterprise-grade web application built with Next.js 16 (App Router), React 19, TailwindCSS v4, and Framer Motion physics.

Key Capabilities & Routes

  • Mission Control (/dashboard): Asymmetric bento grid with real-time /healthz liveness, /readyz component readiness, SQLite conversational thread counts, and OpenMetrics request rates.
  • Agent Playground (/playground): Real-time Server-Sent Events (SSE) token streaming, dynamic model selection (llama-3.3-70b-versatile), inline Human-in-the-Loop authorization gates, and LangSmith feedback scoring.
  • Session History (/conversations): Persistent multi-turn conversation storage with turn-by-turn inspection, checkpoint restore, and one-click JSON transcript download.
  • Workflow Topology (/workflows): Interactive visualization of the 12-agent LangGraph supervisor hierarchy, execution stages, and deterministic routing paths.
  • HITL Approvals (/approvals): Dedicated supervisory queue allowing human operators to preview search results, approve/reject web queries, and maintain an immutable decision audit log.
  • Knowledge RAG (/knowledge): Explains and simulates the 6-stage Hybrid RAG pipeline (Reciprocal Rank Fusion with $K=60$) alongside an interactive SVG Knowledge Graph entity explorer.
  • MCP Tool Registry (/tools): FastMCP tool execution sandbox with JSON schema inspection for web_search and calculate.
  • Platform Telemetry (/observability): Real-time Prometheus OpenMetrics parser with HTTP status code distributions (2xx, 3xx, 4xx, 5xx), average latency, and memory monitoring.
  • Settings & Security (/settings): Session identity management, PBKDF2 parameters, inference preferences, and theme toggle.

Frontend Quick Start

cd frontend
npm install
npm run dev

Open http://localhost:3000 in your browser.

Run automated integration tests:

npm test

Create production build:

npm run build

🚀 Quick Start

Clone

git clone https://github.com/Roshanrameshhub/AgentForge.git
cd AgentForge

Install

pip install -r requirements.txt

Start API

python run_service.py

Start UI

streamlit run streamlit_app.py

Optional full infrastructure

docker compose up -d --build

The documented local stack includes FastAPI, Streamlit, Prometheus, and Grafana.


📚 Feed AgentForge Your Own Knowledge

Place PDFs inside:

rag_docs/

Run:

python scripts/ingestion/ingest_local_rag_pdfs.py \
  --pdf-dir rag_docs \
  --reset

Then ask:

local: summarize the uploaded documents

For graph-oriented documents:

graph_rag_docs/
python scripts/ingestion/ingest_graph_rag_pdfs.py \
  --pdf-dir graph_rag_docs \
  --reset

🌌 Roadmap

AgentForge is evolving toward a broader autonomous AI workbench.

✅ Foundation

  • LangGraph supervisor
  • Multi-agent workflow
  • Hybrid RAG
  • Knowledge graph reasoning
  • MCP tools
  • Human-in-the-loop
  • Persistent graph state
  • Conversation persistence
  • Streaming
  • Monitoring
  • Authentication

🚧 Intelligence

  • Dynamic task planning
  • Parallel agent execution
  • Long-term memory
  • Context compression
  • Agent verification
  • Self-correction
  • Better evaluation
  • Advanced retrieval

🔮 Next Generation

  • Multimodal AI
  • Realtime voice
  • Computer-use workflows
  • Expanded MCP ecosystem
  • Agent/plugin marketplace
  • Autonomous multi-step execution

The existing codebase already provides the supervisor, retrieval, HITL, persistence, MCP, and monitoring foundations on which these capabilities can evolve.


⚠️ Current Limitations

The current implementation is strong as an orchestration foundation but is still evolving.

Known limitations include:

  • heuristic evaluation rather than complete factual verification
  • clarification requiring a subsequent user turn
  • possible routing errors for ambiguous inputs
  • retrieval quality depending on ingestion, embeddings, chunking, and reranking quality

🎯 What AgentForge Is For

AgentForge can serve as a foundation for building:

Research Agents
Developer Copilots
Knowledge Assistants
Document Intelligence
Enterprise AI Workflows
Autonomous Task Systems
Multi-Agent Applications

The underlying goal is simple:

Don't build another chatbot. Build a system that can reason through work.


🤝 Contributing

Contributions are welcome across:

  • new agents
  • retrieval strategies
  • MCP integrations
  • security
  • evaluation
  • observability
  • agent state
  • multimodal workflows
  • performance optimization

⚖️ License

AgentForge is distributed under the MIT License.

See LICENSE for the full terms.

This project is derived from an MIT-licensed open-source foundation and has been renamed and substantially extended as AgentForge. Original license and copyright notices are retained.


⚡ AgentForge

From prompts → to reasoning → to autonomous workflows.

The orchestration layer for the next generation of AI applications.

About

Build autonomous AI workflows with multi-agent orchestration, intelligent routing, hybrid RAG, knowledge graphs, MCP tools, persistent memory, and human-in-the-loop execution.

Topics

Resources

Stars

9 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages