AgentOps Gateway

AgentOps Gateway is an AI agent control center for reviewing AI-proposed tool calls before they affect real systems.

It is a portfolio/demo project about safe AI-agent workflow design. The app shows how an LLM can propose a tool-call plan while deterministic application logic validates the output, applies policy, requires human approval for sensitive actions, blocks dangerous actions, and records an auditable trace.

This project does not execute real tools. It does not include auth, a database, persistence, or real customer data.

Overview

AgentOps Gateway models a control layer between a tool-using AI planner and operational systems.

The main screen is Run Review, where a user can:

• Enter or edit a goal.
• Generate a proposed tool-call plan through OpenRouter or deterministic mock data.
• Compare each AI-proposed action against the gateway verdict.
• Approve or reject sensitive actions locally.
• See blocked actions clearly marked as non-actionable.
• Preview the trace that explains what happened.

The core product idea:

The LLM proposes a tool-call plan.
The gateway validates the output.
Deterministic policies decide what is allowed, what needs human approval, and what is blocked.
Humans review sensitive actions.
The full run remains traceable.

Why this project exists

AI agents become more useful when they can call tools, but tool access introduces risk. A model can draft a useful plan, but it should not be the authority on safety, permissions, approvals, or execution.

This project explores the control layer around AI tool use:

• What should the model be allowed to do?
• Where should deterministic validation happen?
• Which actions require human review?
• Which actions should never proceed?
• How can a reviewer understand the run after the fact?

The interesting part is not the LLM output by itself. The interesting part is the gateway that governs the output before anything sensitive can happen.

Core workflow

User goal
-> OpenRouter or mock planner proposes tool calls
-> API route validates and sanitizes planner output
-> gateway evaluates each tool call
-> policy engine returns allowed / requires approval / blocked
-> user approves or rejects sensitive actions locally
-> trace viewer records the run

In the UI, this is centered around:

AI proposed action -> Gateway verdict -> Human review

Features

• Run Review workflow for evaluating AI-proposed actions.
• OpenRouter live planner path.
• Deterministic mock planner and fallback path.
• AGENT_PLANNER_MODE=auto | openrouter | mock.
• Server-side /api/planner route.
• Zod request and planner-output validation.
• Sanitized tool calls limited to the known tool registry.
• Deterministic gateway policy engine.
• Local approve/reject controls for approval-required actions.
• Read-only Tool Registry.
• Read-only Policies section.
• Trace/audit viewer.
• Light and dark mode.
• Custom AgentOps visual system and typography.

Architecture

At a high level:

• src/app/api/planner/route.ts receives planner requests, validates request shape with Zod, and calls the planner service.
• src/lib/agent-planner/planner-service.ts chooses the planner mode and handles OpenRouter fallback.
• src/lib/agent-planner/openrouter-agent-planner.ts calls OpenRouter through the AI SDK, validates structured output with Zod, and normalizes known tool calls.
• src/lib/agent-planner/mock-agent-planner.ts produces deterministic demo plans from predefined scenarios.
• src/lib/data/mock-tools.ts defines the available tool registry.
• src/lib/gateway/policy-engine.ts maps tool categories to deterministic gateway outcomes.
• src/lib/gateway/build-gateway-run.ts combines planner output, policy decisions, approvals, metrics, registry metadata, and trace events.
• src/lib/gateway/build-trace-events.ts creates the data-driven trace timeline.
• src/components/dashboard/sections/run-review-section.tsx presents the primary demo workflow.

The frontend keeps approval decisions in local React state. No approval is persisted and no tool is executed.

AI planner modes

Configure planner behavior with:

AGENT_PLANNER_MODE=auto
OPENROUTER_API_KEY=
OPENROUTER_MODEL=openrouter/free

Planner modes:

Mode	Behavior
auto	Uses OpenRouter when OPENROUTER_API_KEY exists; otherwise falls back to the deterministic mock planner.
openrouter	Explicitly attempts the live OpenRouter planner path. If the API key is missing or the live call fails, the app returns the mock fallback result with fallback metadata.
mock	Uses deterministic mock scenarios only. No OpenRouter call is attempted.

Fallback is intentional. It keeps the demo usable without an API key and makes the UI reliable during portfolio reviews or interviews.

Safety model

The safety boundary is explicit:

• The model does not decide what is safe.
• The model does not approve tool calls.
• The model does not execute tools.
• The model only proposes a plan.
• The deterministic gateway owns validation, policy decisions, approval requirements, and blocked actions.

OpenRouter receives a system prompt that restricts it to proposing known tool calls. The server validates structured output and normalizes only tool calls that exist in the registry. The gateway then evaluates each normalized call with deterministic application logic.

Policy rules

Tool type	Gateway behavior
Read-only	Allowed
Internal write	Allowed / medium risk
External side effect	Requires human approval
Financial action	Requires human approval
Destructive action	Blocked
Unknown tool	Blocked

These rules are visible in the Policies section. There is no policy editor yet; this version is intentionally read-only.

Tech stack

• Next.js App Router
• React
• TypeScript
• Tailwind CSS
• shadcn-style UI primitives
• Radix UI primitives
• lucide-react icons
• next-themes for light/dark mode
• next/font for typography
• Zod for validation
• AI SDK with OpenRouter provider

Running locally

Install dependencies:

npm install

Create a local environment file:

cp .env.example .env.local

Start the dev server:

npm run dev

Then open the local URL printed by Next.js, usually:

http://localhost:3000

Environment variables

.env.example:

AGENT_PLANNER_MODE=auto
OPENROUTER_API_KEY=
OPENROUTER_MODEL=openrouter/free

Notes:

• Leave OPENROUTER_API_KEY empty to use mock fallback in auto mode.
• Set AGENT_PLANNER_MODE=mock for fully deterministic local demos.
• Set AGENT_PLANNER_MODE=openrouter to explicitly exercise the live planner path.
• Never commit a real API key.

Useful commands

npm run dev
npm run lint
npm run typecheck
npm run build

On Windows PowerShell, if npm run lint or npm run typecheck is blocked by script execution policy, use:

npm.cmd run lint
npm.cmd run typecheck

Project structure

src/app/
  api/planner/          Server-side planner route
  globals.css           AgentOps theme tokens and global styles
  layout.tsx            App shell providers and fonts

src/components/
  dashboard/            App shell, navigation, sections, metrics
  agent/                Goal form and planner output UI
  tools/                Tool Registry UI
  trace/                Trace/audit timeline UI
  approvals/            Local approval cards
  gateway/              Gateway decision display components
  theme/                Theme provider and mode toggle
  ui/                   Reusable UI primitives

src/lib/
  agent-planner/        OpenRouter planner, mock planner, scenarios, types
  data/                 Mock tool registry
  gateway/              Policy engine, decision builder, trace builder
  types/                Shared domain types

Key implementation decisions

• The planner proposes tool calls only; it does not govern safety.
• The OpenRouter path runs server-side so the API key is not exposed to the browser.
• Planner output is structured and validated with Zod.
• OpenRouter tool calls are sanitized to known registry tools before gateway evaluation.
• Gateway decisions are deterministic and derived from tool category/risk.
• Approval decisions are local UI state to keep the demo focused and infrastructure-light.
• The trace is built from run data, planner metadata, gateway decisions, and local approval state.
• Run Review is the main product workflow because it shows the full control path in one screen.

Limitations

• No real tool execution.
• No database.
• No auth.
• No persistence.
• No persistent audit log.
• Approvals are local UI state.
• No organization/team permission model.
• No configurable policy editor.
• OpenRouter output depends on model availability and behavior.
• The project is designed as a portfolio/demo architecture, not production infrastructure.

What I would improve in production

• Persistent audit logs.
• Real user authentication.
• Organization and team permissions.
• Configurable policy rules with review history.
• Real tool execution sandbox.
• Approval webhooks and notifications.
• Rate limiting.
• Provider observability and retry controls.
• Background jobs for long-running runs.
• A broader automated test suite.
• Deployment monitoring.
• Integration with real internal tools or MCP servers.

Interview notes

I built AgentOps Gateway to explore how AI agents can be safely connected to tools.

The key design decision is that the LLM is not trusted as the safety layer. OpenRouter can propose a plan, but deterministic application logic validates and governs the plan before anything sensitive can happen.

In a walkthrough, I would focus on:

• Run Review, because it shows the full product workflow.
• The planner service, because it demonstrates live AI plus mock fallback.
• The OpenRouter normalization step, because it keeps the model inside the known tool registry.
• The policy engine, because it owns the allowed / approval-required / blocked outcomes.
• The trace builder, because it explains the run after the decision.

The project is intentionally scoped: it demonstrates the control layer around AI tool use, without pretending to be production infrastructure.