Skip to content
BranchesTags

Repository files navigation

ApprovalFlow

AI-assisted invoice approval with a hard safety boundary: an LLM agent advises, deterministic Python decides, humans review the risky slice, and no payment moves without trusted server-created authority.

ApprovalFlow is an event-driven microservice system that automates invoice and expense approvals. It auto-approves the simple, low-risk majority of invoices, escalates ambiguous, risky, or high-value cases to a human reviewer, and executes payment through a trusted-authority chain with budget reservation and saga-style compensation. An LLM agent analyzes every invoice against company policy — but it is advisory only: deterministic rules own every route, every enforcement decision, and every dollar.

Highlights

  • Deterministic authority over advisory AI — the LLM recommends; Python rules make the final route. Explicit policy violations and the autonomy ceiling cannot be overridden by AI output, browser input, or human notes.
  • Bounded LLM tool-call agent loop — the agent reads policy facts through read-only tools backed by deterministic code and must return schema-validated structured output. Every failure path (invalid tool, bad arguments, invalid schema, exceeded rounds) degrades safely to human review — never to payment.
  • Event-driven microservices — three services (Workflow, Decision, Payment) communicating over Dapr pub/sub with Redis-backed state, behind an NGINX gateway with rate limiting.
  • Trusted payment chain — payment authority is created server-side by Decision and validated by Payment before any effect; ETag-guarded budget reservation, idempotent payment, and durable compensation on failure.
  • Human-in-the-loop escalation — reviewers see the agent's recommendation, confidence, and cited policy rules; approver identity is server-side; human approval cannot override deterministic hard stops.
  • Auditable end to end — full per-invoice audit history, an append-only decision ledger, correlation-id lookup, and optional trace evidence.
  • Tested and verified — 613 unit and integration tests plus one-command verification gates with committed evidence and reproduce commands.

Quick Start

docker compose up --build

Then open http://localhost:8080:

Page Role
/submitter.html Submit an invoice and track its status
/approver.html Review queue — approve, reject, or send back
/auditor.html Audit trail and decision metadata
/controller.html Dashboard: throughput, auto vs. escalation, money split

The default mode is fully offline (LLM_PROVIDER=stub) — no API key required. See Environment Modes for live-LLM advisory mode and First-Time Setup for running tests.

How It Works

Service Path Owns
Workflow services/workflow/ Intake, status, linked corrections, review queue, full audit history, final notifications, dashboard, and UI-facing API.
Decision services/decision/ Policy checks, AI advisory, deterministic routing, duplicate checks, Decision ledger, trusted authority.
Payment services/payment/ Authority validation, Dapr ETag budget reservation, idempotent payment, outcome publication, compensation.

Infrastructure: NGINX gateway with rate limiting, Dapr sidecars, Redis-backed Dapr state and pub/sub, Dapr local secret store, Docker Compose, and a static browser UI.

flowchart LR
    USER["Browser roles"] --> GATEWAY["NGINX gateway\nUI + /api + rate limit"]
    GATEWAY -->|"submit · status · review · correction · audit · notifications · dashboard"| WF["Workflow\nowns the full process"]
    WF -->|"DecisionEvaluationRequested.v1"| DEC["Decision\nAI advice + rule-based route"]
    DEC -->|"DecisionEvaluated.v1"| WF
    WF -->|"authority request"| DEC
    WF -->|"PaymentRequested.v1"| PAY["Payment\nchecks authority, reserves, pays, recovers"]
    PAY -->|"authority validation"| DEC
    PAY -->|"payment outcome"| WF
    WF --> VIEW["Full audit history\n+ one final notification"]
Loading

A submission is acknowledged immediately with a tracking id, then flows asynchronously through decision and payment:

sequenceDiagram
    participant UI as UI
    participant WF as Workflow
    participant BUS as Dapr pub/sub
    participant DEC as Decision
    participant PAY as Payment

    UI->>WF: POST /submissions
    WF-->>UI: 202 Accepted + tracking_id
    WF->>WF: Store RECEIVED + SUBMISSION_RECEIVED audit
    WF-->>BUS: workflow-processing intent
    BUS-->>WF: /submissions/process

    WF->>WF: Store DECISION_PENDING
    WF-->>BUS: DecisionEvaluationRequested.v1
    BUS-->>DEC: deliver DecisionEvaluationRequested.v1
    DEC->>DEC: Validate, normalize, call advisory, route deterministically
    DEC-->>BUS: DecisionEvaluated.v1
    BUS-->>WF: deliver DecisionEvaluated.v1
    WF->>WF: Add decision details to audit history

    alt auto approve
        WF->>DEC: Create or validate authority (sync)
        DEC-->>WF: authority_id
        WF->>WF: Store PAYMENT_PENDING
        WF-->>BUS: PaymentRequested.v1
        BUS-->>PAY: deliver PaymentRequested.v1
        PAY->>DEC: Validate authority (sync)
        DEC-->>PAY: valid or invalid
        PAY->>PAY: Reserve, pay, compensate if needed
        PAY-->>BUS: payment outcome event
        BUS-->>WF: deliver payment outcome
        WF->>WF: Store final state + payment audit + one notification
    else human review
        WF->>WF: Store HUMAN_REVIEW + open review + audit
    else reject or duplicate
        WF->>WF: Store final state + decision audit + one notification
    end
Loading
Service context map — boundaries and published events 
flowchart LR
    subgraph GW["Gateway / UI Context"]
      UI["Static UI + public API facade + rate limit"]
    end
    subgraph WF["Workflow Context"]
      W["Submission · correction · status · review · audit · notifications"]
    end
    subgraph DEC["Decision Context"]
      D["Policy · AI advisory · deterministic router · ledger · authority"]
    end
    subgraph PAY["Payment Context"]
      P["Reserve · pay · idempotency · compensation"]
    end

    UI --> W
    W -->|publishes DecisionEvaluationRequested| D
    D -->|publishes DecisionEvaluated| W
    W -->|publishes PaymentRequested| P
    P -->|publishes PaymentCompleted / PaymentFailed| W
    P -->|publishes PaymentCompensationRequested / PaymentCompensated| W
    P -->|validate authority request/response| D
Loading
State models — Workflow, Decision, and Payment lifecycles 
stateDiagram-v2
    direction LR

    state "Workflow lifecycle" as Workflow {
        [*] --> W_RECEIVED
        W_RECEIVED: RECEIVED
        W_DECISION_PENDING: DECISION_PENDING
        W_HUMAN_REVIEW: HUMAN_REVIEW
        W_PAYMENT_PENDING: PAYMENT_PENDING
        W_PAID: PAID
        W_REJECTED: REJECTED
        W_DUPLICATE: DUPLICATE
        W_RETURNED_FOR_CORRECTION: RETURNED_FOR_CORRECTION
        W_PAYMENT_FAILED: PAYMENT_FAILED
        W_COMPENSATION_PENDING: COMPENSATION_PENDING

        W_RECEIVED --> W_DECISION_PENDING: processing intent
        W_DECISION_PENDING --> W_HUMAN_REVIEW: human route
        W_DECISION_PENDING --> W_PAYMENT_PENDING: auto authority
        W_DECISION_PENDING --> W_REJECTED: reject route
        W_DECISION_PENDING --> W_DUPLICATE: exact duplicate
        W_HUMAN_REVIEW --> W_PAYMENT_PENDING: approval allowed + authority
        W_HUMAN_REVIEW --> W_REJECTED: reject
        W_HUMAN_REVIEW --> W_RETURNED_FOR_CORRECTION: send back
        note right of W_RETURNED_FOR_CORRECTION
          A correction creates a new linked Workflow at RECEIVED.
          The original Workflow does not change.
        end note
        W_PAYMENT_PENDING --> W_PAID: payment completed
        W_PAYMENT_PENDING --> W_PAYMENT_FAILED: no reservation or released failure
        W_PAYMENT_PENDING --> W_COMPENSATION_PENDING: release failed
        W_COMPENSATION_PENDING --> W_PAYMENT_FAILED: later release succeeds
    }

    state "Decision lifecycle" as Decision {
        [*] --> D_RECEIVED
        D_RECEIVED: RECEIVED
        D_NORMALIZED: NORMALIZED
        D_DUPLICATE: DUPLICATE
        D_AI_ADVISORY_READY: AI_ADVISORY_READY
        D_AUTO_APPROVE_AUTHORIZED: AUTO_APPROVE_AUTHORIZED
        D_HUMAN_REVIEW_REQUIRED: HUMAN_REVIEW_REQUIRED
        D_REJECTED: REJECTED
        D_AUTHORITY_CREATED: AUTHORITY_CREATED

        D_RECEIVED --> D_NORMALIZED: normalize invoice
        D_NORMALIZED --> D_DUPLICATE: exact duplicate
        D_NORMALIZED --> D_AI_ADVISORY_READY: advisory ready
        D_AI_ADVISORY_READY --> D_AUTO_APPROVE_AUTHORIZED: deterministic auto route
        D_AI_ADVISORY_READY --> D_HUMAN_REVIEW_REQUIRED: escalation route
        D_AI_ADVISORY_READY --> D_REJECTED: hard reject
        D_AUTO_APPROVE_AUTHORIZED --> D_AUTHORITY_CREATED: payment authority
        D_HUMAN_REVIEW_REQUIRED --> D_AUTHORITY_CREATED: allowed human approval
    }

    state "Payment lifecycle" as Payment {
        [*] --> P_RESERVED
        P_RESERVED: RESERVED
        P_PAID: PAID
        P_PAYMENT_FAILED_RELEASED: PAYMENT_FAILED_RELEASED
        P_COMPENSATION_PENDING: COMPENSATION_PENDING

        P_RESERVED --> P_PAID: provider succeeds
        P_RESERVED --> P_PAYMENT_FAILED_RELEASED: provider fails + release succeeds
        P_RESERVED --> P_COMPENSATION_PENDING: provider fails + release fails
        P_COMPENSATION_PENDING --> P_PAYMENT_FAILED_RELEASED: idempotent release succeeds
    }
Loading

The accepted architecture decisions and their trade-offs are recorded in ADR.md; full diagrams, states, records, and flows are in docs/ARCHITECTURE.md.

AI Safety Architecture

The LLM is a judgment-capable advisor locked inside a deterministic cage:

  • It receives a system prompt, a structured response schema, message history, and read-only tool definitions — no write tools, payment tools, approval tools, or workflow-mutation tools exist in its registry.
  • Policy facts come through read-only tools backed by deterministic Python policy code — the policy document is never stuffed into the prompt.
  • Every response is validated against the InvoiceAdvisory contract; invalid JSON, missing fields, out-of-range confidence, or blank reasoning fails safely.
  • The tool-call loop is bounded, and every failure exit routes to human review — an AI provider outage can never cause an automatic payment.
flowchart TD
    A[Decision service needs AI advisory] --> B[Build prompt, schema, invoice context, and read-only tools]
    B --> C[Call LLM provider]
    C --> D{Tool call or final advisory?}

    D -->|Tool call| E[Validate tool name]
    E --> F{Allowed read-only tool?}
    F -->|No| X[Fail safely: AI_PROVIDER_ERROR]
    F -->|Yes| G[Validate tool arguments]

    G --> H{Arguments valid?}
    H -->|No| X
    H -->|Yes| I[Run deterministic read-only tool]

    I --> J{Tool succeeded?}
    J -->|No| X
    J -->|Yes| K[Append tool result to message history]

    K --> L{Max rounds exceeded?}
    L -->|Yes| X
    L -->|No| C

    D -->|Final advisory| M[Parse and validate InvoiceAdvisory]
    M --> N{Schema valid?}
    N -->|No| X
    N -->|Yes| O[Return advisory only]

    O --> P[Deterministic router makes final route]
Loading

One-Command Verification and Demo Runbook

Run the Single-Command Verification (D5)

This is the canonical D5 verification command:

tools/verify_d5.sh

It runs all four required journeys (INV-1001, INV-1003, INV-1007, and INV-1012), verifies at least two approvals complete without human action, checks that an "approve me" note cannot change the decision, and prints a clear pass/fail result. A nonzero exit status means D5 failed.

tools/verify_d5_v2.sh is the expanded V2 safety and architecture gate. It includes the D5 scenarios, but it does not replace the canonical D5 command above.

Follow the Recording Runbook (D7)

Use docs/RECORDING_SCRIPT.md to produce the required 2-5 minute screen recording through the working browser UI. The runbook identifies the pages, fixtures, actions, and outcomes to show, including all four required D5 journeys.

The runbook is instructions, not the finished D7 artifact. D7 is complete only when the recording has been produced, uploaded, and made accessible by URL.

The Autonomy Dilemma

The central design question: how much money — and which categories of expense — may the agent approve fully autonomously? The full product rationale is in docs/PRODUCT_DILEMMA.md; the evidence index is docs/evidence/final/DILEMMA_EVALUATION.md.

The current approach is:

  • low-risk, in-policy invoices may auto-approve;
  • unclear, risky, expensive, incomplete, or policy-sensitive invoices go to a person for review;
  • exact duplicates stop there; a changed correction is checked again as a new, linked invoice;
  • Python rules choose the final route, enforce safety rules, and protect payment;
  • AI may advise, but cannot approve, reject, pay, create authority, or mutate Workflow state.

Active autonomy values come from config/policy-config.json:

Control Value
Global no-human approval ceiling $250.00
Absolute autonomy safety maximum $750.00
Hardware capital review threshold $1,000.00
FX hard-stop threshold $1,000.00

The $1,000.00 values are policy-review thresholds, not autonomy targets.

Fraud and Risk Controls

Decision applies the configured GLOBAL-FRAUD signals deterministically. A matching signal prevents no-human approval and routes the invoice to human review with the signal recorded. Current signals include missing line-item detail, a round-number amount of at least $100.00 in $100.00 increments to an unknown vendor, a configured Saturday check based on the invoice date, and a line-item quantity of at least 100. Sunday is currently a normal day. The Decision router also supports a direct submittedAt check for the configured 20:00 through 06:00 window, but Workflow does not accept that client field or forward its trusted timestamp, so this is Decision-level fixture coverage, not current end-to-end enforcement. Exact duplicates remain terminal with no second payment; unknown vendors, missing required receipts, and aggregate invoice-splitting risk also prevent automatic payment.

Workflow records a trusted UTC submitted_at timestamp when it accepts an invoice. By Human Developer decision, that timestamp is currently for audit and traceability only: it is not copied into the invoice sent to Decision and does not change the existing fraud rules. Using the trusted timestamp with a configured company timezone to check submission days or hours has not been built.

Prerequisites

Required for the full local stack:

  • Docker Desktop or compatible Docker Engine
  • Docker Compose v2
  • Python 3.12 recommended
  • POSIX shell for helper scripts

Required for local Python tests:

  • Python virtual environment support
  • pip

Optional:

  • Live LLM credentials for LLM_PROVIDER=maf
  • LLM_API_KEY
  • LLM_MODEL
  • Optional LLM_CHAT_CLIENT in module:Class format

No Dapr CLI install is required for the normal Docker Compose run; the stack uses Dapr containers.

First-Time Setup

Create a virtual environment and install the three services plus test dependencies:

python3 -m venv .venv
.venv/bin/python -m pip install -e ./services/workflow -e ./services/decision -e ./services/payment pytest requests

Create a local .env only to override defaults (for example, a live LLM_API_KEY):

cp -n .env.example .env

Notes:

  • .env and dapr/secrets.json are ignored and should stay local.
  • The deterministic stub verification path does not require a live LLM key.
  • Docker Compose mounts dapr/secrets.example.json (the committed fake smoke-secret) by default, so a fresh checkout starts with no real secret file. Create an ignored dapr/secrets.json only to override that value locally.

Run The System

Start the full local stack:

docker compose up --build

Open the UI:

http://localhost:8080

Useful UI routes:

  • http://localhost:8080/submitter.html — submit an invoice and check its status
  • http://localhost:8080/approver.html — review queue; approve, reject, or send back
  • http://localhost:8080/auditor.html — audit trail and decision metadata
  • http://localhost:8080/controller.html — dashboard (throughput, auto vs escalation, money)

Stop the stack:

docker compose down

Check running services:

docker compose ps

View service logs:

docker compose logs workflow
docker compose logs decision
docker compose logs payment
docker compose logs nginx

Environment Modes

Default deterministic mode:

LLM_PROVIDER=stub docker compose up --build

Safe provider-failure mode:

LLM_PROVIDER=failing docker compose up --build

Live MAF/LLM advisory mode:

LLM_PROVIDER=maf docker compose up --build

Live mode requires local .env values:

LLM_API_KEY=...
LLM_MODEL=...
LLM_CHAT_CLIENT=optional.module:ClientClass

Provider failures are safe: deterministic hard stops still apply, and otherwise payable cases route to human review.

Test And Verify

Requirement-by-requirement coverage — status, evidence, and the command to reproduce each — is in docs/REQUIREMENTS_TRACEABILITY.md (the requirement-level view; pair it with docs/VERIFICATION.md for the command-level view).

Fast local suite:

tools/test.sh -q

Assignment D5 gate:

tools/verify_d5.sh

Integrated D5/V2 gate:

tools/verify_d5_v2.sh

Post-1:1 edge-case hardening gate:

tools/verify_edge_cases.sh

Version-scoped gates:

tools/verify_v1.sh
tools/verify_v2.sh

Docker integration and gateway e2e tests:

tools/test.sh tests/integration -m integration -q

Platform smoke against a running stack:

BASE_URL=http://localhost:8080 sh scripts/smoke_platform_stack.sh

Review-console smoke against a running stack:

sh scripts/smoke_review_console_flow.sh

Optional live LLM wiring check:

sh scripts/live_llm_check.sh

The command catalog, fixture map, and final evidence file descriptions are in docs/VERIFICATION.md (the command-level view; pair it with docs/REQUIREMENTS_TRACEABILITY.md for the requirement-level view).

Six UI Story Evidence

The stored browser screenshots, raw gateway responses, and limited-scope VALID & HEALTHY certifications for all six UI stories are under docs/evidence/ui-stories/. See tests/ui-stories/README.md for the story map, evidence checks, prerequisites, and clean live-MAF reproduction commands.

Trace Evidence

Tracing is off by default. Enable local JSONL trace evidence:

ENABLE_OTEL=true docker compose up --build

When enabled, services write local trace logs under logs/:

logs/workflow-trace.jsonl
logs/decision-trace.jsonl
logs/payment-trace.jsonl

Regenerate the stitched trace summary:

.venv/bin/python tools/gen_trace_evidence.py

Committed sample trace evidence:

docs/evidence/final/trace_inv_1001.json

API Surface

Main external API paths through NGINX:

  • GET /api/workflow/health
  • GET /api/decision/health
  • GET /api/payment/health
  • POST /api/submissions
  • GET /api/submissions/{tracking_id}
  • POST /api/submissions/{tracking_id}/corrections
  • GET /api/reviews
  • POST /api/reviews/{workflow_id}/actions
  • GET /api/audit/{tracking_id}
  • GET /api/audit?correlation_id={correlation_id}
  • GET /api/notifications/{tracking_id}
  • GET /api/dashboard
  • GET /api/openapi.json

The project also has read-only health, safety, protocol, smoke-test, and demo recovery endpoints. See docs/ARCHITECTURE.md and infra/nginx/nginx.conf.

The demo operational recovery routes (/api/ops/*, used by the Controller page's recovery buttons) are gated by DEMO_OPS, which defaults to false in the services. docker-compose.yml sets DEMO_OPS=true, so they work in the local demo; when the flag is off they return 404. The gateway publishes only on 127.0.0.1:8080, so ops routes are never reachable from another machine. The /api/smoke/secret check reports whether the Dapr secret store is reachable without returning the secret value.

Workflow validates submitted invoices against a server-side field allowlist: only known invoice and line-item fields are stored, and any extra field (including fixture-only expected/scenario) is dropped at intake with just its name logged, so it never reaches durable state, the Decision request, the AI prompt, or logs. Request bodies larger than 64 KiB are rejected with 413. The INV-1012 payment-failure/compensation demo is triggered by a trusted server config, PAYMENT_DEMO_FAIL_INVOICE_IDS (default empty; set to INV-1012 in the demo Compose) — no submitted invoice field can make a payment fail.

Repository Map

services/workflow/    Workflow service
services/decision/    Decision service
services/payment/     Payment service
config/               Policy, fixtures, trusted vendor, and attachment data
dapr/                 Local Dapr components and example secret store
infra/nginx/          Gateway config
infra/openapi/        OpenAPI document served by NGINX
ui/                   Static browser UI
tools/                Test, verification, report, and evidence helpers
scripts/              Smoke and live-provider checks
tests/                Integration, tool, and script tests
docs/                 Requirements, scope, architecture, plans, and evidence

Status and Origin

The system is complete: every mandatory requirement is implemented and verified, with committed evidence and a reproduce command per requirement in docs/REQUIREMENTS_TRACEABILITY.md.

ApprovalFlow was built as the capstone project for two courses — Microservice Architecture and AI Engineering. That origin is why the documentation tracks stable requirement IDs (F functional, M must-have, D dev-process) and keeps per-requirement verification evidence. The project is intentionally small and explainable rather than production-hardened; see Non-Goals.

Main Documents

Document Role
docs/REQUIREMENTS.md Top project authority — stable assignment-derived and human-approved requirements.
docs/REQUIREMENTS_TRACEABILITY.md Every assignment requirement → status, how it is verified, evidence file, and reproduce command.
docs/SCOPE.md Included and deferred scope.
docs/ARCHITECTURE.md Current architecture, diagrams, states, records, and flows.
ADR.md Accepted architecture decisions and trade-offs.
docs/VERIFICATION.md Test commands, datasets, final evidence files, and what each proves.
docs/PRODUCT_DILEMMA.md Product rationale for the autonomy posture.
docs/evidence/final/DILEMMA_EVALUATION.md Evaluator-facing evidence index for the autonomy posture and the integrated verification checkpoint.
docs/EDGE_CASES.md Edge-case catalogue and verification map.
docs/planning/BASELINE_PLAN.md Frozen V1 baseline plan.
docs/planning/V4_PLAN.md Implemented and verified V4 work (T0–T2).
docs/planning/V2_PLAN.md / V3_PLAN.md Older improvement plans kept for history.

Troubleshooting

If docker compose up --build fails because Docker is unavailable, start Docker Desktop first.

If http://localhost:8080 does not respond:

docker compose ps
docker compose logs nginx

If service health fails:

curl -fsS http://localhost:8080/api/workflow/health
curl -fsS http://localhost:8080/api/decision/health
curl -fsS http://localhost:8080/api/payment/health

If live LLM mode fails, switch back to deterministic stub mode:

LLM_PROVIDER=stub docker compose up --build

If tests appear to import the wrong service package, use tools/test.sh; it runs service suites in separate processes to avoid same-named package collisions.

Local Cleanup

Ignored local artifacts can be removed at any time:

  • .DS_Store
  • __pycache__/
  • .pytest_cache/
  • logs/*.jsonl
  • tmp/
  • .env.backup

Do not commit .env, local secrets, virtual environments, generated caches, or trace logs.

Non-Goals

  • No production payment integration.
  • No production metrics, alerting, hosted tracing, or repair console.
  • No full transactional outbox.
  • No Kubernetes, multi-tenant platform, or production identity system.
  • No generalized workflow engine, saga framework, policy engine, provider registry, command bus, or extra business service.
  • No LLM write tools or authority-granting tools.

About

Backend-focused invoice approval system demonstrating deterministic policy enforcement, AI-assisted analysis, human-in-the-loop escalation, and trusted payment authorization across three services.

drive.google.com/file/d/1-KHeoircBfzuyS5XoGFq66v4aPrwKWH3/view?usp=drive_link

Topics

ai-agentsbackenddaprdistributed-systemsdocker-composeevent-drivenfastapihuman-ai-collaborationhuman-in-the-loopmicroservicesnginxopenapipythonredisworkflow-automation

Resources

Readme
MIT license
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages