Skip to content

About

Ask questions about a CSV/Excel file: Google ADK agents, a deterministic router, a figure verifier and guardrails.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

67 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

InsightForge

Upload a CSV or Excel file and ask questions about it in plain English. A fixed router decides which Google ADK agents a question needs, the agents call pandas tools, and a verifier checks every figure in the answer against the tool output before it is shown.

Built by Ameema Rashid. Originally built as a Google ADK capstone project; this README describes the current system.

Live demo:

Answer card Answer card for "Which region has the highest amount?" on the bundled messy demo file: computed headline, verifier badge, route chips, then the supporting figures.

Executive summary The summary route (Data Scout, Analyst, Report Writer): Key findings, Risks and Recommendations.

How this was worked out The collapsed "How this was worked out" section: route, agents, tools, timings, model calls and verifier result. This screenshot was taken before the $-escape fix, so \$2307.51 shows a stray backslash that the current code no longer produces.

What it does

  • Reads messy CSV and Excel files (encoding and delimiter detection, ragged rows, Excel title rows above the header, first sheet).
  • Scores data quality 0-100 with a letter grade and readable flags: missing or unreadable cells, duplicate rows, case and space variants, cells containing the file delimiter, currency-like text.
  • Answers questions about the file (top item, totals, rates, charts, quality, executive summary), choosing the value column from the file's own column names.
  • Shows which route and agents answered, which tools ran, and whether every figure was found in the tool output.
  • Runs with no API key in a labelled Offline demo mode (the same ADK steps with deterministic stand-in models), so it can be tried without any account.

Architecture

flowchart TD
    UI["Streamlit app"] --> IG["Input guardrails<br/>length, injection, rate limit"]
    IG --> R["Deterministic router<br/>fixed whole-word rules"]
    R -->|off-topic| SC["Fixed scope reply<br/>0 model calls"]
    R -->|in scope| F["InsightForgeFlow<br/>custom ADK BaseAgent"]
    F --> DS["Data Scout"]
    F --> AN["Analyst"]
    F --> RW["Report Writer"]
    DS --> T["Tools (pandas)<br/>quality, stats, group breakdown, chart"]
    AN --> T
    DS --> V
    AN --> V
    RW --> V["Verifier<br/>figures + attribution"]
    V --> OG["Output guardrails<br/>prompt leak, scope, sanitizer, PII"]
    OG --> C["Answer card"]
    SC --> C
    V -.->|"retry failed"| FB["Direct calculation<br/>labelled fallback"]
    FB --> OG
    OFF["Offline stand-in models<br/>no API key"] -.->|"replace Gemini, same steps"| F
Loading

Only the agents the route needs are run. adk web still opens the original LLM-routed Orchestrator with its original prompts. A separate stdio MCP server exposes the three data tools; the agents do not use it.

How a question is answered

Route Agents that run Model calls
quality Data Scout 2
stats (figures) Data Scout (short) → Analyst 4 (5 when an extra group_breakdown call is needed)
chart Data Scout (short) → Analyst 4
summary Data Scout → Analyst → Report Writer 5
off-topic none (fixed scope reply) 0

Counts are for the scripted/offline stand-ins and scripted live runs; the verifier's one retry can add calls. Handoffs between agents are structured data in ADK session state, not chat text. If an agent skips its tool, the flow runs it and the trace says so.

Guardrails

Layer Check File Test
Input length ≤ 500 chars security/guardrails.py (check_question_length), app.py test_limits.py
Input prompt-injection regex, friendly reply, logged as check: injection without text security/guardrails.py, app.py test_security.py, test_app.py
Input questions / session / minute: 10 offline, 3 live security/guardrails.py (RateLimiter), agents/settings.py test_polish_e.py, test_limits.py
Routing off-topic → fixed scope reply, 0 model calls agents/router.py, agents/flow.py test_router.py
Routing missing column → "not available" + suggestions, 0 model calls agents/answers.py (not_available), agents/flow.py test_polish_a.py, test_fuzz_shapes.py
Tools per-route tool scope; hallucinated tools → error result agents/pipeline.py (ToolScopePlugin) test_polish_b.py
Data instruction-like text in cells/headers redacted agents/pipeline.py (ToolResultPlugin) test_data_injection.py
Data PII redaction in tool output agents/tools.py (_redact_pii) test_day1.py
Tools path allowlist, session file only, path-free errors agents/tools.py test_uploads.py, test_handoffs.py, test_polish_a.py
Output computed headline (quality/figures/chart) agents/answers.py, agents/flow.py test_polish_a.py
Output figures + attribution verifier, one retry, live fallback to the direct calculation agents/verifier.py, agents/flow.py, agents/pipeline.py test_verifier.py, test_polish_a.py
Output prompt leak (8-word run from any instruction), also on quoted agent text security/output_checks.py, app.py test_polish_d.py
Output scope (column, value or verified file figure, else scope reply) security/output_checks.py test_polish_d.py
Output sanitizer + PII; per-answer check line in "How this was worked out" security/guardrails.py, security/output_checks.py, ui_helpers.py (checks_line) test_output_sanitizer.py, test_polish_d.py
Budget 13 calls/min, 400/day; offline fallback security/guardrails.py, agents/pipeline.py test_limits.py
Model failures loop-safe client, 5xx retry, timeouts, any live exception → labelled offline answer agents/models.py, agents/pipeline.py test_models.py
Logs JSON steps; fields check and count hold names/counts only agents/tracing.py test_tracing.py, test_polish_d.py

The injection checks are regex lists, not a complete defence.

Evaluation

Question files are frozen and hashed (eval/MANIFEST.sha256) before scoring runs. Expected answers come from plain pandas.

Offline (stand-in models: fixed templates over pandas, so this measures routing, wiring and the tools, not a language model):

Set Score Label
dev 12/12
held-out 8/8 It was 6/8 when first frozen; 8/8 after fixing the two issues it found, so it is not a blind measure.
routing 22/22 The routing rules were written by the same author as the set.
injection 8/8 Says nothing about whether Gemini follows injected text.
fuzz 68/68 17 generated file shapes × 4 questions; every figure recomputed with pandas except the quality score, which the verifier checks.

Live (gemini-3.5-flash-lite, 12 Nov 2026, image insightforge:final3, HEAD 3aea154; run by python -m eval.run_live):

Set Score Latency median / slowest Answers with figures that passed the verifier
dev 12/12 5.1 s / 12.3 s 10/10
held-out 8/8 7.4 s / 14.4 s 6/6
injection 8/8 12.0 s / 54.0 s 5/5
  • 77 of the 120 allowed model calls were used.
  • The tool backstop ran in 6 of 21 questions that made model calls. Backstop: if an agent skips its tool, the flow runs it and the trace says so.
  • The slowest answer (54.0 s) was an injection-set question (inj-05).
  • These are small question sets written by the project author.

Before the router, a quality question took 34.3 s in a live check on 11 Nov (reported by the author, not re-measured).

Real files, checked by the author and not re-run here (no Kaggle data is in this repo):

  • Kaggle Superstore: West is the top region by Sales (651,930.77), Technology the top category (779,451.74), and the sheet notice is shown.
  • Kaggle Telco churn: churn rate by contract is 42.7% (month-to-month), 11.3% (one year) and 2.8% (two year).

Engineering numbers

Measured in Docker with --network none and no API keys, image insightforge:final4 (after the last code change):

Metric Value
Tests 552 passed, 0 failed (inside the deploy image)
Coverage, product code as configured (.coveragerc: every .py except tests/) 94.91% (3455 statements, 176 missed)
Coverage, no omits (test files counted) 96.97% (6601 statements, 200 missed)
Ruff 0.11.13 0 findings
Image size 1.01 GB as reported by docker images (size on disk)
Container non-root (uid 10001), listens on $PORT (health "ok" on PORT=9090)

A GitHub Actions workflow (.github/workflows/ci.yml) runs on every push: ruff, the test suite with a 70% coverage gate, and a Docker build, boot and health check.

Bugs found by testing

  • Offline budget locked the site. A fresh container refused the second question of any minute as "busy", because offline runs reserved 8 of the 10 per-minute model calls. Offline mode now uses no shared budget.
  • ADK's sync runner swallowed exceptions. Runner.run executes agents in a background thread, and a test that made the model fail "did not raise". The pipeline uses run_async so failures surface.
  • "Event loop is closed" in live mode. ADK caches one Gemini client per model, but the app runs each question in a new event loop, so a pooled connection from a closed loop failed the next question. Reproduced against a loopback stub; fixed with a client that is rebuilt per loop (LoopSafeGemini). A live run then still logged "Task exception was never retrieved", because the old client's async close ran on the new loop; the client is now closed inside its own loop before it ends.
  • Agents answered from chat history and skipped their tools. With ADK's default content mode every request carried the whole session. On the third question the agents answered from earlier turns. The Scout and Analyst now see only the current turn, function calling is restricted to the route's tools, and the flow runs a skipped tool itself.
  • The scope check replaced an in-scope answer. After the skipped tools left no evidence, the scope check had nothing to compare with and showed the off-topic reply. Evidence is now only what tools wrote during this question, and in live mode a failed output check falls back to the direct calculation.
  • A pandas 3 validity bug. Text columns have dtype str in pandas 3, which the loader counted as typed. The demo file scored 96.4 (A); with the regression test and fix it scores 87.1 (B).

Run locally

Offline, no key (Docker, nothing else needed):

docker build -t insightforge:local .
docker run --rm -e PORT=9090 -p 127.0.0.1:9090:9090 insightforge:local   # open http://127.0.0.1:9090

Or from source (Python 3.11+): python -m venv .venv, activate it, pip install -r requirements.txt, streamlit run app.py.

Live, with your own Gemini key. Put GOOGLE_API_KEY in an env file that is not in the repo and pass it at run time; never commit it:

docker run --rm --env-file <path-to-your-env-file> -e PORT=9090 -p 127.0.0.1:9090:9090 insightforge:local

The default model is gemini-3.5-flash-lite (GEMINI_MODEL changes it). With no valid key the app falls back to Offline demo mode and says so. DEPLOYMENT.md lists every setting and the Cloud Run steps (not yet run against a real project).

adk web from the repo root still opens the original LLM-routed Orchestrator.

Tests (no network, no keys):

docker run --rm --network none -e COVERAGE_FILE=/tmp/.coverage -v "${PWD}\tests:/src/tests:ro" insightforge:local `
  sh -c "cp -r /app /tmp/w && cp -r /src/tests /tmp/w/ && cd /tmp/w && python -m pytest -p no:cacheprovider --cov --cov-fail-under=70"

Limitations

  • Quality score validity. Validity is the share of columns with a numeric or date type, so clean, text-heavy files score lower (for example 89.8 with 0 missing values on Superstore, reported by the author).
  • Spelling variants are flagged, not merged. Merged records such as a "North;South" group are flagged too, not split.
  • Currency text is flagged, not converted ($2307.51, 1.200,50). EU decimals are not parsed. Mixed date formats are not converted.
  • First sheet only. The sheet used is shown; a sheet picker is not built.
  • The verifier checks numbers and the item they belong to, not reasoning.
  • Gemini free-tier limits. For gemini-3.5-flash-lite: 15 requests/minute and 500 requests/day (checked by the author on 11 Nov 2026; Google changes them). The app caps itself at 13 calls/minute and 400/day. Counters are in memory and reset when the instance restarts.
  • Mobile layout was checked in browser dev tools, not on a real device.
  • Offline mode answers from fixed templates; it is not a language model.

Credits

Built by Ameema Rashid. Originally built as a Google ADK capstone project. docs/WRITEUP.md is the original capstone write-up.

License: CC BY 4.0 (see LICENSE).

About

Ask questions about a CSV/Excel file: Google ADK agents, a deterministic router, a figure verifier and guardrails.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages