A multi-vendor LTE network telemetry pipeline: scheduled ingestion → SQLite storage → FastAPI REST layer → live operational dashboard.
Built to modernize a legacy, email-based data extraction workflow into a proper API-driven pipeline with a queryable data layer and real-time KPI reporting.
Large telecom networks report performance data (RRC connection success rates, attempt/success counts, etc.) from multiple equipment vendors, each with their own export format and schema. This project demonstrates an end-to-end pattern for consolidating that kind of multi-vendor telemetry into a single, queryable, API-accessible data layer:
Vendor Telemetry Sources (Ericsson / Nokia / Samsung)
│
│ Scheduled HTTP extraction
▼
Ingestion Script (Python + requests + pandas)
│
│ Per-vendor tables
▼
SQLite Database
│
│ Read-only queries
▼
FastAPI REST Backend (main.py)
│ │
│ JSON endpoints │ Live HTML dashboard
▼ ▼
External consumers Operational KPI view
(dashboards, tools) (auto-refreshing)
- Multi-vendor ingestion: pulls hourly eNodeB-level performance data from three independent vendor sources into a unified SQLite schema.
- REST API (FastAPI): exposes the dataset as JSON —
GET /api/rrc/summary— row counts across all vendor tablesGET /api/rrc/{vendor}— filterable per-vendor records (by date, with pagination)GET /api/rrc/schema/{vendor}— schema introspection, useful when onboarding a new vendor whose export format isn't known ahead of timeGET /docs— interactive Swagger UI (auto-generated by FastAPI)
- Auto-detecting KPI computation: rather than hardcoding a column name
per vendor, the API scans each table's actual columns for a name that
looks like an RRC success-rate metric (matching on patterns like
RRC+SR/RATE/ACC/%). This lets the same code correctly compute a real metric across vendors whose CSV exports name that column differently — a common real-world problem when consolidating multi-vendor data. - Live operational dashboard: a self-contained HTML dashboard
(
/dashboard) showing per-vendor success rates, record counts, and operational status, styled and rendered server-side with no separate frontend build step.
- Python (pandas, requests) — extraction and transformation
- SQLite — lightweight embedded storage, one table per vendor
- FastAPI — REST API layer and dashboard rendering
- Uvicorn — ASGI server
This public repository uses a synthetic data generator
(generate_synthetic_rrc_dataset.py) in place of the real production
extraction script. The real version of this pipeline connects to live
internal vendor telemetry servers on a corporate network; for obvious
confidentiality reasons, that script and its credentials are not included
here.
The synthetic generator produces realistic-but-fake hourly data across the same three vendors, using genuinely different column-naming conventions per vendor — the same way real vendor CSV exports differ from each other — so the API's auto-detecting KPI logic can be exercised and verified end-to-end without touching any real system.
# 1. Install dependencies
pip install fastapi uvicorn pandas numpy
# 2. Generate a synthetic dataset
python generate_synthetic_rrc_dataset.py
# 3. Start the API
python main.py
# 4. View it
# API root: http://localhost:8000/
# Dashboard: http://localhost:8000/dashboard
# Swagger docs: http://localhost:8000/docs- Persist historical snapshots for trend analysis over time (currently reflects only the latest ingested data)
- Replace the hardcoded fallback KPI value with an explicit "unavailable" state when a vendor's success-rate column can't be auto-detected
- Add authentication for the REST endpoints
- Swap SQLite for a proper cloud data warehouse (e.g. BigQuery, Snowflake) for larger-scale, concurrent access
