Skip to content

About

FastAPI-based ingestion service replacing legacy email-scraped reporting — pulls hourly multi-vendor telemetry, stores it queryable, and exposes it via REST endpoints plus a live dashboard.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

Multi-Vendor-Network-Telemetry-API

A multi-vendor LTE network telemetry pipeline: scheduled ingestion → SQLite storage → FastAPI REST layer → live operational dashboard.

Built to modernize a legacy, email-based data extraction workflow into a proper API-driven pipeline with a queryable data layer and real-time KPI reporting.

Screenshot

RAN Operational KPI Dashboard

Overview

Large telecom networks report performance data (RRC connection success rates, attempt/success counts, etc.) from multiple equipment vendors, each with their own export format and schema. This project demonstrates an end-to-end pattern for consolidating that kind of multi-vendor telemetry into a single, queryable, API-accessible data layer:

 Vendor Telemetry Sources (Ericsson / Nokia / Samsung)
                    │
                    │  Scheduled HTTP extraction
                    ▼
        Ingestion Script (Python + requests + pandas)
                    │
                    │  Per-vendor tables
                    ▼
              SQLite Database
                    │
                    │  Read-only queries
                    ▼
         FastAPI REST Backend (main.py)
           │                    │
           │ JSON endpoints     │ Live HTML dashboard
           ▼                    ▼
     External consumers    Operational KPI view
     (dashboards, tools)   (auto-refreshing)

Features

  • Multi-vendor ingestion: pulls hourly eNodeB-level performance data from three independent vendor sources into a unified SQLite schema.
  • REST API (FastAPI): exposes the dataset as JSON —
    • GET /api/rrc/summary — row counts across all vendor tables
    • GET /api/rrc/{vendor} — filterable per-vendor records (by date, with pagination)
    • GET /api/rrc/schema/{vendor} — schema introspection, useful when onboarding a new vendor whose export format isn't known ahead of time
    • GET /docs — interactive Swagger UI (auto-generated by FastAPI)
  • Auto-detecting KPI computation: rather than hardcoding a column name per vendor, the API scans each table's actual columns for a name that looks like an RRC success-rate metric (matching on patterns like RRC + SR/RATE/ACC/%). This lets the same code correctly compute a real metric across vendors whose CSV exports name that column differently — a common real-world problem when consolidating multi-vendor data.
  • Live operational dashboard: a self-contained HTML dashboard (/dashboard) showing per-vendor success rates, record counts, and operational status, styled and rendered server-side with no separate frontend build step.

Tech Stack

  • Python (pandas, requests) — extraction and transformation
  • SQLite — lightweight embedded storage, one table per vendor
  • FastAPI — REST API layer and dashboard rendering
  • Uvicorn — ASGI server

A note on the data

This public repository uses a synthetic data generator (generate_synthetic_rrc_dataset.py) in place of the real production extraction script. The real version of this pipeline connects to live internal vendor telemetry servers on a corporate network; for obvious confidentiality reasons, that script and its credentials are not included here.

The synthetic generator produces realistic-but-fake hourly data across the same three vendors, using genuinely different column-naming conventions per vendor — the same way real vendor CSV exports differ from each other — so the API's auto-detecting KPI logic can be exercised and verified end-to-end without touching any real system.

Running it locally

# 1. Install dependencies
pip install fastapi uvicorn pandas numpy

# 2. Generate a synthetic dataset
python generate_synthetic_rrc_dataset.py

# 3. Start the API
python main.py

# 4. View it
# API root:      http://localhost:8000/
# Dashboard:     http://localhost:8000/dashboard
# Swagger docs:  http://localhost:8000/docs

Possible extensions

  • Persist historical snapshots for trend analysis over time (currently reflects only the latest ingested data)
  • Replace the hardcoded fallback KPI value with an explicit "unavailable" state when a vendor's success-rate column can't be auto-detected
  • Add authentication for the REST endpoints
  • Swap SQLite for a proper cloud data warehouse (e.g. BigQuery, Snowflake) for larger-scale, concurrent access

About

FastAPI-based ingestion service replacing legacy email-scraped reporting — pulls hourly multi-vendor telemetry, stores it queryable, and exposes it via REST endpoints plus a live dashboard.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages