AI-powered crowd intelligence platform for real-time crowd monitoring, stampede risk detection, congestion prediction, and operational decision support — built for stadiums, metro stations, airports, malls, concerts, and public venues.
CrowdSense doesn't just count people. It estimates density per zone, tracks how a crowd is moving (not just how big it is), forecasts risk 2–45 seconds ahead per zone, and pushes a Slack alert with specific redirection advice the moment a zone crosses a danger threshold — before a human operator would necessarily notice.
~7,400 lines of Python across 22 files. Built solo in ~4 days.
- Frame in — from an uploaded video, a live webcam, or a synthetic simulation.
- Person detection — YOLOv8n (bounding boxes) and CSRNet (density map) both run; an auto-switch picks whichever is trustworthy for the current density (see below).
- Optical flow — Farneback dense flow between consecutive frames, extracting how chaotically or how uniformly the crowd is moving.
- Zone analysis — the frame is split into venue zones (gates, corridors, platforms, etc.), each gets its own count, corrected for camera perspective warp.
- Risk scoring — a weighted formula combining density, rate-of-change, and motion disorder, adaptive per venue type (see Infrastructure Safety Score below).
- Flow forecasting — three LSTM models predict where risk is heading 2s, 5s, and 45s out, per zone, from the last 10 frames of features.
- Alert system — state-machine detects SAFE → ELEVATED → WARNING → STAMPEDE transitions and pushes a consolidated Slack situation report with zone-specific redirection advice.
- Persistence — every frame, alert, and session is logged to SQLite for replay and trend analysis.
Frame → YOLOv8 + CSRNet (auto-switch) → Optical Flow → Zone Analysis
→ Risk Scoring (ISS-adaptive) → LSTM Forecasting → Alert System → Slack
This is the part that goes beyond "count people in a box." Every analyzed frame produces a feature vector per zone — [density, rate_of_change, motion_disorder] — and the last 10 frames (3.3s of history at 3 FPS) feed three LSTM forecasters running in parallel, each trading off horizon length against confidence:
| Model | Horizon | Params | Out-of-distribution MAE | Out-of-distribution correlation |
|---|---|---|---|---|
| CrowdRiskLSTM | 2s | 13,217 | 0.070 | r = 0.925 |
| StandardCrowdRiskLSTM | 5s | 13,217 | 0.064 | r = 0.960 |
| EvidentialCrowdRiskLSTM | 45s | 51,204 | 0.194 | r = 0.398 |
The 45s model is deliberately weaker on raw accuracy — it outputs the 4 parameters of a Normal-Inverse-Gamma distribution (μ, ν, α, β) instead of a single number, decomposing its uncertainty into epistemic (the model hasn't seen enough like this) and aleatoric (the situation is inherently noisy). That's the honest tradeoff: a 45-second stampede forecast that also tells you how much to trust it is more useful operationally than a falsely-confident point estimate. All three were trained on 320 synthetic sequences (16 scenario configs × 5 seeds × 4 augmentation methods — temporal jittering, Gaussian noise, synthetic surge injection) and validated on scenario configurations never seen during training, not just a held-out split of the same distribution.
At inference, all three forecasts run in under 1ms combined — the bottleneck is the vision models, not the forecasting.
Every 5 seconds during active monitoring, CrowdSense:
- Analyzes all zones together, not just the worst one
- Separates danger zones from safe zones
- Generates a crowd-management action plan with zone-specific redirection advice
- Sends one consolidated Slack message — not a flood of per-zone pings
Alert state machine: detects transitions between SAFE → ELEVATED → WARNING → STAMPEDE, and also fires on the first non-SAFE observation so an already-elevated venue doesn't get missed on startup. 5-second batching prevents notification spam during a fast-moving situation.
Redirection advice engine: zone-name-aware, rule-based (16 keyword rules) — a zone named "gate" gets "redirect crowd to alternate gate, close this gate temporarily"; a "corridor" gets "open adjacent exits, deploy staff to manage flow direction"; a "platform" gets "hold incoming trains, redirect passengers to alternate platform." This is a real decision-support output an operator can act on immediately, not just a number going red.
Channels: Slack (Block Kit formatted, color-coded by severity), Discord (webhook-compatible), a generic JSON webhook for anything else, and a persistent in-app notification panel.
Infrastructure Safety Score (ISS) adapts detection thresholds to the venue, because "dangerous density" means something different in a bottleneck than an open field:
ISS = (num_exits × avg_exit_width) / (area_m² × congestion_factor)
| ISS Range | Level | Venue type | Warn (p/m²) | Critical (p/m²) |
|---|---|---|---|---|
| >0.05 | 1 | Open Ground (park, field) | 3.0 | 4.5 |
| 0.02–0.05 | 2 | Stadium (concert, sports) | 2.5 | 4.0 |
| 0.01–0.02 | 3 | Street/Market (bazaar, festival) | 2.0 | 3.0 |
| 0.005–0.01 | 4 | Corridor/Bridge (walkway, platform) | 1.5 | 2.5 |
| <0.005 | 5 | Bottleneck (temple entrance, tunnel) | 1.0 | 2.0 |
Thresholds based on Fruin's Level of Service research and Keith Still's crowd safety studies — cited, not derived independently.
Risk formula, combining three signals with tunable weights (default α=0.5, β=0.3, γ=0.2):
Risk = α × (Density / Density_critical)
+ β × (|Rate_of_Change| / Rate_critical)
+ γ × (Motion_Disorder / Disorder_critical)
| Risk Score | Alert Level | Action |
|---|---|---|
| 0.0–0.49 | SAFE | Monitor |
| 0.50–0.74 | ELEVATED | Prepare |
| 0.75–0.99 | WARNING | Act |
| ≥1.00 | STAMPEDE | Emergency |
Motion disorder (circular variance of optical flow vectors) distinguishes an orderly crowd (0.0, everyone moving the same direction) from a panicking one (1.0, chaotic random directions) — computed as 1 - |Σ(magnitude × e^(i·angle))| / Σ(magnitude). A directional surge (precursor to stampede) is flagged when mean velocity > 1.2 and circular variance < 0.2 — fast, uniform movement, which is what a crowd rushing toward one exit looks like.
CSRNet (density estimation) — VGG16 frontend (13 layers, 3 max-pool → 8× downsampling) + dilated-convolution backend (6 layers, dilation rate 2, preserves resolution while capturing multi-scale context) + 1×1 conv output producing a density map whose pixel sum is the estimated count. 16.3M parameters, trained on ShanghaiTech Part A, 137ms per frame on CPU with test-time-augmentation (horizontal flip averaging, +3–5% accuracy). Strong in dense, overlapping crowds where individual detection fails; degrades on scene types very different from its training distribution and produces no individual boxes.
YOLOv8n (person detection) — single-shot detector, COCO class 0 only, 29ms per frame on CPU. Fast and accurate for sparse-to-moderate crowds with individual person locations; fails under heavy occlusion in dense crowds.
Auto-switch logic — runs both models every frame and picks the more trustworthy one:
if yolo_count > threshold:
use CSRNet # YOLO detected many → definitely crowded
elif csrnet_count > yolo_count * 1.5 and csrnet_count > 15:
use CSRNet # CSRNet sees more people than YOLO → dense crowd YOLO missed
else:
use YOLO # Sparse scene → YOLO is more preciseVerified on real test images:
| Image | YOLO count | CSRNet count | Auto choice | Result |
|---|---|---|---|---|
| Moderate density (~40 GT) | 19 | 27 | YOLO (19) | Reasonable |
| Moderate density (~45 GT) | 16 | 34 | CSRNet (34) | Improved |
| Dense crowd (~150 GT) | 12 | 154 | CSRNet (154) | Accurate |
| Empty scene | 0 | 0 | YOLO (0) | Correct |
In the dense case, YOLO undercounted by 12× — it can't separate heavily overlapping people. CSRNet doesn't need to separate them; it estimates density directly. Cross-checking both catches exactly this failure mode.
A real-time 2D SVG venue map, not a video overlay — 5 built-in venue templates (Stadium: 8 zones, Metro Station: 6, Mall Entrance: 7, Concert Venue: 8, Single Zone for basic single-camera analysis), each rendering:
- Zone polygons colored live by risk level
- Crowd count + density labels per zone
- Animated crowd particles (Brownian drift normally, uniform surge during directional events)
- Pulsing hotspot circles on WARNING/STAMPEDE zones
- Flow direction arrows sourced from the optical-flow data
- A risk-score gauge and wall/exit markers
Honest about single-camera limits: in real-video mode, only the total count is shown on the main zone — CrowdSense doesn't pretend one camera can map a full multi-zone venue. Simulation mode distributes counts across zones with weighted variance instead, for demoing multi-zone behavior without needing multiple calibrated cameras.
Four physically-motivated scenario generators, each a different curve shape:
| Scenario | Duration | Count range | Alert pattern | Generator |
|---|---|---|---|---|
| Normal Day | 180s | 40–120 | Stays SAFE | Sinusoidal, dual-frequency |
| Building Crowd | 180s | 20–350 | SAFE → ELEVATED → WARNING | Sigmoid growth with dip events |
| Stampede | 120s | 50–550 | SAFE → ELEVATED → WARNING → STAMPEDE | 4-phase: calm → buildup → rapid escalation → surge spike → aftermath |
| Evacuation | 180s | 27–450 | WARNING → STAMPEDE → WARNING → SAFE | Exponential decay with bottleneck stall events |
Plus a Custom Builder for hand-defined timed events (surge/dip/ramp/steady, with magnitude and width) and Replay from DB to re-run any saved session through the Digital Twin. Signal smoothing (moving averages, window sizes 9–15) prevents alert flicker from single-frame noise.
SQLite in WAL mode, thread-safe (threading.Lock), three core tables: sessions, frames (every analyzed frame, per zone), alerts (every state transition with before/after level and risk score). Configurable data-retention policies, cross-session trend analysis, zone heatmap aggregation, Prometheus-format metrics export (Grafana-compatible), CSV/JSON export per session.
| Component | Time | Notes |
|---|---|---|
| CSRNet inference | 142 ms | 384×384, with TTA |
| YOLOv8n inference | 29 ms | Any resolution |
| Optical flow | 50–80 ms | Farneback, 640×480 |
| LSTM forecasting (all 3 horizons) | <1 ms | Batch of 1 |
| Risk scoring | <0.1 ms | Pure math |
| Zone analysis | <1 ms | Mask + sum |
| SVG rendering | ~5 ms | Digital twin frame |
| Total pipeline | ~250–350 ms | Sustained ~3 FPS |
3 FPS is a deliberate design point, not a limitation to hide — crowd density doesn't spike meaningfully frame-to-frame, so trend detection doesn't need video frame rate. It's not full real-time video analysis, and the README says so directly.
vs. pure YOLO counting: YOLO alone is faster (29ms vs ~300ms) but has no spatial density heatmap, undercounts dense crowds, and has no stampede detection at all.
vs. pure CSRNet: CSRNet alone can't detect individuals, overestimates sparse crowds, and has no motion analysis, forecasting, or alerting.
vs. commercial crowd-monitoring SaaS: those are typically cloud-dependent, closed-source, and vendor-locked on data. CrowdSense runs fully local on CPU (no GPU required), keeps all data in local SQLite, and ships with multi-horizon uncertainty-aware forecasting and a built-in digital twin — features usually sold as premium add-ons.
pip install -r requirement.txt
streamlit run crowdsense.pyOpen http://localhost:8501. Tabs: Monitor, Counting, Digital Twin, Webcam, Simulation, Analytics.
Input modes: image/video upload, live webcam (single or multi-camera), synthetic simulation with Digital Twin visualization, or session replay from the database.
Optional — retrain the LSTM forecasters (pre-trained weights already included) or evaluate CSRNet against ShanghaiTech:
python training/extract_train_multi.py --model_type lstm --augment
python training/evaluate_auto.py --dataset_dir /path/to/shanghaitech
CSRNet density estimation on ShanghaiTech test image (estimated: 154 people)
Live simulation run — Stadium template, Stampede scenario:
Digital Twin risk gauge (89/100) and live per-zone notification feed as multiple zones cross SAFE → WARNING → STAMPEDE in real time.
Automated density projections and zone-specific redirection advice ("open adjacent exits, deploy staff," "INITIATE EVACUATION PROTOCOL") firing live as the simulation escalates.
Full screen recording of a live run (Stadium template, Stampede scenario, Digital Twin + notifications panel): files/demo/simulation_demo.mov
crowdsense.py ← Main dashboard entry point (Bloomberg Terminal-style UI)
core/
model.py ← CSRNet architecture
stampede_config.py ← ISS formula, alert thresholds, zone config
stampede_detector.py ← Detection pipeline orchestrator
notifications.py ← Alert state machine, Slack/Discord broadcasting
db.py ← SQLite persistence layer
models/
lstm_forecaster.py ← Single-horizon LSTM
lstm_multi_horizon.py ← Multi-horizon LSTM + Transformer + Evidential variants
ui/
digital_twin.py ← SVG venue map renderer
simulator.py ← Synthetic crowd scenario engine
training/ ← CSRNet + LSTM training and evaluation pipelines
legacy/ ← Earlier single-model prototypes, kept for reference
| Layer | Technology |
|---|---|
| ML/DL | PyTorch 2.0.1, torchvision 0.15.2 |
| Object Detection | ultralytics (YOLOv8) |
| Computer Vision | OpenCV 4.8+ (optical flow, perspective warp) |
| Dashboard | Streamlit 1.58 |
| Forecasting | LSTM, Transformer, Evidential Deep Learning |
| Notifications | Slack/Discord webhooks, HTML5 Audio |
| Database | SQLite (WAL mode, thread-safe) |
| Visualization | Inline SVG (Digital Twin), Matplotlib |
- CSRNet trained on ShanghaiTech only — accuracy degrades on scene types very different from that dataset (indoor vs outdoor, aerial vs ground-level)
- Single-camera limitation — one camera shows one perspective; can't truly map a multi-zone venue without multiple calibrated cameras (real-video mode is honest about this, only reporting the main zone)
- CPU-only inference — no GPU acceleration yet (ONNX/TensorRT planned)
- LSTM forecasters trained on synthetic data — real-world crowd dynamics may differ from simulated patterns
- Streamlit's re-run model means every widget interaction triggers a full page re-execution — not ideal for true real-time streaming
- No person re-identification — multi-camera mode can't track individuals across cameras
Near-term: 3D Digital Twin (Three.js), cross-camera person re-identification, pose estimation for fall/panic detection, ONNX export for edge deployment. Medium-term: FastAPI backend (decouple ML from UI), mobile app for crowd marshals, RL-based evacuation routing, PostgreSQL for production deployments. Long-term: Multi-venue SaaS platform, privacy-preserving federated learning, audio-visual fusion (crowd noise + density), natural-language querying via LLM integration.
Built by a 4-person team at RVCE. This repo is the team's shared codebase — individual component ownership isn't broken out here since that wasn't confirmed for public attribution.
Satvik Krishna — satvikkrishna06@gmail.com
MIT License