An asynchronous HTTP/HTTPS forward proxy written in Python (asyncio). It intercepts client traffic, handles RFC 7231 CONNECT tunneling for HTTPS, inspects TLS Server Name Indication (SNI) packets, evaluates requests against configurable domain and category policies, logs access events asynchronously, and provides a web-based administration console.
A forward proxy handles outbound client traffic rather than incoming server traffic. Aegis Gate is structured into five decoupled modules:
+---------------+ +-------------------------------------------------------+ +----------------------+
| Client Device | -----> | AEGIS GATE | -----> | Destination Server |
| (Browser/CLI) | | +-------------------------------------------------+ | | (e.g. example.com) |
+---------------+ | | TCP Listener (asyncio.start_server, port 8080) | | +----------------------+
| +-------------------------------------------------+ |
| | |
| +-------------------------------------------------+ |
| | Policy Engine (Exact, Wildcard, Regex, Category)| |
| +-------------------------------------------------+ |
| | (Allowed) | (Blocked) |
| v v |
| +-------------------+ +---------------------+ |
| | CONNECT Tunnel / | | 403 Forbidden | |
| | Upstream Forward | | HTML Block Page | |
| +-------------------+ +---------------------+ |
| | | |
| +-------------+--------------+ |
| v |
| +-------------------------------------------------+ |
| | Asynchronous Batch Queue -> SQLite (proxy.db) | |
| +-------------------------------------------------+ |
+------------------------|------------------------------+
v
+-----------------------+
| Web Console (port 8081|
+-----------------------+
- Proxy Core (
proxy/): Listens for incoming client TCP sockets, reads initial HTTP request lines, evaluates policies, and handles either regular HTTP proxying or bidirectionalCONNECTtunneling. - TLS SNI Parser (
proxy/sni_parser.py): An in-memory binary parser for TLS 1.2/1.3ClientHellopackets (RFC 5246, RFC 6066). Extracts the destination hostname directly from the binary stream without terminating TLS. - Policy Engine (
policy/): Evaluates requests in order: per-user rules -> exact domain matches -> wildcard matches (*.reddit.com) -> regex rules -> threat category database (malware,social-media,gambling, etc.) -> default fallback action. - Storage & Audit Layer (
storage/): Implements an in-memory queue (asyncio.Queue) drained by a background worker into SQLite viaexecutemanybatches, preventing synchronous disk I/O from stalling the proxy event loop. - Admin Dashboard (
dashboard/): Anaiohttpweb application providing real-time log inspection, 1-click category enforcement toggles, custom rule management, and a policy evaluation simulator.
The benchmark suite (benchmark/benchmark.py) tests two distinct code paths under 50 concurrent async workers:
- Policy Rejection Path (Blocked Traffic): Measures the proxy's in-memory policy lookup and fast 403 response generation without upstream socket creation.
- Upstream Forwarding Path (Allowed Traffic): Measures end-to-end proxying against a local mock HTTP server, including upstream TCP socket connection, request forwarding, response piping, and socket teardown.
| Metric | Policy Rejection Path (Blocked) | Upstream Forwarding Path (Allowed) |
|---|---|---|
| Throughput | 2,616 requests/sec | 1,392 requests/sec |
| Median Latency ( |
18.89 ms | 35.13 ms |
| 90th Percentile ( |
21.57 ms | 38.99 ms |
| 95th Percentile ( |
22.31 ms | 40.08 ms |
| 99th Percentile ( |
22.60 ms | 40.39 ms |
| Average Latency | 18.42 ms | 34.35 ms |
| HTTP Status Code | 100% 403 Forbidden
|
100% 200 OK
|
To reproduce these numbers locally:
python benchmark/benchmark.py -c 50 -n 500 --mode allBeing honest about architectural boundaries is essential when discussing network systems:
-
No Full TLS Interception (No MITM / Deep Packet Inspection): Aegis Gate does not terminate TLS sessions. It inspects only the outer
CONNECTrequest and the unencryptedClientHelloSNI. Consequently, it cannot inspect inner HTTP request methods, query strings, headers, or POST payloads on HTTPS traffic without deploying a custom private root CA to every client machine. -
Encrypted Client Hello (ECH / RFC 8744): Modern browsers and TLS implementations are transitioning to Encrypted Client Hello (ECH), which encrypts the SNI extension inside an outer
ClientHello. When ECH is negotiated, passive SNI sniffing cannot extract the inner destination hostname unless ECH DNS records (HTTPS/SVCBresource records) are stripped or blocked at the network resolver. -
SNI Spoofing and Domain Fronting: While the proxy checks both the HTTP
CONNECTtarget and the TLS SNI extension for consistency, a sophisticated client can technically craft mismatched SNI fields or route through shared CDNs (domain fronting). Mitigating this without full TLS decryption requires strict DNS-to-IP validation or IP-layer egress firewalls. -
SQLite Concurrency Ceiling: Access logs are batched into SQLite via an asynchronous queue to avoid event loop blocking. While effective for single-instance workloads, SQLite relies on file-level write locking. Scaling to multi-instance or high-throughput production would require replacing SQLite with a distributed write-optimized store like Redis, PostgreSQL, or ClickHouse.
-
Single-Process Event Loop: Python's Global Interpreter Lock (GIL) and single-threaded event loop bind processing to a single CPU core. Multi-core scaling requires spawning multiple worker processes behind a Layer 4 load balancer using socket reuse (
SO_REUSEPORT).
- Python 3.10+ (tested on Python 3.12)
- Git
git clone https://github.com/kr307/secure-forward-proxy.git
cd secure-forward-proxy
pip install -r requirements.txtpython main.py- Proxy Server:
http://0.0.0.0:8080 - Admin Dashboard:
http://localhost:8081
curl -x http://127.0.0.1:8080 -i http://facebook.com/Response:
HTTP/1.1 403 Forbidden
Content-Type: text/html; charset=utf-8
X-Proxy-Action: BLOCKED
X-Proxy-Category: social-media
...
[HTML Policy Block Page]curl -x http://127.0.0.1:8080 -I http://example.com/curl -x http://127.0.0.1:8080 -I https://www.bbc.com/python -m pytest tests/ -vAll 12 unit and integration tests verify:
- Binary TLS ClientHello SNI parsing and truncation handling
- Subdomain inheritance and wildcard precedence
- HTTP GET blocking and custom header verification
- HTTPS CONNECT blocking and connection rejection
secure-forward-proxy/
├── proxy/
│ ├── server.py # Asyncio TCP socket listener & worker pool
│ ├── handler.py # HTTP/CONNECT protocol parser & block page generator
│ ├── sni_parser.py # Binary TLS ClientHello SNI extractor (RFC 6066)
│ └── tunnel.py # Bidirectional TCP streaming pipe
├── policy/
│ └── engine.py # Multi-tier policy & category evaluation engine
├── storage/
│ └── db.py # Asynchronous SQLite queue & query layer
├── dashboard/
│ ├── app.py # Admin dashboard REST API (aiohttp)
│ └── templates/
│ └── index.html # Web management console
├── config/
│ ├── config.yaml # Server ports, buffer, and auth settings
│ └── categories.yaml # Categorized domain intelligence lists
├── benchmark/
│ └── benchmark.py # Concurrency and latency load testing tool
├── tests/
│ ├── test_proxy.py # HTTP GET & HTTPS CONNECT interception tests
│ ├── test_policy.py # Category & wildcard rule precedence tests
│ └── test_sni.py # TLS binary packet parsing tests
├── docs/
│ └── dashboard_preview.png # Dashboard console interface preview
├── Dockerfile # Container definition
├── docker-compose.yml # Container orchestration
├── requirements.txt # Python dependencies
├── ARCHITECTURE.md # Protocol mechanics and sequence diagrams
└── main.py # Application entry point
Distributed under the MIT License. See LICENSE for details.
