Real-Time Endpoint Behavioral Anomaly Detection & Threat Hunting Engine
Bridging Low-Level Windows Telemetry, Deep Sequence Modeling, and Live Adversary Emulation.
Modern cyber threats—such as living-off-the-land binaries (LOLBins), in-memory execution, credential dumping, and ransomware—routinely bypass traditional static antivirus signatures and hash blacklists.
Hawkeye is a lightweight, host-based intrusion detection and threat hunting engine (HIDS / micro-EDR). It continuously monitors raw operating system events via Windows Sysmon, normalizes kernel- and userland telemetry in real time, and passes process behavior through an unsupervised LSTM Autoencoder to detect anomalous execution sequences without requiring prior malware signatures.
+---------------------------------------+
| Rich Terminal UI |
| (ui_dashboard.py) |
+---------------------------------------+
^
| TCP :5056
+-----------------------------------+ |
| Windows Event Log | TCP :5055 |
| (Microsoft-Windows-Sysmon/ | ------> [observer_win.py]
| Operational) | |
+-----------------------------------+ v
+---------------------------------------+
| Detection Brain |
| (brain/brain.py) |
| |
| PyTorch LSTM Autoencoder |
| - Tokenized sequence window (L=10) |
| - Cross-entropy loss thresholding |
| - Blame attribution & confidence |
| - Conservative online learning |
+---------------------------------------+
|
v
+---------------------------------------+
| Persistent Alert Log & Status |
| (eye_alerts.log) |
+---------------------------------------+
-
Telemetry Collector (
observer_win.py)- Subscribes to the live
Microsoft-Windows-Sysmon/Operationalchannel via Windows Event API (win32evtlog). - Normalizes XML payloads into a lean event schema (
pid,name,id), mapping Sysmon Event IDs (ProcessCreate, NetworkConnect, ImageLoad, CreateRemoteThread, ProcessAccess, FileCreate, Registry, WMI, FileDelete, etc.). - Emits structured, newline-delimited JSON streams over resilient local TCP sockets.
- Subscribes to the live
-
Neural Detection Core (
brain/)- Sequence Autoencoder (
model.py): An encoder-decoder LSTM that projects deterministic event tokens into dense embeddings and attempts sequence reconstruction. - Behavior Engine (
brain.py): Slides a 10-event evaluation window, scores reconstruction loss, attributes blame to the suspicious process, and triggers alerts when loss exceeds the anomaly threshold. - Online Adaptation: Gradually updates weights during validated benign periods to adjust to natural baseline drift.
- Sequence Autoencoder (
-
Terminal Operations Console (
ui_dashboard.py)- Live split-pane interface styled with Rich.
- Displays real-time throughput (events/second), total processed events, active threat counters, and severity-highlighted process alerts.
- Behavioral vs. Static: Flags threats based on how actions unfold temporally rather than file hashes or static strings.
- End-to-End Pipeline: From low-level Windows ETW/Sysmon XML parsing to PyTorch tensor processing and live socket IPC.
- Edge-Ready Performance: Sub-millisecond to 3ms inference latency per sequence window on standard CPU (~300–680 windows/second).
- Adversary Emulation Suite: Includes a built-in multi-stage PowerShell simulation script (
demo_anomalies.ps1) covering 5 core MITRE ATT&CK tactics.
Hawkeye was validated against real-world public enterprise datasets from the Splunk Threat Research Team and the Open Threat Research Forum (Mordor):
| Dataset / Source | Attack Archetype | Total Events | Evaluated Windows | Mean Loss | Anomaly Detection Rate | Latency (CPU) |
|---|---|---|---|---|---|---|
| Splunk Attack Data | T1003.001: LSASS Credential Dump | 7,945 | 7,845 | 4.01 | 60.08% | 1.48 ms (678 eps) |
| Mordor (OTRF) | T1059.005: Empire C2 VBS Launcher | 1,190 | 1,140 | 3.70 | 69.30% | 3.33 ms (301 eps) |
| Mordor (OTRF) | T1003.001: Outflank-Dumpert Direct Syscalls | 95 | 75 | 3.70 | 70.67% | 5.48 ms (182 eps) |
| Mordor (OTRF) | T1218.005: Mshta LOLBin Execution | 892 | 842 | 3.69 | 70.07% | 2.43 ms (411 eps) |
| Mordor (OTRF) | T1547.001: Elevated Run Key Persistence | 129 | 109 | 4.16 | 77.06% | 1.67 ms (598 eps) |
Complete benchmark scripts and raw telemetry logs are included in
benchmark_evaluation.pyandbenchmark_data/.
- OS: Windows 10/11 or Windows Server (Sysmon requires Windows)
- Python: 3.10 or higher
- Sysmon: Installed and running (Sysinternals Sysmon)
-
Clone the repository:
git clone https://github.com/PixWizard/Hawkeye.git cd Hawkeye -
Install dependencies:
pip install -r brain/requirements.txt pip install pywin32
-
Verify or train the model (a pre-trained baseline is included):
python brain/train.py
Open three terminal windows to experience the full streaming pipeline:
python brain/brain.pyListens on 127.0.0.1:5055 for incoming normalized events and monitors sequence loss.
python ui_dashboard.pyListens on 127.0.0.1:5056 and renders the real-time split-pane terminal UI.
python observer_win.pyHooks into Microsoft-Windows-Sysmon/Operational and streams live telemetry to both the Brain and the Dashboard.
In a separate terminal, trigger the adversarial simulation suite to observe Hawkeye detecting attacks in real time:
powershell -ExecutionPolicy Bypass -File .\demo_anomalies.ps1The script simulates 5 real-world attack behaviors:
- Mass File Creation (Ransomware simulation)
- Network Connection Burst (C2 beaconing simulation)
- Fork Burst / Injection (Child process spawning simulation)
- Registry Spam (Persistence simulation)
- Mass File Deletion (Data wiper simulation)
To evaluate Hawkeye against pre-recorded enterprise attack datasets:
python benchmark_evaluation.pyThis runs automated sliding-window inference across all datasets in benchmark_data/ and outputs metrics to benchmark_results.json.
- Systems & OS Telemetry: Windows Event Log APIs (
pywin32), Event Tracing for Windows (ETW), Sysmon XML stream parsing. - Deep Learning: PyTorch, LSTM Sequence Autoencoders, embedding tokenization, loss-based anomaly scoring.
- Network & Concurrency: Non-blocking TCP socket streaming, client-server IPC, rate-limited online training loops.
- Terminal UI: Rich library, split-pane layout design, dynamic live tables and pulse indicators.
- Security Engineering: Threat hunting, MITRE ATT&CK framework mapping, Atomic Red Team testing.
- Per-Process Sequence Buffers: Decouple OS-wide global queues into per-PID sliding windows to minimize concurrent process interleaving.
- Provenance Graphing: Correlate parent-child process lineage (
ProcessGuid) for complete attack chain graphing. - Transformer-based Baselining: Experiment with lightweight Transformer encoders for longer-range temporal dependencies.
This project is licensed under the MIT License.