Adaptive Fault-Tolerant Control System for a Portable Radiation Detector

An edge-AI system that fuses multi-sensor telemetry from a portable radiation-detection instrument, predicts incipient faults with an LSTM, and uses a Double DQN + Prioritized Experience Replay agent to make real-time control decisions (recalibrate, throttle, boost power, flag for maintenance) that keep the instrument accurate, powered, and safe — trained in a custom simulation, quantized to INT8, and deployed through a C++ sensor-bus layer onto Jetson-class edge hardware with a hardware-in-loop test harness.

Why this project

Built specifically to demonstrate hands-on (not tutorial-level) depth across the full RL-to-edge-deployment stack: reward engineering in a from-scratch simulation environment, an LSTM encoder feeding an RL agent's state, INT8 quantization with both PTQ and QAT paths, and the C/C++ sensor interfacing (I2C/UART/CAN) real embedded deployment requires.

Architecture

 [I2C: power monitor, temp, detector FE]  ─┐
 [UART: signal-quality telemetry]         ─┼─► sensor_interface (C++) ─► fused JSON ─► jetson_inference.py (ONNX Runtime / TensorRT)
 [CAN: vibration sensor, actuator cmds]    ─┘                                              │
                                                                                            ▼
                                                                              Double DQN policy (INT8)
                                                                                            │
                                                                              action -> CAN actuator cmd

Training side:

synthetic multi-sensor data ─► leakage-safe split ─► LSTM pretrain (fault classification)
                                                            │
                                                    encoder plugged into
                                                            ▼
                                    custom Gymnasium env (DetectorFaultControlEnv)
                                                            │
                                          Double DQN + Prioritized Experience Replay
                                                            │
                                          INT8 PTQ (DQN) / QAT (LSTM) ─► ONNX + TorchScript export

Repo layout

Path	What it is
data/generate_synthetic_data.py	Physically-motivated multi-sensor simulator with 4 injected fault classes
data/data_pipeline.py	Cleaning, rolling-median denoise, leakage-safe split (by session, not row), windowing
env/detector_env.py	Custom Gymnasium env — state/action/reward built from scratch
models/lstm_encoder.py	LSTM fault classifier + .encode() used as the RL state feature
models/dqn.py	Q-network
train/replay_buffer.py	Prioritized Experience Replay — real sum-tree implementation, not a sorted list
train/pretrain_lstm.py	Supervised LSTM pretraining
train/train_dqn.py	Double DQN + PER training loop, LSTM plugged in as the env's fault-prob provider
quantization/quantize_ptq.py	Dynamic INT8 PTQ on the DQN + ONNX/TorchScript export
quantization/quantize_qat.py	QAT fine-tuning for the LSTM classifier head + dynamic INT8 LSTM body
deployment/jetson_inference.py	ONNX Runtime / TensorRT inference bridge (Python side)
deployment/sensor_interface/*.cpp	I2C / UART / CAN readers + main HIL control loop (C++, compiles clean)
deployment/hil_test_harness.py	Replay + live hardware-in-loop validation, with false-trip/missed-fault diagnostics
benchmarks/latency_power_profile.py	Latency distribution + Jetson power-rail hook

Results from this build

• LSTM fault classifier: 98.2% validation accuracy (leakage-safe split verified — no sequence overlaps train/val/test).
• QAT fine-tuned classifier head retains 98.1% accuracy post-fake-quant.
• INT8 dynamic quantization: ~3x smaller state dict, 97% action-selection agreement with FP32 on the DQN.
• Full pipeline latency (ONNX Runtime, CPU): sub-millisecond per inference — well within a 5 Hz control loop budget.
• C++ sensor interface compiles clean (-Wall -Wextra, zero warnings) and the full C++ → Python → ONNX chain was run end-to-end.
• Double DQN + PER shows a clear improving reward trend during training (verified on a short run; the repo defaults to 400 episodes for a real training pass).

Honest limitations / what a longer run would fix

The DQN policy shipped in this repo's smoke test was trained for a short run to keep iteration fast — the HIL harness correctly reports weak fault-catch performance at that checkpoint, which is exactly the kind of pre-deployment signal this harness exists to catch. Run train/train_dqn.py for the full 400+ episodes (or longer, with reward-shaping tuning) before treating the policy as production-ready. Real hardware I/O (I2C/UART/CAN reads) is stubbed with documented fallback behavior since this was built without physical sensors attached — the C++ code is written to run unmodified once real device paths (/dev/i2c-1, /dev/ttyTHS1, can0) are wired to actual parts; only the register map in i2c_reader.cpp needs the real datasheet values swapped in.

Running it

pip install -r requirements.txt --break-system-packages

python3 data/generate_synthetic_data.py       # synthetic multi-sensor dataset
python3 train/pretrain_lstm.py                # supervised LSTM pretraining
python3 train/train_dqn.py                    # Double DQN + PER (set N_EPISODES env var)
python3 quantization/quantize_ptq.py          # INT8 PTQ + ONNX/TorchScript export
python3 quantization/quantize_qat.py          # QAT fine-tune for the LSTM
python3 deployment/hil_test_harness.py replay # validate the deployed policy
python3 benchmarks/latency_power_profile.py   # latency/throughput benchmark

# C++ sensor interface (compiles on Jetson or any Linux box with the headers):
cd deployment/sensor_interface
g++ -std=c++17 -O2 -c *.cpp && g++ *.o -o sensor_loop -lpthread
./sensor_loop | python3 ../jetson_inference.py --stdin-loop