Yusuf Guenena

I build the reliability layer for learned robot systems.

M.S. Robotics Engineering @ Wayne State University · Runtime safety for learned policies · Sim-to-real on quadrupeds

Learned policies fail quietly. A locomotion network saturates, a perception model stops perceiving, a simulator's counterfactual stops being faithful, and nothing in the stack raises its hand. Most of what I build is the layer that notices: fault detection and recovery on a live robot, deploy-time parity gates, out-of-distribution monitoring on policy internals, and acceptance evidence you can re-verify a year later.

I work across the stack: C++ control loops, Python perception and RL, ROS 2 architecture, embedded firmware, CAD, and the GUIs on top. Most of it runs on a Unitree GO2. Every repo below states what has actually run on hardware and what has not, because that distinction is the whole job.

A self-healing runtime for ROS 2 robots, built as a four-tier loop. Sense: lifecycle nodes emit structured FaultEvents from a rolling Z-score detector, a heartbeat monitor, and a log parser. Diagnose: deterministic rules turn faults into RecoveryHints. Recover: a single cmd_vel publisher behind a strict allowlist and cooldown. Explain: an advisory local LLM that is never on the safety-critical path. The C++ port of the hot-path anomaly detector has landed (helix_sensing_cpp): a 30-minute hardware parity run measured -56% RSS and -60% CPU against the Python node, and it stays launch-gated behind use_cpp_anomaly=false until parity is re-confirmed.

Validated on a live Unitree GO2 and Jetson Orin NX across eight hardware lab sessions. The Session 8 bag runs the loop end to end: a 439 s idle run producing 30 anomalies into 14 recovery hints into 14 audited recovery actions, with no allowlist or cooldown violation. dashboard/helix_console.html replays that session tier by tier from the extracted telemetry, offline and with no build step.

Open limitations, named rather than buried: /helix/cmd_vel had zero downstream subscribers during Session 8, so STOP_AND_HOLD is a real, audited decision landing on a topic nothing is listening to. "The robot holds" is proven through the software path only; wiring it through a twist_mux fallback to the motors is the next hardware task. That same idle session produced 9 stops in 7m19s, which is the false-positive rate to assume until the detector work lands in the shipped config.

ROS 2 Humble C++17 Lifecycle Nodes Jetson Orin NX Unitree GO2 Local LLM

Closed-loop sim-to-real learning for the GO2. A locomotion policy trains in Isaac Lab, exports to ONNX through a torch/onnxruntime parity gate that refuses to ship a checkpoint whose deploy-time numerics drift outside tolerance, then runs behind a fail-closed ROS 2 safety layer with a shared slew cap. Failures captured on hardware replay in simulation under randomized physics and feed a fine-tuning curriculum. The deploy stack has run end to end on the real robot, on the Jetson. The shipped stand policy (stand-v3-h25) evaluates at 32/32 success in simulation, with per-step slew saturation at 3.30% nominal and 2.91% under full domain randomization against a <5% gate.

Where it stands: on-robot locomotion validation (Gate 7) is open; the last live run saturated at 33% slew and no on-robot stand has cleared the gate. An export audit found pre-audit checkpoints silently dropped observation normalization, so every checkpoint owes a re-export and a fresh parity check before the retry. The adaptation loop has not yet closed once on real failure data: the replay and fine-tune path is wired and unit-tested, but no hardware failure Parquets exist to feed it. EVIDENCE.md is the verified / inferred / not-validated ledger.

Isaac Lab PPO ONNX ROS 2 Sim-to-Real Unitree GO2

Recall, closed on the real robot. Someone says "come here" from outside the camera's field of view and the dog has to solve the whole problem: hear it, work out where it came from, turn, find the person, walk over, stop, sit. The ReSpeaker array's firmware DOA register gives the bearing (software SRP-PHAT was tried on the mounted array and failed, which is itself the finding), a bounded whole-token Whisper matcher gates the wake so ordinary lab talk does not trigger it, the turn closes on /utlidar/robot_odom yaw, then YOLOv8 takes over for acquisition, alignment, approach, and a bounding-box stop at 75% of frame height.

Two live end-to-end successes on the GO2, one of them a blind call with the caller's position undisclosed beforehand: bearing -59°, turn -56° in 1.16 s, acquisition at +22°, 0.66 m of approach, sit, about 9.6 s from wake word to seated. A systemd unit brings the stack up at boot, listening roughly 45 s after power-on with no laptop, no SSH, and no terminal, and it refuses to arm unless the robot is already standing.

What is still open: right-side bearings are inconsistent (the same spot has read +141°, -100°, and -90° across runs), and the full-circle camera scan that backstops a bad bearing has never been needed live, so it is unproven on hardware. Five consecutive successes on one frozen config are owed before the demo video. The armed boot path has not been watched end to end. The work above lives on the demo-doa branch; main is the earlier camera-only version.

ROS 2 Humble ReSpeaker 4-Mic Array Whisper ASR YOLOv8 systemd Jetson Orin NX Unitree GO2

Offline acceptance and evidence for simulator experiments. You declare what "unchanged behavior" means in a YAML file (which signals, which tolerances and units, which controls must hold, what the minimum sample is). IVF checks the experiment was even valid before letting any result decide a verdict, then seals the manifest, signals, reasoning, provenance, and per-file SHA-256 digests into one auditable bundle.

The flagship case is a real PhysX versus Newton/MJWarp cart-pole comparison whose recorded verdict is FAIL, with 5 of 9 oracles passing and 22 validity checks (18 pass, 3 unverifiable, 1 not applicable, 0 failed). Anyone can re-verify the shipped 13-file bundle from its seal, CPU only, with no GPU and no simulator install. IVF is not a benchmark: it does not measure throughput and does not designate a reference engine.

Python Isaac Sim 6.0 PhysX Newton / MJWarp SHA-256 provenance CLI

A distribution-shift teardown of the neural network that drives openpilot, an L2 driver-assistance system deployed on public roads. One question: presented with input outside its training distribution, does it fail conspicuously or silently?

Silently. The study rests on a parity-controlled reimplementation of v0.9.7 inference that agrees with comma's own reference output on 100% of 1159 real frames within ±0.5 m/s², so the negative result is attributable to the model and not the harness. Under shift, 8 of 10 tracked output readouts fall below 1% of real activity, the recurrent state contracts to a point, and the exported uncertainty heads rise only 1.20-1.84x and never leave their nominal real-driving range. An internal recurrent signal does encode the failure and is recoverable, but the model never exposes it. The result is also version-specific: on v0.9.6 the silent freeze does not reproduce, and the model fails by chaotic amplification instead. Full writeup, not submitted.

ONNX Runtime CARLA OOD Detection Recurrent State openpilot

A guide dog has to be summonable. If the handler puts the harness down, sits on a bench, and then wants the dog back, the dog finds them by voice, not by an app or a joystick. This repository closes the recall half of that interaction on real hardware: a four-channel mic array gives a GCC-PHAT bearing, Whisper parses the command, YOLOv8 plus depth back-projection gives 3D person poses, and a fused audio-visual score must hold across five consecutive frames before the target locks and publishes a Nav2 goal.

ROS 2 Humble GCC-PHAT Whisper ASR YOLOv8 RealSense D435i Nav2 C++ Lifecycle Node

A GO2 you talk to, with the brains running entirely on local hardware: no API key, no subscription, no cloud. You say "Fetch, describe the room." and the dog turns through a yaw-only scan, sends each view to a local vision-language model, aggregates the views into one grounded description, and says it out loud through its own speaker. The scene question is asked as closed yes/no prompts rather than open description, because an open "scene" field made the model return empty objects on live frames, and a per-view verification pass re-asks each view's top objects on the same image, which cut false objects on blank walls from 10 to 0 without losing a real one.

The important part is what owns the wheels. A safety envelope is the only publisher on /cmd_vel: the language layer proposes, the envelope disposes, a wireless deadman on the remote's L1 gates motion by default, and the whole stack refuses to arm without it unless you pass an explicit flag that prints an error banner. Voice-triggered end to end on the real GO2 in 31.2 s with no degraded subsystems, after a live fix to the microphone gate that is now the launch default.

What is still open: heading ends about 12° off after a full scan (clockwise undershoot). Object naming on low-texture views is a carried risk. The spoken scan is verified; navigation to a named place is not the demonstrated path. The source stays private while the work is in progress.

ROS 2 Humble Local VLM (Ollama) Whisper ASR ReSpeaker 4-Mic Array Safety Envelope Jetson Orin NX Unitree GO2

A connected body of work turning a stock quadruped into something that learns, navigates, and recovers on its own.

The layer that tells you when your robot is about to do something stupid and catches it on the way down.

Some projects exist because the idea did not survive contact with measurement. They stay public with their full evidence trail, because the measurement is the contribution.

Where I started, and still the most satisfying when it physically moves.

Website · LinkedIn · yusuf.a.guenena@gmail.com