Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Locomanipulation game

An Isaac Lab external project for the Unitree H1-2 with Magpie grippers: one robot, two MAPPO agents (skrl). The legs walk on a velocity command the way Unitree's sport mode does; the arms track a wrist pose each. Policies deploy to the real robot through GOLEM's lmg_deploy package.

The recipe copies sport mode, as measured in its bags (docs/experiments/sportmode_ablations.md): its PD gains on every joint (with action scales matched to them), its stance (knee 0.70 rad), its command ranges (vx 0.55, vy 0.54 m/s, wz 0.97 rad/s), its 0.5 s command low-pass and 0.70 s stride, stepping on any command. The legs are paid for following the command, sport mode's 6 cm foot lift and joint swing, and small rewards for the feet, knees and pelvis height inside their bands (no posture or alive terms); a quarter of the commands are zero. The policy runs at 100 Hz over 500 Hz physics with an explicit PD and 0-20 ms of action delay; the actors are LSTMs with a capped cell state; episodes have drawn lengths (uniform, 90-300 s); the arm goals start once training's own sport-mode tracking metrics pass. These are the defaults: train.py with no overrides trains the recipe.

Layout

  • source/locomanipulation_game/locomanipulation_game/
    • assets/h1_2.py: the robot: names, the sport stance, the PD actuators and their matched action scales.
    • data/arm_target_tables_v3.pt: the arm goals' tables (Git LFS).
    • tasks/locomanip_marl/: the tasks, on one env. LocoManip-LegsOnly-v0 (the legs walk, no arm goals, the arms held at rest; legs_only_env_cfg.py), LocoManip-ArmsOnly-v0 (the pelvis pinned, every task an arm goal; arms_only_env_cfg.py) and LocoManip-Marl-Direct-v0 (both at once on a free pelvis, each agent on exactly its own task, neither's depending on the other's; marl_direct_env_cfg.py).
      • locomanip_marl_env_cfg.py: everything configurable; scene_cfg.py: the scene.
      • locomanip_marl_env.py: a DirectMARLEnv that runs Isaac Lab's managers, the warm start and the arm goals' odometry.
      • mdp/: commands (velocity with the sport-mode metrics; arm goals with their kernel curriculum), rewards, actions, observations, events, terminations.
      • recurrent.py: the LSTM actors and recurrent MAPPO; golem_safety.py: GOLEM's e-stops; run_files.py: a run's saved state, task and overrides.
  • scripts/skrl/: train.py, play.py, eval_scorecard.py (with scorecard_metrics.py and its check against the sport-mode bags, test_scorecard_sportmode.py), eval_arms.py (the arms' reach, hold and vibration), export_marl.py, deploy_parity.py.
  • scripts/fit_sport_gait.py: builds the sport-gait reference under data/ from its source.
  • scripts/train_candidate.sh: train seeds, then score, export and check each one.
  • agent_scripts/: the CPU checks of an export: hw_stress_mujoco.py (MuJoCo under measured hardware effects), hw_bag_replay.py (the deploy controller on recorded robot bags), and hw_deploy_template.yaml.
  • mujoco_evals/: the policy loop in MuJoCo (mujoco_sim.py) and the run settings the tools share (common.py).
  • logs/: everything the code writes (untracked). Runs go to logs/skrl/locomanip_marl/<time>_<name>/.

Setup

See environment.yml: the conda env, Isaac Lab from source with skrl, then

git submodule update --init --recursive   # third_party/CL_Assets: the robot's USD, URDF and MJCF
git lfs pull                              # the arm-goal tables
conda activate env_isaaclab
pip install -e source/locomanipulation_game

The CPU checks run in a second python with mujoco, torch, numpy, pyyaml, pin and rosbags, and need GOLEM (~/GOLEM, for lmg_deploy).

Train, score, export, check

# one run (600k policy steps = 12500 updates of 48, about 21 h on an RTX 4090)
python scripts/skrl/train.py --headless --max_iterations 12500 --name <name> --seed 42
# the other tasks: --task LocoManip-LegsOnly-v0 or --task LocoManip-ArmsOnly-v0
# or the whole pipeline for seeds 42 and 43: train, scorecard, export, deploy parity, MuJoCo stress, bag replays
bash scripts/train_candidate.sh <name> 42 43        # SMOKE=30 for a 30-update smoke test

# the sport-mode scorecard and the wrist phase (motors nominal) -> <run>/checkpoints/scorecard_<N>.{json,md}
python scripts/skrl/eval_scorecard.py --checkpoint <run>/checkpoints/agent_<N>.pt --headless \
    'env.events.actuator_scales.params.gain_range=[1.0,1.0]' 'env.events.actuator_scales.params.strength_range=[1.0,1.0]'
# export for lmg_deploy: legs.pt, arms.pt, h12.yaml (the template), export.yaml (what the run trained with); empty
# policies/h12 first: it holds one policy
python scripts/skrl/export_marl.py --checkpoint <run>/checkpoints/agent_<N>.pt --out ~/GOLEM/core_ws/src/lmg_deploy/policies/h12
# the deploy controller against the env, step by step
python scripts/skrl/deploy_parity.py --checkpoint <run>/checkpoints/agent_<N>.pt --export <export dir> --headless
# CPU: MuJoCo stress tests and the replay of a robot bag
python agent_scripts/hw_stress_mujoco.py --export <export dir> --label <name>
python agent_scripts/hw_bag_replay.py --bag <bag> --deployed mid360v2 \
    --policy mid360v2=logs/deploy_baselines/mid360v2 --policy <name>=<export dir> --sim_seeds 4
python scripts/skrl/play.py --checkpoint <run>/checkpoints/agent_<N>.pt

The scorecard's targets are sport mode's: turn sense >= 0.99, speed ratio >= 0.90, turn ratio >= 0.90, cos >= 0.95, angular error <= 0.05 rad/s, linear error <= 0.15 m/s.

The evaluation and export scripts rebuild the env a run trained in: its task (<run>/params/task.txt), the current defaults and its saved overrides (<run>/params/hydra_overrides.txt); an override whose key the current config no longer has is dropped (its value is now the default) and listed. A run from before 2026-11-16 did not record its task: pass --task for a legs-only or arms-only one (the scripts say when they assume the walking task). Runs from before the 2026-10-25 cleanup on other recipes need the pre-cleanup tag.

While training, the robot walks on flat ground, unpushed, with the arms held at rest by their IK alone, until the warm-start gate passes (training's sport-mode metrics: cos >= 0.8, turn sense >= 0.9, linear error <= 0.2 m/s, angular error <= 0.15 rad/s; warm_start_*); then standing-height arm goals and the arms' residual ramp in over 20k steps. <run>/warm_start_gate.yaml changes its thresholds without a restart; TensorBoard shows training's sport-mode metrics under Metrics/sport/.

Deploy

Export into ~/GOLEM/core_ws/src/lmg_deploy/policies/h12/, replacing what is there, build, and launch lmg_sim or lmg_real (GOLEM's lmg_deploy README). The deploy package keeps one policy per robot, one iteration at a time: old ones are deleted, not kept beside it. logs/deploy_baselines/mid360v2 keeps the export the robot bags were recorded with, for the bag replays. The node runs at the policy's control_dt and reads the gains, stance, action scales, command filter and ranges from export.yaml.

Gotchas

  • One Isaac process at a time on the GPU; a small job beside training is fine if memory allows. "fail to launch kernel" in a log means a GPU fault: rerun.
  • Isaac ignores SIGTERM: timeout -s KILL, then check nvidia-smi.
  • Fixed-seed rollouts are deterministic.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages