Skip to content

golem_skills: grasp and pick_place on the G1 - #43

Open
correll wants to merge 8 commits into
mainfrom
g1/skills-manipulation
Open

correll wants to merge 8 commits into
mainfrom
g1/skills-manipulation

Conversation

@correll

@correll correll commented Nov 21, 2026

Copy link
Copy Markdown

For Max: /skill/grasp and /skill/pick_place on the G1, from your list. Eight small commits, one concern each. The H1-2 should behave exactly as before.

What changed and why

  1. golem_skills: read the grasp robot from a per-robot profile. The grasp frames, pinch frames, prep and staging configs and GraspGenX gripper were H1-2 constants spread over base.py, grasp.py, pick_place.py and graspgenx.py. They now come from robots.RobotProfile, picked by a new read-only robot parameter (default h12). No behaviour change.
  2. golem_skills: keep /joint_states per joint, merging its publishers. On the G1 two nodes publish /joint_states (golem_upper_ik's body and dex_3's fingers), so keeping only the last message could start the reach check's IK without the arm joints. Each joint now keeps its own latest position.
  3. golem_skills: open, close and grip-check a Dex3-1 hand. RobotProfile.hand is magpie (the H1-2) or dex3. With dex3, opening and closing call /<arm>/hand/open and /<arm>/hand/close. The grip check reads how far the closing finger joints travelled. Its thresholds were chosen, not measured. golem_skills now exec-depends on dex_3.
  4. golem_skills: refuse grasp options the robot lacks, before moving. A goal aborts before the arm moves if it needs GraspGenX and the profile has no GraspGenX gripper, or asks for visual_servo without hand cameras. The message names the methods that do run.
  5. golem_skills: a G1 profile, grasping with the Dex3-1 from 'home'. robot:=g1 uses the Dex3-1, golem_upper_ik's home as prep and staging, no GraspGenX model and no hand cameras. The G1's URDF has no grasp frame, so the profile adds tool frames on <arm>_hand_palm_link, published on /tf_static: a grasp frame in the GraspGenX convention and a pinch frame at the finger contact point. These frames come from the URDF's joint origins and were not measured on the hand.
  6. golem_skills: run the grasp reach check on the node's robot. ReachModel takes the node's robot: its IK profile, its hand's collision boxes, and tool frames through their parent link. On the G1 no method runs the check yet, because only the GraspGenX methods do.
  7. golem_bringup: start the G1's golem_skills on its G1 profile. g1_core with use_skills:=true passes robot: g1. Before this change it started the H1-2 profile. The real G1 starts no skills, and the H1-2's bringups are unchanged.
  8. docs: golem_skills' robot profiles and the G1's manipulation skills. CLAUDE.md §4, §6 and §7 and docs/g1.

Testing

  • golem_skills unit tests: 153 pass and 1 is skipped. On main it's 124 passing with the same skip. The new tests are test_robots, test_hands, test_joint_states and test_g1_node. test_grasp_reach and test_grasp_synthesis were extended, and test_nav_launch checks the new launch parameter.
  • On the G1 RoboCasa sim on an M4 Pro Mac, at real-time factor 1.00:
    • Grasp motion, from a made-up object. A scratch script, not committed, gave the real grasp code a 5×5×8 cm object in front of the robot. heuristic, antipodal and centroid each reached a grasp, ending within about 2.5 cm of the object, which is the planned stop-short distance. top_down (80°) was out of reach. The steep and outward-leaning candidates were tried and failed first, about 35 s each.
    • Closing the Dex3-1 on nothing read 0.97–1.02 of full close on every joint, reported correctly as a close on nothing.
    • Refusals: on the live node, graspgenx_filtered and visual_servo goals abort with a clear message before the arm moves.
    • /skill/frontier_explore on the G1 profile ran end to end. Nav2 timed out after the robot walked 0.7 m of a 1.04 m goal.
  • Not run: a grasp or pick_place driven by perception on the G1. That needs Gemini, SAM3 and GraspGenX on x86, and GraspGenX has no Dex3-1 model. Also not run: a grasp of a real object, to set the grip check's thresholds; the H1-2 sim on x86; and the real G1.

Found along the way, not fixed here

  • The G1's head camera can't see a counter-top object. The camera is pitched 48° down, so the 0.92 m kitchen counter sits at the top edge of its view. The Battery workcell gave no depth in view. In home, both hands are in the camera image. A perception grasp on the G1 needs a scene it can see, and probably a G1 prep posture that keeps the hands out of view.
  • In an empty-floor scene FAST-LIO's odometry drifted, and a grasp then asked for a 117 m move. frame_task failed to reach it and the arm went back to home, but nothing checks the odometry before a grasp moves the arm.
  • Already on main: test_frontier_explore::test_node_gives_up_at_its_timeout fails about 2 runs in 10. golem_bringup's flake8 test fails on line length in g1_sim.launch.py and g1_real_robot.launch.py, which this branch doesn't touch.
  • Possible follow-ups: candidate ordering for the G1, since the steep candidates waste time; and measuring the Dex3-1 grasp-frame offset.

Worth checking on the lab machine: the H1-2's grasp and pick-and-place still work as before. Then try a perception grasp on the G1 in the x86 sim.

🤖 Generated with Claude Code

curobotlab and others added 8 commits November 20, 2026 20:56
The grasp frames, pinch frames, prep and staging configs and GraspGenX
gripper the manipulation skills drive were H1-2 constants spread over
base.py, grasp.py, pick_place.py and graspgenx.py. They now come from
robots.RobotProfile, picked once by a new read-only `robot` parameter
(default h12, the only profile so far). No behaviour change on the H1-2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SkillsBase kept the last /joint_states message, so with two publishers
(golem_upper_ik's body joints and, on the G1, dex_3's finger joints) the
grasp reach check could start its IK from a message without the arm
joints. Each joint now keeps its latest position and arrival time;
latest_joint_state() returns them all as one JointState, and
joint_positions_since() waits for named joints published after a time
(the Dex3-1 grip check's input). On the H1-2, with one publisher, the
reach check sees the same joints as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RobotProfile.hand names the hand a robot grasps with: 'magpie' (the
H1-2, unchanged) or 'dex3', dex_3's driver. With dex3 the node makes no
Magpie clients; open_gripper and close_gripper call /<arm>/hand/open and
/<arm>/hand/close, with no force limit and no grip-hold loop (the driver
keeps commanding the pose). The grasp's grip check reads how far the six
closing finger joints travelled from dex_3's open pose to its close pose
(hands.dex3_grip_failure): all of them at 0.9 or more is a close on
nothing, the index and middle fingers' mean under 0.2 a close that did
not happen. The thresholds are chosen, not measured. golem_skills now
exec-depends on dex_3 for its poses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A grasp goal now aborts before the arm moves when its method plans with
GraspGenX (GraspMethod.graspgen) and the robot's profile names no
GraspGenX gripper (graspgen_gripper None), or when it asks for
visual_servo and the profile has no hand cameras (hand_cameras). The
message names the methods that do run. graspgenx._plan also refuses a
missing gripper model rather than call the service. The H1-2 has both,
so nothing changes there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
robot:=g1 selects it: the Dex3-1 hand (dex_3's driver), golem_upper_ik's
'home' as prep and staging (the G1 has neither), no GraspGenX gripper
model and no hand cameras. The G1's URDF has no grasp frame, so the
profile defines tool frames on <arm>_hand_palm_link: a grasp frame in
the GraspGenX convention (+Z the approach, along the palm face's normal;
+X the closing axis, the palm's +X; the finger contact point 0.1146 m
along +Z) and a pinch frame at that contact point, 80 mm along the palm
and 40 mm in front of its face, from the URDF's joint origins and not
measured. SkillsBase broadcasts the tool frames on /tf_static and drives
a tool frame's frame_task goal through its parent link
(robots.drive_pose). The H1-2 has no tool frames, so its goals are
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reach check built golem_upper_ik's H1-2 model and tested the Magpie's
volume. ReachModel and its child process now take the node's robot: its
IK profile, its hand's boxes (HAND_BOXES_M per hand, the Dex3-1's from
the URDF's joint origins) and, for a tool frame such as the G1's Dex3-1
grasp frame, the IK follows the frame's parent link as frame_task does.
On the G1 no method runs the check yet (only the GraspGenX methods do,
and the G1 refuses them); the H1-2's check is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With use_skills:=true, g1_core started golem_skills on its default
profile, the H1-2's (Magpie grippers, graspgenx frames). It now passes
robot: g1, so /skill/grasp and /skill/pick_place drive the Dex3-1 hands
and their palm-link grasp frames. Affects only the G1 bringups with
use_skills (sim; the real G1 starts no skills); the H1-2's bringups and
/skill/frontier_explore are unchanged. test_nav_launch checks the
parameter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CLAUDE.md §4, §6 and §7 and docs/g1 describe the `robot` parameter, what
the G1's profile drives (the Dex3-1 through dex_3, tool frames on the palm
links, 'home' as prep), what it refuses (GraspGenX, visual_servo), and
what has run: the grasp's execution in the sim from a synthetic object,
not a perception-driven grasp or pick_place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants