Skip to content

Isaac Sim backend #903

Open
Open

Description

@petercorke

Part of #886. For @alexermolovich — please comment here so you can be assigned.

Scope (milestone 1)

  • backend="isaacsim" starts Isaac Sim via SimulationApp, with a basic scene (camera, lighting)
  • add(robot) for URDF-backed robots, converted to USD with Isaac's URDF importer
  • add() returns a handle that owns its own q, so several instances of one model work
  • Desktop only, Isaac Sim 6.2, optional isaacsim extra with a lazy import; import roboticstoolbox must still work without it

Target

A URDF Panda from robot_descriptions:

panda.plot(panda.qr, backend="isaacsim")
traj = rtb.jtraj(panda.qz, panda.qr, 50)
panda.plot(traj.q, backend="isaacsim", dt=0.05)

A screencast in Discussions → Show & Tell.

Out of scope for now

Notebooks, teach, ellipsoids, ETS/DH robots, physics-driven step.

Open questions (to settle before the PR is reviewed)

  1. Testing. Isaac Sim needs a GPU and isn't open licensed, so it can't run in CI. What's testable without it (mocked SimulationApp, joint-mapping logic), and what's the manual checklist for the rest?
  2. Dispatch. Blocked by Backend registry: single name → constructor dispatch #902 (backend registry). Register the backend there as isaacsim. Once Backend registry: single name → constructor dispatch #902 is merged, you can also develop and test outside the RTB tree with register_backend("isaacsim", ...).
  3. Expanded URDF. Keeping the expanded XML (Xacro args, patches) changes RTB's URDF loading. Please do that as a separate, small PR first.
  4. Python versions. Isaac Sim targets 3.12, while RTB supports 3.10–3.14. The extra should install only where Isaac Sim does.
  5. restart(). If Isaac Sim can't restart in-process, raise a clear error and say so in the docs, rather than half-supporting it.

Process


Generated by Claude Code

Activity

  1. alexermolovich commented on Oct 25, 2026

    @alexermolovich

    Accepted

  2. alexermolovich commented on Oct 30, 2026

    @alexermolovich

    Working on this issue here PR

  3. alexermolovich commented on Nov 13, 2026

    @alexermolovich

    Hey @petercorke, I’ve added a basic smoke test that requires an NVIDIA GPU. The robot now loads successfully and is added to the scene with lighting. I’m currently working on adding more control through the code, but the basics are working now: Isaac Sim instance starts, scene is being setup correctly, and the robot works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions