Scrutexity-Sim

A simulation environment for physical AI construction robotics — built to prove that Scrutexity can make Chinese construction robots (Fangshi, Weibuild) work on US job sites before any hardware ships.

"We can simulate your robot on a US construction site before it ever arrives. Here's a video of your robot finishing a concrete slab in simulation. Here's the code. Here's the data pipeline."

---

What This Is

Scrutexity-Sim is a 14-day proof-of-capability build that demonstrates, end-to-end:

1. A construction site scene (slab, form walls, rebar, equipment) loaded in a real physics engine.
2. A robot model (Franka Emika Panda placeholder for a generic 6-DOF screed arm) operating in the scene.
3. A concrete finishing task with a real reward function (coverage + flatness + time penalty).
4. A PID controller driving the screed head across the slab.
5. A synthetic data pipeline producing labeled images for perception training.
6. An edge deployment story (Docker + ROS 2 + simulated Jetson runtime).
7. One-command demo: python run_demo.py launches simulation, runs the task, writes a video.

Target Stack

Component	Technology	Status in this repo
Simulation engine	Genesis World 1.3.3 (Nyx + Quadrants + unified multi-physics)	✅ Installed & verified on CPU
Training framework	Isaac Lab (GPU RL, 30+ envs)	🟡 Stubbed — requires NVIDIA GPU
Sim-to-real bridge	Sim2RealLab (Jetson ROS 2 runtime)	🟡 Stubbed — requires Jetson
Construction middleware	ROS2-TMS-for-Construction (IoRT)	🟡 Stubbed — requires ROS 2 host
Synthetic data	InternDataEngine / Isaac Replicator	🟡 Custom fallback generator implemented
Orchestration	Docker Compose + NVIDIA OSMO	🟡 Compose file written; needs Docker host
Version control	Git + DVC	✅ Git initialized, DVC config ready

Honest status: This sandbox has no GPU and no Docker. Real Genesis physics runs on CPU. Rendering uses a matplotlib fallback. All "real" code paths (Nyx renderer, Isaac Lab training, Jetson deploy) are written and wired in — flip one env var on a GPU box and they take over from the fallback.

Quick Start

Prerequisites

• Python 3.10–3.13 (tested on 3.12)
• ~3 GB disk for deps
• (Optional) NVIDIA GPU + CUDA for full Nyx rendering and Isaac Lab training
• (Optional) Docker for the edge deployment demo

Install

git clone <repo-url> scrutexity-sim
cd scrutexity-sim
make setup

This installs:

• genesis-world (the actual Genesis World Python package, v1.3.3)
• CPU-only torch (fallback when no GPU is detected)
• matplotlib, numpy, opencv-python, dvc, pyyaml
• (Skipped if absent) GPU torch + isaac-lab + isaac-sim

Run the Demo

make demo
# or
python run_demo.py

This will:

1. Initialize Genesis World on CPU (or GPU if available).
2. Build a US construction site scene: 8 m × 6 m slab, perimeter form walls, rebar grid, equipment props.
3. Spawn a Franka Panda arm with a flat screed plate end-effector.
4. Run a PID controller that drives the screed across the slab in a boustrophedon (lawn-mower) pattern.
5. Compute task reward (coverage %, flatness variance, elapsed time).
6. Render the sim state to demo/concrete_finishing.mp4 (matplotlib animation, 30 fps).

Run the Tests

make test

Tests cover: scene construction, controller convergence, reward function semantics, synthetic data labeling.

Repository Layout

scrutexity-sim/
├── README.md                  ← you are here
├── Makefile                   ← setup / demo / test / data / clean targets
├── docker-compose.yml         ← sim + edge + ros2 services
├── .env.example               ← SCRUTEXITY_RENDER, ROS_DOMAIN_ID, etc.
├── .gitignore
├── pyproject.toml             ← project + dev deps
├── run_demo.py                ← single-command entrypoint
├── assets/
│   ├── sites/
│   │   └── construction_site.py   ← Genesis scene builder
│   ├── robots/
│   │   └── screed_robot.urdf     ← Franka-based generic screed arm
│   └── equipment/
│       └── equipment.py          ← pump, hopper, power trowel props
├── src/
│   ├── genesis_compat.py         ← headless CPU patch + GPU autoselect
│   ├── perception/
│   │   └── synthetic_data.py     ← InternDataEngine-style data generator
│   ├── control/
│   │   └── screed_controller.py  ← PID boustrophedon controller
│   ├── world_model/
│   │   └── slab_state.py         ← concrete height-field state
│   └── edge/
│       └── jetson_runtime.py     ← simulated Jetson deployment shim
├── tasks/
│   └── concrete_finishing.py     ← task spec + reward function
├── scripts/
│   ├── setup.sh
│   ├── run_simulation.sh
│   └── generate_data.sh
├── tests/
│   ├── test_scene.py
│   ├── test_controller.py
│   ├── test_reward.py
│   └── test_synthetic_data.py
├── docs/
│   ├── architecture.md
│   ├── onboarding.md
│   └── roic_model.md
├── deploy/
│   ├── terraform/                ← cloud GPU runner (sketch)
│   └── k8s/                      ← k8s manifests for edge fleet
├── data/
│   ├── raw/                      ← external sensor data
│   ├── synthetic/                ← generated labeled images
│   └── models/                   ← trained policy checkpoints
└── demo/
    └── concrete_finishing.mp4     ← produced by run_demo.py

Architecture (One-Pager)

                   ┌──────────────────────────────────────────┐
                   │           Scrutexity-Sim Orchestration     │
                   └──────────────────────────────────────────┘
                                       │
       ┌───────────────────────────────┼───────────────────────────────┐
       │                               │                               │
       ▼                               ▼                               ▼
┌─────────────┐                ┌──────────────┐               ┌──────────────┐
│  Genesis     │                │  Concrete     │               │  Perception  │
│  World scene │◄──────────────│  Finishing    │──────────────►│  / Synthetic │
│  (physics)   │   sim state    │  Task +       │   scene meta  │  Data Engine │
│              │                │  Reward fn    │               │              │
└─────────┬───┘                └──────┬───────┘               └──────┬───────┘
          │                           │                              │
          │ joint targets              │ coverage/flatness           │ labeled
          │                           │                              │ images
          ▼                           ▼                              ▼
┌─────────────────┐          ┌──────────────────┐         ┌──────────────────┐
│ PID Screed       │          │  Slab height-field│         │  DVC-tracked     │
│ Controller       │─────────►│  (world model)   │         │  dataset         │
│ (boustrophedon)  │          │                  │         │  data/synthetic/ │
└─────────────────┘          └──────────────────┘         └──────────────────┘
          │                                                           │
          │                                                           │ policy
          │                                                           ▼
          │                                                  ┌──────────────────┐
          │                                                  │  Edge Runtime    │
          └─────────────────────────────────────────────────►│  (Jetson + ROS2 │
                                                             │  + trained model)│
                                                             └──────────────────┘

See docs/architecture.md for the full layered breakdown.

Adding a New Robot

1. Drop a URDF or MJCF into assets/robots/.
2. Add an entry to assets/robots/registry.py mapping a slug → file path + end-effector link name.
3. Pass the slug to run_demo.py --robot <slug>. The scene builder will load it.
4. If the new robot has different DOF count, adjust screed_controller.py's IK target chain.

Adding a New Task

1. Create tasks/<task>.py implementing the Task protocol in tasks/base.py.
2. The task must expose: setup(scene), reward(state) -> float, done(state) -> bool, spec().
3. Register it in tasks/__init__.py.
4. Run with python run_demo.py --task <task>.

Honest Limitations

This sandbox runs on CPU only. The following features are scaffolded but not exercised here:

• Nyx / pyrender rendering: requires EGL or a display. src/genesis_compat.py stubs the rasterizer on CPU; visualization falls back to matplotlib. On a GPU box with SCRUTEXITY_RENDER=genesis, the real renderer takes over.
• Isaac Lab RL training: needs Isaac Sim + GPU. The task spec and reward are Isaac-Lab-compatible; integration code is in src/training/ (placeholder).
• Real ROS 2 bridge: needs a ROS 2 host. The bridge code in src/edge/ is structured for rclpy and degrades to a mock when ROS is unavailable.
• Real Jetson deployment: needs Jetson hardware. The Docker image in deploy/ is a representative Jetson image (JetPack 6 / ROS 2 Humble).

Everything else — Genesis physics, task logic, controller, synthetic data, DVC, tests, docs — runs for real in this sandbox.

License

Proprietary — Scrutexity, 2026. See LICENSE (or pending).