SMPL-Unity-Motion-Studio

A versatile Unity visualizer and sequence generator for SMPL (Skinned Multi-Person Linear) 3D human body models, motion capture trajectories, and human-object interactions.

Tailored for computer vision, robotics, and deep learning researchers to visualize motion sequences, inspect human motion prediction models (comparing observed past context with predicted future motions), and produce publication-ready paper figures, stroboscopic sequences, and video demos with minimal setup.

Real-time motion playback: the SMPL body displays current movement, with an optional forward 3D skeleton previewing predicted future motion (toggleable on/off for pure motion playback or prediction analysis).

---

✨ Key Features

• 🔮 Motion Sequences & Human Motion Prediction: Specifically designed to visualize motion sequences and evaluate human motion prediction models. Inspect observed past context versus predicted future motion through forward time steps, dedicated future prefabs, and material/color swapping (materialChangeFrame).
• ⚡ Real-Time Playback: Animate 24-joint SMPL rigs in real time directly from JSON files (MotionController.cs). Supports Neutral, Male, and Female shape scales.
• 📸 Stroboscopic Multi-Pose Sequences: Generate "ghost" sequences (SequencesGenerator.cs) with time-based alpha fading, selective shadow casting, and configurable frame steps for publication figures.
• 🦴 3D Procedural Skeleton: Real-time procedural 3D skeleton visualizer (RuntimeSkeletonVisualizer.cs) displaying joints as spheres and bones as cones/cylinders with customizable colors.
• 📈 Motion Trajectory Splines: Continuous 3D path tracing using Unity Splines (SplineFromJoints.cs) to visualize joint trajectories (pelvis, hands, feet) over time.
• 🪑 Human-Object Interaction (GRAB): Native support for 3D bounding boxes of manipulated objects and obstacles/tables with automated pedestal placement (TablesManager.cs).
• 🎥 Recording Ready: Seamless integration with Unity Recorder and smooth camera rigs for high-resolution video and sequence rendering.
• 🚀 Ready-to-Run Demos: Curated sample motions are bundled directly into the repository so every scene works immediately upon cloning.

---

🚀 Quickstart

1. Clone the repository:

   git clone https://github.com/MichaelVanuzzo/SMPL-Unity-Motion-Studio.git

2. Open in Unity Hub:
   • Unity Version: 2022.3.62f3 (or any modern Unity 2022.3 LTS release).
   • Packages will resolve automatically on first launch via Unity Package Manager.
3. Open any demo scene from Assets/Scenes/ and press Play!

---

🎬 Available Demo Scenes (Assets/Scenes/)

Scene	Preview	Description	Included Samples
01_SMPL_TPose_Skeleton.unity		Canonical SMPL rigged model in T-Pose with procedural 3D skeleton visualization (spherical joints, conical bone links) and runtime material swapping.	SMPL Base Rig
02_Realtime_Motion_Playback.unity		Continuous real-time animated playback of human motion interacting with tables and objects from motion JSON data.	Collaborative Assembly (collaborative_assembly)
03a_TrajectorySpline.unity		Stroboscopic multi-pose sequence with alpha fading and continuous 3D joint trajectory splines at the assembly workstation.	Collaborative Assembly (collaborative_assembly)
03b_ArmTrajectory.unity		Upper-body reaching and arm kinematics visualization tracing shoulder, elbow, and wrist 3D trajectory splines.	Collaborative Assembly (collaborative_assembly)
03c_WorkstationObjects.unity		Workstation assembly context featuring human worker posture, table obstacles, elevated shelves, and 3D bounding boxes.	Collaborative Assembly (collaborative_assembly)
03d_MotionProgression.unity		Discrete stroboscopic pose sequence detailing temporal progression, human kinematics, and step-by-step motion breakdown.	Collaborative Assembly (collaborative_assembly)
04_HumanObject_TablePedestals_GRAB.unity		Studio panorama of human-object grasping trajectories (GRAB benchmark) with 3D oriented bounding boxes and procedural table pedestals.	Object Grasping (grab_examples)
05_HumanObject_BoundingBoxes_GRAB.unity		Publication-ready horizontal breakdown of an object pick-and-lift motion with 3D oriented bounding boxes and table contact surfaces.	Object Grasping (grab_examples)
06_TextToMotion_Gestures.unity		Expressive conversational and text-to-motion gestures rendered across multiple actors against a clean, neutral white studio backdrop.	Conversational Gestures (amass_examples)
07_StudioRoom_MotionGestures.unity		Complete 3D studio environment featuring stage acoustic walls, floor lighting, and multi-actor conversational body motion playback.	Conversational Gestures (amass_examples)

---

📊 Motion Data Format (JSON)

Motions are stored in clean, human-readable JSON files easily exported from Python / PyTorch / NumPy:

{
  "n_frames": 254,
  "n_joints": 24,
  "framerate": 25.0,
  "translations": [
    0.042, 0.596, 0.557,
    ...
  ],
  "rotations": [
    0.0, 0.0, 0.0, 1.0,
    ...
  ]
}

• translations: Flattened array [n_frames * 3] of root position $(x, y, z)$.
• rotations: Flattened array [n_frames * 24 * 4] of quaternions $(x, y, z, w)$ for the 24 SMPL joints in standard kinematic order.

Optional Bounding Boxes (Object Interaction)

For scenes involving objects (e.g. GRAB dataset):

• n_objects: Number of dynamic objects.
• obj_bb: Flattened 8-vertex 3D coordinates [n_frames * n_objects * 8 * 3].
• n_obstacles: Number of static obstacles/tables.
• obs_bb: Flattened 8-vertex 3D coordinates [n_frames * n_obstacles * 8 * 3].

Python Export Helper

import json
import numpy as np

def export_to_smpl_studio(filename, root_trans, joint_quats, fps=25.0):
    """
    root_trans: (N, 3)
    joint_quats: (N, 24, 4) in (x, y, z, w) order
    """
    data = {
        "n_frames": int(len(root_trans)),
        "n_joints": int(joint_quats.shape[1]),
        "framerate": float(fps),
        "translations": root_trans.flatten().tolist(),
        "rotations": joint_quats.flatten().tolist()
    }
    with open(filename, 'w') as f:
        json.dump(data, f)

Place custom JSON files in Assets/Resources/ or a subfolder, then specify the filename or folder in the Unity Inspector!

---

⚙️ Technical Requirements

• Unity LTS: 2022.3.62f3 (or compatible 2022.3.x LTS).
• Required Packages (managed automatically via Package Manager):
  • com.unity.splines (2.8.2+)
  • com.microsoft.mrtk.graphicstools.unity (v0.8.1)
  • com.unity.recorder (4.0.3+)

---

📚 Citation

If you use SMPL-Unity-Motion-Studio in your research, generate figures for a paper, or build upon this work, please cite this repository:

@software{vanuzzo2026smplunitystudio,
  author    = {Vanuzzo, Michael},
  title     = {SMPL-Unity-Motion-Studio: A Unity Visualizer and Sequence Generator for SMPL Models and Motion Sequences},
  year      = {2026},
  publisher = {GitHub},
  url       = {https://github.com/MichaelVanuzzo/SMPL-Unity-Motion-Studio}
}

---

📜 License & Academic Attribution

This software is released under the MIT License.

If you use this visualizer in academic publications or projects, please credit this repository and acknowledge the underlying datasets and body models:

• SMPL Model: Max Planck Institute for Intelligent Systems (SMPL terms)
• AMASS Dataset: Mahmood et al., AMASS: Archive of Motion Capture as Surface Shapes, ICCV 2019 (AMASS terms)
• GRAB Dataset: Taheri et al., GRAB: A Dataset of Whole-Body Human Grasping of Objects, ECCV 2020 (GRAB terms)