Official implementation of SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge. SPARK combines VLM-guided part-level and global image guidance with diffusion transformers to produce high-quality articulated object reconstructions from a single image.
- [2026-11] Image generation moved from Gemini to OpenAI's
gpt-image-2.6-sunburst, and the VLM steps now usegpt-6.1-sol, so only an OpenAI API key is required. We compared the latest GPT image models here.
1. Clone and create the env
git clone https://github.com/YumengHe/SPARK.git
cd SPARK
conda create -n spark python=3.10 -y
conda activate spark2. Install PyTorch + torch-cluster + project dependencies
# PyTorch (CUDA 12.4 wheels)
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu124
# torch-cluster (must match torch + CUDA)
pip install torch-cluster==1.6.3 -f https://data.pyg.org/whl/torch-2.5.1+cu124.html
# Project dependencies (pins numpy etc. before PyTorch3D is compiled)
pip install -r requirements.txt3. Point the build at a CUDA 12.4 toolchain
PyTorch3D compiles CUDA code, so nvcc must match the 12.4 wheel above. Pick one of the two options below.
Option A — use an existing system CUDA 12.4 (lightweight, recommended if available)
export CUDA_HOME=/usr/local/cuda-12.4
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH
nvcc --version # sanity-check: must report release 12.4Option B — install CUDA 12.4 toolkit into the conda env (~3 GB, use if no system CUDA 12.4)
conda install -c "nvidia/label/cuda-12.4.0" cuda-toolkit -y
export CUDA_HOME=$CONDA_PREFIX
nvcc --version # sanity-check: must report release 12.44. Build PyTorch3D
pip install ninja
pip install --no-build-isolation "git+https://github.com/facebookresearch/pytorch3d.git@stable"--no-build-isolation lets the build see the torch we just installed (otherwise pip spins up an isolated env without it).
All checkpoints (PartCrafter, TripoSG, RMBG-1.4) live in
2bidoubi/SPARK and are fetched into
pretrained_weights/<name>/ automatically on first run. To pre-download them:
python -m src.utils.download_weightsCreate an .env file and add API key inside:
# OpenAI: metadata.json prediction, joint-axis correction, prompt generation, part/open-state image generation
OPENAI_API_KEY=
# Meshy: texturing
MESHY_API_KEY=
Place input images in ./image/ and run the full pipeline:
python run.pyrun.py runs nine stages end-to-end (metadata → URDF → per-part image generation → PartCrafter inference → mesh cleanup → texture). Toggle individual steps with the step1 … step9 flags at the top of the file.
See datasets/README.md for the full pipeline. A typical PartNet-Mobility flow:
# Convert PartNet-Mobility (obj+mtl) to GLB
python datasets/merge_to_glb.py mesh/partnet_test mesh/partnet_glb
# Voxelize meshes
python datasets/voxel_surface.py mesh/partnet_glb --subfolder -r 200
# Build PartCrafter training data
CUDA_VISIBLE_DEVICES=0 python datasets/preprocess/preprocess_partnet.py \
--input mesh/partnet_glb --output preprocessed_databash scripts/train/train_partcrafter.sh --config configs/mp16_nt1024.yamlSet WANDB_API_KEY in the script if you want experiment tracking.
Single-image object inference:
python scripts/inference/object.py \
--image_path image/example.png \
--num_parts 4 \
--output_dir results \
--renderPer-part image inference:
python scripts/inference/part_images.py \
--image_folder output/example_case \
--output_dir results \
--renderRender a generated mesh:
python scripts/render/glb.py \
--input results/example/object.glb \
--output_dir render_output \
--gifscripts/inference/: inference entrypointsscripts/render/: mesh and URDF rendering helpersscripts/train/: training launch scriptsscripts/batch/: batch inference runnersscripts/tools/: maintenance utilitiesscripts/legacy/: archived old scripts
For maintainers: see docs/upload_weights.md.
Thanks to Yongfei She for organizing the code for open source release.
If you find SPARK useful for your research, please cite:
@inproceedings{he2026spark,
title={Spark: Sim-ready part-level articulated reconstruction with vlm knowledge},
author={He, Yumeng and Jiang, Ying and Lu, Jiayin and Yang, Yin and Jiang, Chenfanfu},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={7233--7243},
year={2026}
}