Armand Losfeld · Sarah Dury · Gauthier Lafruit · Mehrdad Teratani · Daniele Bonatto
@inbook{Losfeld2026,
title = {$\partial$DIBR: Differentiable Depth Image-based Rendering for Fast Novel View Synthesis},
ISBN = {9783032370105},
ISSN = {1611-3349},
url = {http://dx.doi.org/10.1007/978-3-032-37010-5_22},
DOI = {10.1007/978-3-032-37010-5_22},
booktitle = {Computer Vision – ECCV 2026},
publisher = {Springer Nature Switzerland},
author = {Losfeld, Armand and Dury, Sarah and Lafruit, Gauthier and Teratani, Mehrdad and Bonatto, Daniele},
year = {2026},
pages = {398–416}
}
- 1. System Requirements
- 2. Installing NerfBaselines
- 3. Installing Rendering Methods with Nerfbaselines
- 4. Dataset Download and Preparation
- 5. Training Ours and Baselines Methods
- 6. Interactive Viewer
- 7. Results
- 8. Standalone Installation Without NerfBaselines
- 9. Uninstallation
- Acknowledgments
- License
- Dataset Licenses
- Linux (e.g., Ubuntu 24.04) — strongly recommended
- NVIDIA RTX GPU, Vulkan + Slang support, 10–24 GiB VRAM
- CUDA (version matching your NVIDIA driver)
- Conda (Anaconda or Miniconda)
- apptainer (for containerized installations)
Note
Some methods require GPU‑based Vulkan rendering; ensure proper driver installation. On laptops, explicitly ensure the NVIDIA GPU is used.
For nerfbaseline-integrated installation,
git clone https://github.com/LISA-VR/ddbir.git
git submodule update --init --remote external/nerfbaselinesFor standalone installation,
git clone https://github.com/LISA-VR/ddbir.git
git submodule update --init --remote external/ZoeDepth
git submodule update --init --remote external/DepthAnything3 #OptionalAll evaluated methods (ours and baselines) are integrated into a unified evaluation workflow using the NerfBaselines API. A modified fork containing our integration is provided in external/nerfbaselines. The forked project can also be found at https://github.com/ArmandLfd/nerfbaselines.
git submodule update --init --remote external/nerfbaselines
cd external/nerfbaselines
conda create -n nerfbaselines -y python=3.13
conda activate nerfbaselinespip install -e .Install any supported method:
nerfbaselines install-method --method [gaussian-splatting|2d-gaussian-splatting|triangle-splatting|diff-dibr] --backend [conda|apptainer]Note
We recommend installing methods inside an Apptainer container for reproducibility. Our ∂DIBR method also works in a standard conda environment.
Warning
If you installed mamba instead of conda, please do not use the apptainer backend, use rather the conda backend. Or install conda.
Warning
If you want to use this code on a HPC, please try to remove some bindings from nerfbaselines/backend/_apptainer.py.
Once completed, you can test the installation:
nerfbaselines test-method --method [gaussian-splatting|2d-gaussian-splatting|triangle-splatting|diff-dibr] --backend [conda|apptainer] --data external://llff/fernOn the first run, the tool downloads:
- Required datasets
- LPIPS pretrained weights
Warning
Image pull permissions: Some systems restrict image pulls to privileged users. See: AndreWeiner/ml-cfd-lecture#36.
Note
Vulkan GPU selection: Install vulkan-tools and run vkinfo to verify the active GPU.
If the system selects an integrated GPU, remove MESA Vulkan drivers and reinstall NVIDIA Vulkan drivers.
Automatic download:
nerfbaselines download-dataset --data external://llffThe dataset will be download in the default cache folder, e.g., ~/.cache/nerfbaselines/datasets/llff.
This dataset must be downloaded manually.
- Download the Shiny dataset from shiny_shared_folder.
- Once done, Copy each scene folder into
~/.cache/nerfbaselines/datasets/shiny/{scene_name}. - Run the script resize.py to resize the images.
To run a training/optimization with the default parameters of any method:
nerfbaselines train --method [gaussian-splatting|2d-gaussian-splatting|triangle-splatting|diff-dibr] --backend [conda|apptainer] --data ~/.cache/nerfbaselines/datasets/${DATASET_NAME}/${SCENE_NAME} --output ${OUTPUT}Note
Vulkan Assert failure: Our code can crash with this error: "vulkan Assert failure". Please first check that you have a correct installation of your GPU drivers and Vulkan installation. If the error persists, check if your GPU has enough memory (for full image resolution, at least 24GB of memory). If not, you can use --set scale_factor=${FACTOR} to rescale images and reduce GPU memory (e.g., $FACTOR = 0.5).
Use our script implementing the multi‑stage optimization described in the paper.
To run the script:
chmod u +x scripts/diff-dibr/coarse-to-fine.sh
scripts/diff-dibr/coarse-to-fine.sh --data ~/.cache/nerfbaselines/datasets/${DATASET_NAME}/${SCENE_NAME} --output ${OUTPUT} --backend [conda|apptainer]| Parameters | Description | Type | Required |
|---|---|---|---|
| --data | Path to input data | Path | Yes |
| --output | Output directory | Path | Yes |
| --backend | Backend used to run the code | [conda, apptainer] | Yes |
| --num_iters | Number of iterations in lvl 1 (lvl2 = 5lvl1/3, lvl3 = 2lvl1) | numeric | No |
| --depth_method | Depth initialization | [Zoe, DepthAnything3, NONE] | No |
| --nb_cams | Nb of cams representing the scene | numeric | No |
| --nb_a_cams | Number of active cameras per rendering | numeric | No |
| --l_atv_12 | Depth regularization weight (stages 1 & 2) | numeric | No |
| --l_atv_3 | Depth regularization weight (stage 3) | numeric | No |
Launch the viewer:
nerfbaselines viewer --method [gaussian-splatting|2d-gaussian-splatting|triangle-splatting|diff-dibr] --backend [conda|apptainer] --data ~/.cache/nerfbaselines/datasets/${DATASET_NAME}/${SCENE_NAME} --output ${OUTPUT}/checkpoint-${NUM_ITERATIONS} --port 6064Then, open http://localhost:6064 in your browser.
Note
By default, ${NUM_ITERATIONS} equals 30000 for splatting-based methods. For ours, it equals 40000. For the lower optimization stages, diminish the value to 20000 (lvl 1) or 30000 (lvl 2).
The rendering results can be found in ${OUTPUT}/predictions-${NUM_ITERATIONS}.
While we recommend using NerfBaselines for reproducibility, standalone use is supported.
unzip -o ./diff_dibr.zip
cd diff_dibr
conda create -n diff-dibr -y python=3.13
conda activate diff-dibr
pip install -r requirements.txt
pip install \
torch==2.9.1 \
torchvision==0.24.1 \
--index-url https://download.pytorch.org/whl/cu128
pip install lpips==0.1.4
pip install -e .Each depth module requires a dedicated conda environment named after the module:
- for ZoeDepth, the conda env must be named
zoe. - For DepthAnything3, it must be
depthanything3.
git submodule update --init --remote external/ZoeDepth
cd external/ZoeDepth
conda create -n zoe -y python=3.9.7
conda activate zoe
conda install -y \
cuda=11.7.1 \
h5py=3.7.0 \
hdf5=1.12.2 \
matplotlib=3.6.2 \
matplotlib-base=3.6.2 \
numpy=1.26.4 \
scipy=1.10.0 \
wandb=0.13.9 \
tqdm=4.64.1 \
huggingface_hub=0.11.1 \
timm=0.6.12 \
pip \
-c pytorch \
-c nvidia \
-c conda-forge
pip install opencv-python==4.6.0.66
pip install \
torch==1.13.1 \
torchvision==0.14.1 \
torchaudio==0.13.1 \
--index-url https://download.pytorch.org/whl/cu117git submodule update --init --remote external/DepthAnything3
cd external/DepthAnything3
conda create -n depthanything3 -y python=3.12.12
conda activate depthanything3
conda install -y pip
pip install xformers torch\>=2 torchvision pillow \
--index-url https://download.pytorch.org/whl/cu128
pip install -e .For optimization, please use NerfBaselines.
python ./script/diff-dibr/freeview.py --path $OUTPUT/checkpoint-${NUM_ITERATIONS} --nb_cams_synthesis $NB_ACTIVE_CAMSconda remove --all -n nerfbaselines -ysudo rm -r ~/.cache/nerfbaselinesconda clean --all -y
conda clean --index-cache
conda clean --packagesArmand Losfeld is a FRIA grantee and Sarah Dury is a research fellow of the Fonds de la Recherche Scientifique – FNRS, Belgium. This work was supported by the 24 Top-Tier International Global Collaborative Research Program funded by ETRI; the IITP grant funded by the Korean government (MSIT) (RS-2017-II170072); and the Service Public de Wallonie Recherche – Wal4XR by Win4Excellence (2310144), Belgium.
We also extend our sincere gratitude to the developers and maintainers of the frameworks, methods, and datasets that made this work possible. Their efforts in building high‑quality, open‑source resources have been invaluable for advancing research in neural rendering and novel view synthesis.
- RVS - https://gitlab.com/mpeg-i-visual/rvs
- NerfBaselines - https://github.com/nerfbaselines/nerfbaselines
- 3D Gaussian Splatting — https://github.com/graphdeco-inria/gaussian-splatting
- 2D Gaussian Splatting — https://github.com/hbb1/2d-gaussian-splatting
- Triangle Splatting — https://github.com/trianglesplatting/triangle-splatting
- LLFF Dataset — https://github.com/Fyusion/LLFF
- Shiny Dataset (NeX) — https://github.com/nex-mpi/nex-code
- Slang and Slangpy - https://github.com/shader-slang/slang and https://github.com/shader-slang/slangpy
All original source code (src/, scripts/) is licensed under AGPL-3.0-or-later. Please also read the copyright notice file.
All web design of the github page (index.html, static/js, static/css) is licensed under CC BY-SA 4.0, adapted from eliahuhorwitz/Academic-project-page-template.
All images (static/images/) are licensed under CC BY 4.0. Note: The ULB logo is a registered trademark. Its inclusion in this repository does not grant a license under CC BY 4.0 or CC BY-SA 4.0. It may not be used or reproduced without explicit permission.
- LLFF is under GPL-3.0 license
- Shiny (NeX) is under MIT License