State-of-the-art 4D generation model L4GM and our solution.
State-of-the-art 4D generation model L4GM and our solution.
Current 3D/4D generation methods are usually optimized for photorealism, efficiency, and aesthetics. However, they often fail to preserve the semantic identity of the subject across different viewpoints. Adapting generation methods with one or few images of a specific subject (also known as Personalization or Subject-driven generation) allows generating visual content that align with the identity of the subject. However, personalized 3D/4D generation is still largely underexplored.
In this work, we introduce TIRE (Track, Inpaint, REsplat), a novel method for subject-driven 3D/4D generation. It takes an initial 3D asset produced by an existing 3D generative model as input and uses video tracking to identify the regions that need to be modified. Then, we adopt a subject-driven 2D inpainting model for progressively infilling the identified regions. Finally, we resplat the modified 2D multi-view observations back to 3D while still maintaining consistency. Extensive experiments demonstrate that our approach significantly improves identity preservation in 3D/4D generation compared to state-of-the-art methods.
Our proposed method, TIRE, consists of three stages: Track, Inpaint, Resplat.
(1) Track aims at providing the masks indicating the infilling regions from other viewpoints beyond the given source view.
(2) Inpaint targets at progressively infilling the unobserved regions in other viewpoints with the infilled contents preserving the identity, while the regions are identified by the previous Track step.
(3) Resplat is responsible for unprojecting the 2D infilled observations back to 3D.
Qualitative comparison on image-to-3D generation with SV3D, Wonder3D, LGM, MeshFormer, TRELLIS, and Hunyuan3D-v2.5. Our TIRE better preserves the identity of the subject for the generated 3D assets.
Although TIRE mainly aims to improve subject-driven appearance modeling of the generated 3D/4D assets, it also demonstrates better geometry quality than the baseline methods.
@inproceedings{zheng2025trackinpaintresplat,
title={Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infilling},
author={Zheng, Shuhong and Mirzaei, Ashkan and Gilitschenski, Igor},
booktitle={NeurIPS},
year={2025}
}