A Simple dynamic extension for TRELLIS

• This code aims to provide a training-free strategy for TRELLIS to generate continuous results through time.
• This is a forked version from TRELLIS. For the original work, please refer to the links below.

Example Result

Mesh result becomes obviously better in this algorithm. The overall shape is controlled in a stable shape through time.

The GS results have also improved, but there are still occasional flickers in the algorithm.

📦 Installation

Please follow the original installation in TRELLIS.

🔨 Requirement

System RAM 48G at least, CUDA RAM 10G at least. The strategy used in this repo is very System-RAM consuming, as it saves all the attention to CPU. You can delete the to('cpu') in trellis/modules/sparse/attention/modules.py and trellis/modules/attention/modules.py to exchange System RAM for CUDA RAM if you have a better GPU.

💡 Usage

1. Place all the images under a directory, make sure their names are correct (e.g., 00.png-99.png).
2. Run dynamic.py or dynamic_cpu_efficient.py according to your System RAM.
3. See the result in output. It saves all the frames' results and a final result combining all frames.

Other Cases

Failure Case

⚖️ License

TRELLIS models and the majority of the code are licensed under the MIT License. The following submodules may have different licenses:

• diffoctreerast: We developed a CUDA-based real-time differentiable octree renderer for rendering radiance fields as part of this project. This renderer is derived from the diff-gaussian-rasterization project and is available under the LICENSE.

• Modified Flexicubes: In this project, we used a modified version of Flexicubes to support vertex attributes. This modified version is licensed under the LICENSE.

📜 Citation

If you find this work helpful, please consider citing the original paper:

@article{xiang2024structured,
    title   = {Structured 3D Latents for Scalable and Versatile 3D Generation},
    author  = {Xiang, Jianfeng and Lv, Zelong and Xu, Sicheng and Deng, Yu and Wang, Ruicheng and Zhang, Bowen and Chen, Dong and Tong, Xin and Yang, Jiaolong},
    journal = {arXiv preprint arXiv:2412.01506},
    year    = {2024}
}