Foundation Models for 3D Asset Synthesis

In recent years, thanks to the continuous innovation and progress of diffusion technology, significant advancements have been made in image and video generation. By inputting textual descriptions or images, we can generate high-quality images or videos, which greatly enhance creative efficiency and imagination. However, progress in the 3D generation field has been relatively slow. Initially, optimization routes, represented by DreamFusion, were explored. This was followed by the exploration of reconstruction routes, such as LRM. It was only later that diffusion based on 3D generation techniques, similar to those in image and video generation, were gradually developed. In addition, based on the token-by-token prediction form similar to LLM, 3D generation based on autoregressive method has gradually made significant progress.

Therefore, this tutorial focuses on the topic of 3D asset generation using diffusion and autoregression, specifically including:

1. Geometry generation modeling based on the diffusion paradigm.
2. Geometry generation modeling based on the autoregression paradigm.
3. Texture generation modeling based on the diffusion paradigm.

We would like to express our gratitude to the research community for their contributions to 3D asset generation.