DreamComposer++: Empowering Diffusion Models with Multi-View Conditions for 3D Content Generation
Yunhan Yang, Shuo Chen, Yukun Huang, Xiaoyang Wu, Yuan-Chen Guo, Edmund Y. Lam, Hengshuang Zhao, Tong He, Xihui Liu · 2025
Read DreamComposer++: Empowering Diffusion Models with Multi-View Conditions for 3D Content Generation by Yunhan Yang, Shuo Chen, Yukun Huang, Xiaoyang Wu, Yuan-Chen Guo, Edmund Y. Lam, Hengshuang Zhao, Tong He, Xihui Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-02299.txt.
Introduction
Yunhan Yang and co-authors present a framework for creating new views of objects in a 2025 paper. Diffusion models can invent fresh angles of an object from one photo, but users cannot steer the result well without more input views. Their system, DreamComposer++, feeds several views in as conditions to guide the output.
A lifting module first builds a 3D sense of the object from the given views. A fusion module blends that sense into the features of the wanted view. Those features then enter standard image or video diffusion models for the final synthesis. Tests show the add-on boosts control for leading view-aware models and supports full 3D object rebuilds.
Catalog record
| CREATOR | Yunhan Yang, Shuo Chen, Yukun Huang, Xiaoyang Wu, Yuan-Chen Guo, Edmund Y. Lam, Hengshuang Zhao, Tong He, Xihui Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-02299
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DreamComposer++: Empowering Diffusion Models with Multi-View Conditions for 3D Content Generation Yunhan Yang, Shuo Chen, Yukun Huang, Xiaoyang Wu, Yuan-Chen Guo, Edmund Y. Lam, Hengshuang Zhao, Tong He, Xihui Liu Abstract Recent advancements in leveraging pre-trained 2D diffusion models achieve the generation of high-quality novel views from a single in-the-wild image. However, existing works face challenges in producing controllable novel views due to the lack of information from multiple views. In this paper, we present DreamComposer++, a flexible and scalable framework designed to improve current view-aware diffusion models by incorporating multi-view conditions. Specifically, DreamComposer++ utilizes a view-aware 3D lifting module to extract 3D representations of an object from various views. These representations are then aggregated and rendered into the latent features of target view through the multi-view feature fusion module. Finally, the obtained features of target view are integrated into pre-trained image or video diffusion models for novel view synthesis. Experimental results demonstrate that DreamComposer++ seamlessly integrates with cutting-edge view-aware diffusion models and enhances their abilities to generate controllable novel views from multi-view conditions. This advancement facilitates controllable 3D object reconstruction and enables a wide range of applications. I Introduction Fig. 1 : DreamComposer++ is able to generate controllable novel views and 3D objects via injecting multi-view conditions. We incorporate the method into the pipelines of Zero-1-to-3 [ 1 ] , SyncDreamer (SyncD) [ 2 ] and SV3D [ 3 ] to enhance the control ability of those models. The generation of 3D objects has become a prominent area of study in the fields of computer vision and graphics. This research area has significant implications for various applications, including augmented reality (AR), virtual reality (VR), film production, and the gaming industry. Leveraging 3D object generative models, users and designers are empowered to efficiently produce customized 3D assets using textual or visual cues, minimizing the need for extensive manual labor by human experts. Recently, diffusion models [ 4 , 5 ] have achieved significant success in generating 2D images from textual descriptions. This breakthrough has sparked interest in extending these models to 3D object generation, leveraging 2D diffusion priors [ 6 , 7 , 8 , 9 , 10 , 11 ] as supervision to learn 3D representations via SDS loss . While impressive results have been achieved [ 9 ] , 2D diffusion models suffer from limited view control and lack the ability to ensure view-consistent supervision. This often leads to quality issues in 3D outputs, such as the appearance of multiple faces and blurry details. To address this issue, Zero-1-to-3 [ 1 ] finetunes 2D diffusion models with view conditioning , enabling zero-shot novel view synthesis (NVS) based on a single-view image. This approach facilitates image-to-3D object generation, improving consistency across different views. Follow-up works [ 2 , 12 , 13 , 14 , 15 ] further propose to enhance the cross-view consistency of generated images. However, due to the incomplete information from a single-view input, these methods often struggle with generating unpredictable or implausible shapes and textures when synthesizing novel views. For example, by only observing the side view of the shoes, it is impossible to determine whether there is one shoe or a pair. In other words, novel view synthesis and 3D object generation remain difficult to generate reliably when conditioned solely on a single-view image. Despite efforts to improve synthesis fidelity, current methods [ 16 , 17 , 18 , 19 ] still struggle with producing realistic 3D shapes from sparse or ambiguous inputs, particularly in complex and unconstrained real-world settings. This limitation highlights the need for more robust approaches capable of handling diverse, in-the-wild data distributions and capturing fine-grained 3D structure under limited observations. In this work, we aim to introduce flexible multi-view image conditioning to diffusion models, enabling more controllable novel view synthesis and 3D object generation for real-world objects. For instance, given front, back, and side views of an object provided by designers, the model can generate novel views while maintaining consistency with the input views. It also enables interactive 3D generation, allowing users to provide additional views if the generated 3D objects deviate from their intended design. However, this task poses two major challenges. First, integrating an arbitrary number of input views into a coherent signal that can guide the generation of novel views is non-trivial. Secondly, it is tricky to ensure compatibility across different network architectures, such as Zero-1-to-3 [ 1 ] , SyncDreamer [ 2 ] , and SV3D [ 3 ] . In this work, we introduce DreamComposer++, a scalable and flexible framework that extends existing view-aware diffusion models to accommodate an arbitrary number of multi-view inputs. DreamComposer++ consists of three key stages: target-aware 3D lifting, multi-view feature fusion, and target-view feature injection. (i) Target-Aware 3D Lifting encodes multi-view images into latent space and subsequently lifts these latent features to 3D tri-planes [ 20 ] . This tri-plane representation, enhanced with latent features, is both compact and efficient. The target-view-aware design enables the network to concentrate on constructing 3D features pertinent to the target view. (ii) Multi-View Feature Fusion renders and fuses the 3D features from different views into target-view 2D features using a novel composited volume rendering approach. (iii) Target-View Feature Injection integrates the latent features from the previous stage into the diffusion models using a ControlNet-like structure. This injection module utilizes the relative angle as a condition, allowing for adaptive gating of the multi-view inputs. DreamComposer++ can be seamlessly integrated into existing image and video diffusion models, such as Zero-1-to-3 [ 1 ] , SyncDreamer [ 2 ] , and SV3D [ 3 ] , enhancing their capabilities with multi-view conditioning, as illustrated in Figure 1 . Our contributions are summarized as follows: • We propose DreamComposer++, a flexible multi-view conditioning framework that enhances existing image and video diffusion models for scalable novel view synthesis. • We design three key modules: target-aware 3D lifting, multi-view feature fusion, and target-view feature injection. These modules underpin the scalability and flexibility of DreamComposer++. • Experiments demonstrate that our framework is compatible with existing state-of-the-art view-aware diffusion models, achieving high-fidelity novel view synthesis and controllable 3D object generation. • Benefiting from multi-view conditioning, our framework further supports a wide range of applications, including 3D object editing and character modeling. Compared with the preliminary conference version [ 21 ] , DreamComposer++ introduces several non-trivial improvements. The most notable improvement is the support for video diffusion model, which enables more dense and consistent multi-view generation compared to previous image diffusion-based models. To better adapt to video diffusion models, we introduce two key modules: View-aware Attention , which preserves the consistency of multi-view latent representations, and Shifted Window Cross Attention , which enhances model robustness. By empowering SV3D [ 3 ] with multi-view conditioning, we demonstrate further improvements in both generation quality and controllability. In addition, we conduct more ablation analysis and visualizations to highlight the effectiveness of DreamComposer++. Finally, DreamComposer++ achieves higher resolution and finer texture control, making it suitable for more complex applications than the previous version [ 21 ] . II Related Work Zero-shot Novel View Synthesis. Previous works [ 22 , 23 , 24 ] on novel view synthesis are generally trained on datasets with limited scenes or categories and