UniView: Enhancing Novel View Synthesis From A Single Image By Unifying Reference Features DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 1551-6865-XXXX-X/XXXX/XX CCS: Information systems Mul
Haowang Cui, Rui Chen, Tao Luo, Rui Li, Jiaze Wang · 2025
Read UniView: Enhancing Novel View Synthesis From A Single Image By Unifying Reference Features DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 1551-6865-XXXX-X/XXXX/XX CCS: Information systems Mul by Haowang Cui, Rui Chen, Tao Luo, Rui Li, Jiaze Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-04932.txt.
Introduction
Creating a new view of an object from one photograph is ill-posed because hidden areas could look many ways. Haowang Cui, lead author with four co-authors, propose UniView in a 2025 computer vision paper. Their model borrows prior information from reference images of a similar object during synthesis.
A retrieval system, assisted by a multimodal language model, selects suitable reference images. A plug-and-play adapter then generates reference features for the target views. A decoupled triple attention mechanism aligns those features with the original image details. Experiments on challenging datasets show clear gains over leading methods.
Catalog record
| CREATOR | Haowang Cui, Rui Chen, Tao Luo, Rui Li, Jiaze Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-04932
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
UniView: Enhancing Novel View Synthesis From A Single Image By Unifying Reference Features DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 1551-6865-XXXX-X/XXXX/XX CCS: Information systems Multimedia content creation Haowang Cui, Rui Chen, Tao Luo, Rui Li, Jiaze Wang Abstract . The task of synthesizing novel views from a single image is highly ill-posed due to multiple explanations for unobserved areas. Most current methods tend to generate unseen regions from ambiguity priors and interpolation near input views, which often lead to severe distortions. To address this limitation, we propose a novel model dubbed as UniView, which can leverage reference images from a similar object to provide strong prior information during view synthesis. More specifically, we construct a retrieval and augmentation system and employ a multimodal large language model (MLLM) to assist in selecting reference images that meet our requirements. Additionally, a plug-and-play adapter module with multi-level isolation layers is introduced to dynamically generate reference features for the target views. Moreover, in order to preserve the details of an original input image, we design a decoupled triple attention mechanism, which can effectively align and integrate multi-branch features into the synthesis process. Extensive experiments have demonstrated that our UniView significantly improves novel view synthesis performance and outperforms state-of-the-art methods on the challenging datasets. 1. Introduction Novel view synthesis (NVS) has long been a fundamental task in the field of 3D generation ( Liu et al. 2024b ; Tang et al. 2024b ; Melas-Kyriazi et al. 2024 ) and 3D reconstruction ( Furukawa et al. 2015a ; Schonberger and Frahm 2016 ; Yao et al. 2018a ) . Models for novel view synthesis using multiple images of the same object from different viewpoints have achieved impressive results ( Kerbl et al. 2023 ; Mildenhall et al. 2021 ) . However, this presents a challenge in terms of input data. In practice, it is often difficult for users to capture multiple images of the target object from different perspectives, especially when the input image is generated by text-to-image models ( Ramesh et al. 2021 ; Rombach et al. 2022 ) . In such cases, synthesizing high-fidelity and high-quality novel views from a single RGB image becomes particularly crucial. Consequently, single-image novel view synthesis has emerged as a prominent area of research in computer vision. Figure 1. Visualization results between Uniview and baseline. Compared with models that solely utilize a single image as input, our UniView can significantly enhance single-image novel view synthesis performance under challenging input viewpoints. This enhancement is achieved by incorporating complementary views from similar reference objects, particularly for regions that are entirely occluded in the original input image. Recent advances in generative models, particularly diffusion model-based ( Ho et al. 2020 ; Sohl-Dickstein et al. 2015 ; Song et al. 2020 ) novel view synthesis frameworks, have been widely adopted. These models conceptualize novel view synthesis as an image-to-image generation task ( Chan et al. 2023a ; Watson et al. 2022 ) , achieving both high fidelity and strong generalization capabilities. As a pioneering work, Zero123 ( Liu et al. 2023 ) generalized single-image novel view synthesis models from specific object categories to arbitrary categories, significantly enhancing their generalization capability. Specifically, Zero123++ fine-tuned a pre-trained Stable Diffusion ( Rombach et al. 2022 ) model which already containing substantial 3D priors – using 2D images rendered from 3D object data across different viewpoints. Zero123 ( Liu et al. 2023 ) posits that injecting camera transformation information into the image generation process enables effective extraction of these inherent 3D priors for novel view synthesis. Subsequent improvements ( Shi et al. 2023 ; Liu et al. 2024a ; Lin et al. 2024 ) building upon Zero123 ( Liu et al. 2023 ) have enhanced both the generative quality and spatial consistency in single-image novel view synthesis. However, synthesizing novel views of an object from a single input image, especially when the input perspective is highly challenging, is an extremely constrained and exceedingly difficult task. For regions of the object that are completely invisible in the input view, the model must rely on learned priors to hallucinate their appearance. This often results in significant distortions, such as a shoe being generated with two toe sections instead of one, as illustrated in Figure 1 . Some works have proposed to incorporate additional control conditions into single-image novel view synthesis models in order to address the issue of poor reconstruction in invisible regions. Specifically, TOSS ( Shi et al. 2024 ) introduced the use of a set of text prompts as control conditions to provide global information, which could enhance the quality of back-side generation in single-image novel view synthesis. However, text is inherently limited in its capacity to accurately depict object features, and relying solely on text often results in insufficiently precise control. "Good artists copy, great artists steal." – Pablo Picasso’s adage reveals an essential truth in artistic creation: faithful recreation stems from strategic appropriation. Inspired by Picasso’s philosophy, we propose that: "Good models generate, great models transplant." Acquiring multi-view images of a target object has been challenging in practical scenarios, while obtaining images of arbitrary objects from the same category is relatively straightforward. For instance, when only possessing a single image of a specific shoe, we can utilize complementary viewpoint images which are from different shoes within the same category as reference for novel view synthesis. Following this principle, we present UniView - a novel framework which is capable of transferring complementary visual information from similar reference objects (e.g., the back view of shoe B) to guide novel view synthesis of target objects (e.g., shoe A). As demonstrated in Figure 1 , image-based references from complementary regions of analogous objects serve as a powerful and precise control signal, significantly improving the synthesis quality of unobserved regions in the original input view. Figure 2. The overall architecture of UniView. It retrieves the reference image from pre-constructed database and utilizes a Meta-Adapter to convert the reference image into an adaptive dynamic reference signal, which is then injected into a pre-trained multi-view diffusion model, thereby enhancing the quality of novel view synthesis. However, the inherent misalignment dilemma between reference images and target objects makes it challenging to inject reference information into the pre-trained multi-view diffusion model meanwhile avoiding misleading guidance for novel view synthesis. In some scenarios, reference image acquisition would be a hindrance. To address these issues, as depicted in Figure 2 , our method has designed a complete system which leverages a multimodal LLM to retrieve the most appropriate reference image form the pre-constructed database. Also, UniView introduces an adaptive architecture that dynamically modulates the intensity of the reference signal, which ensures precise and effective control of reference images over the pre-trained single-image novel view synthesis model. The main contributions of this work are summarized as follows: We design a Dynamic Reference Retrieval System which dynamically retrieve the most appropriate reference image from the pre-constructed database. We propose a Meta-Adapter Module that generates adaptive dynamic reference signals from input reference images to enhance novel view synthesis quality for target objects, particularly for completely unobserved viewpoints in the input. We develop a Decoupled