Pixal3D: Pixel-Aligned 3D Generation from Images Journal: TOG Conference: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers; July 19–23, 2026; Los Angeles, CA, USA Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers (SIGGRAPH Conference Papers ’26), July 19–23, 2026, Los Angeles, CA, USA DOI: 10.1145/3799902.3811175 ISBN: 979-8-4007-2554-8/2026/07 CCS: Computing methodologies Mesh geometry models CCS: Computing methodologies Volumetric models
Dong-Yang Li, Wang Zhao, Yuxin Chen, Wenbo Hu, Meng-Hao Guo, Fang-Lue Zhang, Ying Shan, Shi-Min Hu · 2026
Read Pixal3D: Pixel-Aligned 3D Generation from Images Journal: TOG Conference: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers; July 19–23, 2026; Los Angeles, CA, USA Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers (SIGGRAPH Conference Papers ’26), July 19–23, 2026, Los Angeles, CA, USA DOI: 10.1145/3799902.3811175 ISBN: 979-8-4007-2554-8/2026/07 CCS: Computing methodologies Mesh geometry models CCS: Computing methodologies Volumetric models by Dong-Yang Li, Wang Zhao, Yuxin Chen, Wenbo Hu, Meng-Hao Guo, Fang-Lue Zhang, Ying Shan, Shi-Min Hu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10922.txt.
Introduction
Dong-Yang Li and seven colleagues present Pixal3D, an image-to-3D generator that prizes pixel-level faithfulness to the input photo. Current 3D generators build shapes in a canonical pose and inject image cues through attention, leaving the link between pixels and 3D points vague.
Pixal3D instead generates the shape directly aligned to the input view, borrowing the correspondence discipline of 3D reconstruction. The pixel-aligned design sharpens fidelity, the measure of how closely the generated asset matches the photo, which has lagged behind geometry quality and realism. Creators and graphics teams get 3D assets that look like the picture they came from, not just a plausible cousin.
Catalog record
| CREATOR | Dong-Yang Li, Wang Zhao, Yuxin Chen, Wenbo Hu, Meng-Hao Guo, Fang-Lue Zhang, Ying Shan, Shi-Min Hu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37076 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10922
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Pixal3D: Pixel-Aligned 3D Generation from Images Journal: TOG Conference: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers; July 19–23, 2026; Los Angeles, CA, USA Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers (SIGGRAPH Conference Papers ’26), July 19–23, 2026, Los Angeles, CA, USA DOI: 10.1145/3799902.3811175 ISBN: 979-8-4007-2554-8/2026/07 CCS: Computing methodologies Mesh geometry models CCS: Computing methodologies Volumetric models Dong-Yang Li, Wang Zhao, Yuxin Chen, Wenbo Hu, Meng-Hao Guo, Fang-Lue Zhang, Ying Shan, Shi-Min Hu Abstract . Recent advances in 3D generative models have rapidly improved image-to-3D synthesis quality, enabling higher-resolution geometry and more realistic appearance. Yet fidelity, which measures pixel-level faithfulness of the generated 3D asset to the input image, still remains a central bottleneck. We argue this stems from an implicit 2D-3D correspondence issue: most 3D-native generators synthesize shape in canonical space and inject image cues via attention, leaving pixel-to-3D associations ambiguous. To tackle this issue, we draw inspiration from 3D reconstruction and propose Pixal3D, a pixel-aligned 3D generation paradigm for high-fidelity 3D asset creation from images. Instead of generating in a canonical pose, Pixal3D directly generates 3D in a pixel-aligned way, consistent with the input view. To enable this, we introduce a pixel back-projection conditioning scheme that explicitly lifts multi-scale image features into a 3D feature volume, establishing direct pixel-to-3D correspondence without ambiguity. We show that Pixal3D is not only scalable and capable of producing high-quality 3D assets, but also substantially improves fidelity, approaching the fidelity level of reconstruction. Furthermore, Pixal3D naturally extends to multi-view generation by aggregating back-projected feature volumes across views. Finally, we show pixel-aligned generation benefits scene synthesis, and present a modular pipeline that produces high-fidelity, object-separated 3D scenes from images. Pixal3D for the first time demonstrates 3D-native pixel-aligned generation at scale, and provides a new inspiring way towards high-fidelity 3D generation of object or scene from single or multi-view images. Project page: https://ldyang694.github.io/projects/pixal3d/ Figure 1. Pixel-aligned meshes generated by Pixal3D. The foreground displays our results with their corresponding input images in the background. Our back-projection conditioning scheme (bottom-left) explicitly lifts 2D image features into a 3D volume to establish robust 2D-3D correspondence for generation. † † cc-license: by 1. Introduction Automatic creation of high-quality 3D assets from images is a central goal in computer graphics, with profound implications for gaming, AR/VR, and digital manufacturing. Recent advances in 3D generative modeling have achieved remarkable milestones, producing assets with increasingly detailed geometry ( 54 ; 55 ) , realistic appearance ( 66 ; 19 ) and controllable parts ( 28 ; 61 ) , pushing 3D generation towards truly ready-to-use assets. However, a critical bottleneck still limits the broader adoption of current image-to-3D methods: fidelity. Here, fidelity measures how faithfully the generated 3D asset matches the input image. Most existing methods condition on an image but often produce only approximately similar shapes, with noticeable misalignment and loss of fine details. This falls short of user expectations: given an image, one typically wants the generated 3D model to (1) precisely reconstruct the visible surface, and (2) plausibly complete the unobserved regions to form a coherent and usable 3D asset. Achieving high fidelity, in addition to high quality, is a critical next step towards making image-to-3D generation genuinely useful in practice. Interestingly, this fidelity issue is far less prominent in 3D reconstruction, a complementary field whose primary goal is to recover visible 3D structure from 2D observations, whether from multiple views or a single view. We attribute this difference to the explicit 2D-3D correspondence establishment. Correspondence is the fundamentals of reconstruction: multi-view geometry ( 7 ) is built upon pixel correspondences and triangulation, and single-view reconstruction pipelines predict depth ( 60 ; 27 ) , normals ( 6 ; 63 ) , or point maps ( 48 ; 43 ) in a pixel-aligned manner, establishing a direct, clear, one-to-one correspondence between 2D image pixels and recovered 3D. In contrast, existing 3D-native generative methods ( 56 ; 14 ; 54 ) synthesize shapes in a canonical pose, and rely on cross-attention to inject image information into 3D latents. This makes 2D-3D correspondence implicit and nontrivial: cross-attention must effectively "search" for where each image feature should influence the 3D representation, introducing ambiguity and confusion for local details, repetitive parts or among multiple input views, which ultimately manifests as reduced fidelity. To resolve this fidelity issue, we propose Pixal3D, a new Pixel-Aligned 3D generation paradigm that marries the geometric rigor of reconstruction with the creative power of generative models. Unlike previous canonical space generation, Pixal3D directly generates 3D in a pixel-aligned pose consistent with the input image. To make this possible, we introduce a back-projection conditioning scheme that establishes explicit 2D-3D correspondence for injecting pixel information into 3D, replacing the commonly used cross-attention mechanism. Concretely, we back-project image features into 3D volume: every 3D voxel along that ray is assigned the corresponding pixel feature, yielding a pixel-aligned lifted 3D feature volume. This volume is then added to the 3D noise volume as a conditioning signal. We further incorporate multi-scale image features to preserve and propagate fine-grained details. Through these careful designs, we demonstrate that this pixel-aligned 3D generation paradigm is not only feasible and scalable to produce high-quality 3D models, but also significantly improves 3D fidelity over current 3D generation, achieving near reconstruction-level fidelity. Moreover, Pixal3D naturally unifies single-view and multi-view settings under the same formulation. We extend Pixal3D to multi-view 3D generation by back-projecting each view into a pixel-aligned feature volume and aggregating them via averaging, leading to a simple and reliable multi-view generation approach. Finally, we show that this pixel-aligned paradigm also benefits 3D scene generation: we propose a modular pipeline that composes object-level generations into high-fidelity, object-separated 3D scenes, in a spirit similar to recent SAM3D ( 37 ) scene construction. Pixal3D is essentially a 3D generative reconstruction paradigm that represents and formalizes the synergy between reconstruction and generation. It inherits the best of both worlds: the visible surfaces are tightly constrained by the input image through explicit correspondence like reconstruction, while the invisible regions are plausibly completed by learned priors of generative model conditioned on what is observed. Pixal3D provides a simple yet effective paradigm for generating faithful 3D objects and scenes from both single-view and multi-view inputs. Figure 1 shows representative examples. Importantly, Pixal3D is orthogonal to specific 3D generative backbones, and can therefore benefit from ongoing advances in geometry representations, part modeling, texturing, materials, etc., making it a scalable foundation for high-fidelity 3D generation. Our contributions are summarized as follows: (1) We introduce Pixal3D, a pixel-aligned 3D generation paradigm, and demonstrate that pixel-aligned generation is feasible at scale while substantially improving image-to-3D fidelity. (2) We propose a ray back-projection conditioning mechanism that replaces cross-attention with explicit 2D-3D correspondence, enabling direct pixel-to-3D feature lifting and more faithful preservation of image details. (3) We extend Pixal3D from single-view to multi-view generation via simple and effective multi-view feature-volume aggregation. (4) We propose a modular 3D scene generation pipeline based on Pixal3D that produces high-fidelity, object-separated 3D scenes. 2. Related Works 2.1. 3D Generation 3D generation has advanced rapidly ( 46 ) , from distilling 2D diffusion priors into 3D ( 36 ; 51 ) to 3D-native pipelines that learn 3D