A 3DGS-Diffusion Self-Supervised Framework for Normal Estimation from a Single Image

Yanxing Liang, Yinghui Wang *, Jinlong Yang, Wei Li · 2025

Read A 3DGS-Diffusion Self-Supervised Framework for Normal Estimation from a Single Image by Yanxing Liang, Yinghui Wang *, Jinlong Yang, Wei Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05950.txt.

Introduction

Yanxing Liang and co-authors present a framework for estimating surface orientation from one image in a paper from two thousand twenty-five. Knowing which way each surface point faces helps rebuild three-dimensional scenes. Their method, called CLONE, learns without true orientation labels.

The system builds a scene from soft three-dimensional blobs and renders it back to an image, so errors guide learning. A refinement network restores fine detail that the smooth blobs erase. A gating step then blends the two estimates, keeping detail where it fits the geometry. All parts train together under a single image matching goal with rules for geometric consistency.

Catalog record

CREATORYanxing Liang, Yinghui Wang *, Jinlong Yang, Wei Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-05950

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A 3DGS-Diffusion Self-Supervised Framework for Normal Estimation from a Single Image Yanxing Liang, Yinghui Wang *, Jinlong Yang, Wei Li Abstract The lack of spatial dimensional information remains a core challenge in normal estimation from a single image. Although recent diffusion-based methods have demonstrated significant potential in 2D-to-3D implicit mapping, they rely on data-driven statistical priors and miss the explicit modeling of light-surface interaction, leading to multi-view normal direction conflicts. Moreover, the discrete sampling mechanism of diffusion models causes gradient discontinuity in differentiable rendering reconstruction modules, preventing 3D geometric errors from being backpropagated to the normal generation network, thereby forcing existing methods to depend on dense normal annotations. This paper proposes SINGAD, a novel S elf-supervised framework from a single I mage for N ormal estimation via 3D GA ussian splatting guided D iffusion. By integrating physics-driven light-interaction modeling and a differentiable rendering-based reprojection strategy, our framework directly converts 3D geometric errors into normal optimization signals, solving the challenges of multi-view geometric inconsistency and data dependency. Specifically, the framework constructs a light-interaction-driven 3D Gaussian Splatting (3DGS) reparameterization model to generate multi-scale geometric features consistent with light transport principles, ensuring multi-view normal consistency. Additionally, a cross-domain feature fusion module is designed within a conditional diffusion model, embedding geometric priors to constrain normal generation while maintaining accurate geometric error propagation. Furthermore, a differentiable 3D reprojection loss strategy is introduced for self-supervised optimization that minimizes geometric error between the reconstructed and input image, eliminating dependence on annotated normal datasets. Quantitative evaluations on the Google Scanned Objects dataset demonstrate that our method outperforms state-of-the-art approaches across multiple metrics. This work provides a pioneering self-supervised paradigm for normal estimation from a single image, advancing the process from data-driven learning to physics-aware modeling. 1 Introduction 3D surface normal estimation from a single image, as a fundamental task in 3D scene understanding and reconstruction [ 1 , 2 ] , aims to infer geometric structures of object surfaces. The persistent challenge of this task directly correlates with inherent spatial information loss, a critical limitation overshadowing existing computer vision advancements. Early studies [ 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 ] mainly use Convolutional Neural Networks (CNNs) or Vision Transformers to extract multi-scale features with geometric consistency constraints. However, these non-generative approaches could only estimate viewpoint-specific normals. The growing prominence of diffusion models [ 12 , 13 , 14 , 15 ] with implicit prior modeling and progressive denoising mechanisms has successfully achieved multi-view normal generation from a single image. Nevertheless, as diffusion models operate as data-driven 2D image generators lacking explicit geometric representation, they often estimate normals with shape ambiguity in concave/convex regions across views. Figure 1: SINGAD network pipeline. The input image is processed by an MLP to estimate geometric attributes (Geo Attr.) and texture attributes (Tex Attr.) for 3DGS. Multi-scale geometric features are extracted via a Feature Pyramid Network (FPN) and fed into the 3DGS module to generate preliminary geometric descriptors. A conditional diffusion model, guided by a cross-domain feature fusion layer (incorporating attention-based weighting and gated fusion), iteratively denoises the input by combining geometric and texture features. The differentiable rendering pipeline and 3D reprojection loss jointly optimize network parameters. The predicted normal map is generated through this closed-loop optimization. Arrow directions indicate data flow and dependencies between modules. Recent state-of-the-art methods [ 2 , 16 , 17 , 18 ] attempt to integrate diffusion models with 3D reconstruction techniques through multi-view joint probability distributions or Score Distillation Sampling (SDS) strategies. Others [ 19 , 20 , 21 , 22 , 23 , 24 ] jointly learn depth-normal correlations by computing gradient differences from depth maps. While these methods use Neural Radiance Fields (NeRF) [ 25 ] or 3D Gaussian Splatting (3DGS) [ 26 ] for final normal rendering, the discrete sampling mechanism in diffusion models creates discontinuous gradients. This problem prevents effective backpropagation of 3D geometric errors through differentiable rendering modules, isolating 3D reconstruction to a post-processing role rather than participating in normal optimization. Zamir et al. [ 1 ] established that surface normals serve as critical intermediate representations for 3D reconstruction, where estimation accuracy directly impacts geometric fidelity. This motivates us to reexamine the existing approaches: Since the ultimate objective of normal estimation is to enable precise 3D reconstruction, why not exploit the inherent multi-view geometric consistency of 3D models? We could establish closed-loop feedback through "normal generation to reprojection error optimization" with 3D reprojection loss backpropagation, rather than relying on multi-view fusion or vision-language fusion. Notably, the rapid growth of "image-3D model" datasets [ 2 , 16 ] now exceeds "image-normal" datasets in scale, providing practical feasibility for our approach. Building on these insights, this paper proposes SINGAD (see Figure 1 ), a novel S elf-supervised framework from a single I mage for N ormal estimation via 3D GA ussian splatting guided D iffusion. Our method integrates the differentiable rendering properties of 3DGS with the prior constraints of conditional diffusion models, establishing an end-to-end self-supervised learning framework via three core components: 1) A light-interaction-driven 3DGS parameter estimation model via an MLP-based network by the light-interaction modeling with the Gabor kernel, ensuring compliance with the optical reflection principle and diffusion model discretization. Its output features, multi-scale geometric descriptors and preliminary normals, are generated through a Feature Pyramid Network (FPN) augmented with Principal Component Analysis (PCA), functioning as geometric priors (see Section 3.1 ). 2) A cross-domain feature-guided conditional diffusion model incorporates a novel feature fusion layer that injects these priors into the denoising process, aligning geometric and RGB domains to refine local normals while preserving multi-view consistency (see Section 3.2 ). 3) A normal reprojection optimization strategy reconstructs a 3D model from predicted normals via rasterization, computes reconstruction errors through a joint reprojection loss, and performs self-supervised backpropagation to optimize the 3DGS parameter estimation and diffusion modules (see Section 3.3 ). Our contributions are summarized below: • A light-interaction-driven 3DGS reparameterization method that explicitly links diffuse irradiance with normal orientation quantized ellipsoid parameters through the Gabor kernel, maintaining physical light transport principles, improving multi-scale geometric feature accuracy. • A cross-domain feature-guided diffusion model that injects multi-scale geometric features to constrain geometry-aware normal generation while preserving gradient propagation. • A annotation-free self-supervised strategy using differentiable rasterization and 3D reprojection losses to jointly optimize 3DGS and diffusion networks, achieving supervised performance without normal annotations. 2 Related Works CNNs and Transformers. Early CNN-based approaches [ 3 , 4 , 5 , 6 , 7 , 8 ] pioneered multi-scale feature fusion for joint normal estimation and scene understanding. Wang et al. [ 3 ] developed a multi-task framework integrating normal prediction with room layout estimation through hierarchical feature aggregation. Wei et al. [ 8 ] introduced virtual normals by randomly sampling 3D point triplets to construct virtual planes, enhancing robustness against noise. However, these discriminative models suffered from limited generalization due to their heavy reliance on normal annotations. The advent of Vision Transformers [ 9 , 10 , 11 ] brought improved long-range dependency modeling in complex scenes. PlaneTR [ 10 ] combined line segment features with Transformer attention for simultaneous normal and 3D plane recovery, while Bae et al. [ 11 ] incorporated uncertainty-aware learning

Read the complete text