Normal-NeRF: Ambiguity-Robust Normal Estimation for Highly Reflective Scenes

Ji Shi, Xianghua Ying, Ruohao Guo, Bowei Xing, Wenzhen Yue · 2025

Read Normal-NeRF: Ambiguity-Robust Normal Estimation for Highly Reflective Scenes by Ji Shi, Xianghua Ying, Ruohao Guo, Bowei Xing, Wenzhen Yue (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-09460.txt.

Introduction

Ji Shi and co-authors rebuild shiny scenes in 3D without flattening their detail in this 2025 paper. Neural radiance fields render scenes from photos, but mirror-like surfaces fool them about the true shape underneath. Older fixes add shape hints that can over-smooth fine structure.

Surface direction, called the normal, is the key to placing reflections right. The team estimates normals from how light fades through the scene, a signal that stays sound even when shape is ambiguous. A second module keeps normals smooth on faces yet crisp at edges. Paired with a reflection-aware shading model, the system beat leading methods on shiny and intricate test scenes.

Catalog record

CREATORJi Shi, Xianghua Ying, Ruohao Guo, Bowei Xing, Wenzhen Yue
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-09460

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Normal-NeRF: Ambiguity-Robust Normal Estimation for Highly Reflective Scenes Ji Shi, Xianghua Ying, Ruohao Guo, Bowei Xing, Wenzhen Yue Abstract Neural Radiance Fields (NeRF) often struggle with reconstructing and rendering highly reflective scenes. Recent advancements have developed various reflection-aware appearance models to enhance NeRF’s capability to render specular reflections. However, the robust reconstruction of highly reflective scenes is still hindered by the inherent shape ambiguity on specular surfaces. Existing methods typically rely on additional geometry priors to regularize the shape prediction, but this can lead to oversmoothed geometry in complex scenes. Observing the critical role of surface normals in parameterizing reflections, we introduce a transmittance-gradient-based normal estimation technique that remains robust even under ambiguous shape conditions. Furthermore, we propose a dual activated densities module that effectively bridges the gap between smooth surface normals and sharp object boundaries. Combined with a reflection-aware appearance model, our proposed method achieves robust reconstruction and high-fidelity rendering of scenes featuring both highly specular reflections and intricate geometric structures. Extensive experiments demonstrate that our method outperforms existing state-of-the-art methods on various datasets. State Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology, Peking University [email protected], [email protected] Code — https://github.com/sjj118/Normal-NeRF Figure 1: Illustration of our transmittance gradient compared to conventional density gradient for normal estimation. We plot the density \sigma(t) and the transmittance T(t) along a ray passing through a semi-transparent surface (top), and the distribution of the estimated normal vectors (bottom). (a) Since the derivatives of density approach zero near the surface, the directions of density gradients in nearby area can change rapidly. (b) In contrast, the derivatives of transmittance are large near the surface, thereby producing consistent normal estimates that align well with the true surface normals. Introduction Neural Radiance Fields (NeRF) ( Mildenhall et al. 2021 ) has been extensively studied for its powerful 3D scene reconstruction and rendering capabilities. NeRF utilizes multilayer perceptrons (MLPs) to encode a 3D scene into continuous fields of volume density and view-dependent radiance. Despite NeRF’s proficiency in capturing fine geometric structures and smoothly varying view-dependent appearance, it often fails to accurately represent highly specular reflections, as the high-frequency view-dependent appearance prevents NeRF from eliminating the shape-radiance ambiguity ( Zhang et al. 2020 ) . Recent studies attempt to enhance NeRF’s capability in rendering reflective scenes by incorporating reflection-aware appearance models. Ref-NeRF ( Verbin et al. 2022 ) reparameterizes the view-dependent radiance as a function of the reflection direction about the surface normal, rather than the viewing direction itself. Later works further employ advanced reflection-aware appearance models, such as Microfacet Reflectance Model ( Mai et al. 2023 ) and Whitted-Style Ray Tracing ( Zeng et al. 2023 ) . While these advanced reflection models are capable of representing complex reflections, achieving accurate reflection modeling remains a challenge due to the inherent shape ambiguity on highly reflective surfaces. We observe that surface normals are critical in nearly all reflection-aware appearance models. Previous works typically derive surface normals based on the negative normalized gradient of the density field, which we term the “density gradient”. However, this approach becomes unreliable on highly specular surfaces, as NeRF tends to fake specular reflections with foggy artifacts embedded behind the surfaces ( Verbin et al. 2022 ) . This phenomenon prevents the density values from monotonicly increasing towards the object’s interior, resulting inconsistent gradient directions, as illustrated in Figure 1 (a). In this paper, we propose an ambiguity-robust normal estimation pipeline to enhance NeRF’s capability in reconstructing and rendering highly reflective scenes. To address the unreliability in conventional normal estimation approaches, we introduce the concept of “transmittance gradient”, which can provide accurate normal estimates even under conditions of ambiguous shape predictions. Instead of calculating the gradient of the density, we compute the gradient of the transmittance, which is naturally monotonic along any specific ray, as illustrated in Figure 1 (b). Furthermore, we observe that reflection-aware appearance models generally prefer locally smooth surface normals, whereas NeRF tends to reconstruct a steep density field to achieve sharper object boundaries. To address this conflict, we propose a dual activated densities module that accommodates the distinct requirements of both the surface normals and the density field. Additionally, we design a stop-gradient warmup strategy for the predicted normal loss to prevent it from impeding the optimization of the density field. By integrating a reflection-aware appearance model, our model achieves robust reconstruction and high-fidelity rendering in highly reflective scenes. Our key contributions can be summarized as follows: • To the best of our knowledge, we are the first to identify and analyze the inherent limitations of the density gradient traditionally used for NeRF normal estimation. • We introduce the Transmittance Gradient to address inconsistencies and irregularities in normal estimation within NeRF. • We propose a Dual Activated Densities module to effectively bridge the gap between smooth surface normals and sharp object boundaries. Related Work Neural Radiance Fields. Neural Radiance Fields (NeRF) ( Mildenhall et al. 2021 ) is a successful pipeline for novel view synthetic of complex scenes by optimizing the volumetric underlying function using a sparse set of input views. Later studies have improved the rendering quality ( Barron et al. 2021 ; Barron et al. 2022 ; Barron et al. 2023 ) and accelerated the rendering speed ( Müller et al. 2022 ; Sun, Sun, and Chen 2022 ; Chen et al. 2022 ) of NeRF using various techniques. NeRF has also inspired many subsequent works that extend its application, including few-shot rendering ( Chen et al. 2021 ; Yu et al. 2021 ; Jain, Tancik, and Abbeel 2021 ; Yang, Pavone, and Wang 2023 ) , dynamic scene rendering ( Park et al. 2021a ; Park et al. 2021b ; Li et al. 2022 ) , and 3D generation ( Schwarz et al. 2020 ; Jain et al. 2022 ; Poole et al. 2023 ) . Radiance Fields with Reflections. Although the view-dependent radiance function of NeRF enables the modeling of non-Lambertian effects, it often encounters difficulties in accurately capturing the appearance of objects with high-frequency reflections due to the shape-radiance ambiguity ( Zhang et al. 2020 ) . NeRFReN ( Guo et al. 2022 ) and MS-NeRF ( Yin et al. 2023 ) treat specular reflections on planar mirrors as virtual images behind the surface, and model them with separate radiance fields to avoid inconsistency between front and back views of the mirror. Neural Catacaustics ( Kopanas et al. 2022 ) further leverages a neural warp field, enabling the modeling of reflections on non-planar surfaces. However, due to the lack of physically-based modeling of interactions between light and surfaces, these methods face challenges in accurately representing reflections on complex surfaces. Recent works incorporate NeRF with reflection-aware appearance models. Ref-NeRF ( Verbin et al. 2022 ) conditions the view-dependent radiance on the reflected view direction instead of the camera view direction to make the radiance MLP easier to interpolate. Neural Microfacet Fields ( Mai et al. 2023 ) employs a microfacet reflectance model for physically-accurate reflection rendering. Mirror-NeRF ( Zeng et al. 2023 ) models multi-bounce reflections with Whitted-Style Ray Tracing. ENVIDR ( Liang et al. 2023 ) leverages a pretrained neural renderer to enable high-fidelity scene relighting. SpecNeRF ( Ma et al. 2024 ) proposes

Read the complete text