Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields 2062 CCS: Computing methodologies Rendering CCS: Computing methodologies Reconstruction Conference: SIGGRAPH Asia 2025 Conference Papers; December 15–18, 2025; Hong Kong, Hong Kong SIGGRAPH Asia 2025 Conference Papers (SA Conference Papers ’25), December 15–18, 2025, Hong Kong, Hong Kong DOI: 10.1145/3757377.3763971 ISBN: 979-8-4007-2137-3/2025/12

Yohan Poirier-Ginter, Jeffrey Hu, Jean-François Lalonde, George Drettakis · 2026

Read Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields 2062 CCS: Computing methodologies Rendering CCS: Computing methodologies Reconstruction Conference: SIGGRAPH Asia 2025 Conference Papers; December 15–18, 2025; Hong Kong, Hong Kong SIGGRAPH Asia 2025 Conference Papers (SA Conference Papers ’25), December 15–18, 2025, Hong Kong, Hong Kong DOI: 10.1145/3757377.3763971 ISBN: 979-8-4007-2137-3/2025/12 by Yohan Poirier-Ginter, Jeffrey Hu, Jean-François Lalonde, George Drettakis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30861.txt.

Introduction

Yohan Poirier-Ginter, Jeffrey Hu, Jean-François Lalonde and George Drettakis make reflections in captured 3D scenes editable. Radiance fields such as 3D Gaussian Splatting render scenes from photos in real time and show shiny reflections well, but they fake them with phantom geometry hidden behind the mirror. That trick blocks editing: you cannot move the reflected object or roughen the surface. The team's proof of concept reconstructs the reflector and the reflected objects in one representation, so reflections become physical.

Learning-based tools first split each photo into diffuse and specular layers plus geometry and material buffers. A diffuse scene is optimised, and path tracing then generates true specular reflections, aided by a fast ray tracer for Gaussian primitives that handles multiple bounces. Results use ground-truth buffers on synthetic scenes, with early tests on real ones.

Catalog record

CREATORYohan Poirier-Ginter, Jeffrey Hu, Jean-François Lalonde, George Drettakis
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30861

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields 2062 CCS: Computing methodologies Rendering CCS: Computing methodologies Reconstruction Conference: SIGGRAPH Asia 2025 Conference Papers; December 15–18, 2025; Hong Kong, Hong Kong SIGGRAPH Asia 2025 Conference Papers (SA Conference Papers ’25), December 15–18, 2025, Hong Kong, Hong Kong DOI: 10.1145/3757377.3763971 ISBN: 979-8-4007-2137-3/2025/12 Yohan Poirier-Ginter, Jeffrey Hu, Jean-François Lalonde, George Drettakis Abstract . Radiance fields such as 3D Gaussian Splatting allow real-time rendering of scenes captured from photos. They also reconstruct most specular reflections with high visual quality, but typically model them with “fake” reflected geometry, using primitives behind the reflector. Our goal is to correctly reconstruct the reflector and the reflected objects such as to make specular reflections editable; we present a proof of concept which exploits promising learning-based methods to extract diffuse and specular buffers from photos, as well as geometry and BRDF buffers. Our method builds on three key components. First, by using diffuse/specular buffers of input training views, we optimize a diffuse version of the scene and use path tracing to efficiently generate physically-based specular reflections. Second, we present a specialized training method that allows this process to converge. Finally, we present a fast ray tracing algorithm for 3D Gaussian primitives that enables efficient multi-bounce reflections. Our method reconstructs reflectors and reflected objects—including those not seen in the input images—in a unique scene representation. Our solution allows real-time, consistent editing of captured scenes with specular reflections, including multi-bounce effects, changing roughness etc. We mainly show results using ground truth buffers from synthetic scenes, and also preliminary results in real scenes with currently imperfect learning-based buffers. Code and data are available at: https://repo-sam.inria.fr/nerphys/editable-gaussian-reflections/ . Figure 1. Our Gaussian-based radiance-field method allows interactive editing of path traced reflections, with consistent updates. We develop our proof-of-concept method in synthetic scenes with known ground truth inputs and show its extension to a real scene with network predicted inputs. † † cc-license: by Figure 2. Our method takes as input several buffers for every training view (left), and reconstructs a unique gaussian-based scene where reflections in novel views are computed with cached diffuse (middle) and physics-based path tracing (right). This figure shows ground truth input buffers from a synthetic scene, from top to bottom: separated targets (diffuse and non-diffuse), BRDF parameters (base reflectance and roughness), and geometry (depth and normals). 1. Introduction Neural Radiance Fields (NeRFs) ( 37 ) and 3D Gaussian Splatting (3DGS) ( 19 ) allow high-quality novel-view synthesis with only posed photographs as input. One of their major strengths is that they render view-dependent effects such as low- to medium-frequency specular reflections with high visual quality. However, the representation of reflections is entangled with the geometry: they are baked into a directional radiance component, and often are actually “fake mirror geometry” situated behind the reflective surfaces ( 63 ; 36 ) . This precludes consistent editing of scenes with reflections, i.e., where reflections will be correctly updated when changing geometry or materials. We propose the first method that allows interactive reflection-consistent editing of radiance fields ( Fig. 1 ) by using distinct processes to reconstruct each of the diffuse and reflective components of the scene. Our method also allows reconstruction of reflected objects unseen in the input images. In the original versions of NeRF and 3DGS, a directional radiance representation is modeled by a Multi-Layer Perceptron (MLP) and Spherical Harmonics (SH) respectively. However, these low-frequency representations cannot represent reflections accurately, and thus the optimization favors the creation of “fake mirror geometry” (see video). Several improvements exist for better reflection in NeRFs, e.g., by changing parameterizations or encodings ( 47 ; 34 ) . Similarly, several methods attempt to improve reflections in 3DGS, often for distant lighting ( 16 ; 59 ) . These methods still suffer from the entangled reflection representation. Recently, NeRFcasting ( 48 ) and EnvGS ( 55 ) use ray tracing but require a separate scene representation for reflections to allow stable optimization. Representing reflectors and reflected objects as separate scenes prevents consistent editing of the radiance field. To address these problems, our key intuition is to have a unique scene with two distinct—but concurrent—optimizations: one for the diffuse, and one for the specular components of the scene, supervised on diffuse and specular input buffers respectively. In this paper, we use the term specular to mean all non-diffuse reflections, from rough glossy to pure mirror. The diffuse buffer captures diffuse global illumination, including shadows and interreflections, while the specular buffer captures mirror and glossy reflections. We reconstruct the diffuse component of the scene using ray traced Gaussian primitives and compute specular reflections with path tracing while treating the diffuse component similarly to an irradiance cache ( 52 ) . Our reflections support surfaces of different roughness, and multi-bounce effects computed in a physically-based manner, i.e., exclusively with path tracing. They also allow interactive scene editing with physically-correct reflections. Note that transparency is more complicated, since buffers would have to deal with two rays—reflected and refracted; we thus leave it as future work. First, we present a proof-of-concept of our method on synthetic scenes, where rendered images are used in the place of photos, and ground truth values are used for all buffers (diffuse, specular, materials etc.). This solution shows we can duplicate a reflective object in a scene, and all real, captured objects will be correctly reflected in the duplicated object interactively, even with multiple bounces (see Fig. 1 ). Second, we present an editable inverse rendering pipeline on synthetic and real images using buffers predicted by neural networks, albeit with lower quality for novel-view synthesis. In particular, recent intrinsic image decomposition methods can extract the required buffers from photographs ( 42 ; 62 ) , by training on synthetic data where light transport is separated into diffuse and reflective (or non-diffuse) components at the first bounce. That being said, while these methods are improving at outstanding rates, the current quality of predicted buffers from existing methods (e.g., ( 18 ; 62 ; 30 ) ) is only partially sufficient for our goals. We thus focus mainly on our proof-of-concept using synthetic data with perfect buffers to develop our method, and present preliminary results for scenes processed with the best possible existing learning-based methods at the time of writing. Figure 3. We showcase reconstruction of objects viewed indirectly by recovering the cover of a book only visible indirectly through a mirror in the training views (left). From these views alone, our method accurately recovers the cover (right), unlike standard methods like 3DGS (middle). Note that in these results, our method uses ground truth input buffers and additional Gaussians were initialized in a bounding box around the book (see supp.). Book cover by Matt Pharr, Wenzel Jakob, and Greg Humphreys, licensed under CC BY-NC-ND 4.0. In short, our method leverages intrinsic decomposition to separately reconstruct the diffuse and reflected components of a scene (see Fig. 2 ) to make editable specular reflections possible (see video). We optimize a unique representation of the scene, i.e., both the reflectors and reflected objects are represented with the same Gaussian primitives, by carefully designing separate losses and a training schedule. This unique representation makes it possible to reconstruct objects only viewed indirectly through

Read the complete text