GOR-IS: 3D Gaussian Object Removal in the Intrinsic Space
Yonghao Zhao, Yupeng Gao, Jian Yang, Jin Xie, Beibei Wang · 2026
Read GOR-IS: 3D Gaussian Object Removal in the Intrinsic Space by Yonghao Zhao, Yupeng Gao, Jian Yang, Jin Xie, Beibei Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00498.txt.
Introduction
Yonghao Zhao and 4 co-authors remove objects from three-dimensional scenes without breaking the light. Reconstructed scenes, built from many photos by radiance fields or Gaussian splatting, often need an object deleted and the hidden region behind it filled in. Existing editors handle consistency fairly well but ignore global lighting, and they stumble on glossy, view-dependent surfaces, producing results that look physically wrong.
Their framework works in intrinsic space, splitting the scene into material and lighting components and modeling light transport explicitly, so shadows and reflections stay consistent after removal. A dedicated module then paints the missing region directly in those material and lighting terms, which handles shiny surfaces more reliably. On synthetic and real datasets the method improved perceptual similarity and signal quality over existing editors by clear margins, with code released publicly.
Catalog record
| CREATOR | Yonghao Zhao, Yupeng Gao, Jian Yang, Jin Xie, Beibei Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.00498
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GOR-IS: 3D Gaussian Object Removal in the Intrinsic Space Yonghao Zhao, Yupeng Gao, Jian Yang, Jin Xie, Beibei Wang Abstract Recent advances in Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have made it standard practice to reconstruct 3D scenes from multi-view images. Removing objects from such 3D representations is a fundamental editing task that requires complete and seamless inpainting of occluded regions, ensuring consistency in geometry and appearance. Although existing methods have made notable progress in improving inpainting consistency, they often neglect global lighting effects, leading to physically implausible results. Moreover, these methods struggle with view-dependent non-Lambertian surfaces, where appearance varies across viewpoints, leading to unreliable inpainting. In this paper, we present 3D G aussian O bject R emoval in the I ntrinsic S pace (GOR-IS), a novel framework for physically consistent and visually coherent 3D object removal. Our approach decomposes the scene into intrinsic components and explicitly models light transport to maintain global lighting effects consistency. Furthermore, we introduce an intrinsic-space inpainting module that operates directly in the material and lighting domains, effectively addressing the challenges posed by non-Lambertian surfaces. Extensive experiments on both synthetic and real-world datasets demonstrate that our framework substantially improves the physical consistency and visual coherence of object removal, outperforming existing methods by 13% in perceptual similarity (LPIPS) and 2dB in peak signal-to-noise ratio (PSNR). Code is publicly available at https://applezyh.github.io/GOR-IS-project-page/ † † * Corresponding author. † † 1 VCIP, College of Computer Science, Nankai University † † 2 School of Intelligence Science and Technology, Nanjing University 1 Introduction Reconstructing 3D scenes from multi-view images has become a standard practice, largely driven by advances in Neural Radiance Fields (NeRF) 29 and 3D Gaussian Splatting (3DGS) 17 . Removing objects from these scenes is a vital editing task, enabling the creation of diverse environments for applications in virtual reality and embodied intelligence. This task requires a geometrically complete and visually seamless inpainting of the regions previously occluded by the target object. However, the absence of native 3D inpainting models often forces a pipeline of performing 2D inpainting on individual views and then lifting the results into 3D, which frequently leads to multi-view inconsistency. Therefore, achieving cross-view geometry and appearance consistency is a central challenge in this task. Existing methods for object removal have made extensive efforts to improve consistency, typically leveraging depth guidance for geometric completion 30 ; 13 ; 25 ; 47 ; 50 ; 32 ; 40 and employing generative models to enhance appearance coherence 32 ; 40 ; 58 ; 50 ; 46 . While these approaches demonstrate impressive results in some scenes, they overlook the consistency of global lighting effects. As shown in the Fig. 1 , a typical failure case involves reflections on glossy surfaces. When an object is removed, its reflections should also be removed to maintain plausibility. Furthermore, these methods often rely on the strong assumption that the inpainted color is view-independent. Unfortunately, this assumption is frequently violated, particularly in scenes with non-Lambertian materials, where the radiance at a 3D point changes with the viewing angle, leading to obvious artifacts like blurring or ghosting. In this paper, we propose 3D G aussian O bject R emoval in the I ntrinsic S pace ( GOR-IS ), a novel framework for 3D object removal that ensures consistency by explicitly modeling light transport within the scene. Our key insight is to decompose the scene into its intrinsic properties (e.g., materials and lighting) and perform inpainting within this intrinsic space. This approach allows us to directly address global lighting effects. For instance, reflections cast by the target object on glossy surfaces can be easily identified and removed, significantly improving global lighting effects consistency. Since intrinsic material properties like albedo or roughness are inherently view-independent, our method bypasses the flawed view-independence assumption of prior work. By operating in this disentangled space, GOR-IS effectively enhances both geometric and appearance consistency, leading to more coherent and physically plausible object removal. Specifically, we extend 3D Gaussian splatting using physically-based rendering (PBR) materials, enabling material and lighting decoupling. Then we introduce a global illumination model to explicitly model the light transport in the scene, ensuring consistent lighting effects during object removal. Finally, to overcome the limitations of existing methods in non-Lambertian scenes, we propose an intrinsic-space inpainting module that operates in the material and lighting domains, enhancing the appearance consistency of scene inpainting. This module incorporates a material inpainting module that leverages the view-independent nature of material properties to inpaint non-Lambertian surfaces, as well as a lighting-aware masking mechanism, derived from explicit light-transport modeling, to detect and suppress reflection-induced blurry artifacts. Through extensive experiments on synthetic and real-world datasets, we demonstrate that our method achieves state-of-the-art (SOTA) results across quantitative as well as visual evaluations, and outperforms existing object removal approaches by 13% in perceptual similarity (LPIPS) 62 and 2dB in peak signal-to-noise ratio (PSNR). Overall, we are the first object removal method that explicitly considers the consistency of global lighting effects, and our main contributions include: • a novel framework (GOR-IS) that achieves more coherent and physically reasonable object removal through scene intrinsic space, • a material and lighting decoupling module, combined with explicit light transport modeling, improves global lighting effects consistency, and • an intrinsic-space inpainting module that enhances the appearance consistency of scene inpainting. Figure 1 : An example of consistent global lighting effects. When an object (red dashed box) is removed, its reflections (green dashed box) should also be removed to maintain plausibility. 2 Related Work 3D Object removal. Early 3D object removal approaches 49 ; 30 typically rely on manual annotations to generate object masks, followed by 2D inpainting to remove the target object from images. The inpainted results are then lifted into the NeRF representation for 3D object removal. Building upon these methods, subsequent works 57 ; 56 adopt the more advanced segmentation model 18 to improve mask accuracy and reduce manual effort. However, these approaches suffer from the inherent cross-view inconsistency of 2D inpainting, often producing unnatural blurring in the inpainted regions. To address this issue, several studies 23 ; 5 enhance the cross-view consistency of the 2D inpainting model to achieve a more coherent appearance. Reference-based methods such as GScream 47 and 3DGIC 13 inpaint both RGB and depth maps from one or a few reference views to preserve consistency across views. In addition, some approaches 25 ; 50 ; 40 ; 32 ; 58 ; 46 further leverage diffusion priors 12 ; 41 to jointly enhance geometry and appearance consistency. Nevertheless, most existing methods focus solely on object removal while neglecting global lighting effects—such as inter-reflections between the object and its surroundings—thereby limiting their physical realism and general applicability. Figure 2 : Overview of the GOR-IS framework. We use 3DGS with extended material properties as the basic 3D representation, combined with a global illumination model, to decompose the scene into its material and lighting components. This decomposition enables explicit light transport modeling, ensuring consistent global lighting effects. Furthermore, we introduce a specially designed module for glossy reflection modeling. Building upon this, we propose an intrinsic-space inpainting module to maintain the consistent appearance of scene inpainting. This module includes a material inpainting module that effectively restores non-Lambertian surfaces