ReCoGS: Real-time ReColoring for Gaussian Splatting scenes

Lorenzo Rutayisire Nicola Capodieci Fabio Pellacini University of Modena and Reggio Emilia · 2025

Read ReCoGS: Real-time ReColoring for Gaussian Splatting scenes by Lorenzo Rutayisire Nicola Capodieci Fabio Pellacini University of Modena and Reggio Emilia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18441.txt.

Introduction

Lorenzo Rutayisire, Nicola Capodieci, and Fabio Pellacini present ReCoGS, a tool for recolouring scenes built with Gaussian splatting. A fast method for recreating three-dimensional scenes that can be viewed from new angles in real time. Editing such scenes is an active area, but many tools lean on image diffusion models to invent training views, which brings inconsistency between views, coarse control, and heavy computing costs.

ReCoGS focuses narrowly on recolouring. Its pipeline lets a user select a region of a trained scene precisely and change its colour, and an interactive tool demonstrates the edit live. Code is shared in public. For artists and designers working with captured scenes, the tool offers direct control without rebuilding the scene or waiting on a diffusion pipeline.

Catalog record

CREATORLorenzo Rutayisire Nicola Capodieci Fabio Pellacini University of Modena and Reggio Emilia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-18441

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ReCoGS: Real-time ReColoring for Gaussian Splatting scenes Lorenzo Rutayisire Nicola Capodieci Fabio Pellacini University of Modena and Reggio Emilia Abstract Gaussian Splatting has emerged as a leading method for novel view synthesis, offering superior training efficiency and real-time inference compared to NeRF approaches, while still delivering high-quality reconstructions. Beyond view synthesis, this 3D representation has also been explored for editing tasks. Many existing methods leverage 2D diffusion models to generate multi-view datasets for training, but they often suffer from limitations such as view inconsistencies, lack of fine-grained control, and high computational demand. In this work, we focus specifically on the editing task of recoloring. We introduce a user-friendly pipeline that enables precise selection and recoloring of regions within a pre-trained Gaussian Splatting scene. To demonstrate the real-time performance of our method, we also present an interactive tool that allows users to experiment with the pipeline in practice. Code is available at https://github.com/loryruta/recogs . \WsPaper † † year: 2025 † † editors: M. Comino Trinidad, C. Mancinelli, D. Cabiddu, and D. Giorgi † † subject: \p@ConfName ( \p@ConfYear ) 1 Introduction 3D Gaussian Splatting (3DGS) [ KKLD23 ] represents the state-of-the-art method for novel view synthesis, outperforming Neural Radiance Field methods (NeRF) [ MST∗21 ] . Specifically, 3DGS offers faster training time, yet achieving a reconstruction quality comparable to the most recently presented NeRF methods, enabling real-time performance on customer-grade hardware. In NeRF-based approaches the outgoing radiance along a viewing direction is encoded in a Multi-Layer Perceptron (MLP) network, whereas Gaussian Splatting representation is more explicit and consists of a set of 3D gaussians positioned to match the training dataset. Despite being more explicit, the 3DGS representation remains challenging for editing tasks, primarily because 3D gaussians do not represent objects geometry. In the original work, gaussians are optimized with only a photometric loss, which allows them to be placed arbitrarily in space as long as the accumulated radiance matches the ground-truth. However, since most gaussians are initialized at precise 3D keypoints, and since — to account for view-consistency constraints — the optimizer is implicitly encouraged to move them towards actual object surfaces, it is still plausible to assume that the final set of gaussians approximates the scene’s geometry. This assumption is leveraged by editors like [ SPH∗25 ] , [ Pla25 ] , that propose a manual selection of the gaussians, thus enabling various editing operations, including tinting. In these tools, tinting is applied uniformly to the selected gaussians. However, due to the manual selection of the gaussians, and the view-independence of the applied tint, we observe that precise edit application is cumbersome and color bleeding in unwanted scene regions is not uncommon. In order to facilitate the selection of editing regions, previously published approaches are based on the idea of embedding segmentation labels within gaussians [ CFY∗23 ] , [ YDYK24 ] , [ CCZ∗23 ] , or language features for open-vocabulary 3D segmentation [ QLZ∗23 ] . However, these methods require costly re-training of the 3DGS scene, making them unsuitable for in-place editing. Additionally, the granularity of the selections is inherently limited by the chosen segmentation model (e.g. SAM [ KMR∗23 ] ). In this context, another worthy approach to editing is represented by mesh extraction. The idea is that if it is possible to extract a triangulated mesh from the 3DGS representation, then it is possible to edit it with any traditional 3D editing software. Notable meshing works are [ GL24b ] , [ GL24a ] , [ HYC∗24 ] , [ WBK24 ] , while [ XHL∗24 ] only focuses on texture extraction. Additionally, [ CG 25 ] is a plugin for Houdini providing connection between 3DGS and mesh representations. All the aforementioned approaches require re-training, which is generally more costly than the original 3DGS training as additional geometric constraints are introduced to align gaussians to surfaces [ GL24b ] , [ HYC∗24 ] , often degrading reconstruction quality. Modern editing research focuses on leveraging the vast knowledge of 2D diffusion models to generate or edit 3D content, given a textual or image prompt. A milestone work in this direction is [ HTE∗23 ] , in which the authors propose to perform iterative dataset updates (using [ BHE23 ] ) and to converge the 3D scene towards them. This method was tailored for NeRF, follow-up work proposed a similar technique for 3DGS scenes [ VH24 ] , [ CCZ∗23 ] . One major issue of the aforementioned methods is that dataset images are updated independently, leading the scene to be optimized with noisy gradients due to inconsistent views. Several recent works [ WYW∗24 ] , [ WBL∗24 ] , [ CLV24 ] , [ GPJ∗24 ] , [ LPS∗24 ] propose different solutions to mitigate view inconsistencies, but they generally fail at producing pixel-perfect results. This is usually due to the diffusion model operating in a low-resolution latent space which leads the final edited scene to have blurry regions or still showcase inconsistencies. Moreover, the textual prompt selection neglect fine-grain edits, although usually the target region can be constrained with a bounding box. Our recoloring pipeline, to the best of our knowledge, stands out from all the prior work. We propose a novel selection method that allows the user to paint a 2D selection/mask on a reference view at pixel resolution. We then propose a method for unprojecting such 2D selection to a 3D mask exploiting a pre-trained depth estimation model. Such 3D selection is then re-projected to training views onto which the edit is applied. The pre-trained Gaussian Splatting scene is fine-tuned towards the edited dataset in background, leveraging an optimizer implementation tailored to our problem. Considering the pipeline as a whole, our approach showcase great interactivity to fine-grain recolor anything in a Gaussian Splatting scene. We provide the code for a demonstrative editor implementing our method at [this URL]. 2 Related Work Most of the literature on Gaussian Splatting editing aims at leveraging 2D diffusion models to generate or edit 3D content (e.g. [ RBL∗21 ] , [ BHE23 ] ). Hence, recoloring becomes one of the possibly many editing operations available. Coupling Gaussian Splatting training with diffusion models carries a significant additional computation cost that makes these methods unsuitable for real-time editing. 2.1 Non Multi-View Consistent Diffusion A milestone work in 3D editing of NeRF is InstructNerf2Nerf [ HTE∗23 ] , where they propose to update training images using a text-prompted 2D diffusion model [ BHE23 ] , and alternate these updates with optimization iterations of the 3D scene. The adaptation of this work to 3DGS is Instruct-GS2GS [ VH24 ] . The drawback of these approaches is that dataset images are updated independently, which may lead the diffusion model to generate view-inconsistent outputs, hindering the optimization of the 3D scene. In TIP-Editor [ ZKC∗24 ] , authors leveraged a 2D diffusion model [ RBL∗21 ] which can be fed with text and image prompt, and a bounding box. The bounding box defines where the edit is placed and a localization loss is introduced to apply the edit in the target area. Although this method provides an innovative way of selecting the region of interest, hence providing additional granularity for the edits, it still

Read the complete text