FreeSplat++: Generalizable 3D Gaussian Splatting for Efficient Indoor Scene Reconstruction

Yunsong Wang, Tianxin Huang, Hanlin Chen, Gim Hee Lee, [email protected] · 2025

Read FreeSplat++: Generalizable 3D Gaussian Splatting for Efficient Indoor Scene Reconstruction by Yunsong Wang, Tianxin Huang, Hanlin Chen, Gim Hee Lee, [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22986.txt.

Introduction

Yunsong Wang and co-authors stretch a fast blob-scene craft from small nooks to whole rooms in one feed pass. Blob scenes paint a room as a cloud of soft 3D blobs, swift to show but slow to build by old fits. Feed craft skips the fit, yet past feeds took few shots and small spans. Their 2025 build, FreeSplat++, first pools shots cross views at low cost, so long shot chains can run.

A three-view fuse then folds twin blobs down as views stack. A weight trim cuts float blobs, a depth fuse in plain sight. A depth-tied touch-up ends the build. Whole-room feeds can now run in one pass at fair grain and pace.

Catalog record

CREATORYunsong Wang, Tianxin Huang, Hanlin Chen, Gim Hee Lee, [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-22986

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FreeSplat++: Generalizable 3D Gaussian Splatting for Efficient Indoor Scene Reconstruction Yunsong Wang, Tianxin Huang, Hanlin Chen, Gim Hee Lee, [email protected] Abstract Recently, the integration of the efficient feed-forward scheme into 3D Gaussian Splatting (3DGS) has been actively explored. However, most existing methods focus on sparse view reconstruction of small regions and cannot produce eligible whole-scene reconstruction results in terms of either quality or efficiency. In this paper, we propose FreeSplat++, which focuses on extending the generalizable 3DGS to become an alternative approach to large-scale indoor whole-scene reconstruction, which has the potential of significantly accelerating the reconstruction speed and improving the geometric accuracy. To facilitate whole-scene reconstruction, we initially propose the Low-cost Cross-View Aggregation framework to efficiently process extremely long input sequences. Subsequently, we introduce a carefully designed pixel-wise triplet fusion method to incrementally aggregate the overlapping 3D Gaussian primitives from multiple views, adaptively reducing their redundancy. Furthermore, given the fused 3DGS primitives with accumulated weights after the fusion step, we propose a weighted floater removal strategy that can effectively reduce floaters, which serves as an explicit depth fusion approach that is tailored for generalizable 3DGS methods and becomes crucial in whole-scene reconstruction. After the feed-forward reconstruction of 3DGS primitives, we investigate a depth-regularized per-scene fine-tuning process. Leveraging the dense, multi-view consistent depth maps obtained during the feed-forward prediction phase for an extra constraint, we refine the entire scene’s 3DGS primitive to enhance rendering quality while preserving geometric accuracy. Extensive experiments confirm that our FreeSplat++ significantly outperforms existing generalizable 3DGS methods, especially in whole scene reconstructions. Compared to conventional per-scene optimized 3DGS approaches, our method with depth-regularized per-scene fine-tuning demonstrates substantial improvements in reconstruction accuracy and a notable reduction in training time. Fig. 1: Results on whole scene reconstruction. FreeSplat++ rft is the per-scene fine-tuned results. Our model excels in efficiently reconstructing geometrically accurate 3D Gaussian primitives. Furthermore, FreeSplat++ shows superior view consistency, e.g. when rendered from Bird-Eye’s View, demonstrating the significance of generalizable 3DGS for whole scene reconstruction. I Introduction In recent years, differentiable neural rendering has attracted considerable interest for applications in novel view synthesis [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ] , mesh reconstruction [ 9 , 10 , 11 , 12 ] , scene understanding [ 13 , 14 , 15 ] , and embodied AI [ 16 , 17 ] . Among these techniques, Neural Radiance Fields (NeRF) [ 1 , 4 ] is a milestone for learning the geometry and appearance of 3D scenes through an implicit radiance field. However, NeRF is computationally expensive in both training and rendering phases. An alternative approach, 3D Gaussian Splatting (3DGS) [ 6 , 7 ] , represents scenes explicitly using a set of anisotropic 3D Gaussian primitives and optimizes them via differentiable rendering. This method has gained increasing popularity for its ability to produce high-quality renderings in real time. However, despite these advancements, vanilla 3DGS still requires tens of minutes for per-scene optimization, limiting its broader applicability. Fig. 2: Visualization of the floaters challenge in generalizable 3DGS. The noise of predicted depth maps ( red regions) may lead to severe floaters in whole scene reconstruction. To this end, there have been multiple attempts [ 18 , 19 , 20 , 21 ] to transfer the vanilla 3DGS to a feed-forward manner. However, most of them solely focus on sparse view interpolation within a considerable narrow view range, making them less effective than vanilla 3DGS for explicitly reconstructing large-scale 3D scenes. This limitation undermines the full potential of 3DGS as an explicit representation. More recently, FreeSplat [ 22 ] has explored extending generalizable 3DGS for reconstructing large-scale 3D scenes. It introduces an efficient backbone capable of processing long-sequence inputs and a Gaussian fusion mechanism that incrementally fuses global Gaussian triplets with local triplets from input views, effectively reducing redundancy in overlapping regions. FreeSplat surpasses previous baselines in geometric accuracy, rendering quality, and efficiency, particularly in long-sequence reconstruction. Despite these advancements, FreeSplat still faces challenges in reconstructing whole indoor scenes because : when given input images spanning the whole 3D scene, its fusion module cannot remove the floaters largely deviating from the surfaces. In these cases, the number of Gaussian primitives becomes enormous and undermines real-time rendering, and the rendering quality of the overlapping regions is also degraded. A visualization of the floaters challenge of FreeSplat is shown in Figure 2 , in which the model struggles to deal with the noisy depth map from some specific input views, and leaves severe floaters in the whole-scene reconstruction. The main reason of such challenge is the lack of operations to check the multi-view depth consistency and remove the inconsistent regions. However, existing multi-view depth consistency checking methods, e.g. in [ 23 ] , produces holes in the final reconstruction and cannot be directly integrated to feed-forward 3DGS. Therefore, we introduce FreeSplat++, an enhanced generalizable 3DGS framework specifically designed for feed-forward whole-scene reconstruction that achieves high rendering quality, reconstruction accuracy, and efficiency. First, we propose a low-cost CNN-based framework to efficiently aggregate multi-view images through cost volumes, enabling efficient management of whole-scene information. Secondly, drawing inspiration from traditional TSDF Fusion techniques [ 24 , 25 ] , we carefully design an improved Pixel-wise Triplet Fusion (PTF) mechanism to fuse pixel-wise Gaussian primitives under current view, with the closest global Gaussian primitives within a broader range, in order to further reduce the redundancy of 3DGS primitives and remove the floaters during fusion. Additionally, inspired by the voxel-based averaging of TSDF values in TSDF Fusion — which effectively mitigates noise in depth maps — we introduce a Weighted Floater Removal (WFR) strategy. This strategy incrementally projects the global 3DGS onto the input views, and adjusts the Gaussian opacities according to predicted/rendered depth maps and the Gaussian weights accumulated in the PTF process. Subsequently, to further improve the rendering quality while maintaining the geometric accuracy, we introduce a depth-regularized per-scene fine-tuning step to efficiently optimize the feed-forward predicted 3DGS. According to our experiments, our improved feed-forward pipeline can effectively remove floaters and efficiently reconstruct complex large 3D scenes, and our per-scene fine-tuned results show significant advantages over vanilla 3DGS in terms of both reconstruction accuracy and efficiency. Overall, our contributions can be summarized as: 1. This work represents a pioneering effort in generalizable 3D Gaussian Splatting (3DGS) for large scale reconstruction. To the best of our knowledge, it is the first framework to effectively apply generalizable 3DGS for indoor whole-scene reconstruction. 2. To address the challenge of feed-forward whole scene reconstruction, we improve the fusion mechanism in FreeSplat [ 22 ] , and propose an effective weighted floater removal strategy that can be seamlessly integrated into the framework to further remove the floaters. 3. We introduce a depth-regularized per-scene fine-tuning step after feed-forward 3DGS prediction. Leveraging the efficient and geometrically accurate initialization of 3DGS from our feed-forward pipeline, our whole-scene reconstruction framework provides a competitive alternative to vanilla 3DGS, achieving a significant improvement on reconstruction accuracy within considerably shorter training time. II Related Work Novel View Synthesis. Novel View Synthesis aims to generate images from unseen views that are consistent with the given multi-view images. Traditional attempts in novel view synthesis mainly employed voxel grids

Read the complete text