ULSR-GS: Urban Large-scale Surface Reconstruction Gaussian Splatting with Multi-View Geometric Consistency

Zhuoxiao Li †, Shanliang Yao†, Taoyu Wu, Yong Yue, Wufan Zhao, Rongjun Qin, Ángel F. García-Fernández, Andrew Levers, Jason Ralph, Xiaohui Zhu cor1 · 2024

Read ULSR-GS: Urban Large-scale Surface Reconstruction Gaussian Splatting with Multi-View Geometric Consistency by Zhuoxiao Li †, Shanliang Yao†, Taoyu Wu, Yong Yue, Wufan Zhao, Rongjun Qin, Ángel F. García-Fernández, Andrew Levers, Jason Ralph, Xiaohui Zhu cor1 (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01402.txt.

Introduction

Zhuoxiao Li and co-authors present ULSR-GS, a system that rebuilds city surfaces at huge scale from aerial photos. Gaussian splatting renders scenes fast and pulls out fine surfaces for small areas. Stretched over whole city districts, though, past methods lose accuracy and detail.

Their fix splits the scene photo by photo, picking the best training views for each sub-region. During training, a densification step guided by multi-view shape agreement sharpens surface detail. Tests on large aerial photogrammetry benchmarks put ULSR-GS ahead of rival splatting methods. Gains show most in tangled urban zones with tall, mixed structures. City modelers, map makers, and disaster planners all need such faithful large-scale surfaces, and this work pushes that craft forward.

Catalog record

CREATORZhuoxiao Li †, Shanliang Yao†, Taoyu Wu, Yong Yue, Wufan Zhao, Rongjun Qin, Ángel F. García-Fernández, Andrew Levers, Jason Ralph, Xiaohui Zhu cor1
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01402

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ULSR-GS: Urban Large-scale Surface Reconstruction Gaussian Splatting with Multi-View Geometric Consistency Zhuoxiao Li †, Shanliang Yao†, Taoyu Wu, Yong Yue, Wufan Zhao, Rongjun Qin, Ángel F. García-Fernández, Andrew Levers, Jason Ralph, Xiaohui Zhu cor1 Abstract Recent advances in 2D Gaussian Splatting (2DGS) have demonstrated compelling rendering efficiency and mesh extraction capabilities. However, its application to large-scale aerial photogrammetry, especially using oblique UAV imagery, remains limited due to three primary challenges: (1) suboptimal image selection in scene partitioning strategies failing to scale effectively; (2) densification pipelines that rely primarily on single-view constraints resulting in over-smoothed reconstructions and loss of fine geometric detail; and (3) the absence of multi-view geometric consistency constraints leading to surface artifacts and inconsistencies. To address these limitations, we propose ULSR-GS, a novel method tailored for high-resolution surface reconstruction in urban-scale environments. Firstly, we propose a point-to-photo partitioning strategy that segments the scene based on the sparse SfM point cloud and assigns only the most relevant images to each sub-region, which resolves key scalability bottlenecks. Secondly, we propose a multi-view guided densification strategy that enforces adaptive geometric consistency across views, overcoming the limitations of single-view-based densifications. Lastly, we introduce consistency-aware loss functions that explicitly regulate depth and normal alignment across views, significantly enhancing surface fidelity. Extensive experiments on large-scale aerial benchmark datasets demonstrate that ULSR-GS consistently outperforms existing single- and multi-GPU Gaussian Splatting methods. Furthermore, compared to MVS pipelines, our approach achieves comparable or superior geometric quality while being substantially more time-efficient, making it a practical solution for scalable 3D modeling in digital twin and urban mapping applications. Project page: https://ulsrgs.github.io . keywords Surface Reconstruction ,Gaussian Splatting ,Large-Scale Scenes ,Aerial Photogrammetry ,Urban Scene Reconstruction † † corresponding: Corresponding author: [email protected] † † corresponding: †Equal Contribution 1 Introduction High-fidelity 3D surface reconstruction at urban scale remains a longstanding challenge in photogrammetry and computer graphics, with critical applications in urban planning, simulation, and digital twins [ 51 , 42 , 12 ] . Traditionally, urban-scale surface reconstruction from aerial oblique imagery has relied heavily on Multi-View Stereo (MVS) techniques [ 44 , 24 , 56 , 57 , 3 , 59 , 71 , 41 , 32 , 60 , 14 , 31 , 70 , 28 ] , which reconstruct dense point clouds from image correspondences and subsequently extract meshes through surface reconstruction algorithms, such as Poisson Surface Reconstruction [ 17 , 18 ] or Delaunay triangulation-based methods [ 23 ] . While effective, these traditional pipelines often encounter computational inefficiencies when scaling to urban-scale datasets comprising thousands of high-resolution images. Recent advancements in novel view synthesis (NVS) have led to from Neural Radiance Fields (NeRFs) [ 58 , 69 , 4 , 1 , 2 ] to the emergence of 3D Gaussian Splatting (3DGS) [ 19 ] , a breakthrough approach enabling fast photorealistic rendering quality with unprecedented efficiency by directly optimizing sparse 3D Gaussian primitives. In addition to achieving compelling visual realism, recent studies have demonstrated that 3DGS and its variants can serve as effective tools for extracting geometry-rich meshes [ 16 , 5 , 27 ] . Scaling Gaussian Splatting (GS) to city-scale surface reconstruction tasks is both an engineering and algorithmic challenge. Issue (1): The vanilla GS training pipeline [ 19 , 16 , 64 , 66 ] does not inherently support multi-GPU parallel processing. For instance, even high-performance GPUs like the RTX 4090 (24GB VRAM) can manage only approximately 8 million primitives, limiting applications to relatively small datasets such as the Mip-NeRF 360 dataset [ 2 ] . Consequently, current large-scale GS approaches predominantly rely on a divide-and-conquer strategy by partitioning extensive scenes into smaller sub-scenes based on camera projections or clustering of camera poses obtained from Structure-from-Motion (SfM), subsequently distributing these partitions across multiple GPUs [ 29 , 20 , 9 , 33 , 34 ] . While effective for controlled, uniform camera distributions such as five-directional aerial surveys [ 35 , 54 ] , these image-based methods struggle with close-range oblique photogrammetry scenarios, where irregular and complex camera trajectories result in uneven or inadequate training data distribution. Issue (2): Existing approaches necessitate merging sub-scenes before mesh extraction, a process requiring substantial computational resources. For example, methods employing TSDF fusion with a 0.2-meter voxel size consume over 300GB RAM for reconstructing merely 1,000 images, making scalability to city-scale scenes with tens of thousands of images impractical. Additionally, extracting meshes separately introduces further complications, such as determining optimal sub-scene boundaries, often leading to unnecessary computational overhead and fragmented meshes. Issue (3): Another significant challenge in GS-based large-scale reconstruction is the densification problem. GS explicitly represents scenes using discrete Gaussian primitives, meaning the richer and denser the primitives, the more accurately high-frequency geometric details can be captured [ 64 , 10 , 27 , 13 ] . However, GS often struggles with accurately reconstructing low-frequency or textureless regions, resulting in incomplete or overly smooth surfaces [ 16 , 13 , 5 ] . Issue (4): Existing GS-based methods predominantly utilize single-view geometric consistency constraints during training [ 64 , 10 , 27 , 13 ] , which fail to leverage comprehensive multi-view information. This single-view approach can cause inconsistencies and inaccuracies, such as floaters or misaligned primitives, particularly in complex urban scenarios. To address these critical issues, we propose ULSR-GS. Our key insight is combining detail-preserving optimization within each divided sub-region and enforcing multi-view consistency during training, which makes it possible for 2DGS to independently reconstruct each tile with high geometric fidelity. These sub-regions can then be seamlessly integrated into a globally consistent urban-scale surface without additional fusion or post-processing steps. Unlike other methods that partition scenes solely by camera positions [ 33 , 67 , 29 , 34 ] , we propose a point-to-photo partitioning strategy driven by the spatial distribution of sparse SfM points and their associated views. For each candidate sub-region, we meticulously select an optimal subset of images for each SfM point, ensuring comprehensive and relevant coverage. We then introduce a multi-view constrained densification approach within the optimization process for each partitioned sub-region. This strategy explicitly enforces geometric consistency across multiple views by penalizing discrepancies in depth and normal orientations of corresponding points across different images. Consequently, our multi-view densification preserves essential fine details, which are preserved when sub-regions are optimized independently. Collectively, the innovative integration of our point-to-photo partitioning and multi-view densification strategies positions ULSR-GS as a highly effective and scalable solution for accurate and detailed large-scale urban surface reconstructions. Our contributions are summarized as follows: • We present ULSR-GS, a novel method specifically designed to overcome the limitations of existing GS-based works in large-scale surface reconstruction tasks. • A partitioning strategy that supports extracting the geometry of each trained GS sub-region separately, without the need for merging and then extracting. • An densification method based on multi-view depth aggregation, which can improve rendering and geometry reconstruction quality. • Two efficient multi-view consistency constraints for generating detailed and accurate reconstructions in large-scale urban environments. 2 Related work 2.1 Large-Scale Novel View Synthesis Recent developments in novel view synthesis [ 1 , 7 , 69 , 39 ] include several innovative approaches tailored for large-scale environments [ 53 , 46 , 38 , 47 ] . BungeeNeRF first addresses the challenge

Read the complete text