GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors

Qilin Zhang, Jinyu Zhu, Olaf Wysocki, Benjamin Busam, Boris Jutzi, [0.5em] Technical University of Munich (TUM) · 2026

Read GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors by Qilin Zhang, Jinyu Zhu, Olaf Wysocki, Benjamin Busam, Boris Jutzi, [0.5em] Technical University of Munich (TUM) (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11401.txt.

Introduction

Zhang and four colleagues add CityGML urban meaning to Gaussian splatting scenes with GS4City. Two-pass raycasting of level-of-detail-3 models yields image-aligned masks. Parent-child relations validate and recover fine facade elements, and these geometry-grounded masks fuse with foundation-model predictions for consistent instances. Each Gaussian learns a compact identity encoding under 2D and 3D supervision. On TUM2TWIN and Gold Coast data, GS4City beats 2D-driven baselines by up to 15.8 IoU points on coarse building segmentation and 14.2 mean points on fine-grained semantics.

Catalog record

CREATORQilin Zhang, Jinyu Zhu, Olaf Wysocki, Benjamin Busam, Boris Jutzi, [0.5em] Technical University of Munich (TUM)
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35013
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11401

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors Qilin Zhang, Jinyu Zhu, Olaf Wysocki, Benjamin Busam, Boris Jutzi, [0.5em] Technical University of Munich (TUM) Abstract Recent semantic 3D Gaussian Splatting (3DGS) methods primarily rely on 2D foundation models, often yielding ambiguous boundaries and limited support for structured urban semantics. While city models such as CityGML encode hierarchically organized semantics together with building geometry, these labels cannot be directly mapped to Gaussian primitives. We present GS4City, a hierarchical semantic Gaussian Splatting method that incorporates city-model priors for urban scene understanding. GS4City derives reliable image-aligned masks from Level of Detail (LoD) 3 CityGML models via two-pass raycasting, explicitly using parent-child relations to validate and recover fine-grained facade elements. It then fuses these geometry-grounded masks with foundation-model predictions to establish scene-consistent instance correspondences, and learns a compact identity encoding for each Gaussian under joint 2D identity supervision and 3D spatial regularization. Experiments on the TUM2TWIN and Gold Coast datasets show that GS4City effectively incorporates structured building semantics into Gaussian scene representations, outperforming existing 2D-driven semantic 3DGS baselines, including LangSplat and Gaga, by up to 15.8 IoU points in coarse building segmentation and 14.2 mIoU points in fine-grained semantic segmentation. By bridging structured city models and photorealistic Gaussian scene representations, GS4City enables semantically queryable and structure-aware urban reconstruction. Code is available at https://github.com/Jinyzzz/GS4City . 1 Introduction Figure 1 : GS4City augments a photorealistic Gaussian scene with structured CityGML priors to produce hierarchical urban understanding. It enables appearance-robust segmentation, geometry-aligned boundaries, open-vocabulary querying, and city-model metadata integration within a unified 3DGS scene. 3D Gaussian Splatting (3DGS) 13 has quickly evolved from real-time novel-view synthesis to a practical representation for large-scale scene reconstruction and rendering 42 ; 21 ; 24 ; 37 ; 6 . Recent work has further extended Gaussian scene representations beyond appearance modeling toward semantic segmentation, grouping, and open-vocabulary querying 38 ; 25 ; 27 ; 32 ; 43 ; 29 . For urban environments, however, a key challenge remains: how to equip photorealistic Gaussian scenes with structured urban semantics that are queryable, geometrically grounded, and hierarchically organized. Most current semantic 3DGS methods still rely primarily on 2D foundation models 27 ; 32 ; 28 ; 16 ; 22 , which are effective for generic object discovery but less reliable when facade elements exhibit repeated patterns, reflective materials, cluttered foregrounds, or strong occlusions 33 . As a result, semantic boundaries often become ambiguous, and building-level part-whole structure is difficult to preserve consistently 41 . Yet buildings are the primary structured objects in urban environments, where urban understanding requires semantics that are both boundary-accurate and hierarchically organized across levels such as windows, doors, walls, roofs, and whole buildings 1 ; 3 ; 33 ; 30 . Structured 3D city models such as CityGML 9 ; 17 provide a complementary source of supervision. They encode explicit object identities, part-whole relations, and multi-level geometric detail that are not readily available from image-based supervision 2 . However, these semantics are attached to explicit polygonal surfaces, whereas 3DGS represents scenes with alpha-composited Gaussian primitives. This mismatch makes direct label transfer unreliable and limits the direct use of city-model semantics for Gaussian scene understanding. As illustrated in Figure 1 , GS4City addresses this problem by using city models as structured supervision in image space and then lifting the resulting identities into the Gaussian field. The method combines two-pass raycasting, multi-source mask fusion, and per-Gaussian identity learning to connect building entities, image masks, and Gaussian primitives within a scene-consistent semantic representation. This design preserves the building hierarchy while remaining compatible with open-vocabulary querying and interactive exploration of urban scenes. We evaluate GS4City on TUM2TWIN 36 and Gold Coast LoD3 dataset 15 . Across both datasets, GS4City improves coarse building segmentation and fine-grained semantic segmentation over existing 2D-driven semantic 3DGS baselines, including LangSplat 27 and Gaga 25 , showing that structured city-model priors can effectively extend Gaussian scenes from photorealistic rendering toward hierarchical urban understanding. Our main contributions are as follows: • We propose GS4City, a semantic Gaussian Splatting method that incorporates structured city-model priors for hierarchical urban understanding. • We introduce a robust supervision pipeline based on two-pass raycasting and multi-source mask fusion, which integrates structured city-model semantics into reliable image-aligned supervision and establishes scene-consistent instance correspondences. • We formulate semantic lifting through a compact Gaussian identity encoding with joint 2D supervision and 3D spatial regularization, enabling accurate coarse-level building segmentation, fine-grained semantic segmentation, and open-vocabulary querying in urban scenes. 2 Related Works This section reviews recent progress in semantic scene understanding for 3D Gaussian Splatting (3DGS) and the use of structured urban priors in city-scale modeling. Semantic Scene Understanding in 3DGS. Recent research has extended 3DGS 13 from photorealistic rendering to semantic scene understanding 26 ; 8 ; 18 , mainly along two directions: feature distillation and mask lifting 11 . One line of work distills pretrained 2D features (e.g., CLIP 28 ) into Gaussian representations 43 ; 27 ; 29 ; 32 . Although effective for open-vocabulary querying, these continuous feature fields make it difficult to extract explicit object identities and precise geometric boundaries. Another line of work focuses on mask lifting, where 2D segmentation masks are associated with Gaussian primitives for sharper object-level separation 38 ; 25 ; 4 ; 5 . However, both directions mainly recover category- or instance-level semantics from 2D observations. They do not explicitly model the hierarchical part-whole relations required in structured urban environments 3 ; 14 . This limitation is particularly restrictive for applications such as urban digital twins and city-scale asset management, where semantics are required to be accurately aligned with geometry and consistently linked across structural levels (e.g., windows, walls, and whole buildings) 2 . Urban Priors for Gaussian Scene Understanding. In parallel, recent urban-scale 3DGS methods have shown that additional priors are essential for robust large-scale reconstruction 21 ; 24 ; 20 . Most of these works focus on geometric or scene-structure stabilization using LiDAR, depth, or proxy geometry, rather than structured semantic understanding 37 ; 31 ; 7 ; 12 ; 40 . Standardized 3D city models, such as CityGML and CityJSON 9 ; 17 , offer a compelling alternative by providing hierarchical representations of the built environment. While lower-level models (LoD1 and LoD2) are widely available worldwide 35 and serve as valuable, lightweight geometric priors 39 ; 23 , fine-grained urban scene understanding demands LoD3 models. Unlike standard meshes, LoD3 city models provide georeferenced, boundary representations (B-Rep) that explicitly encode detailed facade elements (e.g., windows and doors) alongside precise object-to-object relationships 30 . However, these models are typically represented as simplified polygonal meshes, lacking the photorealistic appearance and complex visual details naturally captured by 3DGS. Consequently, integrating the structural and semantic richness of LoD3 models with the visual fidelity of 3DGS is crucial for comprehensive urban digital twins. Yet, this integration poses a significant challenge due to a fundamental difference in data representation: city models attach semantics to discrete, explicit polygonal surfaces, whereas 3DGS relies on alpha-composited Gaussian primitives without explicit surface ownership. Bridging this gap combines the rigorous, object-level semantics of LoD3 city models with the photorealistic rendering and open-vocabulary querying capabilities of recent semantic Gaussian representations 10 ; 19 ; 32 ; 26 . 3

Read the complete text