RoofSeg: An edge-aware transformer-based network for end-to-end roof plane segmentation Journal: ISPRS Journal of Photogrammetry and Remote Sensing

Siyuan You, Guozheng Xu, Pengwei Zhou, Qiwen Jin, Jian Yao, Li Li · 2025

Read RoofSeg: An edge-aware transformer-based network for end-to-end roof plane segmentation Journal: ISPRS Journal of Photogrammetry and Remote Sensing by Siyuan You, Guozheng Xu, Pengwei Zhou, Qiwen Jin, Jian Yao, Li Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19003.txt.

Introduction

Siyuan You and five co-authors describe RoofSeg in a 2025 paper on mapping roofs from the air. Aircraft scanners record clouds of points over cities, and software must split each roof into its flat planes. That split is a key step in building detailed three-dimensional city models.

Older methods often needed hand-built rules or extra steps after the main model ran. RoofSeg works end to end in one pass, using a transformer network that predicts each roof plane directly. A special edge-aware module sharpens the borders where two planes meet, which is where errors tend to appear. The design also uses the flat geometry of roof planes to guide training. Tests on standard data show cleaner and more exact roof maps.

Catalog record

CREATORSiyuan You, Guozheng Xu, Pengwei Zhou, Qiwen Jin, Jian Yao, Li Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-19003

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RoofSeg: An edge-aware transformer-based network for end-to-end roof plane segmentation Journal: ISPRS Journal of Photogrammetry and Remote Sensing Siyuan You, Guozheng Xu, Pengwei Zhou, Qiwen Jin, Jian Yao, Li Li Abstract Roof plane segmentation is one of the key procedures for reconstructing three-dimensional (3D) building models at levels of detail (LoD) 2 and 3 from airborne light detection and ranging (LiDAR) point clouds. The majority of current approaches for roof plane segmentation rely on the manually designed or learned features followed by some specifically designed geometric clustering strategies. Because the learned features are more powerful than the manually designed features, the deep learning-based approaches usually perform better than the traditional approaches. However, the current deep learning-based approaches have three unsolved problems. The first is that most of them are not truly end-to-end, the plane segmentation results may be not optimal. The second is that the point feature discriminability near the edges is relatively low, leading to inaccurate planar edges. The third is that the planar geometric characteristics are not sufficiently considered to constrain the network training. To solve these issues, a novel edge-aware transformer-based network, named RoofSeg, is developed for segmenting roof planes from LiDAR point clouds in a truly end-to-end manner. In the RoofSeg, we leverage a transformer encoder-decoder-based framework to hierarchically predict the plane instance masks with the use of a set of learnable plane queries. To further improve the segmentation accuracy of edge regions, we also design an Edge-Aware Mask Module (EAMM) that sufficiently incorporates planar geometric prior of edges to enhance its discriminability for plane instance mask refinement. In addition, we propose an adaptive weighting strategy in the mask loss to reduce the influence of misclassified points, and also propose a new plane geometric loss to constrain the network training. Extensive experiments demonstrate that the proposed RoofSeg can effectively segment roof planes with accurate edges, minimal misclassified points, and high geometric fidelity, while significantly outperforming the current competitive approaches. We will release the source code of this work at https://github.com/Li-Li-Whu/RoofSeg . 1 Introduction Airborne light detection and ranging (LiDAR) can rapidly acquire large-scale three-dimensional (3D) data with high geometric precision, namely point clouds, which have been commonly employed in 3D building reconstruction ( Zhang et al. 2018 ; Huang et al. 2022 ; Liu et al. 2024 ; Sun et al. 2024 ) at levels of detail (LoD) 2 and 3. Given that the roof structure of a urban building is generally composed of geometrically varied planar patches, roof plane segmentation is a key procedure in 3D building reconstruction. Accurate segmentation of roof planes can significantly improve the final 3D reconstruction quality. However, despite many efforts in roof plane segmentation studies ( Khaloo and Lattanzi 2017 ; Wang et al. 2019 ; Liu et al. 2023 ; Li et al. 2024 ) , extracting accurate roof plane segments from the input point clouds remains challenging for existing 3D reconstruction systems ( Yu and Lafarge 2022 ) . The existing traditional approaches for roof plane segmentation are predominantly based on point feature grouping mechanism. Guided by some manually designed features (e.g., normal vectors or certain geometric distance), they utilize several grouping techniques, such as region growing ( Khaloo and Lattanzi 2017 ; Wu et al. 2019 ) , RANdom SAmple Consensus (RANSAC) ( Xu et al. 2015 ; Canaz Sevgen and Karsli 2020 ) or clustering ( Miao et al. 2023 ; Chen et al. 2024 ) , to iteratively group coplanar points with similar features. However, the mannally designed features may lack robustness and discriminativity. To solve this problem, some deep learning-based approaches ( Zhang and Fan 2022 ; Li et al. 2024 ; Zeng et al. 2025 ) are proposed to automatically learn more powerful and representative point features through deep networks. These approaches have demonstrated superior performance compared to traditional approaches. Nevertheless, there are still three unsolved issues in current deep learning-based approaches. The first issue is that most of them can not achieve truly end-to-end roof plane segmentation. They still rely on some geometric voting and clustering techniques ( Zhang and Fan 2022 ; Li et al. 2024 ) , or other post-processing steps (e.g., edge refinement) ( Zeng et al. 2025 ) to generate the final roof segments. This leads to two main limitations: (1) the inherent difficulty in hyperparameter tuning, and (2) the suboptimal segmentation results caused by error accumulation across multiple intermediate processes. The second issue is that these approaches still struggle to learn instance-discriminative features near the edges, which may lead to inaccurate edge segmentation. The third issue is that they do not fully exploit the geometric characteristics of planes to supervise the network training, which may cause the final plane segments with relatively low geometric fidelity. To resolve the aforementioned issues, we develop an end-to-end edge-aware transformer-based network, named RoofSeg, for segmenting roof planes from airborne LiDAR point clouds. Inspired by transformers ( Cheng et al. 2021 ; Schult et al. 2023 ; Lai et al. 2023 ; Roh et al. 2024 ) for instance segmentation, we utilize a set of plane queries, learned from transformer-based encoder-decoder architecture, to directly detect the potential planes and predict their instance masks from the point clouds. Thus, we can avoid parameter-sensitive intermediate processing and enable end-to-end roof plane segmentation. To achieve accurate segmentation near the edge regions, we design an Edge-Aware Mask Module (EAMM) for further plane mask refinement. Specifically, we utilize the tangent distances from edge points to the predicted initial plane as the key geometric cues to enhance the learning of edge features, thereby improving the instance discriminability of points near the edges. Subsequently, based on the learned edge-aware point features, the EAMM can predict refined plane masks with accurate edges. Moreover, we propose a novel loss function for supervising the plane mask predictions. Based on the conventional mask loss formulation, we introduce an adpative weighting strategy to eliminate the influence of misclassified points. To ensure the geometric fidelity of the predicted plane segments, we design a plane geometric loss to constrain the network training. The extensive experimental results on three benchmarks proves that the proposed RoofSeg generates the roof plane segmentation results with accurate edges, minimal misclassified points and high geometric fidelity, while surpassing the existing competitive approaches. The main contributions of this work are concluded as bellow: 1. An end-to-end edge-aware transformer-based network, named RoofSeg, is developed for segmenting roof planes from airborne LiDAR point clouds in this work. Without any carefully designed clustering or post-processing method, RoofSeg can detect all the potential planes and predict their instance masks from 3D point clouds. 2. To address the challenge of accurate edge segmentation, we design an Edge-Aware Mask Module (EAMM) that integrates the critical geometric priors to enhance the discriminative edge features learning, which facilitates the further plane mask refinement. 3. Based on the existing instance mask loss formulation, we propose a novel loss function that incorporates the adaptive weighting strategy and plane geometric constraint, to ensure the plane mask predictions with minimal misclassified points and high geometric fidelity. 4. The extensive experiments conducted on three benchmarks demonstrate that our network achieves the new state-of-the-art performance compared to the current leading approaches. 2 Related work In general, the

Read the complete text