BuildMamba: A Visual State-Space Based Model for Multi-Task Building Segmentation and Height Estimation from Satellite Images
Sinan U. Ulu, A. Enes Doruk, I. Can Yagmur, Bahadir K. Gunturk · 2026
Read BuildMamba: A Visual State-Space Based Model for Multi-Task Building Segmentation and Height Estimation from Satellite Images by Sinan U. Ulu, A. Enes Doruk, I. Can Yagmur, Bahadir K. Gunturk (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08523.txt.
Introduction
Sinan Ulu and colleagues present BuildMamba, a single model that maps buildings and their heights from one overhead photo. Past routes bleed across edges and shrink tall blocks, since one flat view leaves depth unclear. Their design scans the whole scene in linear time, sharpens edges with a mask-aware height pass, and sets a new bar on three test sets.
Catalog record
| CREATOR | Sinan U. Ulu, A. Enes Doruk, I. Can Yagmur, Bahadir K. Gunturk |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67436 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.08523
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BuildMamba: A Visual State-Space Based Model for Multi-Task Building Segmentation and Height Estimation from Satellite Images Thanks: Sinan U. Ulu, A. Enes Doruk and Hasan F. Ates are with the Graduate School of Engineering, Ozyegin university, 34794 Istanbul, Türkiye. (e-mail: [email protected]). Thanks: I. Can Yagmur is with the Department of Computer Science, University of Rochester, Rochester, NY 14627 USA. Thanks: Bahadir K. Gunturk is with the School of Engineering and Natural Sciences, Istanbul Medipol University, 34810 Istanbul, Türkiye. Thanks: Oguz Hanoglu is with Huawei Turkiye R&D Center, 34768 Istanbul, Türkiye. Sinan U. Ulu, A. Enes Doruk, I. Can Yagmur, Bahadir K. Gunturk Abstract Accurate building segmentation and height estimation from single-view RGB satellite imagery are fundamental for urban analytics, yet remain ill-posed due to structural variability and the high computational cost of global context modeling. While current approaches typically adapt monocular depth architectures, they often suffer from boundary bleeding and systematic underestimation of high-rise structures. To address these limitations, we propose BuildMamba, a unified multi-task framework designed to exploit the linear-time global modeling of visual state-space models. Motivated by the need for stronger structural coupling and computational efficiency, we introduce three modules: a Mamba Attention Module for dynamic spatial recalibration, a Spatial-Aware Mamba-FPN for multi-scale feature aggregation via gated state-space scans, and a Mask-Aware Height Refinement module using semantic priors to suppress height artifacts. Extensive experiments demonstrate that BuildMamba establishes a new performance upper bound across three benchmarks. Specifically, it achieves an IoU of 0.93 and RMSE of 1.77 m on DFC23 benchmark, surpassing state-of-the-art by 0.82 m in height estimation. Simulation results confirm the model’s superior robustness and scalability for large-scale 3D urban reconstruction. I Introduction Computer vision has advanced rapidly, enabling large-scale analysis of satellite imagery and raising expectations for urban analytics. In this context, building footprints and heights constitute fundamental variables for numerous downstream applications. They support urban planning and land-use analysis by characterizing building density and spatial organization [ 1 ] , infrastructure monitoring and large-scale assessment of the built environment using building inventories and 3D city models [ 2 , 3 ] , and risk and vulnerability assessment through the quantification of hazard exposure based on building geometry [ 4 ] . Building height and footprint information are also essential for shadow modeling and urban climate analysis [ 5 ] , communication network design and line-of-sight modeling [ 6 ] , and high-resolution population and asset mapping at regional and continental scales [ 7 , 8 ] . Fig. 1 : Overview of the proposed BuildMamba architecture for joint building segmentation and height estimation. Satellite-based remote sensing is uniquely suited to provide such information at regional and global scales, but it operates under a fundamentally different imaging geometry than ground-level vision. From a near top-down viewpoint, buildings appear primarily as two-dimensional radiometric patterns rather than explicit three-dimensional structures. This requires structural attributes such as height and precise footprint boundaries to be inferred indirectly from appearance cues, including texture, tone, and spatial context. As a result, joint building segmentation and height estimation from single-view RGB satellite imagery is inherently ill-posed and highly sensitive to illumination changes, shadows, atmospheric effects, roof materials, and heterogeneous urban layouts, leading to ambiguity and instability widely observed in monocular height estimation studies [ 9 , 10 , 11 ] . Building height estimation from monocular satellite imagery is commonly addressed by adapting monocular depth estimation architectures, including encoder–decoder CNNs [ 9 , 10 ] , ordinal-regression models [ 12 ] , and building-specific transformer-based predictors [ 11 ] . Recent refinements incorporate adaptive binning strategies [ 13 , 14 ] and hybrid classification-regression formulations [ 15 ] to mitigate ambiguities in monocular height retrieval and better handle long-tailed height distributions. Despite these improvements, most RGB-only approaches remain appearance-driven and rely on implicit correlations between image texture and height, with limited structural understanding of building shapes. Consequently, height prediction becomes highly sensitive to homogeneous rooftop textures, shadows, and radiometric distortions, frequently producing boundary artifacts, systematic underestimation of tall buildings, and limited generalization across cities and sensors. These issues are further exacerbated by noise in DSM (Digital Surface Model) or LiDAR-derived supervision caused by resolution mismatch, temporal misalignment, and registration errors. Some works attempt to mitigate these limitations by incorporating additional modalities such as SAR [ 16 , 17 , 18 ] or auxiliary supervision such as land-use masks [ 19 ] . However, such information is frequently unavailable at inference time, restricting practical deployment in large-scale RGB-only settings. Moreover, in many joint segmentation–height estimation frameworks [ 20 , 21 ] , height estimation is treated as a secondary output rather than a structurally constrained task, leaving the dependency between building footprints and heights under-exploited. These observations suggest that robust monocular height estimation requires both strong global contextual reasoning and precise local spatial modeling, together with explicit structural coupling between segmentation and height prediction. Motivated by these limitations, recent remote sensing frameworks increasingly adopt dual-path encoders that separate global contextual modeling from local spatial feature extraction [ 22 , 23 , 24 ] . Such architectures aim to reconcile the need for fine-grained spatial detail with long-range contextual reasoning, benefiting both building footprint extraction [ 25 ] and joint segmentation–height estimation tasks [ 20 , 26 ] . Global pathways often rely on transformer-based encoders to model scene-level context, while local pathways employ convolutional layers to preserve spatial locality and edge structure. More recently, state-space-driven visual models such as VMamba [ 27 ] have emerged as efficient alternatives to attention-based transformers [ 28 ] and hierarchical variants such as SwinTransformer [ 29 ] . By enabling long-range contextual modeling with linear computational complexity while preserving spatial locality through structured recurrence, VMamba offers a favorable balance between global reasoning and efficiency for high-resolution remote sensing imagery. In parallel, multi-task and multi-modal formulations jointly learn building segmentation and height estimation using complementary data sources beyond RGB. DSMNet [ 21 ] employs a two-stage design for joint prediction and refinement, while MFTSC [ 16 ] integrates transformer-based encoding with local feature extraction and pyramid pooling to impose semantic constraints on height regression. Joint contrastive learning approaches [ 30 ] further promote feature consistency between segmentation and height prediction. Despite these advances, existing methods often rely on auxiliary modalities, rigid fusion strategies, or benchmark-specific assumptions, and remain sensitive to domain shifts caused by heterogeneous sensors and geographic variability. Moreover, height prediction is frequently weakly coupled to explicit building footprints and boundaries, limiting geometric consistency and robustness in practical RGB-only settings. As a result, several core challenges remain unresolved in monocular satellite-based building height estimation. Existing methods struggle with tall-building prediction due to severe data imbalance and the scarcity of high-rise examples, leading to systematic underestimation. Boundary ambiguity, shadows, and structural variability introduce noise around complex roof geometries, while domain shifts across cities and sensors degrade generalization. Furthermore, reliance on auxiliary modalities limits scalability in practical RGB-only deployments. Addressing these limitations requires architectures that jointly model long-range context and fine-grained spatial detail, while explicitly coupling segmentation and height estimation. To address these limitations, we propose BuildMamba, a unified multi-task framework for joint building segmentation and height estimation from single RGB satellite images. As summarized in Figure 1 ,