Enhancing Monocular Height Estimation via Weak Supervision from Imperfect Labels

Sining Chen, Yilei Shi, and Xiao Xiang Zhu · 2025

Read Enhancing Monocular Height Estimation via Weak Supervision from Imperfect Labels by Sining Chen, Yilei Shi, and Xiao Xiang Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02534.txt.

Introduction

Sining Chen and co-authors train height maps from flawed labels in a 2025 remote sensing paper. Guessing ground height from a single image is cheap and useful, but fine labels exist mostly in rich regions. Labels from elsewhere arrive incomplete, rough, or wrong, and naive training on them drags models down.

Their pipeline pairs with any height network and learns from the noise instead of choking on it. Balanced soft losses keep bad labels from shouting, and ordinal constraints teach the model that taller should stay taller even when exact values wobble. On two benchmark sets, cross-region error fell by up to about a fifth against standard training. Taking the pipeline apart piece by piece confirmed that each design choice earned its place. The authors aim at global maps built from uneven data.

Catalog record

CREATORSining Chen, Yilei Shi, and Xiao Xiang Zhu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-02534

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enhancing Monocular Height Estimation via Weak Supervision from Imperfect Labels Thanks: S. Chen and X. Zhu are with the Chair of Data Science in Earth Observation, Technical University of Munich (TUM), 80333 Munich, Germany and also with the Munich Center for Machine Learning (e-mail: [email protected]; [email protected]). Thanks: Y. Shi is with the School of Engineering and Design, Technical University of Munich (TUM), 80333 Munich, Germany (e-mail: [email protected]). Thanks: The work is supported by German Federal Ministry for Economic Affairs and Climate Action in the framework of the ”national center of excellence ML4Earth” (grant number: 50EE2201C) and by Munich Center for Machine Learning, by the TUM Georg Nemetschek Institute for Artificial Intelligence for the Built World as part of the AI4TWINNING project and by the Munich Center for Machine Learning. Sining Chen, Yilei Shi, and Xiao Xiang Zhu Abstract Monocular height estimation is considered the most efficient and cost-effective means of 3D perception in remote sensing, and it has attracted much attention since the emergence of deep learning. While training neural networks requires a large amount of data, data with perfect labels are scarce and only available within developed regions. The trained models therefore lack generalizability, which limits the potential for large-scale application of existing methods. We tackle this problem for the first time, by introducing data with imperfect labels into training pixel-wise height estimation networks, including labels that are incomplete, inexact, and inaccurate compared to high-quality labels. We propose an ensemble-based pipeline compatible with any monocular height estimation network. Taking the challenges of noisy labels, domain shift, and long-tailed distribution of height values into consideration, we carefully design the architecture and loss functions to leverage the information concealed in imperfect labels using weak supervision through balanced soft losses and ordinal constraints. We conduct extensive experiments on two datasets with different resolutions, DFC23 (0.5 to 1 m) and GBH (3 m). The results indicate that the proposed pipeline outperforms baselines by achieving more balanced performance across various domains, leading to improvements of average root mean square errors up to 22.94 %, and 18.62 % on DFC23 and GBH, respectively. The efficacy of each design component is validated through ablation studies. Code is available at https://github.com/zhu-xlab/weakim2h . I Introduction Three-dimensional perception in remote sensing is attracting much attention, as the acquired 3D information is of great significance and is crucial for many applications. In particular, modeling buildings in 3D improves our understanding of the built environment, because building heights and the building volumes that can be derived from them correlate well with urban anthropogenic factors, such as population density [ 1 , 2 , 3 ] , energy consumption [ 4 , 5 ] and solar potential [ 6 , 7 , 8 ] , and urban heat island effects [ 9 , 10 , 11 ] . Moreover, monitoring forests in 3D enhances our knowledge of the global carbon reservoir [ 12 , 13 , 14 ] , and plays an important role towards climate adaptation [ 15 ] . Among all the means of 3D perception in remote sensing, methods based on single images are the most efficient and cost-effective. These advantages encourage large-scale applications of such methods in order to benefit the Global South. Technically, the 3D information is retrieved from single remote sensing images via monocular height estimation. Since the emergence of deep learning, there has been extensive research on deep learning-based monocular height estimation. While generic fully convolutional neural networks are adopted for such purposes [ 16 , 17 , 18 ] , various customized networks are proposed towards better performance [ 19 , 20 , 21 , 22 , 23 , 24 ] . As more and more novel techniques have been proposed and rapidly popularized, they have also been adapted for monocular height estimation. Nonetheless, current research in monocular height estimation still relies on sufficient data with well-annotated labels, and does not address one major issue in real-world cases, that is, the limited data with perfect labels and plenty of data with imperfect labels. To elaborate, perfect labels for monocular height estimation are only possible with LiDAR observations, which are costly. As a result, such labels are mainly available in developed regions such as North America and Europe. Models trained on these data usually perform poorly on the unseen areas, which are the high-demand developing regions ( e.g. , South America and Asia), where only imperfect labels acquired by other means are available. Only by incorporating imperfect labels can models work reasonably well in those areas. There have been attempts to train networks with imperfect data through weakly supervised learning [ 25 , 26 , 27 , 28 ] . However, the solutions are tailored to one specific task and problem setting, and thus cannot be adopted directly. To design a framework specifically for monocular height estimation, several aspects should be taken into account. First, there are different levels of label qualities in monocular height estimation—each requiring a distinct treatment because of different label properties. Second, given the availability of high-quality labels corresponding to the development status, the domains vary with different label qualities. Third, as one of the thorny issues, the long-tailed distribution of height values also exists in these imperfect labels [ 29 ] . To address these issues, we propose an ensemble-based framework that deals with all types of label quality. We design the training loss functions with careful consideration of the challenges. In summary, our contributions are as follows. • To the best of our knowledge, we tackle the problem of introducing imperfect labels in training monocular height estimation networks for the first time. • We propose an ensemble-based pipeline, compatible with all kinds of label qualities. • We propose loss functions tailored to best exploit the supervisory information from the imperfect labels. This includes the balanced soft height loss and the ordinal constraints. We also propose the ground truth augmentation module to mitigate the blurring effect in the outputs. • We conduct experiments on two datasets, and we demonstrate the efficacy of our proposed pipeline in learning from the imperfect labels and balancing the performance across various domains. The remainder of the article is organized as follows. Section II gives an overview of the related works. The proposed method is described in detail in Section III , followed by Section IV describing the experiments and Section V showing the experimental results. Discussions and ablation studies are presented in Section VI . Conclusions are drawn, and further research directions are described in Section VII . II Related Works II-A Monocular Height Estimation Monocular height estimation is the process of predicting height values from single optical images. Early works explicitly exploit shadows as cues to infer the heights of ground objects [ 30 ] . However, they are restricted to cases when shadows are clearly seen in the images, and therefore do not perform well in dense urban areas. As deep learning has achieved state-of-the-art performance in many tasks, deep learning-based monocular height estimation methods stand out due to their flexibility and capability. Monocular height estimation, in the context of this work, refers to pixel-wise height estimation methods, which estimate a height value for each pixel. It requires neural networks for dense

Read the complete text