Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth

Viktor Kocur, Sithu Aung, Gabrielle Flood, Yaqing Ding, Lukas Bujnak, Torsten Sattler, Zuzana Kukelova · 2026

Read Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth by Viktor Kocur, Sithu Aung, Gabrielle Flood, Yaqing Ding, Lukas Bujnak, Torsten Sattler, Zuzana Kukelova (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19797.txt.

Introduction

Kocur and colleagues offer Depth2Pose, a way to judge single image depth estimators by downstream usefulness instead of raw depth error. Depth predictions feed geometric solvers alongside feature matches, and camera pose accuracy becomes the proxy for depth quality. Only camera poses are needed, rather than costly dense ground truth. Their D2P dataset adds hard scenes outside common training distributions. Methods strong on standard error metrics also score well on pose metrics, but do not always carry over to harder scenes.

Catalog record

CREATORViktor Kocur, Sithu Aung, Gabrielle Flood, Yaqing Ding, Lukas Bujnak, Torsten Sattler, Zuzana Kukelova
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71114
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19797

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth Viktor Kocur, Sithu Aung, Gabrielle Flood, Yaqing Ding, Lukas Bujnak, Torsten Sattler, Zuzana Kukelova Abstract Monocular depth estimation has improved significantly in recent years, driven by increasingly powerful models and large-scale training data. Predicted depth is increasingly used as an input signal for downstream tasks such as Structure-from-Motion (SfM), visual localization, and SLAM. However, monocular depth estimators (MDEs) are still primarily evaluated in terms of depth accuracy. Standard metrics aggregate errors globally and may not reflect the usefulness of depth for downstream geometric tasks. We therefore propose Depth2Pose, a framework for evaluating MDEs in the context of downstream tasks. By combining depth predictions with feature correspondences in depth-aware geometric solvers, we use relative camera pose estimation accuracy as a task-driven proxy for depth quality. Traditional benchmarks require dense ground truth in the form of per-pixel depth, which is expensive to obtain. In contrast, our formulation requires only camera poses, which can be estimated efficiently, e.g., using Structure-from-Motion pipelines. As a result, our framework can be applied to scenes where ground-truth depth is difficult to obtain, for example due to large scene scale or heavy occlusions (e.g., vegetated environments). Leveraging this, we introduce the D2P dataset, which contains challenging scenes outside the distribution of commonly used training data. We show that methods performing well under standard depth error metrics on existing benchmarks also perform well under our pose-based metric when evaluated on the same datasets, but do not necessarily generalize to our more challenging dataset. Finally, we provide a simple and extensible evaluation framework. The dataset and code are available at kocurvik.github.io/depth2pose . 1 Introduction Monocular depth estimation has become a central problem in computer vision, with rapid progress driven by deep learning and the availability of large-scale training data. Contemporary methods achieve strong performance on established benchmarks and are increasingly deployed as components within larger systems. In particular, predicted depth is starting to be used as an additional cue in downstream tasks such as Structure-from-Motion (SfM) [ 37 , 43 , 81 ] , visual localization [ 1 , 6 ] , Simultaneous Localization and Mapping (SLAM) [ 39 , 61 , 82 , 32 , 12 ] , and dense 3D reconstruction [ 22 , 77 , 16 , 23 ] , where monocular depth maps help improve robustness and reduce ambiguities. Despite their growing importance for downstream tasks, monocular depth estimation methods are still only evaluated in terms of their depth accuracy. In order to produce scalar metrics, existing evaluation protocols globally aggregate errors [ 17 ] . Common measures such as absolute relative error or root mean squared error aggregate discrepancies over all pixels and treat all regions equally. Threshold-based metrics ( e . g ., the percentage of pixels exceeding a given error) partially address this limitation by highlighting regions with large errors. However, they still do not explicitly account for the differing geometric importance of regions in downstream tasks. Another limitation of existing and widely used benchmarks [ 19 , 56 , 10 , 55 , 63 , 60 , 7 , 34 , 62 , 33 , 72 , 49 ] lies in their dependence on per-pixel ground truth depth values. Obtaining this ground truth depth itself is challenging, restricting these benchmarks to scenes where specialized hardware ( e . g ., RGB-D cameras or LiDAR) can be employed, where multi-view stereo approaches can be used to obtain pseudo-ground truth [ 34 , 33 ] , or to synthetic scenes [ 7 , 72 , 49 ] . As a consequence, many challenging scenarios, including large-scale outdoor scenes, reflective or transparent surfaces, and unconstrained environments, are underrepresented in current evaluation protocols. Furthermore, because ground-truth depth is typically used both for training and evaluation, benchmarks are often closely aligned with the training data distribution. This overlap limits the assessment of generalization and makes it difficult to quantify performance in diverse, real-world environments. For many downstream geometric tasks, such as SfM or SLAM, it is not necessary for depth to be uniformly accurate across the entire image. Instead, it is often sufficient for depth to be highly precise in regions that are geometrically informative, e . g ., areas with strong texture, stable feature correspondences, or favorable viewing geometry. Errors in less informative regions may have only a minor impact on final task performance. Consequently, two depth estimators with similar global or threshold-based error metrics can exhibit substantially different behavior when integrated into geometric pipelines. This discrepancy is largely invisible under standard evaluation protocols, which do not capture the task-dependent utility of depth. At the same time, obtaining accurate camera poses is often significantly easier than acquiring dense ground-truth depth. Mature SfM pipelines can reliably reconstruct camera trajectories across a wide range of environments, including those where depth sensing is impractical. This observation motivates a shift in perspective: rather than evaluating monocular depth directly against ground truth, one can assess its quality indirectly through its impact on downstream geometric estimation. In this work, we introduce Depth2Pose , a novel evaluation framework that measures the quality of monocular depth predictions via their contribution to relative pose estimation. Specifically, we employ modern depth-aware geometric solvers, such as those introduced in the RePoseD framework [ 13 ] , which jointly leverage point correspondences and predicted depth to estimate relative camera motion. Thus, Depth2Pose measures depth quality based on the usefulness of the predicted depth maps for accurate pose estimation. Importantly, this formulation does not require ground-truth depth and can therefore be applied to a much broader range of datasets. Moreover, by evaluating depth through its influence on geometric estimation, our framework naturally emphasizes regions of the scene that are most informative for many downstream tasks, allowing it to capture cases where depth is locally accurate and practically useful despite higher global error. We validate our framework on several established benchmarks and show that pose-based evaluation correlates strongly with standard depth metrics, confirming its reliability as an evaluation signal for depth quality. In order to investigate the generalization capabilities of monocular depth estimators, we present the D2P dataset , which consists of challenging real-world scenes that lie outside the distribution of commonly used training and evaluation data. We show that good performance under depth- and pose-based metrics on standard depth accuracy benchmarks does not necessarily translate to good performance on our new dataset. As such, our dataset contributes to a more realistic assessment of monocular depth estimation methods. The result further validates our evaluation framework, as it can be used to easily create benchmark datasets that reflect real-world deployment scenarios without requiring per-pixel ground truth depth values. In summary, our contributions are as follows: (1) we introduce Depth2Pose , a task-driven evaluation framework for monocular depth estimation based on depth-aware relative pose estimation. Depth2Pose represents a benchmarking paradigm that replaces ground-truth depth with ground-truth poses, enabling evaluation in diverse real-world environments. (2) we validate our framework on existing benchmarks for measuring depth accuracy. Our results show that pose-based metrics correlate with standard depth metrics, while additionally capturing task-relevant, spatially non-uniform depth quality. (3) we present a new dataset of challenging real-world scenes beyond the distribution of existing benchmarks. We

Read the complete text