GeoDiff: Geometry-Guided Diffusion for Metric Depth Estimation

Tuan Pham, Thanh-Tung Le, Xiaohui Xie, Stephan MandtUniversity of California, Irvine · 2025

Read GeoDiff: Geometry-Guided Diffusion for Metric Depth Estimation by Tuan Pham, Thanh-Tung Le, Xiaohui Xie, Stephan MandtUniversity of California, Irvine (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18291.txt.

Introduction

Tuan Pham, with Thanh-Tung Le and two co-authors, presents GeoDiff in a 2025 paper on depth estimation. Depth estimation means judging how far each point in a photo sits from the camera. Single photos give relative depth well, but absolute distance stays ambiguous without a scale cue.

GeoDiff treats depth recovery as an inverse problem, meaning it works backward from images to the hidden scene. It pairs a pretrained diffusion model, which generates images by gradual denoising, with stereo geometry constraints. The method needs no retraining and plugs into existing depth frameworks. Tests show it matches or beats leading methods, even on shiny and translucent surfaces.

Catalog record

CREATORTuan Pham, Thanh-Tung Le, Xiaohui Xie, Stephan MandtUniversity of California, Irvine
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-18291

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GeoDiff: Geometry-Guided Diffusion for Metric Depth Estimation Tuan Pham, Thanh-Tung Le, Xiaohui Xie, Stephan MandtUniversity of California, Irvine Abstract We introduce a novel framework for metric depth estimation that enhances pretrained diffusion-based monocular depth estimation (DB-MDE) models with stereo vision guidance. While existing DB-MDE methods excel at predicting relative depth, estimating absolute metric depth remains challenging due to scale ambiguities in single-image scenarios. To address this, we reframe depth estimation as an inverse problem, leveraging pretrained latent diffusion models (LDMs) conditioned on RGB images, combined with stereo-based geometric constraints, to learn scale and shift for accurate depth recovery. Our training-free solution seamlessly integrates into existing DB-MDE frameworks and generalizes across indoor, outdoor, and complex environments. Extensive experiments demonstrate that our approach matches or surpasses state-of-the-art methods, particularly in challenging scenarios involving translucent and specular surfaces, all without requiring retraining. Figure 1 : Results on the challenging Booster [ 36 ] dataset. Comparison of depth estimation performance on transparent and reflective objects. From left to right: input image, followed by depth maps from GeoDiff (ours), DepthPro [ 5 ] , Marigold [ 23 ] , and IGEV-Stereo [ 58 ] . Our method generates sharp, accurate metric depth maps in a zero-shot setting, leveraging stereo pairs for improved depth recovery. 1 Introduction Depth estimation is an essential task and play a fundamental role in wide applications, such as 3D reconstruction [ 33 , 25 ] , autonomous driving [ 55 ] , and AI-generated content [ 64 , 29 ] . Recently, monocular depth estimation (MDE) [ 4 , 12 , 13 , 42 ] and stereo depth estimation (StDE) have emerged as the leading methods for depth estimation. MDE approaches generally predict relative depth, which is invariant to scale and shift, whereas StDE methods focus on predicting disparity between two input images, which can be converted to metric depth (in meters) using known camera baseline and focal length. While recent methods attempt direct metric depth prediction from monocular images through large foundation models [ 63 , 61 , 60 ] , these approaches demand extensive synthetic and real data and are computationally expensive to train. Recently, diffusion models have demonstrated potential as robust priors for zero-shot dense prediction tasks, including depth estimation [ 11 , 23 , 21 , 43 , 42 , 14 , 34 ] . Marigold [ 23 ] is among one of the pioneer methods that propose to repurpose diffusion-based image generators for MDE. The main idea is to finetune pretrained latent diffusion models (LDMs) [ 41 ] , which have been trained on extensive text-to-image datasets, to generate depth maps from noise by conditioning on RGB images. The simple diffusion-based MDE (DB-MDE) paradigm works surprisingly well, delivering strong performance across a diverse range of natural images. Subsequent studies [ 14 , 20 , 17 , 11 ] have developed upon this paradigm, establishing diffusion-based MDE as an active research in dense prediction task. However, estimating metric depth from a single image remains an inherently ill-posed and challenging problem, leading most DB methods to concentrate on reconstructing relative depth rather than absolute metric depth. Figure 2 : Top. Prior methods [ 23 , 14 ] focus on fine-tuning diffusion models to estimate relative depth. Bottom. In contrast, our approach, without retraining, combines a pretrained monocular model with geometric guidance from stereo cues to directly predict metric depth in meters, achieving superior accuracy in even challenging scenes such as transparent and reflective surfaces. In this work, we advance this line of research by leveraging pretrained priors from DB-MDE models to achieve metric depth estimation through the incorporation of additional stereo settings. Specifically, we reformulate the depth sampling process as an inverse problem (IP) solved through diffusion models [ 10 , 45 , 7 , 8 , 22 ] . Our approach leverages pretrained LDMs, along with stereo vision-based geometric guidance, to learn the scale and shift for any given scene. Built upon the foundation of diffusion-based MDE (DB-MDE) approaches, such as Marigold [ 23 ] , our method is scene-agnostic—applicable to objects, indoor, and outdoor scenes—and can be integrated with any DB-MDE framework following a similar schema. Extensive experiments across diverse datasets demonstrate that our approach performs effectively on a wide range of scenes and depth scales without requiring re-training for specific use cases. In summary, we propose: • A novel framework that leverages diffusion-based MDE priors and stereo settings to achieve metric depth estimation. • An IP-based approach for depth estimation that introduces a plug-and-play module, seamlessly integrating with any pretrained diffusion-based depth models that use iterative updates. • Extensive experimental evaluations of our method compared to other competing methods across various datasets, encompassing indoor, outdoor, and challenging scenes with translucent or specular surfaces. 2 Related works 2.1 Depth Estimation Depth estimation has various applications in 3D vision [ 23 , 14 , 26 , 27 , 18 , 19 , 50 ] . Monocular depth estimation and stereo matching have both seen significant advancements in recent years. Traditional monocular methods focused on in-domain metric depth estimation but faced challenges in generalization, leading to a shift towards zero-shot relative depth estimation using approaches like Stable Diffusion [ 41 ] for depth denoising and large-scale datasets such as [ 23 , 14 ] . However, these data-driven models are limited by the availability of synthetic dataset, which has prompted methods like DepthAnything [ 60 , 61 ] , DepthPro [ 5 ] to leverage vast amounts of additional real dataset for enhanced robustness. Stereo matching, on the other hand, relies heavily on cost volume filtering techniques, typically employing deep learning models to extract features, build cost volumes, and regress disparities. Models like GCNet [ 24 ] and PSMNet [ 6 ] use 3D CNN architectures to address challenges with occlusions and textureless surfaces, while newer methods, such as GwcNet [ 18 ] and ACVNet [ 57 ] , introduce group-wise correlation and attention mechanisms to improve cost volume expressiveness. RAFT-Stereo [ 30 ] adapts the optical flow network RAFT [ 51 ] with multi-level convolutional GRUs, achieving impressive results. Building on this foundation, IGEV-Stereo [ 58 ] proposes iterative geometry encoding volumes, demonstrating improved robustness. Despite these advancements, the high memory and computational costs of 3D convolutions limit scalability. Our method leverages diffusion priors and geometric guidance, effectively overcoming the limitations of traditional cost volume approaches, particularly for challenging surfaces like transparent or reflective regions. In particular, our approach employs pretrained model from diffusion-based MDE combining with stereo information to estimate metric depth in the wild. To the best of our knowledge, we are the first to employ a training-free approach that combines both monocular and stereo information to solve metric depth estimation effectively. 2.2 Diffusion Models for Inverse Problem Solving Inverse problems (IPs) are ubiquitous and and associated with a wide range of reconstruction problems such as computational image [ 2 , 1 ] , medical imaging [ 49 , 39 ] , and remote sensing [ 31 ] . IPs aim to recover an unknown sample x\in\mathbb{R}^{n} , given observed measurements y\in\mathbb{R}^{m} of the form: y=\mathcal{A}(x)+e , where function \mathcal{A}(\cdot):\mathbb{R}^{n}\rightarrow\mathbb{R}^{m} is the forward measurement operator and e\in\mathbb{R}^{m} is additive noise. In the literature, the traditional

Read the complete text