One Look is Enough: Seamless Patchwise Refinement for Zero-Shot Monocular Depth Estimation on High-Resolution Images
Byeongjun Kwon, Munchurl Kim, https://kaist-viclab.github.io/One-Look-is-Enough_site · 2025
Read One Look is Enough: Seamless Patchwise Refinement for Zero-Shot Monocular Depth Estimation on High-Resolution Images by Byeongjun Kwon, Munchurl Kim, https://kaist-viclab.github.io/One-Look-is-Enough_site (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22351.txt.
Introduction
Byeongjun Kwon and Munchurl Kim present a 2025 method to estimate depth from a single high resolution photo. Models trained on smaller images lose accuracy at full size, yet shrinking the photo blurs the edges in the depth map.
Patch based fixes cut the image into tiles, but the tiles seldom agree where they meet, and reassembly wastes time. Their method refines patches in a single look. Four overlapping patches are processed together, with a loss that forces the overlaps to agree. A masking trick stops the model from picking up biases of synthetic training data. Tests on four benchmarks, in zero shot mode, showed gains in both speed and reach.
Catalog record
| CREATOR | Byeongjun Kwon, Munchurl Kim, https://kaist-viclab.github.io/One-Look-is-Enough_site |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-22351
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
One Look is Enough: Seamless Patchwise Refinement for Zero-Shot Monocular Depth Estimation on High-Resolution Images Byeongjun Kwon, Munchurl Kim, https://kaist-viclab.github.io/One-Look-is-Enough_site Abstract Zero-shot depth estimation (DE) models exhibit strong generalization performance as they are trained on large-scale datasets. However, existing models struggle with high-resolution images due to the discrepancy in image resolutions of training (with smaller resolutions) and inference (for high resolutions). Processing them at full resolution leads to decreased estimation accuracy on depth with tremendous memory consumption, while downsampling to the training resolution results in blurred edges in the estimated depth images. Prevailing high-resolution depth estimation methods adopt a patch-based approach, which introduces depth discontinuity issues when reassembling the estimated depth patches, resulting in test-time inefficiency. Additionally, to obtain fine-grained depth details, these methods rely on synthetic datasets due to the real-world sparse ground truth depth, leading to poor generalizability. To tackle these limitations, we propose Patch Refine Once (PRO), an efficient and generalizable tile-based framework. Our PRO consists of two key components: (i) Grouped Patch Consistency Training that enhances test-time efficiency while mitigating the depth discontinuity problem by jointly processing four overlapping patches and enforcing a consistency loss on their overlapping regions within a single backpropagation step, and (ii) Bias Free Masking that prevents the DE models from overfitting to dataset-specific biases, enabling better generalization to real-world datasets even after training on synthetic data. Zero-shot evaluations on Booster, ETH3D, Middlebury 2014, and NuScenes demonstrate that our PRO can be seamlessly integrated into existing depth estimation models. It preserves the performance of original depth estimation models even under grid-based inference on high-resolution images, exhibiting minimal depth discontinuities along patch boundaries. Moreover, our PRO achieves significantly faster inference speed compared to prior patch-based methods. 1 Introduction Monocular Depth Estimation (MDE) [ 10 , 33 , 36 , 27 , 43 , 13 , 14 ] has been widely investigated as the demand for 3D information continues to grow in autonomous driving, robotics, and virtual reality applications. After Midas [ 30 ] proposed an MDE network trainable with mixed training datasets, numerous zero-shot MDE networks [ 31 , 9 , 2 , 46 , 47 , 19 , 11 ] have been studied to predict depth for unseen real-world images. However, due to the architectural limitation or limited image resolutions of training datasets, most zero-shot MDE networks are trained specifically with the images of low resolutions (e.g. 384 \times 384, 518 \times 518). Consequently, when high resolution images are fed into these models, the resulting estimated depth images tend to contain disrupted overall structures with low-frequency artifacts, although yielding improved edge details [ 26 , 7 ] . Patch-based methods [ 26 , 24 , 23 ] have shown promising results by splitting high-resolution images into patches for DE, mitigating memory consumption issues while achieving remarkable performance. However, they suffer from depth discontinuity problems (e.g. boundary artifacts along grid boundaries) which occur when independent DE patches are reassembled to construct a complete (whole) depth map because depth continuity is maintained within each patch but not between patches. Previous methods [ 24 , 23 ] alleviate this depth discontinuity problem (i) by incorporating a consistency loss during training and (ii) by ensemble averaging at test time which slows down inference speed, making it impractical for real-world applications. Figure 2 : Comparison of performance and efficiency on Middlebury 2014 [ 34 ] . The area of each circle represents the inference time. Circles of the same color represent the same model with different patch numbers ( \mathrm{P}=16 for small-sized yellow and grey circles, \mathrm{P}=177 for large-sized yellow and grey circles). Our model (PRO) achieves the best performance in terms of edge errors ( \mathrm{D}^{3}\mathrm{R} ) and depth errors (AbsRel) while maintaining the fastest inference time. Inference time for all models is measured on the RTX 4090. A common strategy for training high-resolution depth estimation is to train models on real-world high-resolution datasets [ 17 , 20 , 39 , 42 , 34 ] or leverage high-resolution synthetic datasets [ 38 , 41 , 40 , 18 ] . Real-world datasets have the advantage of a smaller domain gap compared to synthetic datasets, but their ground truth (GT) depths are often sparse, particularly around edges, where supervision is crucial. Additionally, the difficulty of acquiring dense depth data in real-world scenarios hinders the training of DE models for high-resolution images. On the other hand, synthetic datasets provide dense GT depth with fine details, but models trained on synthetic datasets often struggle with zero-shot inference on real-world data due to the domain gap [ 21 ] . Moreover, we observe that transparent objects are inaccurately labeled in the UnrealStereo4K dataset [ 38 ] , the only synthetic dataset providing 4K depth annotations, as their depth values correspond to the background behind them. To address the aforementioned problems, we propose Patch Refine Once (PRO), a novel refinement model that enables DE models to generate accurate depth predictions with only a single refinement per patch at inference time (i.e.,“One Look”). Our PRO consists of two strategies: Grouped Patch Consistency Training and Bias Free Masking. Fig. shows a qualitative comparison of patch-based depth estimation models for high-resolution images. As shown, our PRO yields depth refinement results with minimal depth discontinuities along grid boundaries, whereas two recent state-of-the-art (SOTA) models still exhibit noticeable boundary artifacts. Fig. 2 illustrates a comparison of performance and efficiency between recent SOTA depth refinement models and our PRO. As observed, our PRO provides the best performance in terms of \mathrm{D}^{3}\mathrm{R} , AbsRel and inference speed. Our contribution is summarized as: • We propose Grouped Patch Consistency Training (GPCT) strategy that can be well harmonized with existing DE models to mitigate the depth discontinuity problem for patch-wise DE approaches on high-resolution images. The GPCT does not require test-time ensemble, allowing a 12 \times faster inference speed compared to the SOTA patch-based method. • We introduce Bias Free Masking (BFM) to selectively apply supervision signals by masking out unreliable regions in images. The unreliable regions are the regions that have inaccurately annotated depth in GT, which is often observed in the UnrealStereo4K dataset. This issue is effectively addressed by our BFM, which prevents the model from overfitting to domain-specific biases. • Our PRO achieves SOTA performance in zero-shot evaluation on Booster [ 29 ] , ETH3D [ 35 ] , Middlebury 2014 [ 34 ] , and NuScenes [ 4 ] across all metrics, and achieves efficient inference (12 \times faster than PatchRefiner with 177 patches), outperforming recent state-of-the-art depth refinement methods on high-resolution images. 2 Related work 2.1 Zero-shot Monocular Depth Estimation Early MDE models [ 1 , 25 , 10 , 33 , 49 ] were trained individually on each dataset [ 12 , 37 ] , achieving high performance on the training data but poor generalization to unseen datasets due to the domain gap. Zero-shot MDE models that perform well on unseen images have been widely studied to improve generalization. MegaDepth [ 22 ] and DiverseDepth [ 48 ] construct large-scale datasets by collecting Internet images, improving adaptability to images from diverse scenes and conditions. They have inspired further efforts to scale up training datasets to enhance zero-shot performance. MiDaS [ 30 ] proposes a