Winner of CVPR2026 NTIRE Challenge on Image Shadow Removal: Semantic and Geometric Guidance for Shadow Removal via Cascaded Refinement

Lorenzo Beltrame, Jules Salzinger, Filip Svoboda, Jasmin Lampert, Phillipp Fanta-Jende, Radu Timofte, Marco Körner · 2026

Read Winner of CVPR2026 NTIRE Challenge on Image Shadow Removal: Semantic and Geometric Guidance for Shadow Removal via Cascaded Refinement by Lorenzo Beltrame, Jules Salzinger, Filip Svoboda, Jasmin Lampert, Phillipp Fanta-Jende, Radu Timofte, Marco Körner (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16177.txt.

Introduction

Beltrame and six colleagues present the winning entry in the CVPR2026 NTIRE challenge on removing shadows from images. Their three-stage pipeline refines predictions step by step, combining colour appearance with semantic guidance and geometric cues from depth and surface normals. A contraction rule keeps reconstruction error from growing across stages, and their final ensemble topped the hidden WSRD+ 2026 test, taking first place overall.

Catalog record

CREATORLorenzo Beltrame, Jules Salzinger, Filip Svoboda, Jasmin Lampert, Phillipp Fanta-Jende, Radu Timofte, Marco Körner
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36592
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16177

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Winner of CVPR2026 NTIRE Challenge on Image Shadow Removal: Semantic and Geometric Guidance for Shadow Removal via Cascaded Refinement Thanks: Accepted at the CVPR 2026 Workshops (NTIRE 2026 Image Shadow Removal Challenge). Lorenzo Beltrame, Jules Salzinger, Filip Svoboda, Jasmin Lampert, Phillipp Fanta-Jende, Radu Timofte, Marco Körner Abstract We present a three-stage progressive shadow-removal pipeline for the CVPR2026 NTIRE WSRD+ challenge. Built on OmniSR, our method treats deshadowing as iterative direct refinement, where later stages correct residual artefacts left by earlier predictions. The model combines RGB appearance with frozen DINOv2 semantic guidance and geometric cues from monocular depth and surface normals, reused across all stages. To stabilise multi-stage optimisation, we introduce a contraction-constrained objective that encourages non-increasing reconstruction error across the cascade. A staged training pipeline transfers from earlier WSRD pretraining to WSRD+ supervision and final WSRD+ 2026 adaptation with cosine-annealed checkpoint ensembling. On the official WSRD+ 2026 hidden test set, our final ensemble achieved 26.680 PSNR, 0.8740 SSIM, 0.0578 LPIPS, and 26.135 FID, ranked first overall, and won the NTIRE 2026 Image Shadow Removal Challenge. The strong performance of the proposed model is further validated on the ISTD+ and UAV-SC+ datasets. 1 Introduction (a) Single-stage OmniSR (b) Three-stage OmniSR (c) Single-stage OmniSR (d) Three-stage OmniSR Figure 1 : Qualitative comparison on validation samples from WSRD+. The single-stage OmniSR predictions in 1(a) and 1(c) retain noticeable residual errors, including mild colour and illumination bias as well as local boundary artefacts. In contrast, the corresponding three-stage OmniSR predictions in 1(b) and 1(d) produce more consistent restoration with cleaner structures, supporting the use of progressive refinement. Image shadow removal is an important low-level vision problem because shadows corrupt local illumination, suppress visible texture, and reduce the reliability of downstream perception systems. This affects not only generic recognition and scene understanding, but also applications such as video analysis, traffic monitoring, and remote sensing 41 ; 4 ; 2 ; 1 . In natural images, deshadowing is challenging because the model must separate illumination changes from intrinsic scene appearance while preserving texture, geometry, and object boundaries. Recent deshadowing methods increasingly combine transformer-based restoration with auxiliary cues beyond RGB 34 ; 43 ; 8 ; 7 ; 36 , yet even strong single-stage systems such as OmniSR 37 may still retain residual colour shifts, illumination bias, and boundary artefacts after one forward pass. As shown in Fig. 1 , the single-stage OmniSR results in 1(a) and 1(c) exhibit visible residual errors on WSRD+ validation samples 30 , whereas the corresponding three-stage OmniSR results in 1(b) and 1(d) yield a more uniform appearance and cleaner local transitions. These limitations after a single forward pass suggest that shadow removal is better viewed as progressive refinement rather than one-shot prediction, consistent with iterative restoration formulations in inverse problems 33 ; 25 ; 24 ; 26 ; 28 . On this basis, our method extends OmniSR to a three-stage direct-refinement architecture. To stabilise optimisation, we use a contraction-constrained objective that encourages monotonic improvement across stages, as detailed in Section 3.3 . We further adopt staged pretraining across Würzburg shadow removal dataset (WSRD), Würzburg shadow removal dataset plus (WSRD+), and WSRD+2026 30 . Additionally, we also evaluate on the Refined Image Shadow Triplets Dataset (ISTD+) 34 and Refined Uncrewed Aerial Vehicle Dataset for Shadow Correction (UAV-SC+) 20 ( Table 4 and Table 5 ). Our contributions are as follows: • We introduce a three-stage OmniSR-based shadow-removal pipeline for progressive direct refinement, coupled with contraction-constrained multi-stage supervision that stabilises training and promotes monotonic improvement across stages. • We combine RGB appearance with DINOv2 semantic guidance and depth-derived geometric cues for robust deshadowing under challenging illumination conditions in a multi-stage setting. • We present a staged pretraining regime for deshadowing that transfers from earlier imperfectly aligned WSRD data to aligned WSRD+ data and then adapts to the WSRD+ 2026 distribution, yielding consistent gains under dataset shift. 2 Related Work Shadow removal has progressed from hand-crafted illumination models and decomposition approaches to fully learned image-to-image restoration systems. Early work studied the effect of shadows on recognition and scene understanding 41 ; 4 , while later deep methods learned shadow-specific priors directly from paired or synthetic data 10 ; 14 . Auxiliary guidance has remained a recurring design choice in this literature. ST-CGAN jointly predicts a shadow mask and a shadow-free image 34 , BMNet conditions restoration on shadow masks and shadow-invariant color cues 43 , and ShadowDiffusion integrates degradation priors while refining masks during diffusion-based restoration 8 . In parallel, transformer-based restoration frameworks have improved the ability to model long-range structure and non-local illumination effects, as illustrated by ShadowFormer and HomoFormer 7 ; 36 . Among these methods, OmniSR 37 is particularly relevant because it explicitly combines RGB appearance with semantic and geometric guidance for shadow removal under both direct and indirect lighting. These two characteristics are considered crucial in the shadow removal community 31 . 2.1 Semantic and geometric guidance Shadow removal is ambiguous when appearance alone cannot distinguish cast shadows from intrinsically dark materials or low-reflectance surfaces. Earlier auxiliary signals in deshadowing were typically mask- or prior-based, including detection outputs, decomposition cues, colour guidance, and degradation models 34 ; 14 ; 43 ; 8 . More recent work has begun to inject higher-level scene representations directly into the restoration backbone. DINOv2 23 supplies robust semantic representations without task-specific supervision, while monocular depth estimation with Depth Anything V2 39 provides geometric structure from which surface normals can be derived. OmniSR 37 demonstrated the utility of this specific semantic-plus-geometry conditioning scheme for shadow removal, and our method builds directly on that observation. 2.2 Iterative and progressive restoration Many restoration problems benefit from repeated refinement rather than one-shot prediction. This perspective is well established in inverse problems, plug-and-play reconstruction, and regularisation-by-denoising frameworks, where learned priors are repeatedly applied to improve reconstruction fidelity 33 ; 25 ; 24 ; 26 ; 28 . We adopt this view for shadow removal and formulate the proposed system as a multi-stage direct-refinement pipeline in which later stages explicitly correct the residual errors left by earlier ones. 3 Method Figure 2 : Proposed three-stage OmniSR pipeline. The shadowed RGB input \boldsymbol{x} is processed by three cascaded OmniSR stages, where stage k takes \hat{\boldsymbol{y}}^{(k-1)} as input and outputs a refined prediction \hat{\boldsymbol{y}}^{(k)} . DINOv2 semantic features and depth-derived geometric cues are extracted once from \boldsymbol{x} and reused across all stages. The final prediction is \hat{\boldsymbol{y}}^{(3)} . Training supervision is shown in the same diagram for compactness: d_{k}=\left\lVert\hat{\boldsymbol{y}}^{(k)}-\boldsymbol{y}^{*}\right\rVert_{2} denotes the stage-wise error to ground truth, the stage reconstruction loss ( \mathcal{L}_{\text{stage}} ) supervises the intermediate outputs, the reconstruction loss constrains the final output, and the contraction loss encourages non-increasing error across stages. Given a shadowed RGB image \boldsymbol{x}\in\mathbb{R}^{H\times W\times 3} , the goal is to predict a reconstruction \hat{\boldsymbol{y}}\approx\boldsymbol{y}^{*} as similar as possible to the ground-truth shadow-free image \boldsymbol{y}^{*} . Our proposed approach adopts a three-stage transformer-based iterative restoration cascade built on OmniSR 37 . Let S(\boldsymbol{x}) denote frozen semantic features extracted once from the input image using DINOv2 23 , and let G(\boldsymbol{x}) denote geometric guidance derived from monocular depth and surface normals estimated from the same input 39 . Let further f_{k}(\cdot) denote the restoration function implemented by the k

Read the complete text