deSEO: Physics-Aware Dataset Creation for High-Resolution Satellite Image Shadow Removal

Lorenzo Beltrame 1,4, Jules Salzinger 1, Filip Svoboda 2, Phillipp Fanta-Jende 1, Jasmin Lampert 1, Radu Timofte 3, Marco Körner 4,5,6 · 2026

Read deSEO: Physics-Aware Dataset Creation for High-Resolution Satellite Image Shadow Removal by Lorenzo Beltrame 1,4, Jules Salzinger 1, Filip Svoboda 2, Phillipp Fanta-Jende 1, Jasmin Lampert 1, Radu Timofte 3, Marco Körner 4,5,6 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03610.txt.

Introduction

Lorenzo Beltrame and six co-authors address a missing resource in satellite imaging: paired examples of the same ground scene with and without shadows. Shadows from terrain and tall buildings degrade classification, detection, and 3D reconstruction, yet public paired data for shadow removal barely exist. Their deSEO method builds such pairs from an existing shadow detection dataset.

For each tile, deSEO picks a lightly shadowed image as a weak reference and pairs it with shadowed views using time and geometry filters, orientation normalization, and careful registration. A validity mask marks reliably aligned pixels. The team also trains a depth-aware deshadowing model, giving satellite shadow removal a replicable data pipeline.

Catalog record

CREATORLorenzo Beltrame 1,4, Jules Salzinger 1, Filip Svoboda 2, Phillipp Fanta-Jende 1, Jasmin Lampert 1, Radu Timofte 3, Marco Körner 4,5,6
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.03610

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

deSEO: Physics-Aware Dataset Creation for High-Resolution Satellite Image Shadow Removal Thanks: Accepted in the annals track at the ISPRS 2026 Congress. Lorenzo Beltrame 1,4, Jules Salzinger 1, Filip Svoboda 2, Phillipp Fanta-Jende 1, Jasmin Lampert 1, Radu Timofte 3, Marco Körner 4,5,6 Abstract Shadows cast by terrain and tall structures remain a major obstacle for high-resolution satellite image analysis, degrading classification, detection, and 3D reconstruction performance. Public resources offering geometry-consistent paired shadow/shadow-free satellite imagery are essentially missing, and most Earth-observation datasets are designed for shadow detection or 3D modelling rather than removal. Existing deep shadow-removal datasets either target ground-level or aerial scenes or rely on unpaired and weakly supervised formulations rather than explicit satellite pairs. We address this gap with deSEO , a geometry-aware and physics-informed methodology that, to the best of our knowledge, is the first to derive paired supervision for satellite shadow removal from the S-EO shadow detection dataset ( 16 ) through a fully replicable pipeline. For each tile, deSEO selects a minimally shadowed acquisition as a weak reference and pairs it with shadowed counterparts using temporal and geometric filtering, Jacobian-based orientation normalisation, and LoFTR–RANSAC registration. A per-pixel validity mask restricts learning to reliably aligned regions, enabling supervision despite residual off-nadir parallax. In addition to this paired dataset, we develop a DSM-aware deshadowing model that combines residual translation, perceptual objectives, and mask-constrained adversarial learning. In contrast, a direct adaptation of a UAV-based SRNet/pix2pix architecture fails to converge under satellite viewpoint variability. Our model consistently reduces the visual impact of cast shadows across diverse illumination and viewing conditions, achieving improved structural and perceptual fidelity on held-out scenes. deSEO therefore provides the first reproducible, geometry-aware paired dataset and baseline for shadow removal in satellite Earth observation. \DeclareSourcemap \maps [datatype=bibtex, overwrite] \map \step [fieldset=editor, null] namesep= ,pairsep= and ,listsep=, ,lastsep= and ,tpairsep= and ,tlistsep=, ,tlastsep=, and ,notesep= ,rangesep= to ,type=book,Name-sg=Book,name-sg=book,Name-pl=Books,name-pl=books,type=part,Name-sg=Part,name-sg=part,Name-pl=Parts,name-pl=parts,type=chapter,Name-sg=Chapter,name-sg=chapter,Name-pl=Chapters,name-pl=chapters,type=section,Name-sg=Section,name-sg=section,Name-pl=Sections,name-pl=sections,type=paragraph,Name-sg=Paragraph,name-sg=paragraph,Name-pl=Paragraphs,name-pl=paragraphs,Name-sg-ab=Par.,name-sg-ab=par.,Name-pl-ab=Par.,name-pl-ab=par.,type=appendix,Name-sg=Appendix,name-sg=appendix,Name-pl=Appendices,name-pl=appendices,type=page,Name-sg=Page,name-sg=page,Name-pl=Pages,name-pl=pages,rangesep=–,rangetopair=false,type=line,Name-sg=Line,name-sg=line,Name-pl=Lines,name-pl=lines,type=figure,Name-sg=Figure,name-sg=figure,Name-pl=Figures,name-pl=figures,Name-sg-ab=Fig.,name-sg-ab=fig.,Name-pl-ab=Figs.,name-pl-ab=figs.,type=table,Name-sg=Table,name-sg=table,Name-pl=Tables,name-pl=tables,type=item,Name-sg=Item,name-sg=item,Name-pl=Items,name-pl=items,type=footnote,Name-sg=Footnote,name-sg=footnote,Name-pl=Footnotes,name-pl=footnotes,type=endnote,Name-sg=Note,name-sg=note,Name-pl=Notes,name-pl=notes,type=note,Name-sg=Note,name-sg=note,Name-pl=Notes,name-pl=notes,type=equation,Name-sg=Equation,name-sg=equation,Name-pl=Equations,name-pl=equations,Name-sg-ab=Eq.,name-sg-ab=eq.,Name-pl-ab=Eqs.,name-pl-ab=eqs.,refbounds-first-sg=,(,),,refbounds=(,,,),type=theorem,Name-sg=Theorem,name-sg=theorem,Name-pl=Theorems,name-pl=theorems,type=lemma,Name-sg=Lemma,name-sg=lemma,Name-pl=Lemmas,name-pl=lemmas,type=corollary,Name-sg=Corollary,name-sg=corollary,Name-pl=Corollaries,name-pl=corollaries,type=proposition,Name-sg=Proposition,name-sg=proposition,Name-pl=Propositions,name-pl=propositions,type=definition,Name-sg=Definition,name-sg=definition,Name-pl=Definitions,name-pl=definitions,type=proof,Name-sg=Proof,name-sg=proof,Name-pl=Proofs,name-pl=proofs,type=result,Name-sg=Result,name-sg=result,Name-pl=Results,name-pl=results,type=remark,Name-sg=Remark,name-sg=remark,Name-pl=Remarks,name-pl=remarks,type=example,Name-sg=Example,name-sg=example,Name-pl=Examples,name-pl=examples,type=algorithm,Name-sg=Algorithm,name-sg=algorithm,Name-pl=Algorithms,name-pl=algorithms,type=listing,Name-sg=Listing,name-sg=listing,Name-pl=Listings,name-pl=listings,type=exercise,Name-sg=Exercise,name-sg=exercise,Name-pl=Exercises,name-pl=exercises,type=solution,Name-sg=Solution,name-sg=solution,Name-pl=Solutions,name-pl=solutions keywords Shadow removal, Satellite imagery, Physics-informed, Dataset, GANs, Remote sensing 1 Introduction The abundance of Earth Observation (EO) data in combination with advanced machine learning techniques is currently revolutionising the field of remote sensing. Among the many physical challenges to aerial and satellite-based remote sensing, terrain-induced shadows constitute an impediment to various forms of analysis ( 11 ) . This is particularly true in high-slope environments such as mountains ( 4 ) , but also in cities where man-made structures can cast strong shadows on other regions of interest ( 1 ) . Physics-based methods, grounded in light transport theory, have long provided solutions for shadow detection and removal. Still, those methods are known to produce artifacts that can interfere with downstream processing ( 10 ) . Following recent trends in various fields, deep learning-based methods have been successfully applied to this task ( 2 ) , yielding improvements in both deshadowing and downstream task performance ( 30 ; 27 ) . Existing shadow-aware EO datasets, such as the Aerial Imagery Shadow Detection (AISD) dataset ( 14 ) and the CUHK-Shadow dataset ( 5 ) , are primarily designed for shadow detection, rather than their removal. They rely on manual annotations, which are costly and potentially subjective, and they do not provide true shadow-free references. While the large-scale Shadow-aware Earth Observation (S-EO) dataset ( 16 ) advances detection with geometry-aware, automatically generated masks, it remains detection-oriented and does not provide paired acquisitions suitable for weakly supervised deshadowing. More broadly, current resources seldom control for seasonal shifts, viewing geometry differences, or illumination changes, all of which introduce confounding factors that hinder the training and fair evaluation of shadow-removal models in high-resolution satellite imagery. A further challenge is that, despite recent progress in deep learning for shadow removal, there is currently no public available dataset of geometry-consistent, paired shadowed and shadow-free EO satellite imagery suitable for weakly supervised deshadowing. Existing shadow-removal datasets either focus on ground-level scenes ( 19 ; 25 ; 23 ) , provide paired imagery acquired by unmanned aerial vehicles (UAVs) under controlled conditions ( 13 ) , or adopt unpaired and weakly supervised formulations without true shadow-free references ( 26 ) . None of these resources captures the multi-date, multi-angle satellite imaging geometry, with rational polynomial coefficient (RPC) camera models and digital surface model (DSM) priors, that is required to build reliable cross-view shadow correspondences. Consequently, shadow-removal methods developed for close-range or UAV imagery do not transfer to satellite imagery, where parallax, seasonal variability, and radiometric inconsistencies dominate. This motivates the need for a methodology that can transform detection-oriented EO datasets into paired resources for deshadowing through principled geometric filtering and reproducible data processing. To address these limitations, we introduce deSEO , a data processing methodology that leverages multi-temporal views of a scene to translate shadow-detection datasets into shadow-removal datasets. Minimally shadowed images are selected as proxy references and paired with shadowed counterparts, enabling weakly supervised training despite the technical impossibility of perfectly shadow-free ground truth. The pairing enforces weak supervision under explicit constraints ( e.g. , seasonal proximity, footprint overlap, and view-geometry similarity) to reduce bias from seasonal and illumination variations while preserving sufficient shadow contrast for learning. In doing so, deSEO reframes existing detection-focused resources into datasets engineered for weakly supervised deshadowing in high-resolution EO imagery. Our contributions are threefold, i.e. , i) a geometry-aware pipeline for constructing paired training samples for satellite deshadowing from a multi-acquisition EO shadow detection dataset through two-stage filtering, orientation normalisation, feature-based registration, and validity-aware pairing, ii) the first paired, geometry-consistent dataset for high resolution satellite shadow removal, derived from S-EO using the deSEO pipeline, as well as iii) a deshadowing model and training strategy tailored to high-resolution satellite imagery, which uses registration-driven validity masks to restrict supervision to reliable correspondences and remain robust to residual misalignment. 2 Methodology We propose a preprocessing and training pipeline that makes the S-EO dataset ( 16 ) suitable for weakly supervised deep learning of shadow removal in high-resolution satellite imagery. The pipeline leverages S-EO’s geometry-aware design and multi-temporal, multi-angle acquisitions to create paired samples that support data-driven deshadowing under realistic acquisition variability. In the second part of this study, we propose a single-stage, weakly supervised network to perform deshadowing, leveraging the physical priors incorporated into S-EO. A diagram of the shadow removal dataset creation deSEO is presented in ?? . Input Dataset Metadata Standardisation Match image/mask/metadata triplets with IDs cf. i) Geometry Extraction off-nadir, look azimuth, sun elevation, sun azimuth cf. i) Window Sampling Sample fixed crops per scene cf. ii) Target Selection • For each window, compute clear fraction • Select the least-shadowed acquisition as target cf. iii) Candidate Pairing • Season match • Footprint overlap • View / illumination similarity • Low no-data • Sufficient shadow contrast cf. iv) DSM Association & Pair Record Creation • Link DSM • Save crop coordinates and metadata in YAML IDs cf. v) - vi) Orientation Normalisation Use RPC Jacobian to detect flips/rotations and correct orientation cf. 2.4 Pair-wise Registration • LoFTR matching • Bidirectional affine estimation with RANSAC cf. 2.5 Feature-based Alignment Keep only reliable aligned pixels for supervision cf. 2.5 Scene-level Splits train/validation/test cf. vii) Target/Input Shadow Masks Target/Input Pansharpened crops Target/Input DSM crops Output Dataset Figure 1 : Diagram of deSEO, the proposed framework for shadow removal dataset creation. The processing pipeline uses

Read the complete text