PhaSR: Generalized Image Shadow Removal with Physically Aligned Priors
Chia-Ming Lee, Yu-Fan Lin, Yu-Jou Hsiao, Jing-Hui Jung, Yu-Lun Liu, Chih-Chung Hsu · 2026
Read PhaSR: Generalized Image Shadow Removal with Physically Aligned Priors by Chia-Ming Lee, Yu-Fan Lin, Yu-Jou Hsiao, Jing-Hui Jung, Yu-Lun Liu, Chih-Chung Hsu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17470.txt.
Introduction
Chia-Ming Lee and five co-authors present PhaSR, a method to remove shadows from images. Shadow removal must split light effects from true surface color. That split fails when physical priors are off line. PhaSR aligns priors at two levels. It aims for robust work from one light source to many. Ambient light from mixed sources is the hard end.
Aligned priors keep colors true once shadows lift. Bad removal can bleach or tint a scene. Good removal aids photo tools, robots, and self-driving sight. The dual-level design targets the root cause, not just the look. It gives vision systems cleaner inputs under real light.
Catalog record
| CREATOR | Chia-Ming Lee, Yu-Fan Lin, Yu-Jou Hsiao, Jing-Hui Jung, Yu-Lun Liu, Chih-Chung Hsu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-17470
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PhaSR: Generalized Image Shadow Removal with Physically Aligned Priors Chia-Ming Lee, Yu-Fan Lin, Yu-Jou Hsiao, Jing-Hui Jung, Yu-Lun Liu, Chih-Chung Hsu Abstract Shadow removal under diverse lighting conditions requires disentangling illumination from intrinsic reflectance—a challenge compounded when physical priors are not properly aligned. We propose PhaSR ( Physically Aligned Shadow Removal ), addressing this through dual-level prior alignment to enable robust performance from single-light shadows to multi-source ambient lighting. First, Physically Aligned Normalization (PAN) performs closed-form illumination correction via Gray-world normalization, log-domain Retinex decomposition, and dynamic range recombination, suppressing chromatic bias. Second, Geometric-Semantic Rectification Attention (GSRA) extends differential attention to cross-modal alignment, harmonizing depth-derived geometry with DINO-v2 semantic embeddings to resolve modal conflicts under varying illumination. Experiments show competitive performance in shadow removal with lower complexity and generalization to ambient lighting where traditional methods fail under multi-source illumination. Our source code is available at https://github.com/ming053l/PhaSR . 1 Introduction Shadows, as natural consequences of light-object interactions, are ubiquitous optical phenomena that profoundly impact multimedia content analysis, degrading performance in tasks ranging from remote sensing 15 , segmentation 25 , tracking 42 , and 3D reconstruction 51 ; 1 to multimedia applications 36 . Removing shadows from images is not only a fundamental computer vision task but also critical for enhancing downstream application performance 5 ; 35 ; 45 . The core challenge lies in accurately distinguishing shadows from intrinsic object darkness and leveraging contextual information to perform physically plausible color correction and content restoration within shadowed areas 22 ; 56 . Figure 1 : Results on indoor-synthesized dataset 53 . Compared with OmniSR 53 and DenseSR 28 , PhaSR with the proposed GSRA achieves more accurate boundary localization and recovers fine reflectance details even under complex indirect illumination. Despite progress in learning-based shadow removal, key challenges persist. First , shadows are easily confused with intrinsic material properties when relying solely on RGB cues, causing color distortion near textured boundaries. Second , while existing methods achieve strong performance on single-light direct shadow benchmarks, emerging applications demand generalization to more complex scenarios—indoor ambient lighting with multiple sources, color shifts, and diffuse indirect illumination—where prior works show limited robustness (Figure 1 ). Third , conventional encoder-decoder frameworks fail to effectively propagate physical priors, as uniform fusion overlooks spatially varying degradation, resulting in blurred edges. As shown in Figure 2 , prior methods lose physical priors through the bottleneck, whereas our framework preserves them under complex illumination and details recovery. These challenges stem from prior misalignment . While physical-guided feature transformation 6 ; 52 ; 8 ; 31 and explicit geometric-semantic prior integration 53 ; 28 ; 54 ; 19 enhance robustness, these priors encode conflicting signals: geometric features respond to local shading variations, while semantic features remain stable across lighting. Without proper alignment, geometric noise disrupts semantic consistency, or semantic over-smoothing erases illumination boundaries—particularly problematic for indirect lighting where geometric and semantic cues must cooperate to disentangle ambient effects from surface properties. We propose PhaSR — Physically Aligned Shadow Removal —addressing prior misalignment through two mechanisms. Physically Aligned Normalization (PAN) performs model-free preprocessing via Gray-world normalization, log-domain Retinex decomposition, and dynamic range recombination, suppressing global chromatic bias. Geometric-Semantic Rectification Attention (GSRA) extends differential attention 58 to cross-modal alignment: computing \mathbf{A}_{\text{rect}}=\mathbf{A}_{\text{sem}}-\lambda\cdot\mathbf{A}_{\text{geo}} across depth-derived geometry (DepthAnything-v2 55 ) and semantic embeddings (DINO-v2 37 ) to harmonize local geometric precision with global semantic stability. This explicit alignment enables accurate interpretation of both direct shadows (geometric-dominant) and ambient lighting (semantic-guided), generalizing from single-light to multi-source illumination scenarios. In summary, the main contributions of this work are threefold: • We introduce Physically Aligned Normalization, a closed-form preprocessing module performing Gray-world normalization, log-domain Retinex decomposition, and dynamic range recombination to suppress chromatic bias. PAN consistently improves existing architectures by 0.15–0.34 dB across diverse lighting conditions. • We propose geometric-semantic rectification attention, extending differential attention to cross-modal prior alignment. By explicitly incorporating depth-derived geometry and DINO-v2 semantic embeddings, GSRA harmonizes physically grounded geometric precision with semantic stability, addressing modal misalignment challenges in ambient lighting normalization. • We demonstrate state-of-the-art performance on challenging shadow removal benchmarks, achieving robust generalization from outdoor direct shadows to indoor indirect and ambient lighting scenarios while maintaining computational efficiency. Figure 2 : Intermediate feature visualization. Existing methods struggle to leverage physical priors without valid shadow masks under complex environmental lighting. In contrast, our PhaSR precisely highlights and restores shadow regions in both bottleneck and decoder stages, demonstrating strong generalization. 2 Related Work Single Image Shadow Removal Single-image shadow removal aims to recover the true appearance beneath shadows. Early traditional methods followed a two-stage pipeline—shadow detection and removal—based on handcrafted features and physical or statistical illumination models 44 ; 60 ; 3 ; 41 ; 23 , but relied on strong priors that struggled with complex lighting and soft shadows. Deep learning has significantly advanced the field: CNNs 40 ; 38 capture multi-scale features but face locality limits; Transformer-based methods 9 ; 52 ; 29 offer better global context, though some rely on explicit shadow masks 9 ; 52 ; Diffusion-based models 10 ; 34 ; 54 achieve high quality at significant computational cost. DeS3 19 pioneered using pretrained DINO 37 priors with diffusion for shadow removal, while RRLNet 31 applied Retinex decomposition to guide diffusion-based texture refinement 7 . OmniSR 53 introduced a synthesized dataset and semantic-geometric aware network for both direct and indirect shadows. DenseSR 28 reframes shadow removal as dense prediction, leveraging geometric-semantic priors and adaptive fusion to overcome ambiguity and boundary blurring. ShadowHack 16 divides shadow removal into luminance recovery and color restoration using rectified outreach attention. Ambient Light Normalization. Beyond conventional shadow removal, recent work has explored the more challenging task of ambient light normalization (ALN) 47 ; 48 , which addresses complex real-world scenarios with multiple light sources, color shifts, and diffuse indirect illumination. ReHiT 6 achieves efficient mask-free shadow removal through Retinex-guided dual-branch decomposition, while IFBlend 47 and RLN2-Lf 48 extend this framework to multi-source white and RGB color lighting respectively. However, the ill-posed nature of disentangling multiple overlapping light contributions remains an open challenge. Our method, while primarily designed for shadow removal, demonstrates strong generalization to ALN scenarios through physically aligned normalization that handles diverse illumination conditions. Figure 3 : Overview of PhaSR: Physically Aligned Shadow Removal. PhaSR achieves physical alignment through two synergistic stages. Stage 1 (Sec. 3.1 ) : PAN performs model-free illumination normalization via Gray-world color correction, log-domain Retinex decomposition ( \log\mathbf{I}=\log\mathbf{R}+\log\mathbf{S} ), and dynamic range recombination, suppressing chromatic bias while preserving reflectance cues. Stage 2 (Sec. 3.2 ) : The multi-scale Transformer encoder-decoder integrates explicit physical priors—frozen DINO-v2 37 semantic embeddings at encoder stages and DepthAnything-v2 55 geometric priors (depth, normals) at the bottleneck—aligned through GSRA’s cross-modal differential attention ( \mathbf{A}_{\text{rect}}=\mathbf{A}_{\text{sem}}-\lambda\cdot\mathbf{A}_{\text{geo}} ). This dual-stage physical alignment—global illumination correction followed by local geometric-semantic rectification—enables robust reflectance recovery under complex lighting without requiring shadow masks. Intrinsic Decomposition and Physical Priors. The formation of shadows originates from the fundamental physics of light transport being occluded by 3D scene geometry 32 ; 27 ; 13 ; 57 . Occlusion of direct illumination leads to sharp, well-defined shadows, while indirect illumination—arising from interreflections and ambient scattering—produces soft, graded shadows that are more challenging to model and recover accurately 53 . From