CFSR: Geometry-Conditioned Shadow Removal via Physical Disentanglement CCS: Computing methodologies Computer vision tasks; Image representations; Appearance and texture representations

Pan Wang, Yihao Hu, Xiujin Liu, Hang Wang · 2026

Read CFSR: Geometry-Conditioned Shadow Removal via Physical Disentanglement CCS: Computing methodologies Computer vision tasks; Image representations; Appearance and texture representations by Pan Wang, Yihao Hu, Xiujin Liu, Hang Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18032.txt.

Introduction

Pan Wang and three colleagues present CFSR, a shadow removal framework that treats restoration as a physics-constrained process. It maps observations into an HVI color space to suppress shadow noise and fuses RGB data with depth priors. A dual guided attention mechanism uses DINO features and surface normals to enforce lighting constraints, while a frozen CLIP encoder adds priors for degraded regions. A frequency collaborative module then harmonizes occlusion boundaries with illumination, reaching state-of-the-art results across benchmarks.

Catalog record

CREATORPan Wang, Yihao Hu, Xiujin Liu, Hang Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43837
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.18032

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CFSR: Geometry-Conditioned Shadow Removal via Physical Disentanglement CCS: Computing methodologies Computer vision tasks; Image representations; Appearance and texture representations Pan Wang, Yihao Hu, Xiujin Liu, Hang Wang Abstract . Traditional shadow removal networks often treat image restoration as an unconstrained mapping, lacking the physical interpretability required to balance localized texture recovery with global illumination consistency. To address this, we propose CFSR, a multi-modal prior-driven framework that reframes shadow removal as a physics-constrained restoration process. By seamlessly integrating 3D geometric cues with large-scale foundation model semantics, CFSR effectively bridges the 2D-3D domain gap. Specifically, we first map observations into a custom HVI color space to suppress shadow-induced noise and robustly fuse RGB data with estimated depth priors. At its core, our Geometric & Semantic Dual Explicit Guided Attention mechanism utilizes DINO features and 3D surface normals to directly modulate the attention affinity matrix, structurally enforcing physical lighting constraints. To recover severely degraded regions, we inject holistic priors via a frozen CLIP encoder. Finally, our Frequency Collaborative Reconstruction Module (FCRM) achieves an optimal synthesis by decoupling the decoding process. Conditioned on geometric priors, FCRM seamlessly harmonizes the reconstruction of sharp high-frequency occlusion boundaries with the restoration of low-frequency global illumination. Extensive experiments demonstrate that CFSR achieves state-of-the-art performance across multiple challenging benchmarks. 1. Introduction Figure 1. Conceptual comparison of shadow removal paradigms. Unlike existing deterministic mapping approaches (red dashed line) that suffer from out-of-distribution artifacts, our method reframes shadow removal as a physics-constrained degradation inversion (green solid line), ensuring the restored image strictly adheres to the latent clean image manifold \mathcal{M}_{\text{clean}} . Shadow removal is a fundamental yet challenging task in computer vision, serving as a crucial pre-processing step for high-level downstream applications such as object detection ( 39 ; 18 ; 44 ) , tracking ( 1 ) , and autonomous driving ( 45 ) . While recent deep learning-based approaches have achieved remarkable progress, the majority of existing networks formulate this task as a deterministic ’image-to-image’ mapping problem. This ’black-box’ paradigm inherently lacks the physical interpretability required to balance spatial detail recovery with global illumination consistency. Operating primarily within the 2D RGB domain, these methods remain oblivious to the underlying 3D physical geometry ( 33 ; 34 ; 11 ; 23 ) . Consequently, they struggle with the ill-posed nature of the task, frequently failing to distinguish between intrinsic dark textures and actual shadows cast by complex geometries. Fundamentally, shadow generation is not merely a localized darkening effect, but rather a geometry-dependent physical phenomenon driven by intricate light absorption and scattering processes ( 9 ) . Existing architectures lack the physics-aware framework necessary to invert these degradations, particularly in the frequency domain. Due to the absence of physics-constrained inversion logic, they often compromise global lighting consistency for local details, resulting in conspicuous visual artifacts ( 25 ) . To break this bottleneck, we argue that shadow removal must be reframed from a deterministic blind pixel regression to a physics-constrained degradation inversion problem . As illustrated in Fig. 1 , rather than adopting a deterministic mapping that easily falls into out-of-distribution states, our ultimate goal is to mathematically reverse the complex light-matter interaction. This paradigm ensures that the restored image adheres strictly to the physical manifold of the latent clean scene ( \mathcal{M}_{\text{clean}} ). Such a physics-guided framework effectively mitigates the global-local trade-off bottleneck. As demonstrated by the spectral analysis in Fig. 2 , compared to existing methods, our degradation inversion strictly reverses the degradation, thereby preserving the structural integrity in the spatial domain and significantly reducing the spectral discrepancy ( |\text{FFT}-\text{GT}| ) in the frequency domain. Figure 2. Visual and spectral comparison against SOTA ( 29 ) . From top to bottom: restored images, corresponding 2D FFT magnitude spectra, and absolute spectral discrepancy ( |\text{FFT}-\text{GT}| , brighter red indicates larger errors). Our physics-constrained inversion paradigm preserves structural integrity and yields the minimal spectral error. To this end, we propose CFSR , a multi-modal prior-driven shadow removal framework that bridges the 2D-3D domain gap by transforming conventional image-to-image mapping into a physics-constrained restoration process . By seamlessly integrating estimated 3D geometric priors with large-scale foundation model semantics, CFSR effectively overcomes the inherent ill-posed limitations of single-modal reconstruction. Specifically, we first employ a pre-trained monocular depth estimator to derive explicit 3D geometric cues. To robustly fuse these cross-domain modalities, we introduce a custom HVI color space equipped with a learnable Intensity Collapse mechanism . This design dynamically suppresses severe noise in deeply shadowed regions while constructing a reliable multi-modal feature base. At the core of our architecture lies the Geometric & Semantic Dual Explicit Guided Attention (GSGA) . Diverging from conventional implicit feature learning, this module utilizes semantic correlation maps from a pre-trained DINO ( 36 ) and planar correlation maps derived from 3D surface normals to explicitly modulate the attention affinity matrix ( 49 ) . This structurally enforces physical lighting constraints during representation aggregation. Furthermore, to combat severe information loss in dense shadows, we leverage a frozen CLIP visual encoder at the deep bottleneck ( 38 ) . By injecting globally robust semantic priors via cross-modal attention, the network is guided to hallucinate and recover lost textures with holistic scene awareness. Finally, the decoding phase is driven by the Frequency Collaborative Reconstruction Module (FCRM) . Instead of monolithic regression, FCRM elegantly decouples high-frequency texture refinement from low-frequency illumination smoothing under continuous 3D geometric conditioning, achieving a mathematically rigorous balance between high-fidelity structural preservation and global color consistency. In summary, the main contributions of this work are three-fold: • Physics-Aware Formulation & Representation: We fundamentally advance shadow removal from an unconstrained 2D mapping task to a physics-conditioned restoration framework. By introducing a custom HVI space with an Intensity Collapse mechanism, we enable the dynamic suppression of shadow-induced noise and the seamless fusion of degraded RGB observations with explicit 3D geometric priors. • Geometric & Semantic Guided Extraction: We propose the Dual Explicit Guided Attention mechanism. By aligning dense 2D semantics (DINO) with spatial 3D surface normals, we explicitly inject physical lighting and structural constraints directly into the attention affinity matrix, effectively bridging the 2D-3D modality gap. • Geometry-Conditioned Collaborative Decoding: We design the Frequency Collaborative Reconstruction Module (FCRM) to disentangle the synthesis process into specialized high- and low-frequency branches. Coupled with a CLIP-guided global semantic injection, this module explicitly incorporates 3D planar constraints to resolve geometry-dependent illumination degradation, ensuring an optimal balance between photorealistic texture recovery and physical consistency. 2. Related Work 2.1. Image Shadow Removal Shadow removal has been extensively studied due to its critical role in enhancing scene understanding and downstream vision tasks. Early traditional methods primarily relied on physical illumination models and hand-crafted priors, such as gradient domain manipulation ( 30 ) , illumination transfer ( 50 ) , and region-based color matching ( 14 ) . While theoretically sound, these methods often fail in complex real-world scenes due to the fragility of hand-crafted features. With the advent of deep learning, convolutional neural networks (CNNs) and Generative Adversarial Networks (GANs) have dominated this field. Representative works ( 43 ; 17 ; 19 ) formulate shadow removal as a deterministic image-to-image translation problem, employing various attention mechanisms and multi-scale feature fusions to map shadowed inputs to shadow-free outputs. Recently,

Read the complete text