LightsOut: Diffusion-based Outpainting for Enhanced Lens Flare Removal
Shr-Ruei Tsai Wei-Cheng Chang Jie-Ying Lee Chih-Hai Su Yu-Lun Liu National Yang Ming Chiao Tung University · 2025
Read LightsOut: Diffusion-based Outpainting for Enhanced Lens Flare Removal by Shr-Ruei Tsai Wei-Cheng Chang Jie-Ying Lee Chih-Hai Su Yu-Lun Liu National Yang Ming Chiao Tung University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-15868.txt.
Introduction
Shr-Ruei Tsai, Wei-Cheng Chang, Jie-Ying Lee, Chih-Hai Su, and Yu-Lun Liu improve lens flare removal when the light causing the flare sits outside the frame. Flare degrades images and harms tasks such as object detection and driving vision. Single-image removal methods falter when off-frame sources are missing or incomplete, since the flare pattern depends on them.
Their framework, LightsOut, uses diffusion-based outpainting to rebuild the missing off-frame sources first. A multitask regression module plus a LoRA-tuned diffusion model keep the invented surroundings realistic and physically consistent. Across hard scenes, LightsOut lifts the results of existing removal methods without any retraining of those methods. The five authors present it as a plug-and-play preprocessing step that can sit before any flare remover.
Catalog record
| CREATOR | Shr-Ruei Tsai Wei-Cheng Chang Jie-Ying Lee Chih-Hai Su Yu-Lun Liu National Yang Ming Chiao Tung University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-15868
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LightsOut: Diffusion-based Outpainting for Enhanced Lens Flare Removal Shr-Ruei Tsai Wei-Cheng Chang Jie-Ying Lee Chih-Hai Su Yu-Lun Liu National Yang Ming Chiao Tung University Abstract Lens flare significantly degrades image quality, impacting critical computer vision tasks like object detection and autonomous driving. Recent Single Image Flare Removal (SIFR) methods perform poorly when off-frame light sources are incomplete or absent. We propose LightsOut, a diffusion-based outpainting framework tailored to enhance SIFR by reconstructing off-frame light sources. Our method leverages a multitask regression module and LoRA fine-tuned diffusion model to ensure realistic and physically consistent outpainting results. Comprehensive experiments demonstrate LightsOut consistently boosts the performance of existing SIFR methods across challenging scenarios without additional retraining, serving as a universally applicable plug-and-play preprocessing solution. Project page: https://ray-1026.github.io/lightsout/ Figure 1 : Illustration of our diffusion-based outpainting method. Given an input image with incomplete or missing off-frame light sources, existing Single Image Flare Removal (SIFR) models struggle to effectively remove lens flare artifacts due to incomplete context. Our proposed approach accurately predicts and outpaints the off-frame light sources, allowing subsequent SIFR models to perform significantly better. As demonstrated, integrating our outpainting strategy as a plug-and-play preprocessing step substantially enhances flare removal quality and visual realism. 1 Introduction Lens flare, categorized into reflective flares, scattering flares, and lens orbs (backscatter) Li et al. 2021 ; Wu et al. 2021 ; Hullin et al. 2011 , significantly degrades image quality and negatively impacts computer vision tasks such as object detection and autonomous driving. Traditional flare removal methods Asha et al. 2019 ; Chabert 2014 ; Vitoria and Ballester 2019 relied on handcrafted cues like intensity thresholding or template matching but struggled with complex artifacts. Recent deep learning approaches, such as U-Net Wu et al. 2021 and Uformer Dai et al. 2022 , have achieved substantial improvements due to dedicated datasets. Nevertheless, Single-Image Flare Removal (SIFR) remains challenging, particularly in nighttime scenarios, due to limited real-world paired training data. Recent SIFR advances primarily focus on dataset construction and architectural improvements. Wu et al. Wu et al. 2021 introduced a flare dataset with captured and simulated images. Flare7k Dai et al. 2022 and its enhanced version, Flare7k++ Dai et al. 2024 , provide extensive synthetic and real flare data. Recent methods like Difflare Zhou et al. 2024 and MFDNet Jiang et al. 2024 employ sophisticated architectures, such as diffusion models and multi-scale processing, achieving state-of-the-art performance. Despite these advancements, current SIFR methods still struggle when images lack complete views of off-frame light sources. As illustrated in Fig. 2 , existing methods significantly degrade in such scenarios, resulting in increased flare artifacts and reduced realism. Metrics like PSNR and LPIPS Zhang et al. 2018a confirm that complete light source information is crucial for effective flare removal. To address this, we propose LightsOut , a diffusion-based outpainting method designed for accurate completion of missing off-frame light sources. Our approach integrates a multitask regression module to predict light source parameters precisely, alongside LoRA Hu et al. 2022 fine-tuning of a stable diffusion inpainting model explicitly conditioned on these predictions. This ensures that generated content aligns closely with real-world flare and illumination distributions. As shown in Fig. 1 , our method seamlessly integrates as a plug-and-play preprocessing step with existing SIFR frameworks, significantly enhancing performance in challenging scenarios. Extensive quantitative and qualitative evaluations confirm that LightsOut consistently improves the realism and effectiveness of flare removal across various state-of-the-art methods. Our contributions are summarized as follows: • We identify and address a key limitation of SIFR methods dealing with incomplete off-frame light sources through a specialized decomposition strategy. • We propose a LoRA fine-tuned diffusion-based model that accurately reconstructs physically consistent off-frame light sources and flare artifacts. • We introduce a plug-and-play preprocessing framework that universally enhances existing SIFR models without additional retraining. Figure 2 : Motivation for outpainting incomplete off-frame light sources. (a) With complete off-frame light sources, state-of-the-art SIFR methods effectively remove lens flare artifacts. (b) In scenarios lacking complete views of off-frame sources, these methods degrade significantly, leaving noticeable artifacts. This highlights the importance of complete light source context, motivating our proposed outpainting solution. 2 Related Work Lens Flare Removal. Physical lens modifications Asha et al. 2019 become ineffective with strong light sources Dai et al. 2023 . Early approaches followed a flare detection-then-removal pipeline Asha et al. 2019 ; Chabert 2014 ; Vitoria and Ballester 2019 , focused on veiling glare removal Reinhard et al. 2005 ; Talvala et al. 2007 and reflective flare removal Chabert 2014 ; Asha et al. 2019 ; Vitoria and Ballester 2019 ; Nussberger et al. 2015 ; Nussberger et al. 2015 . Deep learning approaches were limited by paired data scarcity. Wu et al. Wu et al. 2021 developed a dataset with a U-Net-based Ronneberger et al. 2015 approach. Dai et al. Dai et al. 2022 expanded this with physics-based simulation Hullin et al. 2011 ; Lee and Eisemann 2013 , later enhanced as Flare7K++ Dai et al. 2024 . Qiao et al. Qiao et al. 2021 used unpaired data with a CycleGAN-inspired Zhu et al. 2017 framework. Recent architectures like MFDNet Li et al. 2021 and Zhou et al.’s approach Zhou et al. 2022 further improved performance. Despite advances, current models degrade significantly when off-frame light sources are incomplete or absent, which is the limitation our work addresses. Adapting Diffusion Models. Diffusion models Ho et al. 2020 ; Song et al. 2021 ; Nichol and Dhariwal 2021 have been adapted for image generation, editing, and restoration Rombach et al. 2022 ; Nichol et al. 2021 ; Saharia et al. 2022a ; Saharia et al. 2022b ; Yeh et al. 2024 ; Hsiao et al. 2024 ; Chen et al. 2024 ; Wang et al. 2024b ; Chao et al. 2022 . Early applications to inpainting by Sohl-Dickstein et al. Sohl-Dickstein et al. 2015 and Song et al. Song et al. 2021 showed promise. Guided synthesis approaches Choi et al. 2021 ; Meng et al. 2021 offered alternative conditioning strategies but have limitations for outpainting tasks. Several approaches focus on fine-tuning pre-trained diffusion models Ruiz et al. 2023 ; Yang et al. 2023 or text embeddings Gal et al. 2022 ; Voynov et al. 2023 using a single or a few reference object images. Additionally, to improve fine-tuning efficiency, researchers have proposed methods that simplify the adaptation process of diffusion models Han et al. 2023 ; Qiu et al. 2023 ; Hu et al. 2022 . Low-Rank Adaptation (LoRA) Hu et al. 2022 has emerged as an efficient fine-tuning approach. However, existing diffusion models lack physics-based modeling of optical phenomena Hullin et al. 2011 ; Holladay 1926 and struggle with contextual extrapolation beyond image boundaries. Image Completion and Outpainting. Traditional image completion used low-level cues Bertalmio et al. 2000 ; Ballester et al. 2001 ; Bertalmio et al. 2003 ; Hays and Efros 2007 . GAN-based methods Goodfellow et al. 2014 ; Pathak et al. 2016 introduced encoder-decoder architectures with innovations including dilated convolutions Iizuka et al. 2017 ; Yu and Koltun 2015 , partial/gated convolutions Liu et al. 2018 ; Yu et al. 2019 , contextual attention Yu et al. 2018 , edge maps Nazeri et al. 2019 ; Xiong et al. 2019