DreamLight: Towards Harmonious and Consistent Image Relighting
Yong Liu, Wenpeng Xiao, Qianqian Wang, Junlin Chen, Shiyin Wang, Yitong Wang, Xinglong Wu, Yansong Tang · 2025
Read DreamLight: Towards Harmonious and Consistent Image Relighting by Yong Liu, Wenpeng Xiao, Qianqian Wang, Junlin Chen, Shiyin Wang, Yitong Wang, Xinglong Wu, Yansong Tang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-14549.txt.
Introduction
Yong Liu and co-authors introduced DreamLight in 2025, a model that relights a subject to fit a new background. A cut-out subject pasted into a scene often clashes in light and color. The new scene can come from a photo or from a text prompt.
Past work leaned on costly scene maps or treated the job as plain image translation, which gave weak light effects between subject and scene. DreamLight puts inputs into one format and uses the sense of a pretrained diffusion model to guide natural results. A light adapter gathers light cues by direction from the background and applies them to the subject with masked attention. A final processing step is also described.
Catalog record
| CREATOR | Yong Liu, Wenpeng Xiao, Qianqian Wang, Junlin Chen, Shiyin Wang, Yitong Wang, Xinglong Wu, Yansong Tang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-14549
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DreamLight: Towards Harmonious and Consistent Image Relighting Yong Liu, Wenpeng Xiao, Qianqian Wang, Junlin Chen, Shiyin Wang, Yitong Wang, Xinglong Wu, Yansong Tang Abstract We introduce a model named DreamLight for universal image relighting in this work, which can seamlessly composite subjects into a new background while maintaining aesthetic uniformity in terms of lighting and color tone. The background can be specified by natural images (image-based relighting) or generated from unlimited text prompts (text-based relighting). Existing studies primarily focus on image-based relighting, while with scant exploration into text-based scenarios. Some works employ intricate disentanglement pipeline designs relying on environment maps to provide relevant information, which grapples with the expensive data cost required for intrinsic decomposition and light source. Other methods take this task as an image translation problem and perform pixel-level transformation with autoencoder architecture. While these methods have achieved decent harmonization effects, they struggle to generate realistic and natural light interaction effects between the foreground and background. To alleviate these challenges, we reorganize the input data into a unified format and leverage the semantic prior provided by the pretrained diffusion model to facilitate the generation of natural results. Moreover, we propose a Position-Guided Light Adapter (PGLA) that condenses light information from different directions in the background into designed light query embeddings, and modulates the foreground with direction-biased masked attention. In addition, we present a post-processing module named Spectral Foreground Fixer (SFF) to adaptively reorganize different frequency components of subject and relighted background, which helps enhance the consistency of foreground appearance. Extensive comparisons and user study demonstrate that our DreamLight achieves remarkable relighting performance. 1 Introduction With the rapid development of generative models in recent years sd ; dit ; diffusion ; ddim , image composition has received increasing attention owing to its capacity for controlled generation anydoor ; composer ; composition3 . However, since the implanted foreground and the new background originate from different sources, this discrepancy can easily lead to an unrealistic perception of the composite image. In this paper, we focus on a challenging task named universal image relighting, which aims to seamlessly composite a subject into a new background while maintaining realism and aesthetic uniformity in terms of lighting and color tone. The background can either be specified by provided natural images (image-based relighting), or be created from unlimited text prompts (text-based relighting). This task ensures that users and digital elements coexist naturally within any environment and has wide application in virtual reality and intelligent editing for the film and advertising industries relightharm ; app2 . Early researches about relighting tend to focus on the image condition, e.g. , image harmonization and portrait relighting, while with scant exploration into text-based scenarios. Portrait relighting methods switchlight ; lightpainter ; holo ; tr mainly take the idea of physics-guided design that explicitly models the image intrinsics and formation physics. They decompose the input images into several components and leverage a phased pipeline to incrementally learn image components such as surface normals and albedo maps. Some harmonization methods Intrinsic ; ih also take similar disentanglement idea. Despite the promising results achieved by these approaches, obtaining training data with pairs of high-quality relighting images and their corresponding intrinsic attributes from light stage is expensive and difficult. Most harmonization methods pct ; inr ; dccf ; cdtnet take this task as an image-to-image translation paradigm and propose to perform pixel-level transformation based on autoencoder architecture. However, they lack object semantic guidance and tend to overlook distinct variations in illumination. Recently, some works graffer ; relightharm ; dilightnet exploit the powerful semantic modeling ability of generative diffusion models sd to enhance relighting effect, but their light source relies on the given environment maps and such maps are not always feasible to acquire in real world. IC-Light iclight is proposed to address both image-based and text-based relighting for natural images without introducing other signal guidance. It imposes light alignment training strategy to enhance the light consistence. However, IC-Light takes two separate models for different conditions and does not tailor the structure. Simply concatenating foreground, background, and noise limits the model’s understanding of foreground-background interactions, leading to severe color bleeding and foreground distortions in some scenarios. To address the above challenges, we present a model named DreamLight that can perform both image-based and text-based relighting. In DreamLight, to generate natural interaction effects of light and color tone between the foreground and the background, we propose a Position-Guided Light Adapter (PGLA), which condenses light information from different directions in the background into several groups of light query embeddings and selectively modulates the foreground area with direction-biased masked attention. In addition, we present an effective Spectral Foreground Fixer (SFF) as a post-processing module to enhance the consistency of foreground appearance and avoid subject distortion. Based on the wavelet transform, SFF is trained to learn dynamic calibration coefficients for high-frequency textures of input foreground and relighted low-frequency light. By adaptively reorganize these information, SFF can output stunningly consistent foreground. Besides, to facilitate the training of our model, we develop different kinds of data generation processes, e.g. , 3D rendering and training relighting lora, to produce diverse training samples. Figure 1: Our DreamLight is a unified image relighting model capable of performing relighting guided by arbitrary text or natural background images without any other prior such as HDR maps. The top left of each set of figures is the original picture of the foreground, the bottom left is the given condition, and the right is the generated result. We have performed extensive quantitative and qualitative comparisons. Experiment results demonstrate that our DreamLight exhibits superior generalization and performance on the universal relighting of natural images. Related ablations also prove the rationality and effectiveness of the proposed designs. Furthermore, we observe that thanks to the unified learning of text and image conditions in a single model, our DreamLight can generate results with the guidance of both conditions. Our contributions can be summarized as follows: • We propose a model named DreamLight for universal image relighting, which can seamlessly composite a subject into a new background with either image or text conditions. We also develop high-quality data generation process to benefit the training of our model. • We introduce a Position-Guided Light Adapter (PGLA), which is designed to enable foreground elements at different positions to interact with background light from various directions in a tendentious manner for generating more natural lighting effects. • We propose a Spectral Foreground Fixer (SFF) module to adaptively reorganize different frequency components of the input and relighted subjects, which helps enhance the consistency of foreground textures. 2 Related Work Image-based Relighting: There are two principal sets of related image-based relighting methods. The first is portrait relighting approaches lightpainter ; switchlight ; holo ; portrait1 ; portrait2 ; portrait3 ; portrait4 ; relightharm ; tr ; composite . They mainly take the idea of physics-guided model design that typically involve the intermediate prediction of surface normals, albedo, and a set of diffuse and specular maps with ground truth supervision. To achieve that, some methods rely on the paired training data acquired with the light stage system lightstage and a target HDR environment map as the external light source. However, the dependence on light stage data and HDR