Multi-illuminant Color Constancy via Multi-scale Illuminant Estimation and Fusion

Hang Luo, Rongwei Li, Jinxing Liang · 2025

Read Multi-illuminant Color Constancy via Multi-scale Illuminant Estimation and Fusion by Hang Luo, Rongwei Li, Jinxing Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-02021.txt.

Introduction

A photo taken under mixed lighting, say daylight plus a warm lamp, carries uneven color tints that cameras struggle to remove. Hang Luo, Rongwei Li, and Jinxing Liang tackle that clean-up task in this 2025 paper on multi-illuminant color constancy. The goal is to estimate the light color at every pixel and undo its cast.

Older learning methods mapped a photo straight to its lighting map, ignoring the photo's scale. The authors instead build the map as a blend of parts read at several scales. A three-branch network reads fine, medium, and coarse views, and an attention module merges the three lighting maps, weighting each as needed. In their tests the blend beat prior methods and set a new best score. The work underlines that lighting judgment needs both detail and the broad scene.

Catalog record

CREATORHang Luo, Rongwei Li, Jinxing Liang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-02021

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-illuminant Color Constancy via Multi-scale Illuminant Estimation and Fusion Hang Luo, Rongwei Li, Jinxing Liang Abstract Multi-illuminant color constancy methods aim to eliminate local color casts within an image through pixel-wise illuminant estimation. Existing methods mainly employ deep learning to establish a direct mapping between an image and its illumination map, which neglects the impact of image scales. To alleviate this problem, we represent an illuminant map as the linear combination of components estimated from multi-scale images. Furthermore, we propose a tri-branch convolution networks to estimate multi-grained illuminant distribution maps from multi-scale images. These multi-grained illuminant maps are merged adaptively with an attentional illuminant fusion module. Through comprehensive experimental analysis and evaluation, the results demonstrate the effectiveness of our method, and it has achieved state-of-the-art performance. a]School of Computer Science and Artificial Intelligence, Wuhan Textile University, No.1 Sunshine Avenue, Jiangxia District, Wuhan, Hubei Province, China keywords Color constancy, U-Net, Convolutional networks † † authorinfo: Corresponding author: Jinxing Liang ( [email protected] ) 1 Introduction The visual system of human beings can maintain relatively stable color perception even under varying illuminations, and this ability is called color constancy [ 1 ] . However, cameras inherently don’t possess this ability, which results in reddish or blueish color casts in an image. Color constancy method aims to mimic the visual color constancy of human beings to discount the color cast within an image. It is typically used to fulfill the built-in auto white balance step in a camera [ 2 ] , which can significantly enhance image quality and improve the illuminant robustness of downstream vision tasks [ 3 ] . The mechanism underlying visual color constancy is not yet fully understood. To simplify the problem, existing works approximately simulate the visual color constancy by transforming an image from arbitrary illuminants to canonical white light using von Kries chromatic adaptation transform [ 4 ] . Specifically, it involves two critical steps: (1) estimating the chromaticity of the illuminant, i.e. the illuminant color, and (2) correcting image color with diagonal transform. The key problem is how to accurately estimate the illuminant color, and existing works are dedicated to solve this problem. A large number of studies further simplify the problem by assuming that there is only one illuminant in the scene, so they are called single-illuminant color constancy method. Classical single-illuminant methods impose additional assumptions on the scene and use image statistics for illuminant estimation [ 5 , 6 , 7 , 8 , 9 , 10 , 11 ] . Another line of research has incorporated machine learning techniques into the field, such as neural networks [ 12 ] , k-nearest neighbor [ 13 ] and Bayesian estimation [ 14 ] . In recent years, deep learning has revolutionized the field, enabling diverse methods based on convolutional neural networks [ 15 , 16 , 17 , 18 , 19 , 20 ] , contrastive learning [ 21 ] , transfer learning [ 22 ] , etc. Despite these advancements, single-illuminant methods face an inevitable limitation that their assumption is rarely satisfied in practice. Natural scenes typically contain multiple illuminants and single-illuminant methods cannot simultaneously correct all local color casts brought by distinct illuminants. Multi-illuminant color constancy methods relax the single-illuminant assumption and aim to estimate the illuminant pixel-wisely, resulting in an illuminant map. An intuitive idea is segmenting an image into regions with distinct illuminants and estimating the illuminant for each region [ 23 , 24 , 25 , 26 , 27 ] , but these methods are limited to coarse-grained local illuminant estimation. In recent years, owing to the emergence of large scale multi-illuminant datasets [ 28 ] , many methods have used deep learning to achieve fine-grained pixel-wise illuminant estimation [ 29 , 30 , 31 , 32 , 33 ] . We can suppose that the illuminant distribution is more uniform at small scales and more diverse in large scales. However, current methods have overlooked the scale-dependent variation of illuminant distribution, consequently limiting their ability to effectively capture diverse illuminant distribution features for precise pixel-wise illuminant estimation. In this paper, we investigate the impact of image scales and propose a coarse-fine-decomposed framework for pixel-wise illuminant estimation. Specifically, we regard an illuminant map as the linear combination of multi-grained components and build a convolutional framework with three branches, each using an U-Net [ 34 ] to estimate the illuminant map from an image of a certain scale. Since larger images provide finer details, these branches can capture multi-grained illuminant distribution features, resulting in multi-grained illuminant maps. At the final stage, an attentional illuminant fusion module adaptively integrates these multi-grained illuminant maps by generating pixel-wise weight maps and performing a linear combination. Our contributions are summarized as follows: (1) We propose that an illuminant map can be decomposed into a group of multi-grained components and propose a framework to estimate multi-grained illuminant maps from multi-scale images; (2) To adaptively identify and enhance the most relevant illuminant for each pixel, we construct an attentional illuminant fusion module to automatically assign pixel-wise weights for these illuminant maps estimated from multi-scale images; (3) We conduct extensive experiments to validate and analyze our method, and the experimental results demonstrate the effectiveness of our method. 2 Related works 2.1 Single-illuminant Color Constancy Existing single-illuminant color constancy methods can be roughly categorized into statistics-based and learning-based methods. Statistics-based methods use the statistics of an image as the illuminant color. Gray World (GW) [ 5 ] , one of the most well-known statistics-based methods, assumes that the average reflectance of a scene is gray and employs the mean value of an image as estimation. White Patch (WP) [ 6 ] supposes that there is a white patch in the scene and uses the max value of each channel as the illuminant color. Shade of Gray (SoG) [ 7 ] uses the p-norm of an image as estimation. Gray Edge (GE) [ 8 ] extends SoG to the gradient domain. Weighted GE [ 9 ] further takes into account the diversity of image edges and proposes an adaptive weighting scheme. Cheng et al. [ 35 ] propose to use the first principal component of bright and dark pixels as the estimation. Since gray pixels can directly indicate the illuminant color, Yang et al. [ 10 ] and Qian et al. [ 11 ] respectively propose a method for gray pixel detection. Statistics-based methods are efficient and interpretable, but their generalization ability is limited due to their inherent assumptions. Learning-based methods estimate the illuminant color in an data-driven manner. Cardei et al. [ 12 ] build the mapping between the binarized chromatic histogram and the illuminant color using neural networks. Gijsenij et al. [ 13 ] employs k-nearest neighbor clustering in the Weibull parameter space to determine the most appropriate statistics-based method for an image. Gehler et al. [ 14 ] use Bayesian method for illuminant estimation. Nowadays, deep learning for illuminant estimation is the main direction. Bianon et al. [ 15 ] build the first convolutional neural networks for illuminant estimation. Barron et al. [ 16 ] reduce the regression problem to a problem of locating the illuminant in the log-chroma space. Hu et al. [ 17 ] build a fully

Read the complete text