Vision At Night: Exploring Biologically Inspired Preprocessing For Improved Robustness Via Color And Contrast Transformations
Lorena Stracke, Lia Nimmermann, Shashank Agnihotri, Margret Keuper, Volker Blanz · 2025
Read Vision At Night: Exploring Biologically Inspired Preprocessing For Improved Robustness Via Color And Contrast Transformations by Lorena Stracke, Lia Nimmermann, Shashank Agnihotri, Margret Keuper, Volker Blanz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-24863.txt.
Introduction
Lorena Stracke and co-authors present a 2025 way to help vision models cope with dark or harsh light. Image models beat people on many tests but fail more when light or weather shifts. Most fixes add more training data or change the model, while this team changes the input itself, taking cues from the human eye.
Their fixed first step remaps colour and pulls out local contrast, leaving a sparse map of scene structure. Models trained on city scenes held up better at night and in bad weather with this step. Accuracy stayed steady even when most of the map was cut. The gains point to input design as a cheap path to safer vision.
Catalog record
| CREATOR | Lorena Stracke, Lia Nimmermann, Shashank Agnihotri, Margret Keuper, Volker Blanz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-24863
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Vision At Night: Exploring Biologically Inspired Preprocessing For Improved Robustness Via Color And Contrast Transformations Lorena Stracke, Lia Nimmermann, Shashank Agnihotri, Margret Keuper, Volker Blanz Abstract Inspired by the human visual system’s mechanisms for contrast enhancement and color-opponency, we explore biologically motivated input preprocessing for robust semantic segmentation. By applying Difference-of-Gaussians (DoG) filtering to RGB, grayscale, and opponent-color channels, we enhance local contrast without modifying model architecture or training. Evaluations on Cityscapes, ACDC, and Dark Zurich show that such preprocessing maintains in-distribution performance while improving robustness to adverse conditions like night, fog, and snow. As this processing is model-agnostic and lightweight, it holds potential for integration into imaging pipelines, enabling imaging systems to deliver task-ready, robust inputs for downstream vision models in safety-critical environments. 1 Introduction Neural networks achieve strong performance under standard conditions but remain brittle under distribution shifts [ 29 , 20 , 35 , 37 , 22 , 4 , 3 , 1 , 2 , 7 , 36 , 6 , 5 ] . A particularly challenging case is nighttime perception, where adverse illumination severely degrades segmentation and detection [ 29 , 35 , 37 ] . While training on adverse-condition data may help, available datasets remain limited in scope and annotation quality [ 35 ] , motivating exploration of inductive biases that promote generalization. Unlike artificial systems, the human visual system (HVS) preserves stable perception across lighting conditions [ 15 ] . This robustness arises from early retinal mechanisms that enhance contrast and suppress irrelevant information, notably center-surround receptive fields and color-opponent pathways [ 31 ] . Center-surround filtering amplifies spatial differences via antagonistic receptive fields and can be approximated with a Difference of Gaussians (DoG) function [ 27 ] . In parallel, cone responses are reorganized into opponent channels (red-green, blue-yellow, black-white), where double-opponent cells combine spatial and chromatic antagonism [ 31 , 23 , 39 ] . Orig Image Grayscale Color-opponency Single-color Cityscapes Dark Zurich ACDC Night ACDC Snow Figure 1 : Cropped example images from the in-distribution dataset (Cityscapes [ 13 ] ) and real-world robustness datasets (Dark Zurich [ 34 ] and ACDC [ 35 ] ). Each row shows the original RGB input (‘Orig image’) followed by the DoG preprocessed variants explored in this work: grayscale, color-opponency, and single-color. Single channels also visualized using the color maps shown below including negative values, where white represents 0. Please refer to Fig. 11 for more visualizations and segmentation masks. Inspired by these mechanisms, we investigate biologically motivated preprocessing for semantic segmentation under adverse lighting. As shown in Fig. 1 , our methods transform inputs through grayscale conversion, color opponency, and DoG-based contrast enhancement, without altering model architectures or training procedures. We evaluate these variants on Cityscapes [ 13 ] , Dark Zurich [ 34 ] , and ACDC [ 35 ] , covering both standard and nighttime conditions. Our contributions are: • Exploring DoG-based preprocessing inspired by center-surround & double-opponent mechanisms in the HVS. • Evaluating single-channel, grayscale, and color-opponent contrast variants. • Demonstrating improved robustness to nighttime and weather-induced distribution shifts across multiple architectures and datasets. 2 Related Work Biological Vision As A Model For Robustness. Biologically inspired neural computation has a long tradition in machine learning [ 28 , 14 , 25 ] . Many efforts focus on modeling the early visual system. Neural activations in trained CNNs resemble center-surround receptive fields [ 32 ] . Kim et al. 2016 introduced ON and OFF ReLU units to mimic the behavior of bipolar cells. Babaiee et al. 2021 incorporated center-surround filters and fed them as additional inputs to deeper layers, improving robustness on NORB [ 26 ] . Hasani et al. 2019 applied DoG filters directly to the first activation maps in CNNs, simulating retinal filtering. Tsitiridis et al. 2019 extended this by using double-opponent cells for opponent-channel processing in face biometrics. Building on this, our work integrates DoG-style preprocessing into the input pipeline for segmentation, targeting robustness to nighttime and weather-based lighting shifts. Grayscale Input In Neural Networks. Grayscale input has been explored in various contexts [ 38 ] . Bui et al. 2016 showed that grayscale images can outperform RGB in object recognition under challenging conditions. For facial expression and identity recognition, grayscale performs comparably to RGB [ 10 , 43 ] , although shallow models rely more on color [ 10 ] . RGB remains superior in color-sensitive tasks such as plant disease classification [ 33 ] . Given the dominance of luminance-based perception in the HVS under low-light conditions [ 39 ] , grayscale variants serve as a promising robustness baseline in our study. Robustness To Lighting Variations. Several works have highlighted the vulnerability of neural networks to lighting changes [ 21 ] . Sivaraman and Murthy 2018 demonstrated performance drops under illumination shifts for ImageNet-trained CNNs. Hu et al. 2023 showed that brightness and shadow variations significantly degrade model performance. The large-scale benchmark by Hendrycks and Dietterich 2019 further confirmed that models trained on clean data are sensitive to a wide range of corruptions. Our work complements these by targeting robustness through preprocessing mechanisms that mimic biologically grounded visual filters. Figure 2 : mIoU results of DeepLabv3+ (ResNet-50), SegFormer, and InterImage architectures on Cityscapes, Dark Zurich and ACDC night datasets. All architectures were trained with a depth of 5 using Cityscapes. For other metrics, please refer to Fig. 6 . Figure 3 : mIoU results of DeepLabv3+ (ResNet-50), SegFormer, and InterImage architectures on Cityscapes, Dark Zurich and ACDC night datasets for both grayscale and grayscale green bias variants. All architectures were trained with a depth of 0. For other metrics, please refer to Fig. 7 . 3 Image Preprocessing We explore biologically inspired preprocessing techniques that enhance neural network robustness in adverse lighting and weather conditions, with a focus on nighttime scenarios. We model the HVS’s center-surround and double-opponent retinal cells through two components: (1) color reparameterization using biologically plausible color space transforms and (2) spatial contrast extraction via a DoG-like process . Both are combined into a lightweight, fixed module applied directly before the network. 3.1 Color Reparameterization The color reparameterization is carried out by forming linear combinations of the RGB channels of the original image. These combinations are obtained using Eq. 1 , and are dependent on the 3\times 3 matrix \mathrm{M} : \scriptstyle\begin{bmatrix}\text{channel}_{1}\\ \text{channel}_{2}\\ \text{channel}_{3}\end{bmatrix}=\mathrm{M}\cdot\begin{bmatrix}R\\ G\\ B\end{bmatrix} (1) We examine three types of color reparameterizations: grayscale, color-opponency, and single-color. Grayscale. The grayscale variant is motivated both by human night vision [ 39 ] and findings that show how grayscale images can perform comparably to RGB images (see Section 2 ). For this variant, all coefficients of the corresponding linear combination are set to \frac{1}{3} . In addition, we examine an alternative grayscale variant, referred to as “ Grayscale green bias ”, whose coefficients are: \scriptstyle\mathrm{M}_{\text{grayscale green bias}}=\begin{bmatrix}0.299&0.587&0.114\\ 0.299&0.587&0.114\\ 0.299&0.587&0.114\end{bmatrix}. (2) Grayscale green bias is commonly adopted by many RGB to Grayscale conversion methods [ 17 ] with the intuition that it approximates an average person’s relative perception of the brightness of red, green, and blue light [ 31 ] . Grayscale images can be represented either as a single-channel image or replicated across all three channels. In the case of single-channel, \mathrm{M} is