Zero-Reference Joint Low-Light Enhancement and Deblurring via Visual Autoregressive Modeling with VLM-Derived Modulation
Wei Dong, Han Zhou, Junwei Lin, Jun Chen · 2025
Read Zero-Reference Joint Low-Light Enhancement and Deblurring via Visual Autoregressive Modeling with VLM-Derived Modulation by Wei Dong, Han Zhou, Junwei Lin, Jun Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18591.txt.
Introduction
Wei Dong, Han Zhou, Junwei Lin, and Jun Chen brighten and deblur dark photos with no reference pair to learn from. Real dark shots mix low light, low contrast, noise, and blur at once. Past tools need paired data or miss how light and blur shift across a scene, so they fail to port.
Their frame builds on visual autoregressive modeling, steered by priors from a vision language model. A curve scheme adapts to varied light, guided by visibility scores from that model. New rotary position codes, aware of space and frequency, sharpen the model's grip on blurred structure. A phase domain tweak in a loop cuts artifacts. The zero reference design aims at real world dark shots.
Catalog record
| CREATOR | Wei Dong, Han Zhou, Junwei Lin, Jun Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-18591
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Zero-Reference Joint Low-Light Enhancement and Deblurring via Visual Autoregressive Modeling with VLM-Derived Modulation Wei Dong, Han Zhou, Junwei Lin, Jun Chen Abstract Real-world dark images commonly exhibit not only low visibility and contrast but also complex noise and blur, posing significant restoration challenges. Existing methods often rely on paired data or fail to model dynamic illumination and blur characteristics, leading to poor generalization. To tackle this, we propose a generative framework based on visual autoregressive (VAR) modeling, guided by perceptual priors from the vision-language model (VLM). Specifically, to supply informative conditioning cues for VAR models, we deploy an adaptive curve estimation scheme to modulate the diverse illumination based on VLM-derived visibility scores. In addition, we integrate dynamic and spatial-frequency-aware Rotary Positional Encodings (SF-RoPE) into VAR to enhance its ability to model structures degraded by blur. Furthermore, we propose a recursive phase-domain modulation strategy that mitigates blur-induced artifacts in the phase domain via bounded iterative refinement guided by VLM-assessed blur scores. Our framework is fully unsupervised and achieves state-of-the-art performance on benchmark datasets. Code — https://github.com/LowLevelAI/VAR-LIDE Introduction Noisy and Blurary Low-light Input VAR Output Noisy and Blurry Normal-light Input VAR Output (a) Observations on blurry and noisy inputs with pre-trained VAR models. Extremely Dark Low-light Input Adjustment Iteration n=4 (✗) Adjustment Iteration n=6 (✗) Adjustment Iteration n=8 ( \checkmark ) Low-light Input with Moderate Visibility Adjustment Iteration n=4 (✗) Adjustment Iteration n=6 ( \checkmark ) Adjustment Iteration n=8 (✗) (b) Zero-DCE presents unfavorable enhancement outcomes on low-light inputs with varying illumination conditions using a fixed adjustment iteration. Figure 1: Motivations of our proposed method. (a) We observe that the pre-trained VAR model ( 31 ) exhibits a certain degree of noise suppression and blur reduction, suggesting its potential for joint LLIE and deblurring. However, it struggles to substantially enhance visibility and recover fine structural details, underscoring the need for task-specific modules to better cope with real-world degradations. (b) Although the iterative illumination adjustment is generally effective, Zero-DCE ( 11 ) fails to provide satisfactory results under diverse illumination conditions, often causing under- or over-exposure artifacts due to its reliance on a fixed number of adjustment iterations. The degradation of images captured in real-world dark environments can be formulated as: \mathbf{x}_{LQ}=\gamma f(\mathbf{x}_{HQ},\mathbf{k})+\mathbf{n} , with \mathbf{x}_{HQ} denoting the high-quality (HQ) image and \mathbf{x}_{LQ} its low-quality (LQ) counterpart. Here, \mathbf{n} denotes sensor noise, f is the convolution with the blur kernel \mathbf{k} and \gamma models dynamic range compression and saturation from exposure. Long exposure, a common strategy to improve photon capture, frequently leads to motion-induced blur and elevated noise levels. These combined artifacts degrade image quality and pose significant challenges for both human perception and high-level vision systems ( 39 ) . Similar to other image restoration tasks ( 45 ; 8 ; 7 ; 6 ) , deep learning has led to notable progress in both low-light image enhancement (LLIE) ( 2 ; 15 ; 20 ; 26 ) and deblurring ( 29 ; 12 ; 4 ) , most methods treat them as separate tasks. LLIE models primarily boost brightness and reduce noise but often leave motion blur unaddressed. Conversely, deblurring algorithms typically assume sufficient illumination and perform poorly under low visibility. Although sequential pipelines may appear viable, they often disrupt blur cues during enhancement or fail to recover motion details when visibility is too low. Although recent efforts explore joint LLIE and deblurring, most rely on supervised pipelines that require paired training data ( 10 ; 46 ) , which is difficult to obtain in real-world scenarios. Though diffusion-based unsupervised methods ( 25 ) show promising results, their reliance on lengthy sampling steps ( e.g. , 1,000 iterations) severely limits efficiency, making them unsuitable for practical applications. Recently, visual autoregressive (VAR) ( 33 ) models offer a compelling alternative to diffusion methods by progressively generating high-resolution images via scale-wise token prediction, achieving superior structural fidelity and significantly faster inference without relying on costly iterative denoising. Technically, VAR models effectively preserve bidirectional spatial correlations while aligning with the unidirectional nature of autoregressive modeling, making them particularly suitable for image restoration tasks ( 31 ; 38 ) , where LQ inputs serve as conditioning signals to guide the generation process. As illustrated in Fig. 1 (a), our preliminary experiments indicate that pre-trained VAR models ( 31 ) possess inherent capabilities for noise suppression and partial blur reduction, making them a promising backbone for reference-free joint LLIE and deblurring. On the other hand, we observe that this VAR backbone demonstrates insufficient capacity for illumination enhancement and blur-compensated recovery, motivating the integration of specialized components to address visibility and detail degradation more effectively. An intuitive way to enable VAR with illumination correction capability is to incorporate a lightweight enhancement module prior to the generative process, allowing the model to operate on visibility-improved inputs while leveraging its inherent noise suppression properties. In our preliminary exploration, we adopt Zero-DCE ( 11 ) , a lightweight yet highly effective method for real-world LLIE, where deep neural networks are employed to predict the parameters of a differentiable curve-based model that iteratively adjusts image illumination. Once training converges, we observe that most pixels in the predicted curve parameter maps exhibit positive values, implying that increased iteration steps correspond to stronger illumination enhancement. However, this introduces two critical issues: (i) for moderately bright images, the default setting of 8 iterations tends to cause overexposure; (ii) reducing the iteration number alleviates overexposure but leads to insufficient enhancement on extremely dark images, as illustrated in Fig. 1 (b). These observations indicate that a fixed iteration setting cannot robustly handle the diverse luminance conditions. This motivates further exploration into mechanisms that enable adaptive brightness modulation, ensuring consistent and perceptually compelling enhancement across diverse lighting scenarios. Building upon these observations and insights, we introduce VAR-LIDE , a fully unsupervised generative framework for joint L LI E and DE blurring, which leverages the strengths of VAR models and perceptual guidance from the vision-language model (VLM). To effectively condition the VAR backbone, we develop a VLM-informed conditioning module that predicts adaptive enhancement curves based on VLM-assessed visibility, enabling robust performance under varying lighting conditions. Furthermore, we augment the pre-trained VAR backbone with dynamic and frequency-aware Rotary Positional Encodings to better model spatial structures degraded by motion blur. To mitigate motion-induced repeated edge artifacts in the Fourier phase domain, we introduce a recursive modulation mechanism that progressively refines the phase via a bounded parametric update, guided by blur-related VLM assessments. We summarize our contributions as following: \diamond We introduce VAR-LIDE , a fully unsupervised VAR-based framework that jointly addresses low-light image enhancement and deblurring, leveraging perceptual priors. \diamond Based on perceptual priors derived from VLM, we develop a VLM-informed conditioning module to support informative conditioning for VAR backbone, and design a recursive phase refinement mechanism to suppress blur-induced edge artifacts in the Fourier domain. \diamond We enhance the VAR backbone with content-aware spatial-frequency rotary positional encodings to better capture structural information under blur degradation. \diamond Our VAR-LIDE relaxes the reliance on paired supervision and achieves compelling performance on challenging real-world low-light benchmarks. Related Works Figure 2: The overall framework of our proposed VAR-LIDE method, which adopts the pre-trained VAR model ( 31 ) as the backbone. We first