GramSR: Visual Feature Conditioning for Diffusion-Based Super-Resolution
Fabio D’Oronzio, Federico Putamorsi, Leonardo Zini, Marcella Cornia, Lorenzo Baraldi · 2026
Read GramSR: Visual Feature Conditioning for Diffusion-Based Super-Resolution by Fabio D’Oronzio, Federico Putamorsi, Leonardo Zini, Marcella Cornia, Lorenzo Baraldi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25457.txt.
Introduction
D'Oronzio and four colleagues put forth GramSR, a one-step diffusion super-resolution frame that swaps text conditioning for visual traits from the low-resolution input, pulled by DINOv3. Text captions give only high-level meaning and lack spatially matched detail. GramSR trains three LoRA modules in order: pixel-level for degradation removal, semantic-level for perceptual detail, and texture-level for trait correlation consistency. At run time, guidance scales control degradation removal, semantic gain, and texture keeping. Tests on standard benchmarks show GramSR beats past one-step diffusion ways.
Catalog record
| CREATOR | Fabio D’Oronzio, Federico Putamorsi, Leonardo Zini, Marcella Cornia, Lorenzo Baraldi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36326 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.25457
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GramSR: Visual Feature Conditioning for Diffusion-Based Super-Resolution Fabio D’Oronzio, Federico Putamorsi, Leonardo Zini, Marcella Cornia, Lorenzo Baraldi Abstract Despite recent advances, single-image super-resolution (SR) remains challenging, especially in real-world scenarios with complex degradations. Diffusion-based SR methods, particularly those built on Stable Diffusion, leverage strong generative priors but commonly rely on text conditioning derived from semantic captioning. Such textual descriptions provide only high-level semantics and lack the spatially aligned visual information required for faithful restoration, leading to a representation gap between abstract semantics and spatially aligned visual details. To address this limitation, we propose GramSR, a one-step diffusion-based SR framework that replaces text conditioning with dense visual features extracted from the low-resolution input using a pre-trained DINOv3 encoder. GramSR adopts a three-stage LoRA architecture, where pixel-level, semantic-level, and texture-level LoRA modules are trained sequentially. The pixel-level module focuses on degradation removal using ℓ 2 \ell_{2} loss, the semantic-level module enhances perceptual details via LPIPS and CSD losses, and the texture-level module enforces feature correlation consistency through a Gram matrix loss computed from DINOv3 features. At inference, independent guidance scales enable flexible control over degradation removal, semantic enhancement, and texture preservation. Extensive experiments on standard SR benchmarks demonstrate that GramSR consistently outperforms existing one-step diffusion-based methods, achieving superior structural fidelity and texture realism. The code for this work is available at: https://github.com/aimagelab/GramSR . 1 Introduction Single-image super-resolution (SR) aims to recover high-resolution images from their low-resolution counterparts, remaining a fundamental problem in computer vision. Since SRCNN [ 9 ] , deep learning-based SR methods have primarily optimized pixel-level fidelity metrics such as PSNR and SSIM [ 36 ] under simplified degradation assumptions. Subsequent works explored deeper and more expressive architectures [ 20 , 30 ] , but these approaches often fail to generalize to real-world scenarios involving complex degradations. This limitation has motivated research on realistic degradation modeling [ 4 , 37 ] , perceptual losses [ 13 ] , and GAN-based training [ 14 , 34 , 47 ] . Despite improved visual realism, adversarial methods are prone to training instability and artifact generation [ 19 , 41 ] . Diffusion models [ 7 , 26 ] have emerged as a powerful alternative, offering expressive generative priors and stable optimization. Large-scale text-to-image models, particularly Stable Diffusion [ 24 ] , demonstrate remarkable ability in generating semantically rich details, motivating their use in SR [ 33 , 40 , 42 , 43 ] . Further efforts focus on improving efficiency through paired data training [ 45 ] , degradation-aware modeling [ 15 ] , flexible control mechanisms [ 5 , 32 ] , and quality-aware optimization [ 39 ] . However, multi-step diffusion sampling incurs substantial computational cost, motivating one-step approaches [ 2 , 6 , 27 , 29 , 35 , 38 ] that achieve real-time performance via distillation and architectural simplification. A key limitation of existing diffusion-based SR methods lies in their reliance on text conditioning [ 29 , 38 , 40 ] . While effective for text-to-image generation, textual descriptions obtained from captioning models [ 16 , 17 ] or tagging modules [ 40 ] encode only high-level semantic concepts, such as object categories and scene context. They lack spatially aligned structural cues, fine-grained geometry, and local texture patterns that are critical for faithful image restoration. This modality mismatch introduces a semantic gap between abstract linguistic representations and the precise visual guidance required for SR. Moreover, current training objectives mainly emphasize pixel-level fidelity and semantic consistency [ 29 ] , without explicitly modeling texture statistics characterized by repeated patterns and feature correlations. To address these challenges, we propose GramSR , a one-step diffusion-based SR framework with two key innovations. First, we replace text conditioning with dense visual features extracted directly from the low-resolution input using a pre-trained DINOv3 encoder [ 25 ] . This provides spatially aligned, hierarchical visual representations that capture both local details and global context. Second, we extend prior dual-LoRA frameworks [ 29 , 38 ] with a third, texture-level LoRA module optimized via Gram matrix loss to preserve second-order feature statistics. While the pixel-level and semantic-level LoRAs address degradation removal and perceptual detail generation, respectively, the texture-level LoRA enforces feature correlation consistency by aligning Gram matrices between the super-resolved output and the ground truth. This sequential training strategy disentangles complementary restoration objectives, while inference-time guidance scales enable independent control over degradation removal, semantic enhancement, and texture preservation. Extensive experiments on standard SR benchmarks demonstrate that GramSR consistently outperforms existing one-step diffusion-based methods in both quantitative metrics and perceptual quality, achieving notably improved texture fidelity and structural consistency. 2 Related Work Early Super-Resolution Methods. Since the introduction of SRCNN [ 9 ] , deep learning-based methods have become the dominant paradigm for SR. Early approaches mainly optimized pixel-level fidelity metrics [ 36 ] under simplified and known degradations ( e.g. , bicubic downsampling). Later works explored more advanced architectures with improved connectivity, hierarchical representations, and global context modeling [ 30 , 20 ] . Despite strong performance on synthetic benchmarks, these methods often struggle to generalize to real-world low-quality (LQ) images with complex and unknown degradations. To address this limitation, real-world SR has been studied through the collection of paired LQ-HQ datasets [ 4 , 37 ] or the synthesis of more realistic training data via degradation modeling. With the rise of generative approaches, perceptual losses [ 13 ] and GAN-based training [ 14 ] have been widely adopted to enhance visual realism. Representative methods such as BSRGAN [ 47 ] and Real-ESRGAN [ 34 ] employ complex, randomized degradation pipelines to generate realistic LQ-HQ pairs. However, adversarial training is often unstable and prone to artifacts, motivating subsequent efforts to improve robustness and visual quality [ 19 , 41 ] . Diffusion-based Super-Resolution Methods. Diffusion models (DMs) have recently gained attention for SR due to their strong generative priors and stable training dynamics. Early DM-based methods adapt denoising diffusion probabilistic models (DDPMs) [ 7 , 26 ] via gradient guidance, enabling training-free restoration under simple degradations, but showing limited robustness to real-world degradations. More recent approaches leverage large-scale text-to-image models, particularly Stable Diffusion [ 24 ] , as powerful semantic priors. Representative methods include StableSR [ 33 ] , PASD [ 42 ] , and SeeSR [ 40 ] , which introduce different conditioning and feature-integration strategies to improve robustness and detail generation. Other works explore truncating the diffusion process or enhancing semantic consistency, such as CCSR [ 28 ] and SUPIR [ 43 ] . A major drawback of diffusion-based SR is the high inference cost caused by multi-step sampling. To improve efficiency, recent methods investigate alternative training and inference strategies, including paired-data training [ 45 ] , joint degradation estimation [ 15 ] , and flexible control mechanisms [ 5 , 32 ] . Additional efforts incorporate perceptual quality optimization into diffusion training [ 39 ] . Nevertheless, most existing approaches still rely on text conditioning and multi-step sampling, motivating one-step diffusion frameworks with more effective and visually grounded conditioning mechanisms. One-Step Super-Resolution Methods. Motivated by the high computational cost of multi-step diffusion sampling, recent works have explored one-step diffusion-based SR to achieve efficient restoration with a single forward pass. SinSR [ 35 ]