A Lightweight Image Super-Resolution Transformer Trained on Low-Resolution Images Only
Björn Möller, Lucas Görnhardt, Tim Fingscheidt · 2025
Read A Lightweight Image Super-Resolution Transformer Trained on Low-Resolution Images Only by Björn Möller, Lucas Görnhardt, Tim Fingscheidt (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23265.txt.
Introduction
Björn Möller, Lucas Görnhardt, and Tim Fingscheidt train an image upscaling transformer using only low-resolution photos, in a 2025 paper. Sharp training images are often unavailable in real deployments, yet transformers are hungry for data. A strict benchmark allows training on low-resolution images alone, which for a fourfold upscale means learning from a sixteenth of the usual pixels.
The team adapted a training scheme first built for microscopy, adding a multi-scale routine matched to standard downscaling. On five classic test sets the method beat the prior low-resolution-only best, and it lifted a plain convolutional model too. The result suggests careful training can offset missing sharp data.
Catalog record
| CREATOR | Björn Möller, Lucas Görnhardt, Tim Fingscheidt |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-23265
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Lightweight Image Super-Resolution Transformer Trained on Low-Resolution Images Only Björn Möller, Lucas Görnhardt, Tim Fingscheidt Abstract Transformer architectures prominently lead single-image super-resolution (SISR) benchmarks, reconstructing high-resolution (HR) images from their low-resolution (LR) counterparts. Their strong representative power, however, comes with a higher demand for training data compared to convolutional neural networks (CNNs). For many real-world SR applications, the availability of high-quality HR training images is not given, sparking interest in LR-only training methods. The LR-only SISR benchmark mimics this condition by allowing only low-resolution (LR) images for model training. For a 4x super-resolution, this effectively reduces the amount of available training data to 6.25% of the HR image pixels, which puts the employment of a data-hungry transformer model into question. In this work, we are the first to utilize a lightweight vision transformer model with LR-only training methods addressing the unsupervised SISR LR-only benchmark. We adopt and configure a recent LR-only training method from microscopy image super-resolution to macroscopic real-world data, resulting in our multi-scale training method for bicubic degradation (MSTbic). Furthermore, we compare it with reference methods and prove its effectiveness both for a transformer and a CNN model. We evaluate on the classic SR benchmark datasets Set5, Set14, BSD100, Urban100, and Manga109, and show superior performance over state-of-the-art (so far: CNN-based) LR-only SISR methods. The code is available on GitHub 1 1 1 https://github.com/ifnspaml/SuperResolutionMultiscaleTraining . Fig. 1 : Low-resolution-only training : Overview over our proposed approach to train single-image super-resolution (SISR) models only with low-resolution (LR) data, consisting of our proposed multi-scale training method for bicubic degradation ( MSTbic ). It creates pseudo-LR/HR training pairs enabling supervised training of a SwinIR lightweight (lw) transformer model . The LR-only SISR benchmark provides only low-resolution (LR) images during training, while high-resolution (HR) test data is used for evaluation. I Introduction Single-image super-resolution (SISR) models aim to construct a high-resolution (HR) output image from a corresponding low-resolution (LR) input image. Conventionally, the SISR performance is measured in terms of image similarity between an SISR HR output image reconstructed from a degraded LR version and the original HR image. Also, the test images are taken from multiple benchmark datasets. The established benchmarks such as SISR on DIV2K, Set5, Set14, BSD100, Urban100 and Manga109 were long led by convolutional neural networks (CNNs) [ 2 , 43 , 20 , 35 , 22 , 10 ] , but were superseded quite recently by transformer models [ 17 , 6 , 7 , 41 , 21 ] . CNNs’ main component are convolutional layers applying filters across the input to capture local spatial features. By progressively aggregating spatial information in deeper layers, hierarchies of features can be learned. Transformers, originally designed for natural language processing [ 36 ] , excel at modeling long-range dependencies and global context by employing self-attention, which allows to process large image regions or even the entire image at once. The self-attention mechanism can model relationships between distant pixels or image features and dynamically weights their importance. In contrast to CNNs, which rely on local convolutions and hierarchical feature extraction, transformers offer more flexibly and have a lower inductive bias induced by the architectural design. We leverage these advantages and adopt a transformer-based lightweight SwinIR [ 21 ] model in this work. The absence of high-quality HR training images in many real-world SISR applications has led to the exploration of LR-only training methods. For the LR-only task, training data is restricted to LR images, while evaluation is still conducted on HR images, defining an unsupervised task. To still enable SR model training, some approaches [ 32 , 3 , 27 , 26 ] generate pseudo LR/HR training pairs from the available LR images. A recent method [ 27 ] addresses microscopy images, while previous approaches [ 32 , 3 ] for macroscopic images report on the LR-only SISR benchmark datasets and show interesting results that are almost competitive with supervised HR-trained models. We adapt the LR-only multi-scale training method from microscopy [ 27 ] to macroscopic images of the LR-only benchmark data under a bicubic degradation condition and consequently propose MSTbic, see Fig. 1 . So far, only CNN-based models have been reported on the LR-only SISR benchmark, while CNNs had already been outperformed by transformers on other SISR benchmarks. Still, it is not obvious whether this also applies to a transformer model under a limited data condition as present in the LR-only benchmark. As transformers inherently make less assumptions about the data structure, they need to compensate for this low inductive bias by requiring extensive training data. However, in the LR-only SISR benchmark, the amount of training information is reduced to 6.25% of the original HR pixels for a 4x SR task. In addition, training image quality is also lowered by the degradation function creating the LR images. In this work, we first employ a transformer-based lightweight SwinIR model for LR-only SISR. Second, we adopt an LR-only training method from microscopy SISR to macroscopic data. Third, we compare it with reference methods and prove its effectiveness both for a transformer and a CNN model. Finally, we set a new state of the art for the LR-only SISR benchmark. II Related Work CNNs and transformers for image super-resolution Deep learning-based SISR approaches primarily utilized CNNs due to their exceptional ability to extract local patterns. Initially featuring simple models with few layers [ 10 ] , improvements such as adding residual connections [ 22 ] or sub-pixel convolutions [ 31 ] led to high-performance SR models [ 2 , 43 , 20 , 35 , 22 , 10 ] . Although effective, their performance is constrained by the locality of the feature extraction, imposed by the receptive field of the convolutional filters. The essence of the transformer architecture [ 36 ] is the self-attention mechanism, which allows to model long-range dependencies and capture global relationships within the input sequence. Initially developed for natural language processing [ 36 ] and later adopted to high-level vision tasks [ 11 ] , transformers since became predominant in super-resolution [ 17 , 6 , 7 , 41 , 21 ] . Compared to CNNs, Transformers have less inductive bias and therefore need more training data to realize their modeling potential [ 6 ] . However, standard global self-attention scales quadratically with input size leading to high computational costs. To combat the computational demand, many approaches restrict the self-attention to local regions [ 21 , 42 , 8 , 6 , 17 ] . SwinIR [ 21 ] introduced local self-attention within shifting windows to still allow information exchange between neighboring local feature regions. As the lightweight (lw) configuration provides an efficient trade-off between performance and computational cost, we deploy the SwinIR lw in our method. LR-only image super-resolution For some real-world SR applications high-quality, high-resolution ground-truth images cannot be acquired. The LR-only image SR benchmark simulates this unsupervised condition, allowing only LR images during training while testing on HR images. Training approaches [ 32 , 33 , 3 , 27 ] create pseudo-LR/HR training pairs to enable supervision for SR model training. Shocher et al. pioneered the task with a zero-shot super-resolution (ZSSR) [ 32 ] approach, that aims to utilize the internal image statistic