Beyond the Ground Truth: Enhanced Supervision for Image Restoration
Donghun Ryou, Inju Ha, Sanghyeok Chu, Bohyung Han, Computer Vision Laboratory, ECE & IPAI, Seoul National University {dhryou, hij1112, sanghyeok.chu, bhhan}@snu.ac.kr · 2025
Read Beyond the Ground Truth: Enhanced Supervision for Image Restoration by Donghun Ryou, Inju Ha, Sanghyeok Chu, Bohyung Han, Computer Vision Laboratory, ECE & IPAI, Seoul National University {dhryou, hij1112, sanghyeok.chu, bhhan}@snu.ac.kr (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-03932.txt.
Introduction
Image restoration networks learn from ground truth photos, but real-world ground truth is itself imperfect, capped by how the data could be captured. Donghun Ryou and colleagues aim beyond that ceiling in a 2025 computer vision preprint. Their framework improves the reference images first, then trains against the improved versions.
A conditional generator learns frequency masks that blend each original reference with its super-resolved variants, keeping the frequencies that help and dropping the rest. This frequency-domain mixing preserves the scene's meaning while adding perceptual detail, without inventing artefacts that would hurt fidelity. The enhanced references then train a lightweight refinement network that plugs into existing models. Experiments and user studies both favour the approach. Gains come from both halves of the design: better supervision and the refinement step.
Catalog record
| CREATOR | Donghun Ryou, Inju Ha, Sanghyeok Chu, Bohyung Han, Computer Vision Laboratory, ECE & IPAI, Seoul National University {dhryou, hij1112, sanghyeok.chu, bhhan}@snu.ac.kr |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-03932
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond the Ground Truth: Enhanced Supervision for Image Restoration Donghun Ryou, Inju Ha, Sanghyeok Chu, Bohyung Han, Computer Vision Laboratory, ECE & IPAI, Seoul National University {dhryou, hij1112, sanghyeok.chu, bhhan}@snu.ac.kr Abstract Deep learning-based image restoration has achieved significant success. However, when addressing real-world degradations, model performance is limited by the quality of ground-truth images in datasets due to practical constraints in data acquisition. To address this limitation, we propose a novel framework that enhances existing ground truth images to provide higher-quality supervision for real-world restoration. Our framework generates perceptually enhanced ground truth images using super-resolution by incorporating adaptive frequency masks, which are learned by a conditional frequency mask generator. These masks guide the optimal fusion of frequency components from the original ground truth and its super-resolved variants, yielding enhanced ground truth images. This frequency-domain mixup preserves the semantic consistency of the original content while selectively enriching perceptual details, preventing hallucinated artifacts that could compromise fidelity. The enhanced ground truth images are used to train a lightweight output refinement network that can be seamlessly integrated with existing restoration models. Extensive experiments demonstrate that our approach consistently improves the quality of restored images. We further validate the effectiveness of both supervision enhancement and output refinement through user studies. Code is available at https://github.com/dhryougit/Beyond-the-Ground-Truth . † † footnotetext: ∗ indicates equal contribution. 1 Introduction Original GT MUSIQ: 47.82 VisualQuality-R1: 3.50 Enhanced GT MUSIQ: 71.56 VisualQuality-R1: 4.80 Figure 1 : Visualization of our enhanced ground truth (GT). Our enhanced GT not only demonstrates sharper text and superior perceptual quality but also maintains semantic consistency with respect to the original GT. Zoom in for better visualizaiton. Image restoration has achieved remarkable progress through supervised training on paired low-quality and ground truth images using deep neural networks. Across various degradation types, a range of architectures ( 50 ; 15 ; 21 ; 6 ; 48 ; 11 ) and learning strategies ( 18 ; 33 ; 44 ; 53 ; 36 ) have been proposed to align restored outputs closely with ground truth images. Recently, the focus has shifted toward improving perceptual quality of the restored outputs, leveraging advances in generative models to produce visually compelling results ( 34 ; 22 ; 45 ; 38 ) . Despite these advances, improving perceptual quality remains a major challenge in real-world image restoration, where obtaining ideal reference images is inherently difficult due to practical data acquisition constraints. Many existing datasets rely on indirect ways to construct ground truth images. For instance, in deblurring datasets ( 28 ; 31 ; 27 ) , ground truth images are selected from video sequences, which often contain slight camera shake or object movements, limiting the image sharpness. Likewise, in denoising datasets ( 29 ; 1 ; 41 ) , ground truth images are constructed by averaging multiple noisy captures, often resulting in blurred references. As a result, models trained on such suboptimal ground truth images inevitably tend to inherit those imperfections, limiting their ability to achieve high-quality restoration. To address this limitation, we propose a novel supervision enhancement framework designed to improve the perceptual quality of suboptimal ground truth images. The proposed framework consists of two main components: (1) super-resolution using a one-step diffusion model to generate perceptually enhanced ground truth variants, and (2) frequency-domain mixup to produce the final enhanced ground truth images. For the frequency-domain mixup, we introduce a conditional frequency mask generator that adaptively produces masks to guide the optimal fusion of frequency components from the original ground truth image and its super-resolved variants. As illustrated in Figure 1 , the resulting enhanced ground truth image provides clearer details and higher perceptual quality than the original. Building upon the enhanced ground truth images, we design a lightweight output refinement network that can be seamlessly integrated into a wide range of pretrained restoration models without requiring architectural changes or retraining. Experiments show that the refinement network consistently improves the quality of restored images, benefiting from the enhanced supervision provided by our framework. Moreover, the network exhibits strong robustness in out-of-distribution scenarios, effectively removing residual degradations that remain after initial restoration. User studies further confirm the superior quality of both the enhanced ground truth images and the refinement outputs. In summary, our key contributions are as follows: • We identify the limitations of conventional ground truth images as a critical bottleneck in real-world image restoration, and propose a supervision enhancement framework based on frequency-domain mixup of an original ground truth image and its super-resolved variants. This design preserves semantic fidelity while enriching perceptual details, resulting in more reliable supervisory signals. • We introduce a lightweight refinement network that is trained solely on the original and enhanced ground truth images, requiring no additional annotations. The module is model-agnostic, seamlessly integrating with arbitrary restoration backbones without architectural modifications or retraining, and is empirically shown to be robust even under out-of-distribution degradations. • We validate our approach through extensive experiments and user studies, demonstrating consistent improvements in both enhanced ground truth quality and restored image fidelity. 2 Related Works With the rise of deep learning, traditional image restoration methods have largely been replaced by data-driven approaches trained on paired low-quality and ground truth images. A wide range of architectures has been proposed, including Convolutional Neural Networks ( 9 ; 50 ; 6 ) , Transformer-based models ( 47 ; 21 ; 48 ) , Generative Adversarial Networks ( 16 ; 15 ) , and more recently, state-space models such as Mamba ( 11 ; 10 ) . These models have typically been trained to maximize metrics such as PSNR and SSIM, which quantify the pixel-wise similarity to the original ground truth images. While these approaches achieve high performance on standard benchmarks, their outputs often lack perceptual realism of high-quality images. Recently, there is a growing interest in enhancing the perceptual quality of restored images. This has been particularly prominent in image super-resolution, demonstrating significant advancements in generating visually plausible high-frequency details ( 40 ; 8 ; 34 ; 22 ; 45 ; 38 ) . This trend extends to broader restoration tasks, such as deblurring and denoising, where diffusion models have been leveraged to enhance perceptual quality ( 30 ; 12 ; 25 ; 54 ; 46 ; 23 ) . While these methods effectively enhance perceptual quality, they often incur significant inference overhead and, more critically, risk hallucinating details or textures absent in the ground truth. In contrast, our approach aims to enhance perceptual quality while preserving the semantics of the original content by our novel frequency mixup strategy. To assess the perceptual quality of restored images, we employ a combination of deep-learning based image quality assessments ( 13 ; 43 ; 4 ; 52 ; 5 ) , and emerging Vision-Language Model-based methods ( 39 ; 19 ) . 3 Supervision Enhancement Framework In this section, we introduce our supervision enhancement framework, which improves the perceptual quality of ground truth images in existing datasets to provide better supervision for image restoration tasks. The framework consists of two main components: (1) super-resolution using a one-step diffusion model to generate perceptually enhanced ground truth variants, and (2) combining these variants with the original ground truth image through frequency-domain mixup using masks generated by a