MILO: A Lightweight Perceptual Quality Metric for Image and Latent-Space Optimization CCS: Computing methodologies Image processing CCS: Computing methodologies Image representations CCS: Computing methodologies Supervised learning

Uğur Çoğalan, Mojtaba Bemana, Karol Myszkowski, Hans-Peter Seidel, Colin Groth · 2025

Read MILO: A Lightweight Perceptual Quality Metric for Image and Latent-Space Optimization CCS: Computing methodologies Image processing CCS: Computing methodologies Image representations CCS: Computing methodologies Supervised learning by Uğur Çoğalan, Mojtaba Bemana, Karol Myszkowski, Hans-Peter Seidel, Colin Groth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01411.txt.

Introduction

Judging how good an image looks to human eyes is central to image processing, but heavy quality metrics are slow and human labels are scarce. Ugur Cogalan, with four co-authors, presents MILO, a lightweight perceptual metric for full-reference quality assessment, where a clean original is available for comparison. It works at several scales and runs fast enough for real-time use.

MILO is trained with pseudo scores: set distortions are applied to varied images, and an ensemble of recent metrics, accounting for visual masking, supplies the grades. That avoids large human-labeled data sets. Despite its small size, MILO beat existing metrics on standard benchmarks. The team also uses it as a perceptual loss for optimizing images and for latent representations inside a diffusion model, aligning results with human vision.

Catalog record

CREATORUğur Çoğalan, Mojtaba Bemana, Karol Myszkowski, Hans-Peter Seidel, Colin Groth
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-01411

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MILO: A Lightweight Perceptual Quality Metric for Image and Latent-Space Optimization CCS: Computing methodologies Image processing CCS: Computing methodologies Image representations CCS: Computing methodologies Supervised learning Uğur Çoğalan, Mojtaba Bemana, Karol Myszkowski, Hans-Peter Seidel, Colin Groth Abstract . We present MILO (Metric for Image- and Latent-space Optimization), a lightweight, multiscale, perceptual metric for full-reference image quality assessment (FR-IQA). MILO is trained using pseudo-MOS (Mean Opinion Score) supervision, in which reproducible distortions are applied to diverse images and scored via an ensemble of recent quality metrics that account for visual masking effects. This approach enables accurate learning without requiring large-scale human-labeled datasets. Despite its compact architecture, MILO outperforms existing metrics across standard FR-IQA benchmarks and offers fast inference suitable for real-time applications. Beyond quality prediction, we demonstrate the utility of MILO as a perceptual loss in both image and latent domains. In particular, we show that spatial masking modeled by MILO, when applied to latent representations from a VAE encoder within Stable Diffusion, enables efficient and perceptually aligned optimization. By combining spatial masking with a curriculum learning strategy, we first process perceptually less relevant regions before progressively shifting the optimization to more visually distorted areas. This strategy leads to significantly improved performance in tasks like denoising, super-resolution, and face restoration, while also reducing computational overhead. MILO thus functions as both a state-of-the-art image quality metric and as a practical tool for perceptual optimization in generative pipelines. Figure 1. We present MILO , our perceptual metric for full-reference image quality assessment, solely trained on learned perceptual data . MILO processes either RGB or VAE-decoded latent input pairs and outputs a perceptual quality score in the MOS range along with a spatial visibility map. When applied as a perceptual loss function, MILO allows through a curriculum learning strategy for spatially guided and content-aware refinement in image restoration tasks and diffusion-based optimization. 1. Introduction For centuries, people have captured real and imagined scenes through photography and painting. Today, visual expression increasingly relies on computer-generated imagery, especially images synthesized by deep neural networks. As demand for high-quality generation grows, so does the need for reliable metrics to assess visual quality as perceived by the human observer and guide optimization in image synthesis and restoration pipelines. Traditional metrics like Mean Absolute Error (MAE) and Peak Signal-to-Noise Ratio (PSNR) rely on per-pixel comparisons and often fail to align with human perception. Perceptual metrics such as LPIPS ( Zhang et al. 2018 ) and DISTS ( Ding et al. 2020 ) improve on this by leveraging deep features from pre-trained networks like VGG. These methods are trained on human-labeled perceptual data at the patch level, which is costly to collect and may still fall short of capturing perceived quality in the context of full images. Due to the labor-intensive nature of repeated and controlled human judgment collection, perceptual datasets are limited in scope. For example, the widely used KADID-10k dataset ( Lin et al. 2019 ) contains only 81 reference images, each with 25 distortions at five distortion levels, amounting to over 10,000 image pairs that need annotation. This scarcity is particularly problematic given that perceived quality is strongly dependent on image content and local context. As illustrated in Fig. 2 , even large variations in distortion strength can result in identical subjective ratings due to masking or content-based tolerance. However, the underlying challenge extends beyond data scarcity: it lies in the absence of models that capture variations in perceptual sensitivity across spatial structure and frequency content. Effective IQA methods must account for these factors to predict human judgments reliably and support optimization tasks that require fine-grained, spatially aware perceptual guidance. We propose MILO, a compact and efficient multiscale FR-IQA metric that addresses these limitations. MILO is trained using pseudo-MOS labels generated by an ensemble of visual masking-aware metrics applied to synthetically distorted images through a forward model. This strategy enables large-scale supervision without the need for manual annotations. The resulting model predicts both global perceptual scores and spatial visibility maps, which allow for the localization of the distortion visibility based on the underlying content. MILO achieves state-of-the-art performance on common IQA benchmarks while being significantly faster than existing learning-based metrics due to its lightweight structure. Beyond quality prediction, we show that MILO is effective as a perceptual loss in restoration and generative tasks. In particular, our method can operate directly in the latent space of diffusion models. Most contemporary generative pipelines perform image synthesis through iterative refinement of the latent. Applying perceptual guidance in this domain is advantageous, as it avoids repeated decoding into image space and allows perceptual supervision to act on high-level semantic representations. Our method enables spatially resolved loss weighting in latent space, improving the quality and efficiency of diffusion-based optimization. The code for our metric, trained models, and further information are available at https://milo.mpi-inf.mpg.de/ . Our contributions summarize as: • a fast perceptual metric (MILO) that achieves state-of-the-art accuracy in full-reference image quality assessment. • a pseudo-supervision strategy based on reproducible distortions and a metric ensemble to synthesize large-scale, annotated training data. • extension of our metric to operate directly in the latent space of diffusion models for efficient, perceptual optimization. • an effective perceptual loss in restoration tasks through curriculum learning and spatial masking. Figure 2. Three distortion images of the KADID10k dataset that apply significantly different amounts of noise to the image. Yet, the subjective quality (MOS) is rated the same by human observers. 2. Related work This section explores former research and discusses advancements in image quality assessment and data augmentation techniques. 2.1. Image Quality Assessment Image quality assessment (IQA) is primarily divided into full-reference (FR) and no-reference (NR) approaches. In this paper, we mainly focus on FR metrics. Traditional FR-IQA methods, such as MAE, PSNR, and SSIM ( Wang et al. 2004 ) , rely on direct comparisons between reference and distorted images to evaluate quality. FSIM ( Zhang et al. 2011b ) incorporates phase congruency and gradient magnitude to better model human perception. FLIP ( Andersson et al. 2020 ) is a task-specific metric designed to assess perceptual differences in rendered graphics with high spatial precision. However, these traditional metrics often fall short of representing visual quality as perceived by humans because they do not account for image content, e.g., high-frequency vs. low-frequency background, or perceptual phenomena such as masking. In recent years, learning-based models ( Prashnani et al. 2018 ; Zhang et al. 2018 ; Ding et al. 2020 ; Ding et al. 2021 ; Liao et al. 2022 ) have been developed to more accurately predict perceptual quality, leveraging deep image features to better align with human perception. Notably, Zhang et al. 2018 demonstrated that internal image representations from classification networks could be used to assess image similarity. Their Perceptual Image Patch Similarity (LPIPS) index computes image similarity by measuring \ell_{2} distances between features extracted from pre-trained VGG networks. This idea builds on earlier perceptual feature-based losses, such as those used in style transfer and super-resolution networks ( Johnson et al. 2016 ) , where VGG features provided a proxy for perceptual content. Ding et al. 2020 introduced the DISTS metric, which evaluates texture and structure similarity by comparing the global statistics—mean, variance, and correlations—of feature

Read the complete text