Learning local and global prototypes with optimal transport for unsupervised anomaly detection and localization

Robin Trombetta (✉), Carole Lartizien · 2025

Read Learning local and global prototypes with optimal transport for unsupervised anomaly detection and localization by Robin Trombetta (✉), Carole Lartizien (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-12927.txt.

Introduction

Robin Trombetta and Carole Lartizien teach a system to spot defects in images after seeing only flawless examples, in a 2025 paper. Labels for defects are costly in industry and medicine, and preset labels also bias what counts as a flaw.

Their method learns prototype patterns at two scales, local patches and whole images, from normal samples alone. A comparison metric balances what a region looks like against where it sits, using optimal transport to match sets fairly. A structural constraint keeps the prototypes true to how normal samples are organized. On two industrial inspection benchmarks the approach matched strong baselines. The authors note medical imaging as a natural next use.

Catalog record

CREATORRobin Trombetta (✉), Carole Lartizien
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-12927

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning local and global prototypes with optimal transport for unsupervised anomaly detection and localization Robin Trombetta (✉), Carole Lartizien Abstract Unsupervised anomaly detection aims to detect defective parts of a sample by having access, during training, to a set of normal, i.e. defect-free, data. It has many applications in fields, such as industrial inspection or medical imaging, where acquiring labels is costly or when we want to avoid introducing biases in the type of anomalies that can be spotted. In this work, we propose a novel UAD method based on prototype learning and introduce a metric to compare a structured set of embeddings that balances a feature-based cost and a spatial-based cost. We leverage this metric to learn local and global prototypes with optimal transport from latent representations extracted with a pre-trained image encoder. We demonstrate that our approach can enforce a structural constraint when learning the prototypes, allowing to capture the underlying organization of the normal samples, thus improving the detection of incoherencies in images. Our model achieves performance that is on par with strong baselines on two reference benchmarks for anomaly detection on industrial images. The code is available at https://github.com/robintrmbtt/pradot . 1 Introduction Anomaly detection refers to the task of identifying samples that deviate from the distribution of normal cases. In domains such as video analysis or medical imaging, obtaining large-scale annotated datasets is often prohibitively expensive. Furthermore, relying on labeled data inherently biases the model toward detecting only known types of anomalies. As a result, rare or previously unseen anomalies can go undetected. To address these limitations, unsupervised anomaly detection (UAD) has emerged as an alternative approach to supervised learning. Instead of learning from both normal and abnormal examples, UAD models are trained exclusively on normal ( i.e. , anomaly-free) data and learn to capture the underlying distribution or structure of the normal data. At test time, the model computes the distance between the new sample and the learnt data distribution. Based on this measure, it is expected to differentiate between in-distribution samples and samples that deviate from the anomaly-free distribution. The research in the field on UAD for automatic industrial inspection was greatly pushed by the MVTec AD dataset [ 2 ] , which set a robust benchmark for the evaluation of UAD methods. More recently, a new version of the dataset called MVTec AD LOCO [ 1 ] was released to address the shortcomings of the first benchmark. In particular, it introduces anomalies of a more logical nature, which violate the expected structural or semantic organization of the image, the detection of which can require capturing long-range dependencies between the different subparts of the image. To address the challenge of unsupervised anomaly detection, a wide range of methods have been developed, employing different strategies to localize abnormal regions in images. A prominent family of methods relies on the extraction of hierarchical features from a pre-trained encoder to build a memory bank, that is, a set of discrete prototypical features representing normal patterns [ 3 , 6 , 25 , 13 , 23 ] . At inference time, a distance metric quantifies the discrepancy between the latent features of the test image and the learnt prototypes. The abnormal parts of the image are identified when there is a high dissimilarity between a latent feature and the prototypes. Various techniques allow to learn such prototypes such as kNN [ 6 , 13 ] or coreset sampling [ 25 ] . These methods have received particular attention because of their high performance, simplicity, speed and efficiency at low data regime. One of their shortcomings is their limited ability to spot abstract incoherencies in the images, such as misplaced objects, even when employing multi-level feature maps and improved neighbour-awareness mechanisms [ 6 , 25 ] , In this work, we propose using optimal transport to efficiently learn prototypes for features extracted from the pre-trained encoder. Inspired by fused Gromov-Wasserstein [ 35 ] (FGW) distance, we introduce a discrepancy function that balances a feature-based cost and a spatial-based cost. By playing on the ratio between these two components, we are able to learn more local or more global representatives of the normal class, thus capturing diverse patterns on the training images. By applying it to features extracted from a pre-trained backbone, such as ResNet50 [ 14 ] , we propose a novel method for Prototype-based Anomaly Detection with Optimal Transport (PRADOT). Compared to previous state-of-the-art methods that leverage a memory bank [ 6 , 25 ] , our approach, in particular due to the structural constraint enforced by assignment to local prototypes, is shown to better identify incoherencies in images. We evaluate our method on MVTec AD and MVTec AD LOCO, two standard benchmarks for unsupervised anomaly detection and localization on industrial images, where it achieves competitive performances compared to state-of-the-art methods. 2 Related Work Unsupervised anomaly detection. Earliest works focusing on anomaly detection tried to estimate the support of the normal samples with one-class support machine or support vector data description [ 31 , 26 ] . A large family of methods makes use of generative models, in particular Auto-Encoders or Generative Adversarial Nets [ 12 , 27 ] , and expects to localize anomalies with the pixel-wise distance between an input image and its reconstruction. However, recent studies have highlighted the fact that auto-encoders can generalize their reconstruction abilities to anomalous parts of images [ 46 ] , thus limiting the effectiveness of such approaches. To overcome this issue, more advanced auto-encoder based methods have incorporated masking strategies [ 46 , 41 ] , memory mechanisms [ 23 , 11 ] or synthetic data [ 45 , 28 ] . Recently, the generative abilities of diffusion models [ 15 ] have also been employed for unsupervised anomaly detection [ 42 ] . Many UAD methods exploit the representative power of features extracted from pre-trained encoders. Density-based methods learn the distribution of normal features via direct likelihood estimation [ 3 , 19 ] while support-based methods [ 26 , 43 ] aim to spot anomalies as samples with a high distance to the estimated boundaries of the distribution of anomaly-free cases. Normalizing flows have been widely applied in UAD, mainly through network-parametrized density estimation of deep latent features [ 44 , 49 ] . In knowledge distillation, a frozen pre-trained encoder is used as a teacher to guide the training of a student network, and the anomalies are detected by a large discrepancy between the features of the two networks [ 37 , 10 , 47 ] . The methods most closely related to this work are those based on prototypes [ 6 , 23 , 25 , 13 ] . A cornerstone work falling in this category is PatchCore [ 25 ] , which builds a memory bank of locally aware prototypes with coreset sampling and uses nearest neighbours search at inference to evaluate the distance between a feature and the discrete representatives of the normal samples. Prototype learning. Prototypical-based methods are popular in machine learning, one of their advantages being that they provide good explainability. Indeed, their exemplar-driven decision process generally allows for intuitive interpretation. One of the most popular applications of prototype learning is the

Read the complete text