Towards Continual Visual Anomaly Detection in the Medical Domain

Manuel Barusco, Francesco Borsatti, Nicola Beda, Davide Dalle Pezze, Gian Antonio Susto · 2025

Read Towards Continual Visual Anomaly Detection in the Medical Domain by Manuel Barusco, Francesco Borsatti, Nicola Beda, Davide Dalle Pezze, Gian Antonio Susto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-18013.txt.

Introduction

Manuel Barusco and co-authors wrote a paper from 2025 on teaching medical image tools to learn over time without forgetting. Such tools learn from normal scans alone and flag scans that look off, a task where spotting the odd region matters. In clinics, the stream of scans shifts as machines and patients change. A tool trained once can then fade.

The team adapted a known tool, named PatchCore, to a step-by-step learning mode. New kinds of scan arrive in stages, and the tool updates while it guards what it learned before. They tested it on a real medical image set with marks at both the image and pixel level. The adapted tool matched versions trained for one task alone. It forgot less than one percent of its past skill. The team calls this a first proof that step-wise learning fits this corner of medical imaging.

Catalog record

CREATORManuel Barusco, Francesco Borsatti, Nicola Beda, Davide Dalle Pezze, Gian Antonio Susto
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-18013

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Continual Visual Anomaly Detection in the Medical Domain Manuel Barusco, Francesco Borsatti, Nicola Beda, Davide Dalle Pezze, Gian Antonio Susto Abstract Visual Anomaly Detection (VAD) seeks to identify abnormal images and precisely localize the corresponding anomalous regions, relying solely on normal data during training. This approach has proven essential in domains such as manufacturing and, more recently, in the medical field, where accurate and explainable detection is critical. Despite its importance, the impact of evolving input data distributions over time has received limited attention, even though such changes can significantly degrade model performance. In particular, given the dynamic and evolving nature of medical imaging data, Continual Learning (CL) provides a natural and effective framework to incrementally adapt models while preserving previously acquired knowledge. This study explores for the first time the application of VAD models in a CL scenario for the medical field. In this work, we utilize a CL version of the well-established PatchCore model, called PatchCoreCL, and evaluate its performance using BMAD, a real-world medical imaging dataset with both image-level and pixel-level annotations. Our results demonstrate that PatchCoreCL is an effective solution, achieving performance comparable to the task-specific models, with a forgetting value less than a 1%, highlighting the feasibility and potential of CL for adaptive VAD in medical imaging. K eywords Visual Anomaly Detection \cdot Continual Learning \cdot Medical Imaging Correct reference for this work : Manuel Barusco, Francesco Borsatti, Nicola Beda, Davide Dalle Pezze, and Gian Antonio Susto. “Towards Continual Visual Anomaly Detection in the Medical Domain”. (2025). In Proceedings of the International Workshop on Personalized Incremental Learning in Medicine . https://doi.org/10.1145/3746259 1 Introduction Visual Anomaly Detection (VAD) is a critical and important task in Computer Vision and Machine Learning that aims at identifying anomalous patterns in the input images that deviate from the normal distributions. In VAD, the data collection is very critical; indeed, normal data is abundant, and anomalies are very rare and difficult to collect. Because of this, VAD models are trained in a totally unsupervised way by considering only normal data, while in inference, the models are tested on normal and anomalous data. In order to provide explainability of the predictions, the models must return an image-level score but also a segmentation map that pinpoints the anomalous parts of the image. VAD models are employed in a variety of fields spanning from manufacturing [ 1 ] to surveillance [ 2 ] and the medical domain [ 3 ] . In the medical domain, VAD is very delicate, and anomalous patterns may indicate important pathological conditions; this underlines the importance of developing accurate and fast-adapting models in this field. VAD methods have shown strong performance on static scenarios, where a separate model is trained for each considered domain or category. However, VAD models, like every other machine learning model, are prone to Catastrophic Forgetting: when learning a new task, where a task can represent a new category or a new data domain, the model forgets the previous ones. Continual learning (CL) is a branch of Machine Learning that enables models to incrementally learn and adapt to new tasks without forgetting the previous ones and without needing a complete retraining of the model on new and old data simultaneously (which may be impossible or too demanding). In this work, we study for the first time the Continual Visual Anomaly Detection task using the BMAD (Benchmarks for Medical Anomaly Detection) [ 3 ] dataset, a diverse collection of medical imaging datasets spanning multiple medical domains as anatomical regions and image acquisition modalities. In this work, we adopt PatchCore [ 4 ] as Visual Anomaly Detection method, a well-established model for industrial anomaly detection, able to output both image-level anomaly scores and interpretable heatmaps that localize the anomalies in the pixel space. In particular, we test PatchCoreCL, a variant of PatchCore proposed for the continual learning setting [ 5 ] . In summary, the main contributions of this paper are: • We explore for the first time Continual Visual Anomaly Detection setting for the medical field, where new domains are incrementally presented to the VAD model. • Our results are tested on a CL scenario created from the well-established BMAD dataset and combined with PatchCoreCL (a CL variant of the well-known PatchCore model), obtaining a robust baseline for future research. • Our results demonstrate that PatchCoreCL is an effective solution, achieving performance comparable to the task-specific models, demonstrating the feasibility for dynamic medical imaging. Moreover, to promote further research in this new scenario, we provide the code used for the experiments, available on the MoViAD library [ 6 ] . The paper is structured as follows: Section 2 provides an overview of the related works in the fields of VAD and CL, Section 3 describes the details about PatchCore and its continual adaptation, while 4 outlines the experimental setting. Section 5 introduces the considered VAD and CL metrics, and Section 6 comments on the obtained results. In the end, Section 7 draws the conclusions of this work and provides some interesting future work directions. Figure 1: Continual Learning Visual Anomaly Detection for the BMAD medical imaging dataset. 2 Related Work 2.1 Visual Anomaly Detection A lot of research has been conducted in the VAD domain during the last years, and several VAD models have been developed. Most VAD methods fall into reconstruction-based and feature embedding-based methods. Reconstruction-based methods use generative models to learn normal image reconstruction during training: large reconstruction errors during inference indicate anomalies. Approaches like AutoEncoders, GANs and Diffusion Models are prominent in this area. An example of these models are [ 7 ] and [ 8 ] . Feature-based methods rely on data representations generated by pre-trained neural networks. For instance, PatchCore is a feature-based method, which saves features of normal images in a memory bank during training to compare them with test images features during inference. These techniques balance anomaly detection performance and interpretability by providing both image-level and pixel-level scores and visual heatmaps that pinpoint the image areas depicted as anomalies. In the last years, VAD research has moved from implementing better models in terms of accuracy towards new challenges such as tackling the computational and memory constraints posed by edge devices [ 9 ] [ 10 ] as well as dynamic environments, which can benefit from continual learning techniques [ 5 ] . 2.2 Continual Learning Traditional Machine Learning models are trained on fixed datasets, but real-world data often differs from the training data distribution. Continual Learning (CL) enables models to adapt to new data without forgetting previous knowledge. Effective CL methods should minimize forgetting, have low memory usage, and be computationally efficient [ 5 ] . CL techniques are generally categorized into three main approaches. Rehearsal-based approaches, such as Experience Replay [ 11 ] , store and reuse past data samples during training. Even if very effective, they are not always applicable, especially in situations when it is not possible to store old data due to data privacy issues or memory limitations. Regularization-based approaches introduce constraints to the network weights or loss function penalties to retain knowledge of old tasks, often using parameter importance or distillation techniques [ 12 ] [ 13 ] . Architecture-based approaches modify the

Read the complete text