Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis
Alexander Frotscher, Christian F. Baumgartner, Thomas Wolfers · 2025
Read Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis by Alexander Frotscher, Christian F. Baumgartner, Thomas Wolfers (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01534.txt.
Introduction
Alexander Frotscher and co-authors run a large fair test of anomaly finding in brain scans. Deep methods can flag odd spots in MR scans with no lesion labels. That path could help clinics a lot. But past tests were split, with mixed data and mixed scores.
Their benchmark spans many centers with one protocol. It also checks for bias across groups. A shared test shows which methods truly work and which just fit one site. That clarity is key before such tools can aid real diagnosis at scale. Bias checks span age and sex groups, since a detector that works for one group and fails for another is not clinic-ready.
Catalog record
| CREATOR | Alexander Frotscher, Christian F. Baumgartner, Thomas Wolfers |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01534
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis Alexander Frotscher, Christian F. Baumgartner, Thomas Wolfers Abstract Deep unsupervised anomaly detection in brain magnetic resonance imaging offers a promising route to identify pathological deviations without requiring lesion-specific annotations. Yet, fragmented evaluations, heterogeneous datasets, and inconsistent metrics have hindered progress toward clinical translation. Here, we present a large-scale, multi-center benchmark of deep unsupervised anomaly detection for brain imaging. The training cohort comprised 2,976 T1 and 2,972 T2-weighted scans ( ≈ \approx 461,000 slices) from healthy individuals across six scanners, with ages ranging from 6 to 89 years. Validation used 92 scans to tune hyperparameters and estimate unbiased thresholds. Testing encompassed 2,221 T1w and 1,262 T2w scans spanning healthy datasets and diverse clinical cohorts. Across all algorithms, the Dice-based segmentation performance varied between ≈ \approx 0.03 and ≈ \approx 0.65, indicating substantial variability and underscoring that no single method achieved consistent superiority across lesion types or modalities for any task. To assess robustness, we systematically evaluated the impact of different scanners, lesion types and sizes, as well as demographics (age, sex). Reconstruction-based methods, particularly diffusion-inspired approaches, achieved the strongest lesion segmentation performance, while feature-based methods showed greater robustness under distributional shifts. However, systematic biases, such as scanner-related effects, were observed for the majority of algorithms, including that small and low-contrast lesions were missed more often, and that false positives varied with age and sex. Increasing healthy training data yields only modest gains, underscoring that current unsupervised anomaly detection frameworks are limited algorithmically rather than by data availability. Our benchmark establishes a transparent foundation for future research and highlights priorities for clinical translation, including image native pretraining, principled deviation measures, fairness-aware modeling, and robust domain adaptation. 1 Introduction Figure 1: Large-scale benchmarking of interdisciplinary state-of-the-art deep unsupervised anomaly detection. a) Number of volumes in the training and test datasets, split by condition, compared to previous benchmarks. b) Age distribution of the training data. c) Example of a reconstruction-based approach. An encoder and decoder are trained exclusively on healthy data, and during testing anomalies are identified by thresholding the reconstruction error with \bm{\tau} . d) Example of a feature-based approach. A feature map is created by passing the image through a neural network. These features are either (i) reconstructed using an autoencoder, analogous to panel c, or (ii) compared directly to features from a memory bank of healthy images. Anomalies are flagged when the Euclidean distance to healthy features exceeds \bm{\tau} . The human brain is the most complex organ in the body, with billions of neurons forming intricate networks that support cognition, behavior, and perception 39 ; 24 ; 18 . Neuroimaging provides insights into this complexity but necessarily compresses rich biology into high-dimensional, often noisy measurements. Diseases of neurodegeneration, mental disorders, stroke, and tumors are characterized by substantial heterogeneity at the brain level 17 . This heterogeneity arises not only from the underlying pathology itself but is further amplified by demographic factors such as age and sex, genetic predispositions, environmental influences, and lifestyle factors. Moreover, technical variability stemming from magnetic resonance imaging (MRI) acquisition protocols, scanner hardware, and image quality 76 ; 38 adds another layer of complexity, making robust characterization and analysis particularly challenging. In clinical practice, diagnosis of brain lesions and treatment planning generally rely on human evaluation of MRI, including lesion identification 73 ; 5 . Yet, even for experienced clinicians, the heterogeneity of disease-related changes makes manual assessment time-consuming and error-prone 82 . To reduce misdiagnosis, streamline planning, enable longitudinal monitoring, and alleviate workload, automated lesion detection and decision-support systems have emerged 35 ; 15 . State-of-the-art systems typically use supervised deep learning to map images to manually annotated lesion masks. While the performance on curated datasets can be strong, generalization to unseen lesion types and different scanners remains limited 78 ; 25 , and assembling large expert-annotated datasets is costly 23 . This tension underscores the need for methods that reduce dependence on manually created lesion maps. Such approaches must not only generalize across diverse lesion types, sizes, and contrasts but also adapt to the wide range of individual brain anatomies and disease presentations observed in clinical populations while remaining robust to demographic variability and nuisance variables. Unsupervised Anomaly Detection (UAD) has emerged as a widely adopted alternative to supervised approaches in MRI analysis and lesion detection. The central principle of UAD is to model a representation of normal brain structure using clean data and to flag deviations from this learned concept of normality as potential anomalies in unseen cases 63 . By shifting the focus from predefined lesion labels to deviations from normative patterns, UAD provides a framework that can potentially capture unexpected, subtle, or previously uncharacterized pathologies, making it particularly valuable for heterogeneous and poorly understood brain disorders. In brain MRI, recent deep UAD methods typically fall into two main categories: (i) reconstruction-based approaches, which learn to compress and reconstruct healthy images and then use voxel-level residuals between the input and reconstruction to localize anomalies 11 ; 48 ; 22 , and (ii) feature-based approaches, which extract intermediate representations from neural networks and apply a secondary detection model for pixel-level anomaly localization 62 ; 79 ; 20 . Furthermore, training strategies based on synthetic-anomalies have been introduced to further reduce reliance on real lesion maps by altering images of healthy individuals. Often, these approaches incorporate reconstruction-based methods to mitigate either detection or generalization problems 80 ; 59 . Analogous unsupervised anomaly detection techniques have already been established in other domains, such as industrial surface defect detection 12 ; 62 ; 79 ; 20 , autonomous driving 30 , and time series analysis 13 ; 4 , where standardized datasets and evaluation protocols have catalyzed progress. In contrast, progress in medical imaging has long been hampered by the lack of comparable large-scale resources. Only with the advent of initiatives such as the Human Connectome Project 74 ; 70 ; 14 and UK Biobank imaging 57 has this begun to change. Perhaps unsurprisingly, as medical data is inherently sensitive, it is more difficult to acquire, share, and pool at scale 60 . To compensate, some studies have resorted to practices utilizing a “healthy” reference from clean slices of anomalous volumes. This strategy potentially introduces systematic biases and undermines the generalizability and robustness of findings, as non-anomalous slices in brains with lesions can hardly be considered to belong to a healthy cohort of brains 10 . Beyond the challenge of reference definition 29 , evaluations in medical imaging often differ in fundamental aspects, including performance metrics, lesion taxonomies, brain regions analyzed, preprocessing strategies, and the degree of control for scanner and demographic factors that render the comparison of methods impossible. Although valuable steps toward unification have been made 8 ; 45 , these efforts remain fragmented and leave critical aspects inconsistently addressed, such as evaluation with healthy data, the diversity of the lesions tested, the impact of demographics on predictions, and missing thresholding strategies to derive the predictions. As a result, a stringent and clinically grounded benchmark is needed to establish the current state of the field, create a fair basis for comparison, and guide the development of more robust and generalizable methods in the