Unsupervised Patch-GAN with Targeted Patch Ranking for Fine-Grained Novelty Detection in Medical Imaging
Jingkun Chen, Guang Yang, Xiao Zhang, Jingchao Peng, Tianlu Zhang, Jianguo Zhang, Jungong Han, Vicente Grau · 2025
Read Unsupervised Patch-GAN with Targeted Patch Ranking for Fine-Grained Novelty Detection in Medical Imaging by Jingkun Chen, Guang Yang, Xiao Zhang, Jingchao Peng, Tianlu Zhang, Jianguo Zhang, Jungong Han, Vicente Grau (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17906.txt.
Introduction
Jingkun Chen and co-authors spot fine anomalies in medical images without any labeled examples, in work from 2025. Rare abnormalities vary widely and hide as small patches inside normal tissue. Whole-image judgments miss those subtle deviations too often.
Their Patch-GAN first rebuilds masked images to learn what normal looks like in detail. The rebuilt image is then split into patches, each judged real or fake. Patch scores expose small abnormal zones that global scores wash out. A ranking step spotlights the patches with the highest abnormality. Local detail and global structure are thus checked together. The method needs no labeled abnormal cases to train.
Catalog record
| CREATOR | Jingkun Chen, Guang Yang, Xiao Zhang, Jingchao Peng, Tianlu Zhang, Jianguo Zhang, Jungong Han, Vicente Grau |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-17906
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Unsupervised Patch-GAN with Targeted Patch Ranking for Fine-Grained Novelty Detection in Medical Imaging Jingkun Chen, Guang Yang, Xiao Zhang, Jingchao Peng, Tianlu Zhang, Jianguo Zhang, Jungong Han, Vicente Grau Abstract Detecting novel anomalies in medical imaging is challenging due to the limited availability of labeled data for rare abnormalities, which often display high variability and subtlety. This challenge is further compounded when small abnormal regions are embedded within larger normal areas, as whole-image predictions frequently overlook these subtle deviations. To address these issues, we propose an unsupervised Patch-GAN framework designed to detect and localize anomalies by capturing both local detail and global structure. Our framework first reconstructs masked images to learn fine-grained, normal-specific features, allowing for enhanced sensitivity to minor deviations from normality. By dividing these reconstructed images into patches and assessing the authenticity of each patch, our approach identifies anomalies at a more granular level, overcoming the limitations of whole-image evaluation. Additionally, a patch-ranking mechanism prioritizes regions with higher abnormal scores, reinforcing the alignment between local patch discrepancies and the global image context. Experimental results on the ISIC 2016 skin lesion and BraTS 2019 brain tumor datasets validate our framework’s effectiveness, achieving AUCs of 95.79% and 96.05%, respectively, and outperforming three state-of-the-art baselines. Introduction Novelty detection in medical imaging is essential for identifying unseen abnormalities that are not represented in limited labeled datasets, a task critical for early diagnosis and effective treatment planning. In medical practice, manually annotating the extensive variety of possible abnormalities—often irregular in shape, size, and texture—is infeasible, particularly in domains such as dermatology and neuroimaging ( Yang et al. 2025 ; Chen et al. 2022 ; Duan et al. 2024 ) . These abnormalities may be subtle and diverse, requiring unsupervised methods capable of leveraging predominantly normal data to reliably detect new patterns, enabling accurate anomaly identification without exhaustive manual labeling ( Luo et al. 2024 ; Chen et al. 2020 ) . Despite its importance, medical novelty detection faces two fundamental challenges. Small abnormalities within large normal regions are often overlooked in global predictions, as they resemble normal variations ( Gao et al. 2023 ) . This challenge is compounded by current methods’ inability to seamlessly integrate localized detection with global context, which is crucial for accurately identifying these small anomalies in predominantly normal images. Approaches that either emphasize global features or treat patches in isolation often fail to capture the nuanced relationship between local anomalies and the overall image, limiting their effectiveness in medical novelty detection ( Chen et al. 2024 ) . To address these challenges, we propose an innovative unsupervised Patch-GAN framework for medical novelty detection that combines local and global perspectives to enhance both sensitivity and localization accuracy. Our approach reconstructs masked normal images, allowing the model to learn intricate local details representative of the normal class—such as texture, structure, and intensity—without any explicit labels. By partitioning the reconstructed images into patches and evaluating each patch independently, our model identifies fine-grained discrepancies that may indicate anomalies. This patch-level analysis increases the model’s sensitivity to small, isolated abnormalities while simultaneously establishing a coherent link between local patches and the overall image context, a key aspect for accurate anomaly localization. To further strengthen the model’s ability to detect and prioritize abnormal regions, we introduce a novel patch-ranking mechanism. This mechanism ranks patches by their abnormal scores, selectively highlighting areas with the highest likelihood of abnormality and integrating them within the broader image context. By correlating these high-anomaly patches with the global structure of the image, our approach effectively balances detailed local detection with contextual awareness, enabling the model to not only detect but also accurately localize small, subtle anomalies. This innovative strategy addresses the limitations of existing methods by providing a multi-scale analysis that adapts to the complexity of medical imaging data. We validate our framework on two public medical imaging datasets—ISIC 2016 for skin lesions and BraTS 2019 for brain tumors—demonstrating that our method consistently outperforms state-of-the-art approaches in sensitivity and specificity. Our contributions are fourfold: • We introduce an unsupervised Patch-GAN framework that reconstructs masked images to learn comprehensive normal patterns, enabling precise novelty detection at the local level. • We propose a patch-based detection approach that enhances the model’s sensitivity to small abnormalities while maintaining a coherent local-global relationship within each image. • We develop a patch-ranking mechanism that prioritizes high-anomaly patches, improving both detection accuracy and the model’s ability to integrate patch-level insights with the overall image structure. • Experimental results demonstrate the effectiveness of our approach, achieving state-of-the-art performance on multiple datasets. Related Work Novelty detection in medical imaging has evolved from classical unsupervised algorithms to sophisticated deep learning models, each attempting to capture the subtle and often complex anomalies characteristic of medical data. Here, we review these advancements and highlight the distinctions and innovations of our unsupervised Patch-GAN framework. Early methods in novelty detection focused on representing the normal class through a dense, low-dimensional distribution, with anomalies identified as deviations from this normal range. Principal Component Analysis (PCA) ( Shyu et al. 2003 ) was among the foundational approaches, projecting data into a low-dimensional space to isolate outliers. Extending this, Soft PCA ( Aggarwal 2017 ) introduced the Mahalanobis distance to quantify deviations from the normal class center, improving separation of normal and abnormal samples. Other approaches, such as one-class Support Vector Machines (SVM) ( Perdisci, Gu, and Lee 2006 ) and local density-based methods like Local Outlier Factor (LOF) ( Breunig et al. 2000 ) and Local Correlation Integral (LOCI) ( Papadimitriou et al. 2003 ) , used decision boundaries or local density to identify anomalies. Isolation Forest (iForest) ( Liu, Ting, and Zhou 2008 ) offered a different strategy, using randomly partitioned hyperplanes to isolate outlying samples. While these classical methods provide a baseline for novelty detection, their limited capacity to model the complex textures and structural nuances in medical images often leads to reduced performance when detecting subtle or small-scale abnormalities. With the advent of deep learning, reconstruction-based models, particularly those using GANs, have shown promise in capturing richer representations of normal data. AnoGAN ( Schlegl et al. 2017 ) was one of the first to leverage GANs for medical novelty detection, reconstructing normal-class images and identifying anomalies through reconstruction error. However, this pixel-level adversarial learning approach proved computationally expensive, as it required a costly remapping of each sample in the latent space. To address this inefficiency, Efficient-GAN-abnormal ( Zenati et al. 2018 ) optimized the distribution mapping, though the computational burden remained significant. Schlegl et al. ( Schlegl et al. 2019 ) further improved speed by incorporating a pretrained encoder, thereby bypassing the need for remapping, but at the cost of added model complexity. Further advancements in GAN-based models, such as Ganomaly ( Akcay, Atapour-Abarghouei, and Breckon 2018 ) and OCGAN ( Perera, Nallapati, and Xiang 2019 ) , explored the use of compact latent spaces to distinguish normal from abnormal samples. These methods assume that latent spaces trained exclusively on normal data will form a compact distribution, enabling easier identification of anomalies as deviations. However, the high generalization capability of deep models can blur these latent representations, especially for subtle or small abnormalities, limiting detection precision ( Shen et al. 2020