AFiRe: Anatomy-Driven Self-Supervised Learning for Fine-Grained Representation in Radiographic Images
Yihang Liu, Lianghua He, Ying Wen, Longzhen Yang, Hongzhou Chen · 2025
Read AFiRe: Anatomy-Driven Self-Supervised Learning for Fine-Grained Representation in Radiographic Images by Yihang Liu, Lianghua He, Ying Wen, Longzhen Yang, Hongzhou Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10972.txt.
Introduction
Yihang Liu, Lianghua He, Ying Wen, Longzhen Yang, and Hongzhou Chen present AFiRe, a 2025 method for teaching vision models from X-ray images without labels. Standard self-teaching methods sort whole images apart, which misses the fine anatomy that radiology turns on. AFiRe instead builds anatomy into the training itself.
One part lines up image tokens that share structure and class, sharpening fine spatial detail. A second part hides and restores local odd spots, which teaches shape detail. A synthetic lesion mask adds variety without breaking anatomy, unlike plain crops. In tests the method grouped features more cleanly than rivals, beat seven radiology-specific methods on labeling tasks with few labels, and spotted anomalies from image-level tags alone.
Catalog record
| CREATOR | Yihang Liu, Lianghua He, Ying Wen, Longzhen Yang, Hongzhou Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-10972
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AFiRe: Anatomy-Driven Self-Supervised Learning for Fine-Grained Representation in Radiographic Images Yihang Liu, Lianghua He, Ying Wen, Longzhen Yang, Hongzhou Chen Abstract Current self-supervised methods, such as contrastive learning, predominantly focus on global discrimination, neglecting the critical fine-grained anatomical details required for accurate radiographic analysis. To address this challenge, we propose an A natomy-driven self-supervised framework for enhancing Fi ne-grained Re presentation in radiographic image analysis (AFiRe). The core idea of AFiRe is to align the anatomical consistency with the unique token-processing characteristics of Vision Transformer. Specifically, AFiRe synergistically performs two self-supervised schemes: (i) Token-wise anatomy-guided contrastive learning, which aligns image tokens based on structural and categorical consistency, thereby enhancing fine-grained spatial-anatomical discrimination; (ii) Pixel-level anomaly-removal restoration, which particularly focuses on local anomalies, thereby refining the learned discrimination with detailed geometrical information. Additionally, we propose Synthetic Lesion Mask to enhance anatomical diversity while preserving intra-consistency, which is typically corrupted by traditional data augmentations, such as Cropping and Affine transformations. Experimental results show that AFiRe: (i) provides robust anatomical discrimination, achieving more cohesive feature clusters compared to state-of-the-art contrastive learning methods; (ii) demonstrates superior generalization, surpassing 7 radiography-specific self-supervised methods in multi-label classification tasks with limited labeling; and (iii) integrates fine-grained information, enabling precise anomaly detection using only image-level annotations. Code — https://github.com/LYH-hh/AFiRe Introduction Contrastive learning (CL) has emerged as a powerful method in self-supervised learning (SSL), demonstrating its effectiveness without relying on expert annotations ( Caron et al. 2020 ; Caron et al. 2021 ) . In natural image analysis, prevailing CL methods like SimCLR ( Chen et al. 2020a ) and MoCo ( He et al. 2020 ) primarily focus on global discrimination. Explicitly, they consider the entire image and its transformations as positive pairs, while treating different images as negative pairs ( Wu et al. 2018 ; Misra and Maaten 2020 ) . Although these models successfully capture high-level representations across diverse images and achieve invariance to salience changes, their efficacy is limited when applied to radiographic images ( Zhou et al. 2021a ; Haghighi et al. 2022 ) . Natural images exhibit invariance of foreground objects across diverse backgrounds, which directs contrastive tasks to emphasize image-level discrimination ( Misra and Maaten 2020 ) . In contrast, radiographic images present clinical salience dispersed throughout the image ( Li et al. 2024 ) , necessitating the model’s attention to fine-grained discrimination, including the distribution of densities, the arrangement of tissues, and the presence of specific pathologies ( Agu et al. 2021 ; Zhao et al. 2021b ) . For example, faint opacities may indicate early signs of pneumonia, while fine reticular patterns can suggest interstitial lung disease. Therefore, incorporating fine-grained details, such as detailed anatomical structures and complex spatial relationships ( Haghighi et al. 2024 ; Chen et al. 2023 ) , into high-level representations is essential for accurately identifying normal anatomy and detecting various lesions in radiographic images. Figure 1: Conception of the proposed method. (a) Local radiographic structures at the same location exhibit anatomical consistency. (b) By aligning anatomical consistency with the token-processing characteristics of ViT, tokens at the same position within a batch share similar structural semantics, while those at different positions convey distinct ones. Recent advances in medical contrastive learning (MCL) have incorporated fine-grained information by focusing on region discrimination ( Chen et al. 2023 ; Singh, Gorade, and Mishra 2024 ; Li et al. 2024 ) . These approaches aim to embed semantically consistent features while distinguishing them from features with distinct semantics ( Taher, Gotway, and Liang 2024 ) . However, the variability of diseases makes it difficult to comprehensively capture pathological semantics, and additional resources are required to determine the relevant areas (such as pre-selection ( Huang et al. 2021 ) ). Another effective method for enriching fine-grained information is to integrate the pixel restoration with MCL ( Zhou et al. 2021a ; Haghighi et al. 2024 ) . However, these methods typically exhibit a deficiency in anatomical semantics, as they restore pixel-level content directly from latent representations. This plain pixel information limits the model’s capability to focus on salient clinical patterns. In this paper, to circumvent the challenge posed by heterogeneous anomalies, which potentially leads to insufficient learning of semantic information, we highlight that comprehensive learning of normal anatomical patterns is comparatively easier. This advantage stems from the significant semantic similarity observed in normal radiographic images ( Xiang et al. 2023 ) , which are characterized by anatomical consistency and stable focal regions across different individuals (Fig. 1 (a)). Such similarity facilitates the effective decoupling of underlying anomalies from normal structures, thereby enhancing pathological diagnosis performance. Additionally, to avoid region pre-selection, we hypothesize that each distinct token in Vision Transformers (ViTs) ( Dosovitskiy et al. 2021 ) explicitly represents the predominant information of specific local anatomical structures . This hypothesis is inspired by SelfPatch ( Yun et al. 2022 ) , which asserts that ViTs have inherent architectural advantages for enhancing visual representations through the processing of discrete image tokens. Summarily, our core idea is to align anatomical consistency with the unique token-processing characteristics of ViT to enhance fine-grained radiographic representation (Fig. 1 (b)). These alignments are also pivotal in anomaly-specific restoration for preserving pixel-level anatomy-associated ( i.e. , geometrical) information. Based on the analysis above, we propose a novel A natomy-driven self-supervised framework to enhance Fi ne-grained Re presentation in radiographic images ( AFiRe ). Equipped with the designed Synthetic Lesion Masks (SLMs), an anatomy augmentation inspired by AnatPaste ( Sato et al. 2023 ) , AFiRe aligns ViT tokens with local anatomical structures via synergistically performing two SSL tasks: Token-wise anatomy-guided contrastive learning and Pixel-level anomaly-removal restoration. For the contrastive learning task, instead of analyzing entire images, AFiRe processes each disjoint token in ViT as an independent sample. By maximizing mutual information among these tokens within a batch, this task learns fine-grained structural invariance and discriminative semantics based on structural and categorical consistency. To guide the model in comprehensively learning the normal anatomical patterns, we introduce a group of spatial-aware prototypes in this task. These prototypes represent the distribution of different anatomical structures, serving as pseudo-cluster assignments for the predicted token probabilities. For the pixel restoration task, we strategically focus on restoring specific abnormal tokens augmented by SLMs. Concretely, we remove these abnormal tokens by substituting them with trainable masked tokens to restore their corresponding normal pixels. Hence, the latent representations retain both normal and abnormal information while preserving geometrical details closely associated with various anatomical structures. Our extensive experiments demonstrate that AFiRe enhances the model’s capacity to capture fine-grained discriminative information from normal radiographic images, facilitating a more robust representation across different anatomic structures. Compared to different supervised and self-supervised benchmarks, AFiRe achieves superior performance in multi-label classification tasks, indicating its generalization in analyzing real disease images. Furthermore, AFiRe has observable advantages in three anomaly detection tasks, outperforming the state-of-the-art counterparts. Overall, our contributions are as follows: • We design a Token-wise anatomy-guided contrastive learning SSL task, equipped with spatial-aware prototypes to integrate fine-grained anatomical discrimination based on structural and categorical consistency. • We design a Pixel-level anomaly-removal restoration SSL task to preserve detailed geometrical information closely associated with various anatomical structures. • We introduce Synthetic Lesion Mask, an