Entropy-Guided Self-Supervised Learning for Medical Image Classification

Joao Florindo, Viviane Moura · 2026

Read Entropy-Guided Self-Supervised Learning for Medical Image Classification by Joao Florindo, Viviane Moura (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21970.txt.

Introduction

Joao Florindo and Viviane Moura introduce an entropy-guided self-supervised framework for medical image classification, where labeled data are scarce and classes differ subtly. Their ensemble pairs masked autoencoder pre-training on domain data with ImageNet pre-training for general features. Masked autoencoder pre-training sharpens feature learning on medical images, while ImageNet pre-training adds strong general features. The ensemble consistently beats single models and existing methods, reaching state-of-the-art results and showing diverse pre-training strategies work well together.

Catalog record

CREATORJoao Florindo, Viviane Moura
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35781
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.21970

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Entropy-Guided Self-Supervised Learning for Medical Image Classification Joao Florindo, Viviane Moura Abstract Accurate and robust medical image classification is paramount for early disease diagnosis and treatment planning. However, challenges such as limited annotated data, high intra-class variability, and subtle inter-class differences often hinder the performance of deep learning models. This paper introduces a synergistic deep learning framework that leverages the strengths of self-supervised learning and transfer learning for enhanced medical image classification. Our approach employs two distinct ConvNeXt-Tiny models: one pre-trained on a large-scale natural image dataset (ImageNet) and another pre-trained using an entropy-guided Masked Autoencoder (MAE) on the target medical dataset. Both models are then fine-tuned on specific medical image classification tasks. A final ensemble strategy, based on averaging predicted probabilities, is utilized to combine the complementary insights from these two models. Rigorous experimental validation across four diverse medical imaging datasets (Breast Ultrasound Images (BUSI), International Skin Imaging Collaboration (ISIC) 2018, Kvasir, and COVID) demonstrates the superior performance and robustness of our ensemble approach. The MAE pre-training significantly improves feature learning on domain-specific data, while the ImageNet pre-training provides strong generalizable features. The ensemble consistently achieves state-of-the-art results, outperforming individual models and existing methods, highlighting the efficacy of combining diverse pre-training strategies for challenging medical image analysis. 1 Introduction Medical image classification plays a pivotal role in modern healthcare, enabling automated or semi-automated diagnosis of various diseases from diverse imaging modalities such as X-rays, CT scans, MRIs, and ultrasound images [ 47 , 2 ] . The ability to accurately and efficiently classify medical images can significantly assist clinicians in early detection, treatment planning, and monitoring disease progression, thereby improving patient outcomes [ 19 , 50 ] . Deep learning, particularly Convolutional Neural Networks (CNNs), has revolutionized computer vision and subsequently medical image analysis [ 22 , 42 , 13 ] . CNNs excel at extracting hierarchical features, from low-level textures to high-level semantic concepts, due to their inherent inductive biases such as translation invariance and local receptive fields [ 26 ] . However, medical imaging datasets often present unique challenges: they are typically smaller than natural image datasets, highly imbalanced, contain subtle yet critical features, and are susceptible to noise and artifacts [ 40 , 43 ] . These characteristics can limit the generalization capabilities of CNNs trained from scratch or even those fine-tuned from ImageNet pre-trained weights, as ImageNet features may not fully capture domain-specific nuances. In recent years, Vision Transformers (ViTs) [ 10 ] have emerged as powerful alternatives, demonstrating superior performance in various computer vision tasks by leveraging self-attention mechanisms to capture long-range dependencies. Hybrid CNN-Transformer architectures have also gained traction, aiming to combine the local feature extraction strengths of CNNs with the global contextual understanding of Transformers [ 11 , 23 ] . However, ViTs often require vast amounts of data for effective training, and their computational complexity can be a bottleneck, especially for high-resolution medical images [ 24 , 25 ] . Self-supervised learning (SSL) has shown immense promise in mitigating the data scarcity problem in medical imaging by pre-training models on unlabeled data to learn rich, transferable representations [ 3 , 32 ] . Masked Autoencoders (MAEs) [ 12 ] are a prominent SSL technique that involves masking a large portion of input patches and reconstructing the missing pixels. This forces the model’s encoder to learn robust representations from partial observations, which can then be fine-tuned for downstream tasks. Inspired by “diffusion forcing” [ 5 ] , here we propose the “entropy-guided” masked autoencoder. This essentially consists in computing an entropy score for each patch, and then independently applying Gaussian noise over each patch with the Gaussian variance controlled by the entropy score. This turns the masking process into a continuous transform. The extreme case of maximum noise would be equivalent to the classical masking in MAE. The “entropy-guided” aspect of MAE pre-training, as explored in this work, further refines the masking strategy to focus on regions with higher information content or uncertainty, potentially leading to more discriminative feature learning for medical images. Overall, this paper proposes a synergistic deep learning framework for medical image classification that integrates the benefits of ImageNet pre-training, entropy-guided MAE pre-training, and ensemble learning. Our approach utilizes two ConvNeXt-Tiny models [ 25 ] as backbone architectures. One model is initialized with weights pre-trained on ImageNet, providing a strong baseline with generalizable features. The second model undergoes an entropy-guided MAE pre-training phase on the target medical dataset, enabling it to learn domain-specific, robust representations from unlabeled data. Both models are then fine-tuned on the labeled medical image classification task. Finally, an ensemble strategy, which averages the predicted probabilities from these two fine-tuned models, is employed to leverage their complementary strengths and enhance overall classification performance and robustness. Our main contributions are summarized as follows: 1. We introduce an entropy-guided MAE pre-training strategy designed to learn domain-specific, robust feature representations from medical images by focusing on informative regions during masking. 2. We propose a novel synergistic framework for medical image classification that combines ImageNet pre-training, entropy-guided MAE pre-training, and ensemble learning to achieve superior performance and robustness. 3. We conduct extensive experiments on four diverse medical imaging datasets (BUSI, ISIC2018, Kvasir, COVID), demonstrating that our SSL approach consistently outperforms individual models and achieves state-of-the-art results across various metrics. 4. We provide a comprehensive ablation study analyzing the impact of different pre-training strategies and hyperparameter settings on the overall performance. The remainder of this paper is organized as follows: Section 2 reviews related works. Section 3 describes the proposed methodology and architectural details. Section 4 outlines the experimental setup. Section 5 presents and analyzes the experimental results and ablation studies. Section 5.2 provides a comparison with state-of-the-art methods. Finally, Section 6 concludes the paper. 2 Related Works Medical image classification has witnessed significant advancements with the advent of deep learning, yet it continues to present unique challenges due to the inherent characteristics of medical data. This section reviews key developments in deep learning for medical image classification, focusing on CNN-based methods, Transformer-based methods, hybrid architectures, and self-supervised learning approaches, while integrating recent literature and the provided auxiliary documents. 2.1 CNN-based Methods Convolutional Neural Networks (CNNs) have long been the cornerstone of medical image analysis due to their ability to automatically extract hierarchical features and their inductive biases like translation invariance and locality [ 22 , 42 ] . Early architectures like VGG [ 42 ] and ResNet [ 13 ] demonstrated the power of increasing depth and residual connections for improved performance. DenseNet [ 15 ] further enhanced feature reuse through dense connectivity, reducing parameter count and improving generalization. More recently, ConvNeXt [ 25 ] re-evaluated modern CNN design principles, incorporating techniques from Vision Transformers such as Layer Normalization and depthwise separable convolutions, achieving strong performance while maintaining CNN efficiency. In medical imaging, CNNs have been widely applied for tasks such as disease diagnosis and lesion localization. For instance, ResNet-based models have been used for breast cancer histological image classification [ 20 ] . DAFNet [ 4 ] proposes a Dynamic Adaptive Fusion Network that enhances feature extraction and classification performance across diverse medical imaging tasks by integrating channel shuffling, feature transformation, adaptive normalization convolution (ANC), and dynamic attention mechanisms.

Read the complete text