Skin Cancer Classification: Hybrid CNN-Transformer Models with KAN-Based Fusion
Shubhi Agarwal, Amulya Kumar Mahto · 2025
Read Skin Cancer Classification: Hybrid CNN-Transformer Models with KAN-Based Fusion by Shubhi Agarwal, Amulya Kumar Mahto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-12484.txt.
Introduction
Shubhi Agarwal and Amulya Kumar Mahto classify skin cancer images with hybrid models that join convolutional networks and transformers, in a paper from 2025. Telling harmful lesions from harmless ones early guides care and saves lives. Their design lets one part read local detail while the other reads the whole scene.
A fusion block based on Kolmogorov-Arnold networks blends the two feature sets with learnable curves. Transfer learning and heavy data augmentation support training. Tests ran on three data sets with apart mixes and class balances. Accuracy reached 92.81 percent on HAM10000, 97.83 percent on PAD-UFES, and 91.17 percent on BCN20000. The authors present the hybrid as a sturdy base for medical image sorting.
Catalog record
| CREATOR | Shubhi Agarwal, Amulya Kumar Mahto |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-12484
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Skin Cancer Classification: Hybrid CNN-Transformer Models with KAN-Based Fusion Shubhi Agarwal, Amulya Kumar Mahto Abstract Skin cancer classification is a crucial task in medical image analysis, where precise differentiation between malignant and non-malignant lesions is essential for early diagnosis and treatment. In this study, we explore Sequential and Parallel Hybrid CNN-Transformer models with Convolutional Kolmogorov-Arnold Network (CKAN). Our approach integrates transfer learning and extensive data augmentation, where CNNs extract local spatial features, Transformers model global dependencies, and CKAN facilitates nonlinear feature fusion for improved representation learning. To assess generalization, we evaluate our models on multiple benchmark datasets (HAM10000, BCN20000 and PAD-UFES) under varying data distributions and class imbalances. Experimental results demonstrate that hybrid CNN-Transformer architectures effectively capture both spatial and contextual features, leading to improved classification performance. Additionally, the integration of CKAN enhances feature fusion through learnable activation functions, yielding more discriminative representations. Our proposed approach achieves competitive performance in skin cancer classification, demonstrating 92.81% accuracy and 92.47% F1-score on the HAM10000 dataset, 97.83% accuracy and 97.83% F1-score on the PAD-UFES dataset, and 91.17% accuracy with 91.79% F1-score on the BCN20000 dataset highlighting the effectiveness and generalizability of our model across diverse datasets. This study highlights the significance of feature representation and model design in advancing robust and accurate medical image classification. I Introduction Cancer remains one of the most pressing global health challenges, with skin cancer being among the most prevalent and rapidly increasing types. Malignant skin cancers, such as melanoma, pose a significant risk due to their aggressive nature and potential for metastasis. Early detection plays a crucial role in improving patient survival rates, and dermoscopic imaging serves as a primary diagnostic tool. However, differentiating malignant from non-malignant lesions remains challenging due to high intra-class variations, inter-class similarities, and dataset biases. Deep learning, particularly Convolutional Neural Networks (CNNs) , has achieved significant success in medical image classification. While CNNs excel at extracting local spatial features , they struggle to capture long-range dependencies , limiting their ability to analyze complex lesion structures. The inability to model global contextual relationships makes it challenging to distinguish visually similar but clinically different lesions. Medical image datasets are often limited in size, leading to overfitting and reduced generalization. Transfer learning has emerged as an effective solution by allowing pre-trained models to adapt to new datasets, improving performance. However, despite these advantages, CNN-based methods still face difficulties in learning robust representations that generalize well across diverse datasets with varying distributions. To address these limitations, we propose a hybrid CNN-Transformer model , leveraging CNNs for hierarchical spatial feature extraction and Transformers for long-range dependency modeling via self-attention.By combining these architectures, our model enhances feature learning, leading to improved classification performance. Additionally, we introduce the Convolutional Kolmogorov-Arnold Network (CKAN) , which employs learnable activation functions to refine feature fusion and enhance interpretability. Kolmogorov-Arnold Networks (KANs) differ from conventional deep learning models by replacing fixed activation functions with learnable transformations at the network’s edges. In our framework, CNNs extract local features, Transformers model global relationships, and CKAN refines feature fusion , ensuring robust classification. This hierarchical decomposition enhances both accuracy and generalization , making the model more resilient to variations in lesion appearance. I-A Our Contributions The key contributions of this study are as follows: • We propose a hybrid CNN-Transformer architecture for skin cancer classification, combining CNNs for local feature extraction with Transformers for global dependency modeling. • We introduce CKAN-based feature fusion , leveraging learnable activation functions to enhance flexibility and feature representation. • We conduct comprehensive evaluations across multiple datasets (HAM10000, BCN20000, ISIC-2020, PAD-UFES) to assess the generalizability of our approach. • We compare CNN-only and the hybrid models to analyze the trade-off between feature extraction techniques and dataset variations. • We provide a detailed analysis of the impact of different architectures on classification performance, highlighting their strengths and limitations in real-world scenarios. II Related Works Deep learning has significantly advanced medical image analysis, particularly in the classification of skin cancers. CNN-based models have been widely used for feature extraction and classification. The study in [ 1 ] applied transfer learning with VGG16 and VGG19, achieving high precision for lesion classification. However, these methods are constrained by their reliance on single datasets, making it challenging to assess their generalizability across diverse real-world data distributions. Recent works have explored Transformer-based models to address CNNs’ limitations in capturing long-range dependencies, Vision Transformers (ViTs) have been explored as an alternative. The review in [ 2 ] highlighted their advantages in learning spatial correlations and dependencies, making them effective for segmentation, detection, and classification tasks. However, ViTs often require large datasets, which can be a challenge in medical imaging. The ViT-CoMer model [ 3 ] introduced a CNN-Transformer fusion mechanism that enhances multiscale feature interaction and bidirectional information exchange, providing a potential improvement in feature representation. Despite these advantages, the direct application of ViT-CoMer to the classification of skin cancer remains an area of further investigation. Another approach to improving classification performance has focused on enhancing CNN architectures. The work in [ 4 ] proposed a deep CNN model with an extensive preprocessing pipeline, including noise filtering, normalization, and data enhancement, achieving competitive accuracy on HAM10000. However, pre-processing alone may not be sufficient to handle variations across datasets. Meanwhile, methods based on the region of interest (ROI), such as [ 5 ] , have attempted to improve classification precision by extracting melanoma-specific regions from images before training CNN models. Although this approach has shown effectiveness, its dependency on accurate segmentation techniques introduces potential bias when applied to varying lesion characteristics and imaging conditions. Beyond CNNs and Transformers, alternative architectures such as Knowledge-Aware Networks (KANs) have been proposed to enhance feature learning. Unlike traditional CNNs, which use fixed activation functions, KANs employ learnable activation functions at the edges of neural networks, allowing for more flexible feature transformations [ 6 ] . Although KANs have shown promise in various domains, their direct integration into medical image classification remains relatively unexplored. Combining KAN-based feature fusion with existing CNN-Transformer hybrids could provide a more structured approach to improving classification accuracy and interpretability. Hybrid approaches that integrate CNNs and Transformers have been explored to combine local and global feature extraction capabilities. However, many existing models rely on direct concatenation or simple fusion mechanisms, limiting their effectiveness. The ViTO model [ 7 ] applied Transformer-based methods to inverse medical imaging problems, showing strong results for reconstruction tasks. Although not specifically designed for classification, such approaches highlight the potential of structured hybrid models in medical imaging. Prior studies primarily focus on either CNN-based or Transformer-based architectures, with fewer works exploring structured hybrid models. Publicly available datasets such as HAM10000 [ 8 ] , BCN20000 [ 9 ] , ISIC-2020 [ 10 ] , and PAD-UFES [ 11 ] have significantly contributed to advancing skin cancer classification research. Although these datasets provide diverse types of lesion and imaging conditions, challenges such as class imbalance and domain shifts persist. III Methodology The proposed approach follows a structured pipeline for classifying skin lesion images into malignant and non-malignant categories. The process begins with data collection from multiple benchmark datasets, ensuring a diverse set of images. Next, preprocessing steps such as resizing and normalization are applied to standardize the