Robust artificial intelligence frameworks for lung cancer subtyping and malignancy detection on thoracic CT Authors: Mustafa Tahsin Yilmaz, Enes Algul, Ishak Pacal Published: 2026 ABSTRACT AbstractAccurate characterization of thoracic malignancies on computed tomography (CT) remains challenging because histological subtype differentiation and nodule malignancy assessment require different visual cues, ranging from localized structural abnormalities to broader contextual patterns. This study presents a controlled comparative evaluation of convolutional neural network (CNN) and Vision Transformer (ViT) architectures for two CT-based thoracic oncology tasks: four-class histological subtyping and three-class lung cancer classification. A total of 30 contemporary architectures, including 15 CNN-based and 15 ViT-based models, were evaluated under a unified experimental setting using two publicly available datasets. The Chest CT-Scan Images dataset was used for Adenocarcinoma, Large cell carcinoma, Squamous cell carcinoma, and normal classification, while the IQ-OTH/NCCD dataset was used for normal, benign, and malignant classification. Model performance was assessed using accuracy, precision, recall, F1-score, AUC, parameter count, confusion matrices, and Grad-CAM-based visual interpretation. For histological subtyping, DeiT3-Base achieved the strongest performance among the evaluated models, with an F1-score of 0.9732, followed closely by high-performing CNN models such as InceptionNeXt-Base. For lung cancer classification, EfficientNetV2-Small, Swin-Base, and MViTv2-Base achieved the highest F1-score of 0.9816. These findings indicate that attention-based architectures may be advantageous for subtype discrimination requiring broader contextual modeling, whereas efficient CNNs and hierarchical transformers remain highly competitive for localized structural abnormalities. Grad-CAM visualizations provided qualitative evidence that high-performing models attended to relevant lesion and parenchymal regions.Keywords: Deep learning, Lung cancer diagnosis, Vision transformers, Histological subtyping, Pulmonary nodule detection, Explainable AISubject terms: Cancer, Computational biology and bioinformatics, Oncology Introduction Cancer persists as one of the most formidable challenges in global healthcare and remains a leading cause of mortality worldwide. Lung cancer, in particular, represents the most prevalent and lethal malignancy, accounting for approximately 1.8 million deaths annually, as reported by the World Health Organization1. The prognosis for patients is heavily dependent on the stage at diagnosis; while advanced-stage disease often has limited therapeutic options, early and accurate diagnosis can significantly enhance treatment efficacy and survival rates. Medical imaging, particularly computed tomography (CT), plays a central role in the screening, detection, and radiological characterization of thoracic malignancies. Although definitive assignment of lung cancer subtype requires pathological examination, CT images may contain macroscopic imaging cues associated with tumor morphology, lesion distribution, margin characteristics, density variation, and surrounding parenchymal changes. These radiological patterns can support computational models designed to predict histologically defined lung cancer subtypes and to distinguish normal, benign, and malignant cases. However, CT interpretation remains labor-intensive and subject to inter-observer variability, especially when early lesions, subtle margins, or overlapping radiological appearances are present. These challenges motivate the development of reproducible and interpretable deep learning approaches for CT-based thoracic cancer classification. The rapid development of deep learning has substantially advanced medical image analysis by enabling automated feature learning from complex imaging data. Convolutional Neural Networks (CNNs) have been widely used because they can extract hierarchical and spatially localized representations from high-dimensional images. Architectures such as ResNet2, DenseNet3, and EfficientNet4 have shown strong performance in tasks ranging from pulmonary nodule detection to cancer image classification. For instance, Ardila et al.5 developed a CNN-based model for lung cancer screening using low-dose CT scans and reported an AUC of 0.94. These findings demonstrate the value of CNN-based models as strong baselines for automated thoracic image interpretation. Nevertheless, convolutional operations are inherently local, which may limit their ability to model broader spatial dependencies in CT images when subtype-related patterns are subtle or distributed across multiple regions. Similarly, Esteva et al.6 applied CNNs to classify skin lesions, demonstrating dermatologist-level accuracy, which underscores the generalizability and potential of CNN architectures across various medical imaging domains. Despite their remarkable success, CNNs possess inherent limitations, primarily due to the local nature of convolutional operations. This inductive bias restricts their ability to model long-range dependencies and capture global context, which are critical for interpreting complex medical images where pathological indicators may span multiple regions or exhibit subtle inter-regional relationships. To address these constraints, Vision Transformers (ViTs), introduced by Dosovitskiy et al.7, have emerged as a paradigm shift in computer vision. By leveraging self-attention mechanisms, ViTs are capable of modeling global dependencies across the entire image, offering a complementary approach to the local feature extraction of CNNs. ViTs have shown significant promise in medical imaging; for example, Chen et al.8 demonstrated that ViT-based models could outperform traditional CNNs in classifying retinal diseases by effectively modeling global image context. Furthermore, hierarchical variants like the Swin Transformer9 have introduced shifted window attention mechanisms to balance local and global processing, achieving state-of-the-art performance in tasks such as brain tumor segmentation10. Additionally, data-efficient variants like DeiT11 and hybrid architectures like MobileViT12 have been developed to enhance computational efficiency, making these powerful models viable for resource-constrained clinical environments. Recent studies have explored CNN and ViT models for thoracic oncology, yet the literature remains fragmented across diagnostic tasks, datasets, and architectural families. In CT-based histological subtype prediction, prior work has investigated deep learning models for separating pathology-defined lung cancer categories such as Adenocarcinoma and Squamous cell carcinoma. Other studies have used histopathology slide images, which provide microscopic tissue information but differ fundamentally from CT-based non-invasive imaging. In malignancy detection, 3D CNN-based approaches have been applied to pulmonary nodule assessment and prognosis, with reported AUC values ranging from 0.86 to 0.93. However, many previous studies evaluate a single model family or a limited set of architectures, making it difficult to determine whether convolutional locality, hierarchical attention, or global self-attention is more suitable for different CT-based thoracic cancer classification tasks13–15. Challenges, gaps, and contributions Despite the remarkable strides in computer-aided diagnosis for thoracic oncology, the current landscape is characterized by significant fragmentation and methodological impediments. Predominantly, existing literature treats diagnostic tasks in isolation, focusing exclusively on either pulmonary nodule malignancy detection or histological subtyping, thereby restricting a comprehensive understanding of architectural performance across multifaceted clinical workflows. Furthermore, a substantial portion of state-of-the-art methodologies is optimized strictly for curated, homogenous datasets; consequently, their robustness when applied to heterogeneous imaging sources, which are increasingly prevalent in multi-center studies and open-access repositories, remains insufficiently validated. This issue is compounded by the “black-box” nature of many deep learning models, where a scarcity of rigorous visual explainability mechanisms provides an interpretability deficit that hinders clinician trust. Finally, the prohibitive computational complexity of emerging Vision Transformers often precludes their deployment in resource-constrained environments, highlighting a critical need for a rigorous comparative evaluation of high-capacity models against lightweight, efficient architectures suitable for edge computing. In response to these multifaceted challenges, this study establishes a robust, unified deep learning framework designed to benchmark a diverse spectrum of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) against heterogeneous imaging constraints. By integrating rigorous performance metrics with granular explainability analyses, we aim to bridge the gap between theoretical model complexity and practical clinical utility. The primary contributions of this study are as follows. We provide a controlled comparative evaluation of 30 contemporary deep learning architectures, including 15 CNN-based and 15 ViT-based models, across two thoracic oncology tasks: histological subtyping and nodule malignancy detection. We provide a controlled comparative evaluation of 30 contemporary deep learning architectures, including 15 CNN-based and 15 ViT-based models, across two thoracic oncology tasks: histological subtyping and nodule malignancy detection. Rather than proposing a single new architecture, the study addresses an important benchmarking gap by examining how different architectural inductive biases influence diagnostic performance under distinct clinical conditions. Rather than proposing a single new architecture, the study addresses an important benchmarking gap by examining how different architectural inductive biases influence diagnostic performance under distinct clinical conditions. The comparative design enables a direct assessment of convolution-driven local feature extraction and attention-driven contextual modeling within the same experimental framework. The comparative design enables a direct assessment of convolution-driven local feature extraction and attention-driven contextual modeling within the same experimental framework. For histological subtyping, DeiT3-Base achieved the strongest performance, with an F1-score of 0.9732, indicating that self-attention may be beneficial when subtype discrimination depends on subtle inter-class patterns and broader contextual cues. For histological subtyping, DeiT3-Base achieved the strongest performance, with an F1-score of 0.9732, indicating that self-attention may be beneficial when subtype discrimination depends on subtle inter-class patterns and broader contextual cues. For malignancy detection, EfficientNetV2-Small, Swin-Base, and MViTv2-Base achieved the highest F1-score of 0.9816, suggesting that both efficient convolutional representations and hierarchical attention mechanisms can capture discriminative structural abnormalities in lung CT images. For malignancy detection, EfficientNetV2-Small, Swin-Base, and MViTv2-Base achieved the highest F1-score of 0.9816, suggesting that both efficient convolutional representations and hierarchical attention mechanisms can capture discriminative structural abnormalities in lung CT images. Grad-CAM analysis provided qualitative support for the quantitative findings by showing that high-performing models focused on anatomically and radiologically relevant image regions rather than unrelated background areas. Grad-CAM analysis provided qualitative support for the quantitative findings by showing that high-performing models focused on anatomically and radiologically relevant image regions rather than unrelated background areas. The findings do not suggest universal superiority of CNNs or ViTs. Instead, they indicate that both model families offer task-dependent strengths, depending on the diagnostic objective, dataset characteristics, and computational constraints. The findings do not suggest universal superiority of CNNs or ViTs. Instead, they indicate that both model families offer task-dependent strengths, depending on the diagnostic objective, dataset characteristics, and computational constraints. Related work The application of deep learning to medical imaging, particularly for chest and lung cancer classification, has seen significant advancements in recent years, driven by the development of CNNs and, more recently, ViTs. This section provides a comprehensive review of prior work in cancer classification using these architectures, focusing on their application to chest and lung cancer detection, their strengths and limitations, and the gaps addressed by our unified framework. We categorize the literature into CNN-based approaches, ViT-based approaches, and hybrid or multi-modal methods, highlighting key studies and their contributions to the field. CNN-based approaches for cancer classification Convolutional Neural Networks have been the cornerstone of medical image analysis due to their ability to extract spatially localized features through convolutional operations, making them well-suited for tasks like cancer detection and classification. Early work by LeCun et al.16 established CNNs as effective for image recognition, and their adaptation to medical imaging has yielded significant results. For lung cancer, Ardila et al.5 developed a deep learning model based on a 3D CNN for lung cancer screening using low-dose CT scans, achieving an AUC of 0.94 on the National Lung Screening Trial dataset. Their model focused on detecting pulmonary nodules and predicting malignancy risk, demonstrating performance comparable to or better than radiologists. However, the model’s reliance on 3D inputs increased computational complexity, limiting its applicability in resource-constrained settings. For CT-based lung cancer subtype prediction, CNNs have been investigated as non-invasive tools for estimating pathology-defined tumor categories such as Adenocarcinoma and Squamous cell carcinoma. Coudray et al.13 used deep learning for non-small cell lung cancer subtype discrimination from histopathology images, achieving strong performance in distinguishing Adenocarcinoma from Squamous cell carcinoma. This distinction is important because histology slides contain microscopic tissue information, whereas CT captures macroscopic radiological morphology. Other studies have applied CNN-based models to CT-derived lung cancer classification, showing that local texture, shape, density, margin characteristics, and lesion-related imaging patterns may contribute to subtype or malignancy prediction. Nevertheless, many CNN-based studies remain limited by single-task design, restricted model comparison, or insufficient analysis of whether model attention corresponds to radiologically plausible regions17. EfficientNet4 introduced a compound scaling approach that optimizes model depth, width, and resolution, leading to improved performance with reduced computational cost. Hosny et al.14 adapted EfficientNet for lung cancer prognosis using CT scans, achieving an AUC of 0.86 for survival prediction. Their work demonstrated the efficiency of scaled architectures but was limited to prognostic rather than diagnostic tasks. More recent CNN architectures, such as ConvNeXt18, incorporate transformer-inspired features like large receptive fields, achieving competitive performance in medical imaging. For instance, Riquelme et al.15 used a 3D CNN for lung nodule detection and classification, but their model struggled with small or low-contrast nodules, a common challenge in lung cancer datasets. Despite their success, CNN-based approaches often struggle with modeling long-range dependencies, as their convolutional kernels are inherently local. This limitation can hinder performance in tasks requiring global context, such as identifying subtle inter-regional relationships in CT images of chest or lung cancer. Additionally, many CNN studies focus on single-modal datasets or specific cancer types, limiting their generalizability across diverse imaging modalities and histopathological classes. Vision transformer-based approaches The introduction of Vision Transformers (ViTs) by Dosovitskiy et al.7 marked a paradigm shift in image classification, leveraging self-attention mechanisms to model global dependencies across image patches. Unlike CNNs, ViTs treat images as sequences of patches, enabling the capture of long-range relationships, which is particularly valuable for medical imaging tasks where lesions may span multiple regions. Chen et al.8 applied ViTs to retinal disease classification, achieving superior performance compared to CNNs by modeling global image context, with an AUC of 0.95 for multi-class disease detection. Their work highlighted ViTs’ ability to handle complex medical images but required large-scale pre-training on datasets like ImageNet to achieve optimal results. In lung cancer imaging, ViT-based architectures have been explored for detection and classification because self-attention can capture longer-range spatial dependencies than conventional convolutional operations. Hierarchical variants such as Swin Transformer balance local and global feature modeling through window-based attention, making them suitable for high-resolution medical images. For CT-based histological subtype prediction, transformer models may be useful when class separation depends on distributed radiological patterns rather than a single localized lesion feature. However, transformer-based architectures often require careful fine-tuning, sufficient training data, and stable optimization. Their performance may also be affected by public-release image formats, preprocessing differences, and limited acquisition metadata. These constraints highlight the need for controlled comparisons between ViTs and CNNs under the same experimental protocol9,10,19. Data-efficient ViT variants, such as DeiT11, have addressed the high data requirements of traditional ViTs by incorporating distillation techniques, improving performance on smaller datasets. Mehta & Rastegari12 introduced MobileViT, a lightweight hybrid convolutional-transformer model, which achieved competitive performance with significantly fewer parameters, making it suitable for clinical deployment on resource-constrained devices. For instance, Roy et al.18 applied MobileViT to lung cancer classification, highlighting its potential for real-time applications. However, ViT-based models often require extensive computational resources, and their performance can degrade on non-DICOM formats like JPG/PNG, which are common in public datasets but introduce compression artifacts. BeiT20, a self-supervised ViT variant, has shown promise in general image classification but struggled in medical imaging tasks without extensive fine-tuning. Hybrid and multi-modal approaches Recent studies have explored hybrid and multi-modal approaches to combine the strengths of CNNs and ViTs, addressing their individual limitations. He et al.21 proposed a hybrid CNN-ViT model for breast cancer detection in ultrasound images, integrating local feature extraction from CNNs with global context modeling from ViTs. Their approach outperformed standalone CNNs and ViTs, demonstrating the complementary nature of these architectures. Similarly, Hosny et al.22 developed a multi-modal framework combining CNNs and ViTs for lung cancer classification, incorporating both CT images and clinical data. Their work highlighted the potential of multi-modal integration to improve robustness, particularly for heterogeneous datasets. Multi-modal approaches have also incorporated non-imaging data, such as genomic or clinical information, to enhance diagnostic accuracy. For instance, Mobadersany et al.23 integrated histopathological images with genomic data using a CNN-based model, achieving improved survival prediction for lung cancer patients. However, their approach was limited to histological images and required complex data preprocessing. More recently, Hosny et al.22 proposed a multi-modal ViT framework for lung cancer diagnosis, combining CT scans with electronic health records. While promising, these studies often face challenges in data availability, as multi-modal datasets are scarce and require sophisticated integration techniques. Methodology This section outlines the methodology employed to develop and evaluate a unified deep learning framework for classifying chest and lung cancers from CT images. The framework integrates Convolutional Neural Network (CNN) and Vision Transformer (ViT) architectures to leverage their complementary strengths in local and global feature extraction. We describe the datasets, preprocessing steps, model architectures, training protocols, evaluation metrics, and visualization techniques used to ensure robust performance and clinical interpretability. The methodology is designed to address challenges such as non-DICOM image formats, limited dataset sizes, and the need for high recall in clinical applications. Datasets Chest dataset The Chest CT-Scan Images dataset comprises approximately 1,000 CT images categorized into four diagnostic classes: Adenocarcinoma, Large cell carcinoma, Squamous cell carcinoma, and normal cases. In this study, these labels were used for CT-based prediction of histologically defined lung cancer subtypes and normal controls. It should be noted that the model inputs are radiological CT images rather than microscopic histology slides; therefore, the task aims to learn imaging patterns associated with pathology-defined subtype labels rather than direct histopathological structures. The dataset is organized into predefined training, validation, and test folders, and this original split structure was preserved during experimentation. Because the images are publicly available in JPG/PNG format rather than in the original acquisition format, detailed acquisition parameters, reconstruction metadata, and full radiometric information are not accessible24. Lung cancer dataset The lung cancer dataset, sourced from the Iraq-Oncology Teaching Hospital/National Center for Cancer Diseases (IQ-OTH/NCCD), was collected over three months in fall 2019 at specialist hospitals in Iraq. It includes 1,190 CT scan slices from 110 cases, grouped into three classes: normal (55 cases), benign (15 cases), and malignant (40 cases). The CT scans were originally collected in DICOM format using a Siemens SOMATOM scanner with a protocol of 120 kV, 1 mm slice thickness, window width of 350–1200 HU, and window center of 50–600 HU, performed with breath hold at full inspiration. All images were de-identified to protect patient privacy, and written consent was waived by the institutional review board, which approved the study. Each case comprises 80–200 slices, capturing various angles and sides of the human chest. The dataset’s diversity in patient demographics (gender, age, educational attainment, area of residence, and occupation, including employees of Iraqi ministries, farmers, and others from Baghdad, Wasit, Diyala, Salahuddin, and Babylon) enhances its representativeness. For compatibility with our framework, DICOM images were converted to PNG format, preserving essential radiological features while aligning with the chest dataset’s format. Data preprocessing To ensure robust model training and evaluation, both datasets underwent standardized preprocessing. Images were resized to pixels to match the input requirements of most CNN and ViT models, as recommended by Dosovitskiy et al.7 for ViTs and Tan & Le4 for EfficientNet. Pixel intensities were normalized to the range [0, 1] using min-max normalization, defined as: where x is the original pixel intensity, and and are the minimum and maximum intensities in the image. For the Chest CT-Scan Images dataset, the available JPG/PNG images were processed in their native public-release format. For the IQ-OTH/NCCD lung cancer dataset, the original DICOM images were converted to PNG using the pydicom library after applying the reported CT window settings. The clinically relevant Hounsfield Unit window was retained during conversion to preserve visible radiological contrast within the lung and lesion regions22. Although PNG itself is a lossless image format, conversion from DICOM to a windowed image representation can reduce radiometric depth and removes scanner-specific metadata that may be available in the original DICOM files. Therefore, the conversion step was used only to harmonize the input format across datasets and to ensure compatibility with the evaluated CNN and ViT pipelines. Data augmentation was applied to mitigate overfitting and enhance generalizability, particularly given the limited dataset sizes. Techniques included random rotations (± 15°), horizontal and vertical flips, scaling (0.8–1.2×), and color-space adjustments (brightness ± 20%, contrast ± 20%), following the approach of Ardila et al.5. The augmentation pipeline can be mathematically described as a series of transformations T: where I is the input image, and , , , and represent the respective transformations. The datasets were divided into training, validation, and test subsets with a 70:20:10 ratio. For the Chest CT-Scan Images dataset, the predefined train, validation, and test folders were used to preserve the original dataset organization. For the IQ-OTH/NCCD lung cancer dataset, splitting was performed at the patient level rather than at the slice level. Each patient case was first assigned to only one subset, and all CT slices belonging to that patient were then inherited by the same subset. This procedure ensured that anatomically adjacent slices, repeated views, or highly correlated images from the same patient could not appear across training, validation, and test sets. Class balance was considered at the patient level where feasible, and no slice-level randomization was applied before patient assignment. This design was adopted specifically to reduce the risk of data leakage and to provide a more realistic estimate of model generalization. Model architectures The comparative framework evaluates a diverse set of CNNs and ViT architectures to examine how different model families behave in CT-based thoracic cancer classification. CNNs are well suited to capturing spatially localized image features through convolutional operations, which makes them effective for identifying radiological structures such as pulmonary nodules, lesion margins, density variations, and other morphology-related patterns16. In contrast, ViT-based architectures use self-attention mechanisms to model longer-range spatial dependencies, allowing them to capture broader contextual relationships that may be relevant when subtype-related imaging patterns are subtle or spatially distributed7. The selected models were therefore used to compare local feature extraction, hierarchical representation learning, and global contextual modeling under a unified experimental protocol. The architecture set was constructed to provide a balanced benchmark across model families, computational scales, and design principles. The selection criteria included: (1) architectural diversity, covering residual, dense, compound-scaled, inception-based, depthwise separable, hierarchical attention, self-attention, and hybrid convolution-transformer designs; (2) a broad parameter range from lightweight models to high-capacity backbones, enabling analysis of accuracy-efficiency trade-offs; (3) prior relevance or frequent use in medical image analysis; and (4) availability of reproducible PyTorch or timm-based implementations. VGG13 was included as a classical CNN baseline because it preserves the core sequential convolutional structure of the VGG family while using fewer layers than VGG16. It was not intended to represent a modern state-of-the-art model, but rather to provide a reference point for assessing the performance gains of more recent CNN and ViT architectures. For the ViT family, ViT-Base with 16 × 16 and 32 × 32 patch sizes was evaluated to examine the effect of patch granularity under the same backbone configuration. This within-backbone comparison was complemented by structurally distinct transformer variants, including Swin-Base, DeiT3-Base, MViTv2-Base, MobileViT, MobileViTv2, PiT, Tiny-ViT, BeiT, MaxViT, FastViT, RepViT, and GcViT. This design allowed both patch-level sensitivity and broader transformer architecture differences to be assessed within the same comparative setting. The following subsections describe the general mathematical formulations of CNNs and ViTs, followed by the specific architectures evaluated in this study. Convolutional neural networks (CNNs) As illustrated in Fig. 1, CNN-based architectures learn hierarchical image representations through successive convolutional, normalization, activation, and pooling operations. In the early layers, these models typically capture low-level visual structures such as edges, intensity transitions, and texture patterns, whereas deeper layers encode increasingly abstract morphology-related representations that may support CT-based thoracic cancer classification. A convolutional operation can be expressed as: CNN-based feature extraction and classification workflow for CT-based thoracic cancer analysis. where denotes the input image or feature map at spatial position and channel c, represents the learnable weights of the k-th convolutional filter with kernel size , is the bias term, is a nonlinear activation function, and is the resulting output feature map. Pooling layers reduce the spatial resolution of feature maps while retaining the most salient activations. For max-pooling, this operation can be formulated as: where denotes the pooling window and sis the stride. The final representation is then passed through fully connected layers or global average pooling, followed by a softmax function to estimate class probabilities: where is the logit for class c, and Cdenotes the number of target classes, corresponding to four classes for the chest subtyping task and three classes for the lung cancer classification task. The CNN architectures included in this study were selected to cover classical residual and dense designs, efficient mobile-oriented networks, compound-scaled models, inception-style architectures, and modern convolutional backbones. ResNet502 was included as a widely used residual CNN baseline with 23.52 M parameters. Its identity skip connections improve gradient propagation in deeper networks and can be expressed as: where denotes the residual mapping and xis the input feature representation. DenseNet1693, with 12.49 M parameters, employs dense connectivity, in which each layer receives the feature maps of all preceding layers: where denotes feature concatenation and is a composite transformation consisting of operations such as convolution, batch normalization, and nonlinear activation. This design promotes feature reuse and parameter efficiency. EfficientNetV2-Small4, with 20.18 M parameters, uses compound scaling and Fused-MBConv blocks to balance model depth, width, and resolution while maintaining computational efficiency. ConvNeXt-Base25, with 87.57 M parameters, represents a modern convolutional design that incorporates large-kernel depthwise convolutions, inverted bottleneck structures, and layer normalization to strengthen feature representation without explicit self-attention. InceptionNeXt-Base26, with 83.61 M parameters, extends inception-style convolutional modeling with modern design principles to capture multi-scale spatial information. A simplified multi-branch representation can be written as: where denotes a convolutional branch with kernel size k. InceptionV427, containing 41.15 M parameters, was evaluated as a strong inception-based CNN that extracts multi-scale representations through parallel convolutional pathways. ReXNet-20028, with 13.82 M parameters, improves representational efficiency through progressive channel expansion. MobileNetV3-Large-10029, with 4.21 M parameters, uses depthwise separable convolutions and lightweight architectural components to reduce computational cost: RepGhostNet-13030, with 4.2 M parameters, employs reparameterized ghost modules to reduce redundant feature computation while preserving discriminative capacity. Additional CNN models included Xception (20.82 M parameters)31, RegNetY-120 (49.59 M parameters)32, RDNet-Base (85.7 M parameters), EfficientNet-B3 (10.7 M parameters)4, and CspDarkNet53. These architectures provide complementary CNN baselines based on depthwise separable convolution, design-space optimization, dense representation learning, compound scaling, and cross-stage partial connectivity, enabling a broad comparison of accuracy, parameter efficiency, and architectural behavior in CT-based thoracic cancer classification. Vision transformers (ViT) Vision Transformer architectures represent an image as an ordered sequence of non-overlapping patches and use self-attention to model spatial relationships across the entire input. This formulation differs from convolutional models, which primarily encode local neighborhoods through fixed receptive fields. As illustrated in Fig. 2, an input image is divided into patches, where each patch has a spatial size of . Each patch is flattened into a vector and projected into a D-dimensional embedding space. A learnable class token and positional embeddings are then added to preserve sequence-level and spatial information: ViT-based patch embedding and self-attention workflow for CT-based thoracic cancer classification. where is the linear patch embedding matrix, denotes positional embeddings, and is a learnable classification token. The resulting token sequence is processed by stacked transformer encoder blocks, each consisting of multi-head self-attention and a feed-forward network with residual connections: where denotes layer normalization, is multi-head self-attention, and is the position-wise feed-forward network. The attention operation is computed as: where Q, K, and V represent the query, key, and value matrices, respectively, and is the dimensionality of the key vectors. Through this mechanism, each patch token can interact with other tokens in the image, enabling the model to encode long-range contextual dependencies that may be relevant for CT-based subtype discrimination and malignancy classification. The final class token representation is passed to a classification head to produce diagnostic class probabilities. The ViT-based models evaluated in this study differ in patch granularity, hierarchical structure, attention design, multi-scale representation, and computational efficiency. The ViT-based architectures were selected to represent different transformer design principles, including standard patch-based self-attention, hierarchical window attention, multi-scale token aggregation, pooling-based token reduction, hybrid convolution-transformer modeling, and lightweight mobile-oriented designs. ViT-Base-Patch16 and ViT-Base-Patch327 were included to examine the effect of patch granularity under the same backbone configuration, with 16 × 16 and 32 × 32 patch sizes corresponding to 85.8 M and 87.46 M parameters, respectively. Swin-Base9, with 86.75 M parameters, was evaluated as a hierarchical transformer that uses shifted-window attention to reduce computational complexity while preserving local structural information. DeiT3-Base11, with 85.82 M parameters, was included because its data-efficient training strategy is relevant for medical imaging scenarios where annotated datasets are often limited. MViTv2-Base33, with 50.71 M parameters, was selected to assess multi-scale attention, which enables the model to integrate local and broader contextual representations at different spatial resolutions. Lightweight and hybrid transformer variants were also included to evaluate the accuracy-efficiency trade-off. MobileViT-base12 and MobileViTv2-10034, with 4.94 M and 4.39 M parameters, respectively, combine convolutional operations with transformer blocks to retain local feature sensitivity while reducing computational cost. PiT-Base35, with 72.74 M parameters, applies pooling-based token reduction to improve efficiency during transformer processing. Tiny-ViT-11 m36, with 10.55 M parameters, represents a compact transformer design developed for efficient inference. Additional transformer-based models, including BeiT-Base20, MaxViT-Base37, NexViT-base, FastViT-SA3638, RepViT-M2, and GcViT-Base39, were included to broaden the comparison across masked-image-modeling, hybrid attention, reparameterized, and global-context transformer families. These models and their variants have been widely investigated in medical image classification, segmentation, lesion detection, and computer-aided diagnosis studies, supporting their relevance as representative transformer baselines for thoracic CT analysis40–46. Results and discussion This section reports the quantitative performance of the evaluated CNN and ViT architectures and interprets their behavior across the two CT-based thoracic oncology tasks. All results were obtained on held-out test subsets that were not used during model training or hyperparameter selection. The analysis focuses not only on predictive performance, but also on class-wise behavior, model complexity, and visual interpretability. Experimental design All experiments were conducted under a unified training protocol to ensure a fair comparison between CNN- and ViT-based architectures. The computational environment consisted of a high-performance workstation equipped with four NVIDIA RTX 5090 GPUs, each with 24 GB of memory, 64 GB RAM, and a 16-core AMD EPYC processor. The implementation was carried out using Python 3.13, PyTorch 2.7, CUDA 12.8, and torchvision 0.15. Each model was trained for a maximum of 50 epochs with a batch size of 32. Stochastic Gradient Descent (SGD) was used as the optimizer with the default momentum setting of the training configuration and a weight decay of . The initial learning rate was set to for CNN-based architectures and for ViT-based architectures. A lower initial learning rate was used for ViT models because transformer-based backbones are generally more sensitive to optimization instability when fine-tuned on relatively small medical imaging datasets. A cosine annealing learning-rate schedule was applied to gradually reduce the learning rate during training: where , for CNN models and for ViT models, and . Early stopping was applied with a patience of 10 epochs based on validation loss to limit overfitting and reduce unnecessary training after convergence. The same preprocessing and augmentation protocol was used across all models. Training images were resized, normalized, and augmented using random rotations, scaling, and flipping, whereas validation and test images were evaluated without stochastic augmentation to preserve a fixed assessment setting. Performance metrics Model performance was assessed using accuracy, precision, recall, and F1-score. For each class, true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) were computed from the confusion matrix. The metrics were defined as follows: Macro-averaging was used to ensure that each class contributed equally to the reported precision, recall, and F1-score. This was particularly important for the Chest CT-Scan Images dataset because of class distribution differences across the four categories. For the IQ-OTH/NCCD lung cancer dataset, macro-averaging similarly ensured balanced evaluation across the normal, benign, and malignant classes. In addition, confusion matrices were used to examine class-level misclassification patterns, and the number of trainable parameters was reported in millions to support the analysis of accuracy-efficiency trade-offs. Where applicable, AUROC values were reported to assess class separability beyond threshold-dependent metrics. Experimental results The experiments were designed to compare CNN and ViT-based architectures across two CT-based thoracic oncology classification tasks with different diagnostic objectives. The first task used the Chest CT-Scan Images dataset for four-class histological subtyping, including Adenocarcinoma, Large cell carcinoma, Squamous cell carcinoma, and normal cases. The second task used the IQ-OTH/NCCD lung cancer dataset for three-class classification of normal, benign, and malignant cases. A broad set of CNN architectures, including ResNet50, DenseNet169, EfficientNetV2-Small, ConvNeXt-Base, and InceptionNeXt-Base, was evaluated alongside ViT-based models such as ViT-Base-Patch16, Swin-Base, DeiT3-Base, MViTv2-Base, and MobileViT. This experimental design enabled a controlled assessment of how convolutional feature extraction, hierarchical representation learning, and attention-based contextual modeling perform under distinct CT classification settings. The evaluation focused on predictive performance, class-wise behavior, computational efficiency, and visual interpretability rather than making claims of immediate clinical deployment. Chest cancer classification results For the histological subtyping task, the Chest CT-Scan Images cohort was evaluated across four diagnostic categories: Adenocarcinoma, Large cell carcinoma, Squamous cell carcinoma, and normal cases. The held-out test subset was used to assess CNN- and ViT-based architectures under the unified evaluation protocol described above. Tables 1 and 2 summarize the corresponding quantitative results, while the subsequent analysis considers class-specific behavior, parameter efficiency, and Grad-CAM-based interpretability to provide a more complete assessment of model performance. Table 1Results of CNN-based models on chest dataset.Model - metricsAccuracyPrecisionRecallF1 ScoreParams (M)Resnet500.86670.87100.88690.876723.52VGG130.92380.91690.93910.9234128.97DenseNet1690.95560.95440.96170.957612.49EfficientNetv2-Small0.95870.95860.96190.959820.18ConvNeXt-Base0.92060.92580.92930.926687.57 InceptionNeXt-Base 0.9683 0.9684 0.9703 0.9692 83.61 RexNet-2000.93020.93760.92770.932413.82Xception0.90790.90580.92710.910420.82MobileNetv3-Large-1000.89840.91810.91040.90804.21RepGhostNet-1300.93650.93700.95370.94314.2InceptionV40.96190.96360.96710.964941.15Regnety_1200.82220.81860.86410.823049.59Rdnet_base0.92060.93150.93430.929585.7EfficientNet-b30.95870.95940.96500.961610.7CspDarkNet530.93020.92530.94680.932726.62 Results of CNN-based models on chest dataset. Table 2Results of ViT-based models on chest dataset.Model - metricsAccuracyPrecisionRecallF1 ScoreParamsViT-Base-Patch160.93020.93540.93950.935085.8ViT-Base-Patch320.96510.96540.96960.967387.46Swin-Base0.95240.95470.95690.954786.75BeiT-Base0.48250.50170.53370.486185.77MViTv2-Base0.94600.94420.95440.947850.71 DeiT3-Base 0.9714 0.9693 0.9780 0.9732 85.82 MobileViT-base0.93970.95020.94030.94444.94MobileViTv2-1000.93330.92840.94600.93464.39PiT-Base0.95240.94960.96270.954872.74Tiny-ViT-11 m0.94600.94230.94940.944610.55MaxViT-Base0.92060.93390.93140.9302118.7NexViT-base0.91430.91200.93360.918643.8FastViT-SA360.88890.89570.90580.891830.51RepViT-M20.89520.88890.92170.89897.78GcViT-Base0.94920.95170.95500.953189.3 Results of ViT-based models on chest dataset. CNN-based model performance The performance of CNN-based models varied significantly, with InceptionNeXt-Base achieving the highest accuracy of 0.9683, precision of 0.9684, recall of 0.9703, F1-score of 0.9692, closely followed by EfficientNet-B3 (F1-score of 0.9616) and EfficientNetV2-Small (F1-score of 0.9598). In contrast, RegNetY_120 recorded the lowest performance, with an F1-score of 0.8230, highlighting the variability in model efficacy. Comparing the CNN-based models, InceptionNeXt-Base stood out as the top performer, leveraging its hybrid architecture combining inception modules with modern design principles to achieve superior feature extraction. However, its high parameter count (83.61 M) indicate greater computational complexity. EfficientNet-B3 and EfficientNetV2-Small offered a compelling balance of performance and efficiency, with F1-scores of 0.9616 and 0.9598, respectively, and lower parameter counts (10.7 M and 20.18 M). Their compound scaling approach optimizes depth, width, and resolution, making them suitable for resource-constrained environments. Lightweight models like MobileNetV3-Large-100 (F1-score of 0.9080, 4.21 M parameters) and RepGhostNet-130 (F1-score of 0.9431, 4.2 M parameters) prioritized efficiency, making them ideal for real-time applications, though they sacrificed some accuracy. RegNetY-120 showed the weakest performance among the CNN models, indicating that architectural capacity alone did not guarantee improved discrimination on this dataset. Its lower performance may reflect reduced alignment between the model design and the subtype-associated CT patterns present in the public-release images. This result also reinforces the importance of evaluating architectures under the same protocol rather than assuming that larger. InceptionV4, EfficientNet-B3, and DenseNet169 also achieved competitive results, with F1-scores of 0.9649, 0.9616, and 0.9576, respectively. DenseNet169 provided an efficient performance profile because of its dense feature-reuse mechanism and relatively low parameter count. In contrast, VGG13 achieved a lower efficiency-performance balance despite its high parameter count, reflecting the limitations of older sequential CNN designs. Vision transformer (ViT)-based model performance As presented in Table 2 the performance of ViT-based models on the chest cancer dataset exhibited significant variability, with DeiT3-Base achieving the highest metrics, including an accuracy of 0.9714, precision of 0.9693, recall of 0.9780, F1-score of 0.9732, demonstrating exceptional capability in distinguishing between Adenocarcinoma, Large cell carcinoma, Squamous cell carcinoma, and normal cases. ViT-Base-Patch32 and PiT-Base followed closely with strong performances, recording F1-scores of 0.9673 and 0.9548, respectively. In stark contrast, BeiT-Base exhibited the lowest performance, with an F1-score of 0.4861, underscoring the diverse effectiveness of ViT architectures on this dataset. The comparison of ViT-based models highlights distinct strengths and trade-offs in their architectural designs, computational efficiency, and suitability for chest cancer classification. DeiT3-Base emerged as the top performer, achieving an F1-score of 0.9732 and recall of 0.9780, driven by its data-efficient training strategy that leverages distilled knowledge to enhance the modelling of complex patterns through self-attention mechanisms. However, its high parameter count (85.82 M) reflect the computational complexity inherent in transformer-based models. ViT-Base-Patch32 followed closely with an F1-score of 0.9673, benefiting from its larger patch size (32 × 32), which allows effective capture of global context, critical for identifying intricate patterns in chest CT images. Its parameter count (87.46 M) are comparable to DeiT3-Base, indicating similar computational demands. PiT-Base also performed strongly, with an F1-score of 0.9548, utilizing a pooling-based architecture that balances performance and efficiency with a parameter count of 72.74 M, making it a competitive alternative for clinical settings where computational resources are a consideration. Among the more efficient ViT models, MobileViT-base and MobileViTv2-100 stood out for their lightweight designs. MobileViT-base achieved an F1-score of 0.9444 with a low parameter count (4.94 M), leveraging a hybrid convolutional-transformer architecture to maintain competitive performance while reducing computational overhead. Similarly, MobileViTv2-100 recorded an F1-score of 0.9346 with the lowest parameter count (4.39 M), making it ideal for real-time clinical applications where speed is prioritized. However, both models sacrifice some accuracy compared to top performers like DeiT3-Base, reflecting a trade-off between efficiency and precision. In contrast, BeiT-Base was the least effective, with an F1-score of 0.4861, likely due to its self-supervised pre-training strategy based on masked image modeling, which may not align well with the CT features of the chest dataset or the JPG/PNG format. This poor performance suggests that extensive fine-tuning is required to adapt BeiT-Base to medical imaging tasks. Other ViT models, such as Swin-Base (F1-score of 0.9547), MViTv2-Base (F1-score of 0.9478), and GcViT-Base (F1-score of 0.9531), performed robustly but fell short of DeiT3-Base and ViT-Base-Patch32. Swin-Base’s hierarchical architecture with shifted windows enhances local context modeling, while MViTv2-Base’s multi-scale design balances global and local feature extraction, with a lower parameter count (50.71 M). GcViT-Base’s global context modeling contributed to its strong performance, though its higher parameter count (89.3 M) make it less efficient. Less competitive models included MaxViT-Base (F1-score of 0.9302), NexViT-base (F1-score of 0.9186), FastViT-SA36 (F1-score of 0.8918), and RepViT-M2 (F1-score of 0.8989). These models, with varying parameter counts (7.78 M to 118.7 M), underperformed due to architectural limitations or insufficient fine-tuning for the chest dataset. Tiny-ViT-11 m, with an F1-score of 0.9446, and low parameter count (10.55 M), offered a lightweight alternative with performance close to larger models, making it a viable option for resource-constrained environments. For the top-performing DeiT3-Base model, class-specific metrics were analyzed across the four diagnostic categories. The confusion matrix in Fig. 3 showed high true-positive rates with limited misclassification: Adenocarcinoma achieved a precision of 0.9826, recall of 0.9417, F1-score of 0.9617, and AUC of 0.9747; Large cell carcinoma achieved a precision of 0.9273, recall of 1.0000, F1-score of 0.9623, and AUC of 0.9895; Squamous cell carcinoma achieved a precision of 0.9674, recall of 0.9889, F1-score of 0.9780, and AUC of 0.9987; and normal cases achieved a precision of 1.000, recall of 0.9815, F1-score of 0.9907, and AUC of 0.9999. These values indicate strong class separability within the held-out test subset. Because the dataset is retrospective and limited in size, the results should be interpreted as benchmark evidence rather than proof of clinical reliability. Confusion matrix and AUC curve extracted by using DeiT3-base on chest images. The visualizations provided by Figs. 3 and 4, and 5 offer critical insights into the performance and interpretability of the ViT-based models, particularly the top-performing DeiT3-Base and the visualization-focused MViTv2-Base. Figure 3 presents the confusion matrix and AUC curve for DeiT3-Base, illustrating its high true positive rates across all classes with minimal misclassifications. The confusion matrix shows that the model correctly classified the vast majority of Adenocarcinoma, Large cell carcinoma, Squamous cell carcinoma, and normal cases, with only rare errors, such as occasional misclassifications of early-stage carcinomas as normal due to subtle feature overlaps. The AUC curve, with values exceeding 0.97 for all classes, visually confirms the model’s exceptional ability to distinguish between classes, with near-perfect separation, particularly for normal cases (AUC of 0.99). This high discriminative power is crucial for clinical reliability, ensuring that the model can accurately differentiate cancerous from non-cancerous cases. Correct predictions and misclassified predictions by applying DeiT3-base to chest images. Grad-CAM visualizations showing image regions contributing to MViTv2-Base predictions across subtype categories and normal CT cases. Figure 4 illustrates representative correct and misclassified predictions produced by DeiT3-Base. Correct predictions generally corresponded to CT images with more distinctive subtype-associated radiological patterns, including lesion morphology, density variation, margin characteristics, and surrounding parenchymal appearance. Misclassified cases appeared visually more ambiguous, with overlapping CT patterns or low-contrast findings that may reduce subtype separability. These examples highlight the difficulty of predicting histologically defined lung cancer subtypes from CT images, where the model does not directly observe microscopic tumor architecture but instead learns radiological patterns associated with pathology-defined labels. Figure 5 presents Grad-CAM visualizations using the MViTv2-Base model, highlighting image regions that contributed most strongly to the classification decisions. Grad-CAM generates class-discriminative localization maps by weighting deep feature activations according to their contribution to the target class, providing a visual approximation of the image regions used by the model during prediction46. In medical image analysis, such visual explanations are commonly used to examine whether model attention is concentrated on anatomically or radiologically plausible regions rather than irrelevant background patterns. As depicted in Fig. 5, the activation maps were generally concentrated around thoracic structures and lesion-related regions that were visually relevant to the predicted class. For the cancer subtype categories, the highlighted regions corresponded to abnormal parenchymal or mass-associated CT patterns rather than peripheral background areas. For normal cases, the attention maps were more diffusely distributed over non-lesion regions. These findings support the qualitative interpretability of MViTv2-Base in this dataset. However, Grad-CAM remains an auxiliary visual explanation method, and expert reader validation is required before such attention patterns can be linked to clinical decision-making. Lung cancer classification results The lung cancer dataset, comprising normal, benign, and malignant cases from CT scans, was used to evaluate the same set of models, with results summarized in Tables 3 and 4. This section provides a deep analysis of the performance of CNN-based and ViT-based models, including detailed comparisons of their effectiveness, class-specific performance, and visualization insights, with a comprehensive explanation of the associated figures. Table 3Results of CNN-based models on lung dataset.Model - metricsAccuracyPrecisionRecallF1-scoreParams (M) Resnet50 0.95450.93330.88100.901523.51VGG130.93640.87810.86510.8711128.96DenseNet1690.97270.97780.91670.940912.49 EfficientNetv2-Small 0.9909 0.9922 0.9722 0.9816 20.18 ConvNeXt-Base 0.9909 0.9922 0.9722 0.9816 87.57 InceptionNeXt-Base0.98180.98480.94440.961983.61 RexNet-200 0.9909 0.9922 0.9722 0.9816 13.81 Xception0.98180.98480.94440.961920.81 MobileNetv3-Large-100 0.9909 0.9922 0.9722 0.9816 4.21 RepGhostNet-130 0.9909 0.9922 0.9722 0.9816 4.2 InceptionV4 0.9909 0.9922 0.9722 0.9816 41.15 Regnety_1200.96360.94390.90870.923949.59Rdnet_base0.97270.95420.93650.944885.7EfficientNet-b30.96360.94390.90870.923910.7CspDarkNet530.97270.97780.91670.940926.62 Results of CNN-based models on lung dataset. Table 4Results of ViT-based models on lung dataset.Model - metricsAccuracyPrecisionRecallF1-scoreParams (M)ViT-Base-Patch160.94550.95830.85520.887185.8ViT-Base-Patch320.96360.94800.92860.937787.46 Swin-Base 0.9909 0.9922 0.9722 0.9816 86.75 BeiT-Base0.94550.95830.87700.905185.76 MViTv2-Base 0.9909 0.9922 0.9722 0.9816 50.71 DeiT3-Base0.97270.97780.91670.940985.82MobileViT-base0.96360.94390.90870.92394.94MobileViTv2-1000.97270.97780.91670.94094.39PiT-Base0.98180.98480.94440.961972.74Tiny-ViT-11 m0.96360.97100.88890.918210.55MaxViT-Base0.97270.97780.91670.9409118.7NexViT-base0.97270.97780.91670.940943.8FastViT-SA360.97270.97780.91670.940930.51RepViT-M20.98180.98480.94440.96197.78GcViT-Base0.95450.91610.90080.907989.3 Results of ViT-based models on lung dataset. CNN-based model performance The CNN-based models demonstrated remarkable performance on the lung cancer dataset, with several models achieving top-tier results. EfficientNetV2-Small, ConvNeXt-Base, RexNet-200, MobileNetV3-Large-100, RepGhostNet-130, and InceptionV4 all recorded an accuracy of 0.9909, precision of 0.9922, recall of 0.9722, and F1-score of 0.9816, indicating exceptional capability in distinguishing normal, benign, and malignant cases. InceptionNeXt-Base and Xception followed closely with F1-scores of 0.9619 respectively. In contrast, VGG13 exhibited the lowest performance with an F1-score of 0.8711, highlighting significant variability in CNN model efficacy. The comparison of CNN-based models reveals distinct strengths and trade-offs in their architectural designs, computational efficiency, and suitability for lung cancer classification. EfficientNetV2-Small, ConvNeXt-Base, RexNet-200, MobileNetV3-Large-100, RepGhostNet-130, and InceptionV4 emerged as top performers, all achieving an F1-score of 0.9816. EfficientNetV2-Small stood out for its balance of high performance and efficiency, with a moderate parameter count (20.18 M), leveraging its compound scaling approach to optimize depth, width, and resolution. ConvNeXt-Base matched this performance but required a higher parameter count (87.57 M), reflecting its modern convolutional design with transformer-inspired features. RexNet-200, with a lower parameter count (13.81 M), offered a lightweight alternative with identical performance, benefiting from its efficient scaling strategy. MobileNetV3-Large-100 and RepGhostNet-130, with the lowest parameter counts (4.21 M and 4.2 M), were highly efficient, making them ideal for real-time clinical applications, though their lightweight designs slightly compromise robustness compared to EfficientNetV2-Small. InceptionV4, with a parameter count of 41.15 M, leveraged inception modules to capture multi-scale features, achieving top performance but with higher computational demands. InceptionNeXt-Base and Xception, with F1-scores of 0.9619, performed strongly but fell short of the top tier due to slightly lower recall (0.9444). InceptionNeXt-Base’s hybrid architecture (83.61 M parameters) enhanced feature extraction, while Xception’s depthwise separable convolutions (20.81 M parameters) offered efficiency but less discriminative power. DenseNet169 and CspDarkNet53, both with F1-scores of 0.9409, benefited from dense connectivity and efficient feature reuse, with lower parameter counts (12.49 M and 26.62 M). RdNet_base (F1-score: 0.9448) and EfficientNet-B3 (F1-score: 0.9239) performed well but were outperformed by top models, likely due to less optimized architectures for the lung dataset. RegNetY_120 (F1-score: 0.9239) and VGG13 (F1-score: 0.8711) were the least effective, with VGG13’s high parameter count (128.96 M) and outdated architecture limiting its efficiency and performance. The baseline ResNet50 (F1-score: 0.9015) was outperformed by most advanced architectures, underscoring improvements in modern CNN designs. Vision transformer (ViT)-based model performance ViT-based models also showed strong performance on the lung cancer dataset, with Swin-Base and MViTv2-Base achieving the highest metrics (accuracy: 0.9909, precision: 0.9922, recall: 0.9722, F1-score: 0.9816), matching the top CNN models. PiT-Base and RepViT-M2 followed with F1-scores of 0.9619, while ViT-Base-Patch16 recorded the lowest performance (F1-score: 0.8871). The high performance of multiple ViT models underscores their effectiveness in handling the lung dataset’s characteristics. Among ViT-based models, Swin-Base and MViTv2-Base achieved the highest performance (F1-score: 0.9816), matching top CNN models. Swin-Base’s hierarchical architecture with shifted windows enhanced local context modeling, with a parameter count of 86.75 M. MViTv2-Base, with a lower parameter count (50.71 M), balanced global and local feature extraction through its multi-scale design, making it a strong contender. PiT-Base and RepViT-M2, with F1-scores of 0.9619, performed well, with PiT-Base’s pooling-based architecture (72.74 M parameters) and RepViT-M2’s lightweight design (7.78 M parameters) offering efficiency. DeiT3-Base, MaxViT-Base, NexViT-base, and FastViT-SA36 all recorded F1-scores of 0.9409, with varying parameter counts (30.51 M to 118.7 M), but were less effective than Swin-Base due to lower recall (0.9167). MobileViTv2-100 (F1-score: 0.9409, 4.39 M parameters) and MobileViT-base (F1-score: 0.9239, 4.94 M parameters) offered lightweight alternatives, ideal for resource-constrained settings. ViT-Base-Patch32 (F1-score: 0.9377) and Tiny-ViT-11 m (F1-score: 0.9182) were competitive, while ViT-Base-Patch16 (F1-score: 0.8871) and BeiT-Base (F1-score: 0.9051) underperformed, likely due to suboptimal adaptation to the lung dataset. GcViT-Base (F1-score: 0.9079) was also less effective, reflecting challenges in global context modeling. The visualizations provided by Figs. 6 and 7, and 8 offer critical insights into the performance and interpretability of the models, particularly for the top-performing Swin-Base (ViT) and EfficientNetV2-Small (CNN). Figure 6 presents the confusion matrix and AUC curve for Swin-Base, illustrating its high true positive rates across all classes with minimal misclassifications. The confusion matrix shows that Swin-Base correctly classified nearly all malignant, benign, and normal cases, with rare errors primarily occurring in benign cases misclassified as normal due to subtle feature overlaps, such as small nodules resembling normal tissue. The AUC curve, with values of over 0.98 for all classes, visually confirms the model’s exceptional discriminative power, particularly for malignant cases, where clear separation is critical for accurate diagnosis. This high AUC underscores Swin-Base’s reliability for clinical applications, ensuring robust differentiation between cancerous and non-cancerous cases. Confusion matrix and AUC curve extracted by using Swin-base on lung images. Correct predictions and misclassified predictions by applying Swin-base to lung images. Grad-CAM visualizations highlighting the image regions contributing to EfficientNetV2-Small predictions across normal, benign, and malignant lung CT classes. Figure 7 illustrates correct and misclassified predictions by Swin-Base, providing a qualitative understanding of its performance. Correct predictions typically involved images with distinct features, such as well-defined pulmonary nodules in malignant cases, uniform tissue structures in normal cases, or intermediate patterns in benign cases. For malignant cases, the model accurately identified irregular, high-density nodules, while for normal cases, it focused on homogenous lung parenchyma. Misclassifications, though rare, occurred in cases with ambiguous features, such as small or low-contrast nodules in benign cases that resembled normal tissue, or early-stage malignant cases with minimal structural abnormalities. These errors highlight the challenge of classifying cases with subtle differences, suggesting the need for enhanced dataset diversity or advanced feature extraction techniques to improve robustness. Figure 8 presents Grad-CAM visualizations using EfficientNetV2-Small, highlighting the regions of interest influencing classification decisions. For malignant cases, the model focused on regions with pulmonary nodules, characterized by irregular shapes, high density, and spiculated margins, aligning with clinical diagnostic criteria. For benign cases, it emphasized areas with smaller, well-defined nodules or calcifications, distinguishing them from malignant patterns. For normal cases, the visualizations highlighted uniform lung parenchyma, avoiding false positives by focusing on the absence of pathological features. These visualizations confirm that EfficientNetV2-Small attends to clinically relevant regions, enhancing its interpretability and potential for clinical adoption. The alignment between the model’s attention and radiological features, such as nodule size and morphology, underscores its reliability for lung cancer diagnosis. Discussion Task-specific performance of CNN and ViT architectures CNN and ViT architectures are better interpreted as task-dependent model families rather than as approaches with a single dominant winner. Their behavior was strongly task-dependent. Histological subtyping and malignancy detection rely on different visual cues in CT images. Subtype classification requires discrimination of subtle inter-class patterns, whereas malignancy detection is more closely related to localized structural abnormalities such as nodule morphology, density variation, and margin irregularity. This distinction helps explain why different architectures performed best across the two diagnostic settings. For the Chest CT-Scan Images dataset, DeiT3-Base achieved the highest performance among all evaluated models, with an F1-score of 0.9732. This result suggests that attention-based contextual modeling can be advantageous when subtype-related differences are subtle and spatially distributed. InceptionNeXt-Base also achieved strong performance among CNNs, with an F1-score of 0.9692. The small gap between these models is important because it shows that transformer-based designs do not automatically replace modern CNNs. Instead, their benefit appears most evident when broader spatial context contributes to class separation. For the IQ-OTH/NCCD lung cancer dataset, the strongest results were obtained by EfficientNetV2-Small, Swin-Base, and MViTv2-Base, each reaching an F1-score of 0.9816. This pattern suggests that malignancy detection in this dataset can be captured effectively by both efficient convolutional representations and hierarchical attention mechanisms. EfficientNetV2-Small is particularly relevant because it achieved top-tier performance with a moderate parameter count, making it a strong candidate for accuracy-efficiency trade-off analysis. Swin-Base and MViTv2-Base also performed well, likely because their hierarchical designs preserve local structure while incorporating wider contextual information. These results clarify the main contribution of the study. The work does not claim superiority of a newly proposed architecture. Its contribution lies in a controlled benchmark of 30 contemporary CNN and ViT models across two clinically distinct CT-based thoracic oncology tasks. This design allows the influence of architectural inductive bias to be examined under comparable experimental conditions. CNNs remained highly competitive for localized structural patterns, while ViT-based models showed particular strength when broader contextual modeling was beneficial. The evidence therefore supports a complementary interpretation of CNNs and ViTs rather than a universal preference for one model family. Interpretability, reliability, and clinical implications The confusion matrices and Grad-CAM visualizations provided additional context for interpreting the numerical results. For histological subtyping, DeiT3-Base produced high class-wise discrimination across Adenocarcinoma, Large cell carcinoma, Squamous cell carcinoma, and normal images, with errors mainly occurring in visually challenging cases where subtype boundaries may be less distinct. For lung cancer classification, the leading models showed strong separation among normal, benign, and malignant classes, while the remaining errors were mostly associated with subtle or low-contrast findings. These patterns are consistent with the known difficulty of separating benign abnormalities from early malignant or normal-appearing structures in CT images. The Grad-CAM maps supported the quantitative findings by showing that the high-performing models generally focused on lesion-containing or anatomically relevant regions rather than unrelated image areas. This is important because high classification performance alone does not confirm that a model is relying on clinically meaningful evidence. At the same time, Grad-CAM should be interpreted as a qualitative interpretability tool, not as proof of clinical reasoning. Expert reader assessment and prospective validation are still required to determine whether these attention patterns are sufficiently reliable for decision-support use. Several methodological choices were used to reduce optimistic bias. For the IQ-OTH/NCCD dataset, the split was performed at the patient level so that all slices from the same patient remained within a single subset. This is essential for CT-based classification because adjacent slices from one patient can contain highly correlated anatomical information. Training-time augmentation, early stopping, and learning-rate scheduling were also used to reduce overfitting. Nevertheless, the relatively small test sets limit the statistical certainty of model ranking, especially when several architectures achieve closely matched performance. The findings are therefore best interpreted as controlled benchmark evidence rather than clinical validation. The datasets were retrospective and publicly available, and the Chest CT-Scan Images dataset was available only in released JPG/PNG format. Although the IQ-OTH/NCCD images originated from DICOM data, conversion to a windowed PNG representation may reduce radiometric depth and remove scanner-specific metadata. These constraints may affect subtle texture information and limit generalization to native clinical DICOM workflows. Future studies should validate these architectures on larger multi-center DICOM-based cohorts with preserved acquisition metadata, external test sets, confidence intervals, calibration analysis, and reader-study evaluation. Such work is necessary before these models can be considered as candidates for clinical decision-support systems. Limitations and future work Several limitations should be considered when interpreting the findings of this study. First, the experiments were conducted using retrospective, publicly available datasets. Although these datasets provide a useful basis for controlled benchmarking, their scale remains limited, particularly for evaluating 30 high-capacity CNN and ViT architectures. The relatively small test sets increase uncertainty in model ranking, especially when the performance differences between top models are narrow. To reduce optimistic bias, the IQ-OTH/NCCD dataset was divided at the patient level, training-time augmentation was applied only to the training subset, and early stopping with learning rate scheduling was used during optimization. Nevertheless, these safeguards cannot replace validation on larger and more diverse external cohorts. Second, the use of publicly released imaging data introduces source-related constraints. The Chest CT-Scan Images dataset was available only in JPG/PNG format, whereas the IQ-OTH/NCCD dataset required harmonization into the same input pipeline. This design allowed a consistent comparison across architectures, but it may also limit access to acquisition parameters, reconstruction details, scanner-specific metadata, and full radiometric information. Such factors can influence subtle texture and contrast patterns that may be relevant for differentiating benign, malignant, and histological subtype-related features. Therefore, the reported results should be interpreted as benchmark evidence under public-dataset conditions rather than as proof of readiness for direct clinical deployment. Future work should prioritize larger multi-center cohorts with standardized acquisition documentation, preserved imaging metadata, and harmonized preprocessing protocols. External validation should include confidence intervals, calibration analysis, subgroup evaluation, and testing across different scanner settings and patient populations. Reader-study evaluation will also be necessary to determine whether model outputs improve diagnostic consistency, interpretation time, or decision confidence when used alongside radiologists. Methodologically, future studies may examine uncertainty-aware prediction, domain-specific pretraining, carefully controlled CNN-ViT fusion, and computational optimization for resource-limited environments. These steps would provide stronger evidence for whether the evaluated architectures can progress from retrospective benchmarking toward clinically useful decision-support candidates. Conclusion This study provides a controlled comparative benchmark of 30 CNN and ViT architectures for two CT-based thoracic oncology tasks: CT-based prediction of histologically defined lung cancer subtypes and malignancy-oriented lung cancer classification. DeiT3-Base achieved the strongest performance for subtype prediction, with an F1-score of 0.9732, whereas EfficientNetV2-Small, Swin-Base, and MViTv2-Base achieved the highest performance for normal, benign, and malignant lung CT classification, each reaching an F1-score of 0.9816. These findings indicate that architectural performance is task-dependent. Attention-based models were particularly effective when subtype prediction required broader contextual modeling, while efficient CNNs and hierarchical transformers remained highly competitive for localized structural abnormalities, including nodule-related morphology, density variation, and margin characteristics. Grad-CAM visualizations further supported the quantitative results by showing that high-performing models generally focused on lesion-related or anatomically plausible thoracic regions rather than unrelated background areas. The findings therefore suggest that CNNs and ViTs should be viewed as complementary model families for CT-based thoracic cancer analysis rather than as competing approaches with a single universally superior architecture. Nevertheless, the results remain controlled benchmark evidence because the study relied on retrospective public datasets, limited test-set sizes, heterogeneous image formats, and no external clinical validation. Future investigations using larger multi-center imaging cohorts, preserved acquisition and reconstruction metadata, standardized preprocessing, calibration analysis, and reader-study evaluation are required before these models can be considered reliable candidates for clinical decision-support workflows. Acknowledgements The authors thank KAU Endowment (WAQF) and the Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, Saudi Arabia, for their technical and financial support. The authors also acknowledge the contributors and curators of the publicly available IQ-OTH/NCCD Lung Cancer Dataset and Chest CT-Scan Images Dataset. Author contributions Mustafa TAHSIN: Conducted experiments with model architectures, generated visualisations including Figs. 3 and 6–12; Tables 1, 2, 3, 4, 5 and 6, and played a key role in managing computational resources and securing funding for the study.Enes ALGUL: Contributed to drafting the initial manuscript, designed the CNN–ViT ensemble strategy outlined in Sect. 3.5, and generated visualisations to demonstrate model performance and behaviour.Ishak PACAL: Reviewed and revised the manuscript, developed software tools for model evaluation and ensemble integration, managed datasets (including Chest CT-Scan Images, IQ-OTH/NCCD lung cancer dataset, and others), and designed data preprocessing pipelines to ensure data quality and consistency. Funding This work was funded by KAU Endowment (WAQF) at King Abdulaziz University, Jeddah, Saudi Arabia. The authors acknowledge WAQF and the Deanship of Scientific Research (DSR) at King Abdulaziz University for their technical and financial support. Data availability The datasets analyzed during the current study are available in the following public repositories:1. The IQ-OTH/NCCD Lung Cancer Dataset: https://www.kaggle.com/datasets/hamdallak/the- iqothnccd-lung-cancer-dataset2. The Chest CT-Scan images dataset: https://www.kaggle.com/datasets/mohamedhanyyy/chest- ctscan-images. Code availability The source code, preprocessing scripts, model training configuration files, and evaluation scripts used in this study are available from the corresponding author upon reasonable request. Declarations Competing interests The authors declare no competing interests. Ethics approval and consent to participate This study used publicly available and fully de-identified imaging datasets. For the IQ-OTH/NCCD Lung Cancer Dataset, ethics approval and waiver of informed consent were granted by the Institutional Review Board of the Iraq-Oncology Teaching Hospital/National Center for Cancer Diseases (IQ-OTH/NCCD), Baghdad, Iraq. No additional ethical approval was required for the publicly available Chest CT-Scan Images Dataset because the data were already anonymized and released for research use. Human and animal rights This study did not involve animal experiments. The human imaging data used in the study were obtained from publicly available, fully de-identified datasets. The study was conducted in accordance with applicable ethical standards for secondary analysis of anonymized research data. Informed consent For the IQ-OTH/NCCD Lung Cancer Dataset, informed consent was waived by the Institutional Review Board of the Iraq-Oncology Teaching Hospital/National Center for Cancer Diseases (IQ-OTH/NCCD), Baghdad, Iraq. No additional informed consent was required for the Chest CT-Scan Images Dataset because it consists of publicly available anonymized data. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The datasets analyzed during the current study are available in the following public repositories:1. The IQ-OTH/NCCD Lung Cancer Dataset: https://www.kaggle.com/datasets/hamdallak/the- iqothnccd-lung-cancer-dataset2. The Chest CT-Scan images dataset: https://www.kaggle.com/datasets/mohamedhanyyy/chest- ctscan-images. The source code, preprocessing scripts, model training configuration files, and evaluation scripts used in this study are available from the corresponding author upon reasonable request.