A robust stacked ensemble strategy with multi-optimizer CNN models for skin cancer classification Authors: Arvind Panwar, Jyoti Agarwal, Shruti Vashist, Yogita Arora, Achin Jain, Ondrej Krejcar, Robert Frischer, Hamidreza Namazi Published: 2026 ABSTRACT AbstractSkin cancer is one of the most prevalent and potentially life-threatening cancers globally, making it a critical area of focus in medical research. Early detection, followed by timely and appropriate treatment, can significantly enhance patient survival outcomes. In recent years, computer-aided diagnostic systems have gained substantial attention for their ability to support accurate and efficient skin cancer diagnosis. This study presents an optimized stacked ensemble strategy to improve skin lesion classification performance. Experiments were conducted on the Melanoma Skin Cancer Dataset using four different convolutional neural network (CNN) architectures, each trained with different optimizers, including Adam, Stochastic Gradient Descent (SGD), Root Mean Square Propagation (RMSprop), and AdaMax (Adamax). All models were trained using categorical cross-entropy loss and evaluated based on validation accuracy, with hyperparameters such as learning rate, batch size, and number of epochs kept constant. More than 90 model combinations were tested using four ensemble strategies: soft and hard voting, weighted voting (using grid search), and stacked ensemble using various meta-learners (Gradient Boosting (GB), k-Nearest Neighbors (KNN), Support Vector Machine (SVM), Logistic Regression (LR), and Naïve Bayes (NB)). The models and ensembles were assessed using standard performance metrics including accuracy, precision, recall, F1-score, and Area Under the Receiver Operating Characteristic Curve (ROC-AUC). Among the evaluated configurations, the stacked ensemble model with Gradient Boosting (GB) as the meta-classifier and M1+M2+M3+M4 as the deep learning base models achieved a validation accuracy of 97.4% and a robust independent test accuracy of 91.80% with a ROC-AUC of 96.87%. The findings demonstrate that ensemble learning, particularly stacked ensembling, significantly enhances classification accuracy and strengthens the reliability of melanoma detection on unseen datasets.Keywords: Skin cancer, CNN, Ensemble learning, OptimizerSubject terms: Cancer, Computational biology and bioinformatics, Mathematics and computing Introduction Skin cancer is among the most common and rapidly increasing forms of cancer worldwide, with millions of new cases diagnosed annually. The two main types are skin cancers of melanoma and non-melanoma. Among these, melanoma poses a significant public health concern, requiring accurate diagnostic methods for effective treatment1. Traditional diagnostic approaches, which are based on clinical expertise and dermoscopic examination, are often subjective and prone to human error. The American Cancer Society has projected a substantial increase in melanoma cases, emphasizing the urgent need for improved diagnostic tools2. With the growing availability of dermoscopic image datasets and advances in computational resources, artificial intelligence (AI), particularly deep learning (DL), has emerged as a powerful solution for automated and accurate detection of skin cancer. Deep learning has gained great popularity in medical image analysis and has demonstrated remarkable accuracy in the classification of skin cancer3,4. In particular, Convolutional Neural Networks (CNNs) have achieved state-of-the-art results in medical imaging tasks and are now widely recognized as the most effective DL technique for skin lesion analysis. CNNs are capable of learning complex data patterns, enabling fast and accurate predictions at relatively low cost. However, single CNN architectures may suffer from overfitting and limited generalization capability. Furthermore, CNN performance is strongly influenced by optimization algorithms, which play a critical role in improving convergence and stability. Optimization techniques are designed to identify the best possible parameters that minimize the loss function and maximize performance5. These algorithms significantly affect training speed, stability, and model accuracy. Therefore, the choice of optimizer is crucial, and exploring multiple optimizers can further enhance classification performance and reliability3. In addition, ensemble learning has recently gained traction for its ability to combine the strengths of multiple DL models while mitigating individual weaknesses6. In such approaches, weak or base learners are combined using strategies such as averaging, weighted voting, or stacking. Stacked ensembles, in particular, leverage the diversity of multiple models and use a meta-learner to aggregate predictions, thereby improving robustness and overall performance. In this study, we propose an optimized stacked ensemble deep learning framework to improve the accuracy and reliability of melanoma detection from dermoscopic images. For experimentation, we used a publicly available dataset containing 10,605 dermoscopic images labeled melanoma and non-melanoma7. More than 90 model combinations were tested using four ensemble strategies: soft and hard voting, weighted voting (using grid search), and stacked ensemble using various meta-learners (Gradient Boosting (GB), k-Nearest Neighbors (KNN), Support Vector Machine (SVM), Logistic Regression (LR), Naïve Bayes (NB)). Models and ensembles were assessed using standard performance metrics that included precision, precision, recall, F1-score, and ROC-AUC. The key contributions of this study are outlined as follows: Implementation and training of four distinct CNN architectures using multiple optimizers (Adam, SGD, RMSprop, Adamax) on the Melanoma Skin Cancer dataset, ensuring diversity in model learning behavior for ensemble creation. Implementation and training of four distinct CNN architectures using multiple optimizers (Adam, SGD, RMSprop, Adamax) on the Melanoma Skin Cancer dataset, ensuring diversity in model learning behavior for ensemble creation. Design and evaluation of optimized ensemble strategies including soft/hard voting, weighted voting (via grid search), and stacked ensemble with various meta-learners GB, KNN, SVM, LR and NB to enhance skin lesion classification performance. Design and evaluation of optimized ensemble strategies including soft/hard voting, weighted voting (via grid search), and stacked ensemble with various meta-learners GB, KNN, SVM, LR and NB to enhance skin lesion classification performance. Demonstration of improved accuracy, stability and generalization in melanoma detection by integrating optimizer-driven model diversity with ensemble learning techniques. Demonstration of improved accuracy, stability and generalization in melanoma detection by integrating optimizer-driven model diversity with ensemble learning techniques. Extensive performance comparison using standard classification metrics such as accuracy, precision, recall, F1-score, and ROC-AUC to validate the effectiveness of the proposed approach. Extensive performance comparison using standard classification metrics such as accuracy, precision, recall, F1-score, and ROC-AUC to validate the effectiveness of the proposed approach. The remainder of this paper is organized as follows. Section 2 reviews related work, Section 3 describes the proposed methodology, Section 4 presents results and discussion, and Section 5 concludes the study with future directions. Literature review Several researchers have extensively investigated the detection and diagnosis of skin cancer. The increasing incidence of skin cancer, largely attributed to environmental factors such as air pollution and ozone depletion, has accelerated efforts to develop computer-aided systems for early diagnosis. These systems leverage machine learning (ML) and deep learning (DL) techniques to improve accuracy, efficiency, and reliability in the detection of skin cancer from dermoscopic images. This section reviews recent studies and approaches in this domain. A feed-forward neural network was applied to a data set of 326 dermoscopic images, achieving an accuracy of 86%8. In 2022, a CNN-based computational model was developed, obtaining 81% accuracy on a clinical dataset9. Transfer learning approaches using models such as MobileNet, Xception, and ResNet50 have also been used for skin cancer classification10. Similarly, the VGG, CapsNet and ResNet architectures were tested and demonstrated promising results11. Tembhurne12 reported a model with accuracy of 93% and recall scores of 99.7% and 86% for benign and malignant cases, respectively. Ensemble learning has been widely adopted to improve performance by integrating multiple learners. For example, an ensemble of VGG, CapsNet, and ResNet achieved superior accuracy compared to individual models13. In 2023, an IoT-enabled DL model was introduced, ODL-SCDC, which collected abnormal skin images via IoT devices and applied hyperparameter optimization, achieving a sensitivity of 97.74%, specificity of 99.7%, and precision of 99.55%14. Another ensemble of VGG16, CapsNet and ResUNet achieved 86% accuracy on the ISIC dataset, outperforming individual learners (80%, 74%, and 76%, respectively)15. Similarly, a Max Voting ensemble produced a accuracy of 93.18%16. Blending ensemble approaches were also explored for the detection of melanoma mutations, yielding 93% precision17. An ensemble of seven deep learning models achieved 93.36% accuracy on one ISIC dataset and 85.54% on another, outperforming weighted ensembles18. A hybrid model combining Xception and MobileNetV2 reached 97.56% accuracy19. In 2024, stacking ensembles of traditional ML algorithms achieved remarkable results, with SVM (98.61%), Random Forest (97.91%), Decision Tree (95. 13%) and Gradient Boosting (95.83%), while the stacking method outperformed all with 99.3% accuracy20. Natha et al.6 proposed a Max Voting ensemble trained on two datasets, achieving 95.80% accuracy. Other researchers highlighted that existing DL approaches often focused on narrow subsets of skin diseases and were limited by small data sets. To address this, an ensemble model trained on the DermNet dataset achieved 91.73% accuracy21. Combining ResNet101 and ResNeXt101 yielded 96.12% accuracy for early-stage detection22. A hybrid CNN and Grey Wolf Optimization (GWO) model improved segmentation and edge detection, achieving 92.66% accuracy23. Two-stage ensemble approaches have also been explored. For example, a computer-aided diagnosis system fine-tuned three models in the first stage and ensembled them in the second, achieving 91.5% accuracy24. A Max Voting ensemble combining three ML models on HAM10000 and ISIC2018 datasets achieved 94.70% accuracy25. In addition, a fuzzy logic-based ensemble that integrates Xception, InceptionResNetV2 and MobileNetV2 on HAM10000 produced 95.14% accuracy26. In summary, the literature reveals that ensemble learning strategies, whether based on CNNs, hybrid DL models, or ML classifiers, consistently outperform individual learners in the detection of skin cancer. These approaches demonstrate a strong potential for robust, accurate, and reliable melanoma classification. Methodology This section presents the proposed methodology for developing an optimized stacked ensemble of deep learning models, where different optimizers are used to improve the performance of skin cancer detection from dermoscopic images. The approach consists of five main stages: data preprocessing and augmentation, base learner selection, optimizer tuning, stacked ensemble construction, and performance evaluation. The general framework is illustrated in Fig. 1. Flowchart of the proposed methodology. Dataset description and preprocessing The experiments were carried out using the HAM10000 dermoscopic image collection27 accessed through the Kaggle mirror cited in7. Although HAM10000 is originally a seven-class dataset, this study focused on a binary classification task by grouping the diagnostic categories into malignant and benign classes. Melanoma (MEL) and basal cell carcinoma (BCC) were treated as malignant, while dermatofibroma (DF), benign keratosis-like lesions (BKL), melanocytic nevi (NV), vascular lesions (VASC), and actinic keratoses and intraepithelial carcinoma / Bowen’s disease (AKIEC) were treated as benign28.The total data consists of 10,605 dermoscopic RGB images. This data is organized into two distinct folders: a training set consisting of 9,605 images (5,000 benign and 4,605 malignant) and an independent test set containing 1,000 images (500 images for each class) reserved for final evaluation. The dataset was preprocessed by resizing all images to a uniform size of pixels and normalizing pixel intensity values to the range [0, 1] using rescale=1./255. To improve model generalization and reduce overfitting, data augmentation techniques were applied during training, including random rotations of up to , width and height shifts of ±20%, shear distortion with a range of 0.2, isotropic zooming of ±20%, and horizontal flipping. All images used in both the training and independent test folders are standardized to dimensions of pixels and are in RGB color format, ensuring consistency in the input data for the models. Representative samples from the dataset are shown in Fig. 2. Sample images from the Melanoma skin cancer dataset. Convolutional neural network A Convolutional Neural Network (CNN) is an extension of the Multilayer Perceptron (MLP), designed specifically to process two-dimensional data such as images. CNNs are characterized by their deep architectures and are widely applied to image datasets. In CNNs, neurons are associated with weights, biases, and activation functions. Figure 3 illustrates the basic CNN architecture, which consists of a convolutional layer with ReLU activation, a pooling layer for feature extraction, and a fully connected layer with softmax activation for classification. Convolutional neural network architecture. Convolutional layer As shown in Fig. 3, the first layer in the CNN architecture is the convolutional layer, which processes the input image. The input is convolved with filters (kernels) to extract local features, producing a feature map representation. Activation: ReLU The activation function used in CNNs is typically the Rectified Linear Unit (ReLU), which introduces non-linearity into the network and accelerates the training process. ReLU sets all negative input values to zero while keeping positive values unchanged29. Pooling layer The pooling layer progressively reduces the spatial dimensions of the feature maps, thereby decreasing the number of parameters and controlling overfitting. Two common pooling methods are max pooling, which selects the maximum value, and average pooling, which computes the mean of the values within a region. Fully connected layer At the final stage of the CNN, the fully connected (FC) layer integrates all extracted features by connecting every neuron from the previous layers. The multidimensional feature maps are flattened into one-dimensional vectors30, which are then passed through dense layers. A softmax activation function is applied to perform multi-class classification. Hyperparameters All models were trained using categorical cross-entropy loss and evaluated based on validation accuracy. Hyperparameters such as learning rate, batch size, and the number of epochs were kept consistent across all models to ensure a fair comparison. CNN model with optimizers (base learner selection) In this work, CNN models with identical architectures but trained using four different optimizers were implemented as the baseline models. Model M1: CNN trained with the Adam optimizer Model M1: CNN trained with the Adam optimizer Model M2: CNN trained with the SGD optimizer Model M2: CNN trained with the SGD optimizer Model M3: CNN trained with the RMSProp optimizer Model M3: CNN trained with the RMSProp optimizer Model M4: CNN trained with the AdaMax optimizer Model M4: CNN trained with the AdaMax optimizer Adaptive moment estimation (Adam) optimizer Adam is an optimization algorithm that combines the benefits of adaptive learning rate methods and momentum-based techniques31. It is widely used due to its robustness and effectiveness in handling varying gradient scales and momentum behaviors. Adam maintains two exponential moving averages: one for the gradients and another for the squared gradients. Let and denote the first- and second-moment vectors, respectively: In Eq. (2), is the exponential decay coefficient for the first-moment estimate (typically 0.9), is the previous first-moment estimate, and is the current gradient. In Eq. (3), is the decay rate for the second-moment estimate (typically 0.999), is the previous value of , and represents the squared gradient. To reduce initialization bias, Adam applies bias correction as follows: Finally, the parameter update rule is given by: Here, is the learning rate, and is a small constant to prevent division by zero. The use of exponential moving averages ensures stable convergence during the training process. Stochastic gradient descent (SGD) optimizer Stochastic Gradient Descent (SGD) is one of the most fundamental and widely used optimizers in deep learning. It minimizes the loss function by updating model parameters (weights) iteratively in the opposite direction of the gradient of the loss function, computed on small batches of training data32. SGD is simple, easy to implement, and works efficiently on large-scale datasets. The general parameter update rule for SGD is standardized using the notation : where: = model parameters at time step t = model parameters at time step t = learning rate (step size) = learning rate (step size) the gradient is defined as representing the loss function gradient with respect to at time step t. the gradient is defined as representing the loss function gradient with respect to at time step t. RMSProp optimizer Root Mean Square Propagation (RMSProp), proposed by Geoffrey Hinton, is an adaptive learning rate optimization algorithm designed to train deep learning models efficiently, especially on noisy data. The key idea of RMSProp is to maintain a moving average of the squared gradients for each parameter, which acts as an estimate of gradient variance33. Unlike SGD, which uses a fixed learning rate for all parameters, RMSProp dynamically adjusts the learning rate for each parameter based on the magnitude of recent gradients. Using the standardized gradient notation, the update rule for a parameter at time step t is: where: = learning rate = learning rate gradient is defined as at time step t gradient is defined as at time step t = decay rate (commonly 0.9) = decay rate (commonly 0.9) = small constant to avoid division by zero (e.g., ) = small constant to avoid division by zero (e.g., ) AdaMax optimizer AdaMax is a variant of the Adam optimizer that replaces the norm with the infinity norm (), making it more stable when dealing with large or sparse gradients31. Using the standardized gradient notation, let: = gradient at time step t = gradient at time step t = exponential moving average of gradients (1st moment) = exponential moving average of gradients (1st moment) = exponential moving maximum (infinity norm) = exponential moving maximum (infinity norm) = first-moment decay rate (commonly 0.9) = first-moment decay rate (commonly 0.9) = second-moment decay rate (commonly 0.999) = second-moment decay rate (commonly 0.999) The exponential moving average of the first moment (mean of gradients) is: The infinity norm is computed as: Bias correction for the first moment is applied as: Finally, the parameter update rule is: All four CNN models (trained with Adam, SGD, RMSProp and AdaMax optimizers) were trained using categorical cross-entropy loss and monitored using validation accuracy. After evaluating their performance, the two best performing models were selected for ensemble learning. Ensemble learning Ensemble learning has great potential to integrate multiple deep learning models to improve prediction accuracy and dependability. Bagging, boosting, stacking and voting are four main ensemble techniques which fall under supervised ensemble learning where multiple base classifiers are selected and their outputs are combined to improve the accuracy of the prediction34. In this work, the voting and stacking ensemble techniques are chosen to integrate the output of the two main base learners. Voting ensemble Voting ensemble is a widely used aggregation technique in ensemble learning, where multiple base classifiers are trained independently and their predictions are combined to generate the final decision6. Based on the type of outputs aggregated, voting can be categorized into hard voting (majority voting) and soft voting (probability averaging). Hard Voting: In hard voting, each base model outputs a discrete class label. The final prediction is made by selecting the class that receives the highest number of votes from all classifiers. Let M be the total number of classifiers, C be the number of classes, and be the predicted class label by the classifier for an input sample x. The final predicted class is: where is the indicator function, which returns 1 if the condition is true, and 0 otherwise. Soft Voting: In soft voting, each classifier provides a probability distribution over the possible classes. The final decision is obtained by averaging these predicted probabilities. Specifically, soft voting is treated as a case where all weights are equal, . Let denote the predicted probability of class j for input x from the classifier. The formulation is: Soft voting is generally more robust than hard voting as it incorporates the confidence levels of individual classifiers, improving classification performance, especially in complex tasks such as melanoma skin cancer detection. Weighted voting ensemble (grid search) Weighted voting ensemble assigns different importance weights to each base classifier. To distinguish this from soft voting, we explicitly incorporate an optimized weighting parameter into the decision rule: where denotes the assigned weight for classifier i, satisfying the constraint . To determine the optimal weight vector , grid search treats weight optimization as a discrete constrained optimization problem. Let represent the predefined search space of candidate weight vectors such that: The goal of grid search is to select the weight vector that maximizes ensemble performance based on a chosen evaluation metric (e.g., Accuracy, F1-score, or ROC-AUC) using cross-validation: where K is the number of cross-validation folds, and is the validation performance of the ensemble with weight vector W in the fold. Grid search systematically evaluates each candidate and selects the one that yields the highest validation performance. Although computationally expensive, it guarantees the identification of the best weight configuration within the defined search space, thereby improving the ensemble’s robustness and predictive performance for melanoma skin cancer detection. Stacked ensemble Stacked ensemble (or stacking) is a hierarchical ensemble learning technique in which multiple base models (level-0 learners) are trained independently, and their predictions are used as input features for a higher-level meta-classifier (level-1 learner). Unlike voting-based ensembles, which directly aggregate predictions, stacking learns how to best combine the strengths of each base classifier using supervised learning25. Let denote the set of M trained CNN base classifiers. For an input sample x, each classifier generates a prediction vector (either class labels or predicted probabilities). These outputs are concatenated to form a meta-feature vector: This meta-feature vector is then used as input to a meta-classifier , which learns the final decision function: The meta-classifier learns how to optimally combine base model predictions by minimizing a supervised loss function, typically defined as: where is the loss function (e.g., cross-entropy), and y is the ground truth label. The meta-classifiers employed in this work include Gradient Boosting (GB), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Logistic Regression (LR), and Naive Bayes (NB). These models were selected due to their ability to learn complex decision boundaries from meta-level features. Stacking is particularly effective in skin cancer detection, as it leverages the complementary strengths of diverse deep learning models and meta-learners, improving generalization, robustness, and predictive accuracy. Figure 4 and Algorithm 1 shows the proposed architecture. Architecture of proposed model. Proposed model. Results and Analysis This section presents the performance evaluation and analysis of the proposed methodology on the Melanoma Skin Cancer dataset. Following preprocessing, four different CNN architectures were trained using multiple optimizers (Adam, SGD, RMSProp, and AdaMax). The performance of these models, along with ensemble approaches, was assessed using standard classification metrics including Accuracy, Precision, Recall, F1-score, Specificity, and ROC-AUC. Accuracy measures the overall correctness of the model and is computed as: Precision quantifies the proportion of correctly predicted positive cases out of all predicted positives, while Recall (or Sensitivity) measures the proportion of actual positive samples correctly identified. These are expressed as: The F1-score represents the harmonic mean of Precision and Recall, providing a balanced measure of both metrics: Specificity indicates the proportion of correctly identified negative samples and is calculated as: To further evaluate model performance, the Receiver Operating Characteristic (ROC) curve was utilized, which illustrates the trade-off between the True Positive Rate (TPR) and the False Positive Rate (FPR): The Area Under the ROC Curve (AUC) provides a single scalar value summarizing the model’s discriminative capability across all classification thresholds. Higher AUC values indicate better separability between positive and negative classes. The combination of these metrics offers a comprehensive evaluation of the proposed models, capturing both classification accuracy and robustness, particularly critical in medical diagnoses such as melanoma detection. Results from CNN models with different optimizers All input images were resized to pixels and converted to RGB format. Hyperparameters such as learning rate, batch size (32), and number of epochs (10) were kept consistent across all models to ensure a fair comparison. Based on validation accuracy, the top two classifiers selected for ensemble learning were those trained with the RMSProp and Adam optimizers, as they achieved the highest validation accuracy. The performance of CNN models trained with four different optimizers is comprehensively summarized in Table 1. Among these, the CNN model using the RMSProp optimizer achieved the highest accuracy of 90.9%, closely followed by the Adam optimizer at 90.8%, SGD at 90.5%, and AdaMax at 90.2%. Precision was highest for AdaMax at 93.9%, with RMSProp, SGD, and Adam achieving 93.8%, 93.2%, and 92.2%, respectively. Recall peaked with the Adam optimizer at 89.2%, while RMSProp and SGD followed with 87.6% and 87.4%, and AdaMax recorded 86.0%. The F1-scores ranged from 89.8% for AdaMax to 90.7% for Adam, indicating a balanced trade-off between precision and recall across all optimizers. The area under the curve (AUC) values further support these findings, with Adam achieving 0.97 and the other optimizers each attaining 0.96. These comparative results are visually depicted in the radar chart in Fig. 5, which highlights the strengths and trade-offs of each optimization method across key performance metrics. Additionally, the training and validation accuracy and loss curves for each optimizer are presented in Figs. 6, 7, 8 and 9, providing insight into the convergence behavior and stability of the models. The confusion matrices shown in Fig. 10 summarize the classification outcomes, illustrating the distribution of true positives, true negatives, false positives, and false negatives for each optimizer, with the X-axis representing predicted labels and the Y-axis representing true labels. Collectively, these results demonstrate that while all optimizers yield comparable performance, RMSProp and Adam optimizers slightly outperform others in accuracy and F1-score, making them preferable choices for this CNN-based skin lesion classification task. Table 1Performance metrics of CNN models with different optimizers.OptimizerAccuracyPrecisionRecallF1-scoreAUCAdam Optimizer(M1)0.90800.92150.89200.90650.97SGD Optimizer(M2)0.90500.93180.87400.90200.96RMSProp Optimizer(M3)0.90900.93790.87600.90590.96AdaMax Optimizer(M4)0.90200.93890.86000.89770.96 Performance metrics of CNN models with different optimizers. Radar chart comparing Accuracy, Precision, Recall, and F1-score of CNN models trained with different optimizers. Accuracy and loss curves of the CNN model with Adam optimizer during training and validation phases. Accuracy and loss curves of the CNN model with SGD optimizer during training and validation phases. Accuracy and loss curves of the CNN model with RMSProp optimizer during training and validation phases. Accuracy and loss curves of the CNN model with AdaMax optimizer during training and validation phases. Confusion matrices of CNN models with different optimizers (Adam, SGD, RMSProp, AdaMax), showing the distribution of true positives, true negatives, false positives, and false negatives. Results using soft voting ensemble classification The performance of different configurations of the voting ensemble is presented in Table 2. The results clearly demonstrate that the ensemble approach is capable of improving model robustness and overall predictive accuracy compared to individual models. By aggregating predictions from multiple base classifiers, the voting ensemble balances out the weaknesses of individual models, leading to more stable and reliable performance. Among the combinations tested, the M1+M2+M3 ensemble achieved the best overall performance, with an accuracy of 0.921, precision of 0.9432, recall of 0.896, and F1 score of 0.919. This indicates that the ensemble was able to correctly classify the majority of cases while maintaining a strong balance between precision (minimizing false positives) and recall (minimizing false negatives). Notably, although some other combinations such as M1+M2+M3+M4 and M1+M3 yielded competitive results, they slightly lagged behind in terms of recall and F1-score, highlighting the effectiveness of the three-model combination. Overall, these findings suggest that carefully selected ensembles (rather than simply adding more models) can yield superior classification outcomes. The confusion matrix of the best-performing ensemble (M1+M2+M3) is further analyzed to illustrate its class-wise predictive capability. Table 2Performance of different soft voting ensemble configurations.ModelsAccuracyPrecisionRecallF1-scoreM1+M2+M30.9210.94320.8960.919M1+M2+M3+M40.9200.94300.8940.9179M1+M30.9190.93920.8960.9171M1+M3+M40.9190.94290.8920.9168M2+M30.9170.94270.8880.9145M1+M2+M40.9170.93890.8920.9149M1+M20.9160.93510.8940.9141M1+M40.9130.93660.8860.9106M2+M3+M40.9130.94030.8820.9102M3+M40.9120.94590.8740.9085M2+M40.9100.93990.8760.9068 Performance of different soft voting ensemble configurations. Results using hard voting ensemble classification The performance of different configurations of the hard voting ensemble is summarized in Table 3. Hard voting aggregates the predicted class labels from multiple base models and selects the majority vote as the final prediction. This method emphasizes consensus among base classifiers and can be particularly effective when models have complementary strengths. From the results, the M1+M2+M3+M4 ensemble achieved the highest F1-score of 0.9156, while the M1+M2+M3 ensemble achieved the highest precision of 0.9408 with an accuracy of 0.917. These results indicate that hard voting ensembles are capable of maintaining high predictive reliability while slightly trading off recall in some configurations. It is noteworthy that combining all four models (M1+M2+M3+M4) did not substantially improve accuracy compared to the three-model ensemble (M1+M2+M3), but it did yield a higher recall, demonstrating a trade-off between precision and recall depending on the chosen ensemble. Overall, these findings confirm that carefully selecting base models for the hard voting ensemble can enhance classification performance without necessarily including all available models. Table 3Performance of different hard voting ensemble configurations.Base ModelsAccuracyPrecisionRecallF1-scoreM1+M2+M30.9170.94080.8900.9147M1+M2+M3+M40.9170.93170.9000.9156M1+M2+M40.9160.94260.8860.9134M1+M3+M40.9130.93840.8840.9104M2+M40.9120.92390.8980.9108M2+M3+M40.9120.94020.8800.9091M3+M40.9110.92200.8980.9098M1+M40.9090.91310.9040.9085M1+M30.9070.90780.9060.9069M2+M30.9070.91280.9000.9063M1+M20.9030.90540.9000.9027 Performance of different hard voting ensemble configurations. Results using grid search weighted average ensemble The performance of different weighted combinations of base models obtained through grid search is presented in Table 4. Weighted averaging allows assigning different importance to each base model’s prediction, enabling the ensemble to maximize overall accuracy by leveraging the strengths of more reliable models. From the results, the combination M1+M2+M3 with weights 0.40, 0.30, 0.30 achieved the highest accuracy of 0.925 with a precision of 0.9474, recall of 0.900, and F1-score of 0.9231. This indicates that proper tuning of the model weights can further improve ensemble performance compared to standard soft or hard voting approaches. Other combinations, such as M1+M2+M3+M4 (weights 0.40, 0.20, 0.20, 0.20) and M1+M2+M4 (weights 0.60, 0.30, 0.10), also provided competitive performance, achieving F1-scores above 0.920. The results highlight that weighted averaging is effective for fine-tuning ensemble predictions and balancing precision and recall according to the optimal contribution of each base model. Table 4Performance of Grid Search Weighted Average Ensemble with Optimal Weights.Base ModelsOptimal WeightsAccuracyPrecisionRecallF1-scoreM1+M2+M30.40, 0.30, 0.300.9250.94740.9000.9231M1+M2+M3+M40.40, 0.20, 0.20, 0.200.9230.94530.8980.9210M1+M2+M40.60, 0.30, 0.100.9220.94330.8980.9201M1+M30.40, 0.600.9200.94490.8920.9177M1+M3+M40.50, 0.40, 0.100.9200.94120.8960.9180M1+M20.70, 0.300.9190.93920.8960.9171M2+M30.50, 0.500.9170.94270.8880.9145M2+M3+M40.40, 0.50, 0.100.9170.94460.8860.9143M1+M40.70, 0.300.9160.93700.8920.9139M2+M40.40, 0.600.9120.94400.8760.9087M3+M40.50, 0.500.9120.94590.8740.9085 Performance of Grid Search Weighted Average Ensemble with Optimal Weights. Results using various model combinations in stacked ensemble The performance of different meta classifiers combined with various deep learning (DL) model combinations is summarized in Table 5. Stacked ensembles leverage the predictive outputs of multiple base models and feed them into a higher-level meta classifier, which learns to optimally combine these predictions for improved accuracy and generalization. From the results, it is evident that Gradient Boosting consistently outperformed other meta classifiers across almost all DL model combinations. The combination of M1+M2+M3+M4 achieved the highest accuracy of 0.974 with Gradient Boosting, while also maintaining strong performance with other classifiers, leading to an average accuracy of 0.933. This demonstrates the effectiveness of incorporating multiple diverse DL models in the stacked framework. Other model combinations, such as M1+M2+M4 and M1+M3+M4, also showed competitive performance with Gradient Boosting as the meta classifier, achieving accuracies of 0.967 and 0.964 respectively. The overall average performance across all combinations highlights that careful selection of both base models and meta classifiers is crucial to maximize predictive performance. Figure 11 visualizes the average accuracy across different meta classifiers for each DL model combination, illustrating the consistent superiority of Gradient Boosting in this stacked ensemble setup. Table 5Accuracy of Different Meta Classifiers with DL Model Combinations.DL Models UsedGBKNNSVMLogRegNaive BayesAverageM1+M2+M3+M40.9740.9300.9220.9220.9180.933M1+M2+M40.9670.9240.9170.9180.9140.928M1+M3+M40.9640.9240.9220.9180.9180.929M2+M3+M40.9630.9200.9160.9170.9170.927M1+M2+M30.9600.9240.9230.9180.9180.929M1+M20.9570.9320.9130.9180.9150.927M2+M40.9550.9220.9090.9090.9100.921M1+M40.9540.9270.9200.9160.9140.926M1+M30.9540.9310.9240.9180.9160.929M3+M40.9530.9170.9120.9120.9130.921M2+M30.9520.9240.9180.9100.9110.923Average0.9580.9250.9180.9160.9150.926 Accuracy of Different Meta Classifiers with DL Model Combinations. Average accuracy of different meta classifiers across DL model combinations in the stacked ensemble. To further analyze the performance of the stacked ensemble with Gradient Boosting as the meta classifier, Table 6 presents accuracy, precision, recall, and F1-score for each deep learning model combination. This detailed breakdown highlights the effectiveness of different model combinations and allows identification of configurations that optimize both precision and recall. The results indicate that the combination M1+M2+M3+M4 achieves the highest accuracy of 0.974 and precision of 0.9917, with a recall of 0.956 and F1-score of 0.9735, making it the best-performing ensemble overall. Other top-performing combinations such as M1+M2+M4 and M1+M3+M4 also demonstrate high predictive performance while slightly trading off recall for precision. Overall, these findings confirm that Gradient Boosting can effectively leverage the outputs of multiple base models to improve classification performance. Table 6Detailed performance of stacked ensemble using Gradient Boosting as the meta classifier.ModelsAccuracy (A)Precision (P)Recall (R)F1-score (F1)M1+M2+M3+M40.9740.99170.9560.9735M1+M2+M40.9670.98950.9440.9662M1+M3+M40.9640.98740.9400.9631M2+M3+M40.9630.97730.9480.9624M1+M2+M30.9600.98120.9380.9591M1+M20.9570.98930.9240.9555M2+M40.9550.97100.9380.9542M1+M40.9540.97690.9300.9529M1+M30.9540.98090.9260.9527M3+M40.9530.96510.9400.9524M2+M30.9520.96690.9360.9512 Detailed performance of stacked ensemble using Gradient Boosting as the meta classifier. To ensure robust and unbiased performance evaluation of the proposed stacked ensemble, a rigorous out-of-fold (OOF) training strategy was implemented. The ensemble framework utilizes four base convolutional neural networks (CNNs) whose probability outputs serve as meta-features for a Gradient Boosting (GB) meta-classifier. During the training phase, the training dataset was partitioned into five stratified folds to maintain class distribution. For each fold, base models generated predictions on the held-out 20% validation set. This iterative process ensured that every training sample received a prediction from a model instance that had not seen it during training, producing a complete set of OOF meta-features. These features were subsequently used to train the GB meta-classifier. This methodology is critical for preventing data leakage and ensuring that the meta-learner captures the true generalization errors of the base models. The stacked ensemble achieved a mean cross-validation accuracy of 93.39% ± 0.46% with a 95% confidence interval of [93.15%, 93.62%], validated through 5-fold cross-validation repeated 3 times. Additional metrics included precision of 94.19%, recall of 91.89%, F1-score of 93.02%, and AUC of 98.25%, demonstrating consistent and stable performance across folds. The final performance was validated on an independent test set of 1,000 images. Base model predictions on this unseen test data were stacked and processed by the trained GB meta-classifier to arrive at the final diagnostic decision. The proposed stacked approach achieved a final classification accuracy of 91.80%, precision of 93.91%, recall of 89.40%, F1-score of 91.60%, and AUC of 96.87% on the independent test set, confirming its effectiveness in combining model-specific strengths to improve melanoma detection accuracy and generalization. To address the diversity of the hypothesis space, we evaluated an architecturally diverse ensemble comprising a custom CNN, DenseNet, InceptionV3, and ResNet50. This configuration achieved an accuracy of 0.963, demonstrating that combining models with varied inductive biases enhances performance compared to optimization-trajectory diversity alone. Discussion The results in Table 7 and Figs. 12, 13 illustrate the impact of different combinations of models and ensemble strategies on the performance of melanoma prediction. The single base model (M3) trained with the RMSProp optimizer achieved a moderate accuracy of 90.90%, indicating its individual predictive capacity. Both soft and hard voting ensembles that combined M1, M2, and M3 slightly improved performance, with soft voting that yielded 92.1% accuracy and hard voting achieving 91.7%, demonstrating the benefit of aggregating multiple models. The stacking ensemble using Gradient Boosting as meta-classifier significantly outperformed all other methods, achieving 97.4% accuracy and high precision, recall, and F1-score. This highlights the advantage of combining diverse base models with a strong meta-classifier to capture complex patterns in the dataset. Additionally, the Grid Search weighted ensemble improved the initial performance to 92.5%, showcasing how hyperparameter optimization can further fine-tune model predictions for better generalization. Figure 14 shows the AUC Score comparison of different best models of the ensemble where the proposed stacking with GB achieves the best AUC Score of 0.997. To further evaluate the robustness and generalizability of the proposed framework, the best-performing stacked ensemble model was tested on an independent external dataset, the PH dataset35, without any further training or cross-validation. The model achieved a high external validation accuracy of 91.50%, confirming its ability to generalize across different clinical cohorts and imaging sources. In general, these findings emphasize that ensemble techniques, particularly stacking with optimized weights, provide robust and highly accurate predictions in medical data sets such as the detection of skin lesions. Table 7Performance Comparison of Base ML, Voting, Stacking, and Grid Search Optimized Models.ModelAccuracy (A)Precision (P)Recall (R)F1-score (F1)Base ML0.90900.93790.87600.9059Soft Voting0.9210.94320.8960.919Hard Voting0.9170.94080.8900.9147Stacking0.9740.99170.9560.9735Grid Search Weighted Ensemble0.9250.94740.9000.9231 Performance Comparison of Base ML, Voting, Stacking, and Grid Search Optimized Models. Comparison of Best Results from different Ensemble Approaches with Best Base ML Model. Confusion Matrix of Best Ensemble Combinations. Comparative ROC curves of ensemble approaches. The substantial improvement observed with the stacked ensemble, particularly using Gradient Boosting as the meta-learner, can be attributed to its ability to model complex, non-linear relationships between the base CNN models’ predictions. Although the individual CNNs exhibit similar overall performance metrics, they tend to make different errors on specific samples due to variations in feature learning influenced by different optimizers. The meta-classifier effectively learns to weigh these diverse predictions, capturing higher-order patterns and correcting individual model weaknesses, which leads to the observed performance gain. Regarding the consistently higher number of false negatives (FN) compared to false positives (FP) across all models, including the meta-classifier, this reflects the inherent challenge in distinguishing melanoma from benign lesions that share similar visual characteristics, such as atypical nevi. The models adopt a conservative classification approach, prioritizing specificity to reduce over-diagnosis, which unfortunately results in more missed malignant cases. This trade-off highlights the difficulty of detecting clinically subtle melanomas and underscores the need for further research to improve sensitivity without compromising specificity. To rigorously validate the performance gains of the proposed approach, statistical significance testing was conducted by comparing the stacked ensemble against the nearest-performing voting-based methods. As summarized in Table 8, paired t-tests on cross-validation folds indicate that Stacked significantly outperforms Soft Voting, Hard Voting, and Weighted Average, with p-values of 0.000684, 0.000527, and 0.003936, respectively. The corresponding Cohen’s d values are very large, reflecting strong practical effect sizes due to consistent performance differences and low variance across folds. Although the Wilcoxon signed-rank test yielded p = 0.0625, which does not meet the conventional significance threshold, all fold-wise comparisons consistently favored Stacked. This suggests a stable and robust performance improvement despite the limited sample size of cross-validation folds. Table 8Statistical Significance Testing Comparison.Comparison PairMean Diff.t-statisticp-value (T-test)p-value (Wilcoxon)Cohen’s dStacking vs. Soft Voting+0.00989.50520.0006840.0625004.2508Stacking vs. Hard Voting+0.015510.16840.0005270.0625004.5474Stacking vs. Weighted Average+0.00865.97770.0039360.0625002.6733 Statistical Significance Testing Comparison. Comparison with state-of-the-art models The comparison of the proposed stacked ensemble with various state-of-the-art models is presented in Table 9. It is crucial to distinguish between the performance phases: while our stacking approach achieved a validation accuracy of 97.4%, its performance on a genuinely held-out independent test set resulted in an accuracy of 91.80%, precision of 93.91%, recall of 89.40%, and an F1-score of 91.60%, with a ROC-AUC of 96.87%. Traditional TL models and transformer-based models achieve accuracies ranging from 84% to 93%. By identifying 91.80% as the primary result for unseen data, the study ensures a transparent and honest assessment of the model’s generalization capabilities. This underlines the robustness and reliability of the proposed ensemble strategy in clinical diagnostic scenarios. Table 9Comparison of State-of-the-Art Models with Proposed Stacking Ensemble (Average Values).ModelAccuracyPrecisionRecallF1 ScoreProposed - Stacking with GB0.974 (Val.)0.918 (Test)0.992 (Val.)0.939 (Test)0.956 (Val.)0.894 (Test)0.974 (Val.)0.916 (Test)VGG160.9170.9070.9170.906ResNet500.8380.8410.8380.840DenseNet1210.9040.9080.9040.898InceptionV30.8950.8940.8950.893ViT0.9280.9320.9280.928SwinT0.9200.9200.9200.920DeiT0.9260.9300.9260.926 Comparison of State-of-the-Art Models with Proposed Stacking Ensemble (Average Values). 0.974 (Val.) 0.918 (Test) 0.992 (Val.) 0.939 (Test) 0.956 (Val.) 0.894 (Test) 0.974 (Val.) 0.916 (Test) Comparison with existing work The comparative analysis in Table 10 highlights the effectiveness of the proposed approach against existing methods. Earlier works relied on standalone deep learning architectures such as CNN (57–91%) or hybrid approaches like DenseNet121 combined with XGBoost (70%), which showed limited performance due to overfitting or weak generalization capabilities. Ensemble-based approaches with handcrafted features or deep feature embeddings improved results to around 91.4%, but still struggled to consistently balance sensitivity and specificity. In contrast, our proposed stacking ensemble model, which integrates Random Forest, K-Nearest Neighbors, and Decision Tree as base learners with Logistic Regression as a meta-classifier, achieved a significantly higher accuracy of 97.4%. This substantial improvement demonstrates the advantages of combining diverse learners under a meta-learning framework, ensuring both robustness and superior predictive capability. The results indicate that proposed model outperforms state-of-the-art methods and establishes a reliable benchmark for skin cancer detection (Table 11). Table 10Comparison between the Proposed Approach and Related Work.SourceMethodResults (Accuracy)36DenseNet121+XGBClassifier70%CNN57%Ensemble model based on DenseNet12166%Enhancing CNN layer with classification part65.5%ResNet5065.5%37AlexNet with Hair Removal91.3%38AdaBoost61.52%39Ensemble of ML with Deep Feature Embeddings91.4%40CNN91%ProposedStacking of Deep Learning Models with GB as Meta Classifier97.4%(Val), 91.8%(Test) Comparison between the Proposed Approach and Related Work. Table 11Performance Statistics and Training Duration of Converged Base Learners.ModelTotal Epochs RunBest EpochTrain AccuracyTest AccuracySGD (M2)22170.92640.8960Adam (M1)15100.93340.9030RMSprop (M3)1270.91380.9100Adamax (M4)19140.93220.9140 Performance Statistics and Training Duration of Converged Base Learners. Performance analysis with extended training and early stopping Additional experiments were conducted to evaluate model stability and convergence beyond the initial training phase. To this end, all base CNN models were retrained for a maximum of 50 epochs utilizing an early stopping mechanism based on validation loss, ensuring that each model reached an optimal state without overfitting. The SGD-based model (M2) reached convergence at epoch 17 after running for 22 total epochs, achieving a final training accuracy of 0.9264 and a test accuracy of 0.8960. The Adam (M1), RMSprop (M3), and Adamax (M4) models converged at epochs 10, 7, and 14, requiring 15, 12, and 19 total iterations respectively. This extended training protocol established more robust base learners for the subsequent ensemble phase. The evaluation of the retrained models on the held-out test set resulted in a collective performance characterized by a test loss of 0.2145, an accuracy of 0.9120, and an AUC of 0.9703. These baseline improvements directly translated to superior ensemble outcomes, as summarized in Table 12. Specifically, the stacked ensemble with Gradient Boosting as the meta-learner achieved a peak accuracy of 0.971, significantly outperforming voting-based strategies. In contrast, Soft Voting, Hard Voting, and Weighted Average obtained accuracies of 0.916, 0.914, and 0.918, respectively. To maintain clarity in the main text, detailed performance plots are provided in the Appendix: accuracy–loss curves per epoch, ROC, Confusion Matrix for each optimizer (Figs. 15, 16, 17, 18), the confusion matrix for the evaluated ensembles (Fig. 19), and the comparative ensemble ROC curves (Fig. 20). Table 12Comparative Ensemble Performance with Converged Base Learners.Ensemble MethodConfiguration / Meta-ClassifierBase ModelsAccuracyStackedGradient BoostingM1+M2+M3+M40.971Weighted AverageWeights: 0.10, 0.50, 0.40M2+M3+M40.918Soft VotingAverageM2+M3+M40.916Hard VotingMajorityM2+M3+M40.914 Comparative Ensemble Performance with Converged Base Learners. Different Plots of Training CNN with Adam Optimizer using 50 Epochs and early stopping. Different Plots of Training CNN with SGD Optimizer using 50 Epochs and early stopping. Different Plots of Training CNN with RMSProP Optimizer using 50 Epochs and early stopping. Different Plots of Training CNN with Adamax Optimizer using 50 Epochs and early stopping. Confusion Matrix of Various Ensemble using Model Trained on 50 epochs with early stopping. ROC of Various Ensemble using Model Trained on 50 epochs with early stopping. Conclusion and future scope This paper presented an optimized strategy for melanoma skin cancer detection using deep learning and ensemble learning techniques. Four CNN models were trained using different optimizers (Adam, SGD, RMSProp, and AdaMax) to introduce diversity in model behavior, which was later exploited using ensemble learning. Four ensemble strategies: Soft Voting, Hard Voting, Weighted Voting (Grid Search), and Stacked Ensemble with multiple meta-learners (GB, KNN, SVM, LR, NB) were implemented to enhance predictive performance. The experimental results revealed that ensemble strategies consistently outperformed individual CNN models. The proposed stacked ensemble achieved the highest validation accuracy of 97.4%, but when evaluated on a genuinely held-out independent test set the model yielded an accuracy of 91.80%, precision of 93.91%, recall of 89.40%, F1-score of 91.60%, and a ROC-AUC of 96.87%. While the validation result reflects the model’s learning capacity during development, the independent-test metrics represent the primary headline performance on unseen data and are used for fair comparison with prior work. These results confirm that optimizer-driven diversity combined with meta-level learning significantly enhances classification performance and generalization. Although the stacked ensemble shows strong discriminative ability on unseen data, we emphasize the independent-test performance (91.80%, AUC 96.87%) as the honest indicator of the model’s expected real-world behavior. Despite achieving high accuracy and reliability, the proposed model has scope for further improvement. Future research may focus on incorporating larger and more diverse dermoscopic datasets to enhance model scalability and robustness, and on conducting external cross-dataset validation to better quantify generalization. Additionally, integrating transfer learning refinements, domain adaptation, or self-supervised learning techniques could reduce dependency on labeled data. Real-time clinical validation and deployment in mobile or edge-based diagnostic systems could further improve practical applicability. Including explainable AI (XAI) methodologies may also improve interpretability and trustworthiness, making the system more suitable for clinical decision support. Appendix See Figs. 15, 16, 17, 18, 19 and 20. Author contributions A.P. and A.J. conceived the study and designed the methodology. J.A., Y.A and S.V. curated the dataset, performed preprocessing, and conducted the initial model training experiments. A.J implemented the optimization strategies and contributed to the validation of the results. O.K. and R.F. supported the experimental design, computational workflow, and interpretation of ensemblelearning outcomes. H.N. supervised the project, provided critical insights on model evaluation, and reviewed the analytical framework. A.J. and A.P. drafted the manuscript, and all authors contributed to the manuscript revision. All authors reviewed and approved the final version of the manuscript. Funding This study was supported in part by Long-Term Conceptual Development of Research Organization (2025) at Skoda Auto University, Czech Republic. Data availability The data set used in this study is publicly available and can be accessed at https://www.kaggle. com/datasets/hasnainjaved/melanoma-skin-cancer-dataset-of-10000-images. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Contributor Information Achin Jain, Email: achin.jain@bvcoend.ac.in. Hamidreza Namazi, Email: hamidreza.namazi@monash.edu. References Associated Data Data Availability Statement The data set used in this study is publicly available and can be accessed at https://www.kaggle. com/datasets/hasnainjaved/melanoma-skin-cancer-dataset-of-10000-images.