Optimizing cervical cancer diagnosis with a hybrid deep neural network and progressive resizing on pap smear WSIs Authors: Nitin Kumar Chauhan, Amit Kumar, Ankit Jain, Krishna Singh, Dharmendra Kumar, Shashank Sheshar Singh, Harish Kumar Shakya Published: 2026 ABSTRACT Nowadays, computer-aided diagnostic (CAD) systems powered by artificial intelligence (AI) are becoming increasingly prevalent in cervical cancer diagnosis. Automatic selection of features by the deep convolutional neural networks (CNN) is a more prominent substitute than the conventional machine learning (ML) models, as they require handcrafted cell segmentation and extraction. Pre-trained deep models using transfer learning and fine-tuning make these models faster and more efficient, even with the limited data availability. This article proposes a novel hybrid deep network with progressive resizing (HDNPR) for classifying whole slide images (WSI) of pap smear slides. This model trains fine-tuned deep learning (DL) models on pre-trained weights utilizing progressively resized and augmented training data of size 224 × 224, 512 × 512, and 1024 × 1024 pixels, laid over with transfer learning. The hybrid deep features produced by the concatenation of extracted features from two prevalent fine-tuned deep learning networks, VGG-16 and ResNet-152, are applied to the fully connected network (FCN) stage for detecting different classes of cervical cancer. The proposed HDNPR network is evaluated for both multiclass and binary classification, and it is able to attain an accuracy score of 97.45% for 5-class classification and 98.58% for 2-class classification, respectively. Introduction Globally, Cervical cancer has been among the most severe gynaecological threats to females in recent years. This is caused by the unrestrained progression of cells due to contamination by human papillomavirus (HPV) in the cervix, which is the bottom segment of the uterus connecting to the vagina. The Globocan Report 2022 states that Cervical cancer is the fourth most common malignancy in women, causing the average age to decline each year1. Most of these fatalities take place in low and middle-income countries due to lower capital expenditure in the build-up and upkeep of health infrastructure. The reported morbidity is dropping more quickly in high-income countries, and some of these nations are working towards eradicating cervical cancer in the coming decades, contributing to the growing disparities2. To prevent cervical cancer and diagnose anomalies at a preliminary phase, the pap smear test is one of the most prevalent preliminary and non-invasive screening clinical cytology procedures. The Human Papillomavirus Deoxyribonucleic Acid (HPV DNA) test identifies all types of HPV present in the cervical cells that might cause cervical cancer. Pap smear and HPV testing can be conducted simultaneously using the same swab or a separate sample. Patients suspected of having cervical cancer must undergo thorough diagnostic procedures, including a biopsy3. However, these techniques have drawbacks, such as false positives or false-negative findings, interobserver variation, subjective interpretation, etc. Also, the screening procedure heavily relied on clinical specialists, which is tedious and time-consuming, with vulnerability to human error4,5. Computer-aided screening and diagnostic tools are therefore being evolved in the interest of automating the screening process. The advancement of AI-powered computer-aided diagnostic (CAD) systems has been an enormous forward step in the identification of cervical cancer. CAD systems analyze cervical cancer pap smear images to detect probable anomalies, adding another level of assistance for pathologists and cytotechnologists6. The employment of CAD tools has increased the efficacy of cervical cancer detection throughout and decreased the rate of false-negative findings, which is encouraging. The CAD platforms offer to enhance early diagnosis and, eventually, improve patient results by helping to identify minor cellular changes7. At the moment, feature-based algorithms are employed the most often in CAD technologies for cervical cancer detection. Traditional ML models, despite their streamlined computing procedures, still rely on manual feature extraction8. DL provides a thorough and automated solution that efficiently addresses the intricate challenges in cervical cancer detection9. Typically, for anomaly detection, the WSI slides’ cells are first segmented, and then the fragmented cell images are utilized to make the classification. As segmentation of cells is quite complicated, no model consistently results well enough for the segmentation of screening WSI recorded in diverse clinical practices and test settings10. This article introduces a novel deep hybrid model, i.e., HDNPR, for the classification of pap smear WSI slides. The abnormality detection in WSI permits further downstream investigation. Unlike existing hybrid approaches that primarily focus on isolated single-cell images and often require multiple deep networks, the proposed HDNPR framework introduces a progressive-resizing hybrid feature learning strategy on Pap smear WSI. It is important to note that our previous study11 was primarily focused on comparing the performance of various standalone fine-tuned deep learning models for classifying whole slide images (WSI). Based on this, the current study aims to take an additional step by introducing a hybrid architecture. The proposed method combines the complementary deep features of the two best performers from our previous study11, Visual Geometry Group 16-layer Net (VGG-16) and Residual Network with 152 layers (ResNet-152), both of which are fine-tuned for the specific task. The proposed method utilizes progressive resizing and a two-step data augmentation technique to enhance multi-scale feature learning and robustness to varying image sizes. For classification, a fully connected network (FCN) is employed to classify cervical cancer samples using the concatenated hybrid features of the two deep models. The FCN approach can automate and improve cervical cancer screening by learning sophisticated characteristics and improving its diagnostic accuracy with reduced computational burden12. This allows the model to combine the fine details of texture information with the contextual information, thereby achieving improved classification performance without compromising efficiency. The following are the article’s primary contributions: Application of progressively resized training data utilizing comprehensive data augmentation to train the DL models well enough.Two fine-tuned DL networks, i.e., VGG-16 and ResNet-152, are being utilized along with transfer learning for feature extraction.The hybrid feature concatenation strategy is employed, where the deep feature vectors obtained from VGG-16 and ResNet-152 are combined into a single feature representation. Concatenation allows both networks to contribute complementary information. VGG-16 captures fine-grained texture and shape patterns, while ResNet-152 captures deeper contextual and structural details.The concatenated hybrid features are fed into an FCN stage for multiclass (5-class) and binary (2-class) classification of cervical cytology WSIs.The publicly available cervical cancer dataset of WSI slides, i.e., SIPaKMeD, is utilized to test the proposed HDNPR model for multiclass (5-class) and binary (2-class) classification.McNemar’s test is used to assess the p-value for finding the non-parametric statistical decision about the distinctness between the proposed and baseline deep classifiers used in the proposed method. Application of progressively resized training data utilizing comprehensive data augmentation to train the DL models well enough. Application of progressively resized training data utilizing comprehensive data augmentation to train the DL models well enough. Two fine-tuned DL networks, i.e., VGG-16 and ResNet-152, are being utilized along with transfer learning for feature extraction. Two fine-tuned DL networks, i.e., VGG-16 and ResNet-152, are being utilized along with transfer learning for feature extraction. The hybrid feature concatenation strategy is employed, where the deep feature vectors obtained from VGG-16 and ResNet-152 are combined into a single feature representation. Concatenation allows both networks to contribute complementary information. VGG-16 captures fine-grained texture and shape patterns, while ResNet-152 captures deeper contextual and structural details. The hybrid feature concatenation strategy is employed, where the deep feature vectors obtained from VGG-16 and ResNet-152 are combined into a single feature representation. Concatenation allows both networks to contribute complementary information. VGG-16 captures fine-grained texture and shape patterns, while ResNet-152 captures deeper contextual and structural details. The concatenated hybrid features are fed into an FCN stage for multiclass (5-class) and binary (2-class) classification of cervical cytology WSIs. The concatenated hybrid features are fed into an FCN stage for multiclass (5-class) and binary (2-class) classification of cervical cytology WSIs. The publicly available cervical cancer dataset of WSI slides, i.e., SIPaKMeD, is utilized to test the proposed HDNPR model for multiclass (5-class) and binary (2-class) classification. The publicly available cervical cancer dataset of WSI slides, i.e., SIPaKMeD, is utilized to test the proposed HDNPR model for multiclass (5-class) and binary (2-class) classification. McNemar’s test is used to assess the p-value for finding the non-parametric statistical decision about the distinctness between the proposed and baseline deep classifiers used in the proposed method. McNemar’s test is used to assess the p-value for finding the non-parametric statistical decision about the distinctness between the proposed and baseline deep classifiers used in the proposed method. Related work The analysis of cervical image classification entails preprocessing methods, nuclei and cell segmentation, feature extraction, and binary or multiclass classification. Traditionally, filtering methods are used as preprocessing techniques to enhance the pixels of the dataset images. The most discriminatory features are selected, followed by the segmentation of cell nuclei and the extraction of features. Pre-processing operations, including vignetting effect removal, contrast modification, pigmentation, stain removal, image smoothing, standardization, and translation, are employed to improve the productivity of image data to cut down on image noise. These preprocessing steps involve image denoising, enhancement, and rotation; object detection and centralization; visual dictionary learning, base function learning, morphological operation, etc13,14. Segmentation of nuclei from cervix cells involves methods based on models, threshold, edge, region, pixel, and ML/DL approaches. Otsu thresholding15, k-means16, levelset17, watershed18, and active contour model19,20 are some traditional methods of nuclei and cytoplasm segmentation. Super-pixel and CNN21; patch-based CNN22; mask-recurrent neural network (Mask-RCNN)23 and local fully connected conditional random field (LFCCRF)24; attention U-net (ATT-UNet)25; LeNet26; instance relation network (IR-Net)27; Encoder-Weighted W-Net28 are some of the CNN-based segmentation methods. Features are extracted from segmented cells of the cervix WSI, followed by feature extraction. The use of feature extraction permits the representation of a picture in terms of its characteristics (such as geometry, texture, and grey-level attributes), structural aspects (global and local attributes), cognitive importance (low, moderate, and high-level attributes), or extracting methods (handcrafted or automated)29–35. The job of enhancing the extracted features then falls on the data post-processing stage, which invariably employs feature selection, normalization, and sparsification techniques. The objective of selecting prime attributes is to identify the optimal or near-optimal subsets of the initial attributes to perform an effective analysis of high-dimensional data36,37. Following the post-processing stage, classification methods are used to separate various classes of histopathology images. These classification methods include traditional linear and non-linear29,32,38; weight-based classifiers29; artificial neural networks (ANNs)33; and other classifiers29,32,34. Numerous outstanding review articles provide in-depth descriptions of all steps for cervical cell classification, including cell segmentation, feature extraction and selection, and classification39–41. DL-based systems offer remarkable efficacy in automatic identification and classification, proving to be a significant advancement in the healthcare ecosystem. Even though several prior research have suggested combining the DL approach with conventional medical tests, including Pap smear and colposcopy test in the detection of abnormality of the cervix, there is still considerable scope for improvement in accuracy, specifically at an early stage to reach highly accepted levels of prognosis and diagnosis. Fully convolutional neural networks (FCNNs)42; CNN with deep convolutional neural network and AlexNet (CNN-DCNN-AlexNet)43; Convolutional neural network - Extreme learning machine (CNN-ELM)44; DCNN-based ThinPrep cytologic test (DCNN-based TCT)45; dual-tree complex wavelet transform (DTCWT) based modified CNN46; and U-Net with support vector machine (UNet + SVM)47 are a few of the recent applications of DL in cervical cancer detection. In recent years, deep hybrid models have gained popularity among investigators for cervix anomaly detection48–52. DeepCyto, a hybrid framework utilizing four DL models, VGG19, VGG16, XceptionNet, and ResNet50, was proposed with an ensemble classifier and traditional fully connected artificial neural network (ANN) classifier for binary classification on single-cell cevix images of SIPaKMeD data48. The dataset contains a limited set of images, and no augmentation method has been utilized, which may lead to data imbalance, lack of diversity, and overfitting issues. The work49 utilized two DL models, Mobilenet and Darknet models, for feature extraction and distinct ML classifiers for prediction on single-cell pap smear images. An ensemble model50 including three deep networks, DenseNet-169, Xception, and VGG-19 with a Swin transformer was developed for cervical cytology classification of single-cell images. A hybrid DL model51 employing four pre-trained DL networks, Resnet-18, Alexnet, Googlenet, and Resnet-50 for feature extraction with the ML classifier stage was proposed for single-cell pap smear image classification. In another investigation52, the individual fine-tuned models AlexNet, ResNet101, InceptionV3, and ResNet152 were used for feature extraction with classification stages having different ML techniques like Random Forest, Simple Logistic, Naive Bayes, Principal Component Analysis, and Decision Tree for single-cell classification of pap-smear images. All these hybrid models were analyzed for single-cell images of cervical cancer, while this investigation is based on WSI pap smear slides. The present research aimed to reduce the number of DL networks required for feature extraction, along with the benefits of the progressive resizing approach. Progressive resizing is a technique that entails gradually increasing the sizes of images during the training process. Progressive resizing optimizes efficiency and performance in challenging image tasks by improving generalization, accelerating early training, enhancing stability, extracting enhanced characteristics, effective resource utilization, reducing overfitting, and enabling adaptable data augmentation. Materials and methods The experimental data, preprocessing, and methods of the proposed network are all included in this section’s arrangement. Dataset overview This subsection includes the dataset of pap smear WSI slides, i.e., SIPaKMeD utilized in this work to assess the proposed model. The database repository includes 966 WSI slides of pap smear tests and 4,049 images of manually cropped cells53. These images were shot with an optical microscope (OLYMPUS BX53F) compatible charge-coupled device (CCD) camera (Infinity 1 Lumenera). Experienced cytopathologists divide these WSI into 5 classes based on the configuration and morphological arrangement of the cells. To be more precise, normal cells fall into two groups: superficial-intermediate (SI) and parabasal (P), abnormal, although non-cancerous cells into two categories: koilocytes (K) and dyskeratotic (D), and benign cells fall into one category: metaplastic (M). The proposed model is examined for 5 classes (SI, P, K, D, and M) and 2 classes (Normal and abnormal) using WSI slides. Preprocessing This subsection illustrates some of the preprocessing steps utilized in this work including dataset splitting, image resizing, and dataset expansion as shown in Fig. 1. Fig. 1Preprocessing steps in the proposed framework. Preprocessing steps in the proposed framework. Dataset division and image resizing The SIPaKMeD WSI data is first partitioned into training, validation, and testing data, with portions of 60%, 20%, and 20% respectively. As the WSI data of the pap test is having high-definition images of 2048 × 1536 pixels, it is rescaled into sets of 224 × 224, 512 × 512, and 1024 × 1024 pixels. Dataset expansion More training data is needed for the DL architecture since there are more training parameters to learn. Training with insufficient data may cause the model to overfit rather readily. Here we are using two-stage data augmentation to deal with training data scarcity54. At the initial stage of augmentation, a range of augmentation techniques from the ML module ‘ImgAug’ is used, including geometric transformations, perspective transitions, contrast variations, color distortion, region dropout, hue/saturation adjustments, cropping, padding, filtering, etc.55. With a single training sample, these transformations produce 8 augmented samples, rendering the training data 9 times larger. Below is a description of some of the methods of augmentation that have been employed. aug: It is a meta-augmenter used to perform affine transformations involving argument functions ‘rotate’ (ranging from − 45 to 45), ‘scale’ (ranging from 0.8 to 1.2 for both the x-axis and y-axis), ‘translation’ (ranging from − 0.15 to 0.15 for both the x-axis and y-axis), ‘shear’ (ranging from − 2 to 2), ‘h_flip’ (0.5), and ‘v_flip’ (0.5). Some of these arguments are applied to images by using the list augmentor ‘SomeOf’. clahe: This augmenter normalizes histograms inside certain local picture regions by using a technique called CLAHE (Contrast Limited Adaptive Histogram Equalization) on the images. It includes a ‘SomeOf’ augmentor to create a list with the arguments such as ‘clip_limit’ (ranging from 1 to 10), ‘tile_grid_size’ (ranging from 3 to 21), and ‘gamma_contrast’ (ranging from 0.5 to 2.0). edge: This augmentor detects edges within an image by detecting them using white color and applying black color for the remaining background. It involves functions such as ‘EdgeDetect’ (ranging from 0 to 0.7), ‘DirectedEdgeDetect’ (ranging from 0 to 0.7), and ‘direction’ (ranging from 0.0 to 1.0). sharpen: Images are sharpened, and the outcome is overlaid on the source image. The augmenter has argument functions ‘alpha’ (ranging from 0 (no sharpening) – 1 (full sharpening)) and ‘lightness’ (ranging from 0.75 to 1.5). canny: This is used for the detection of edges using a canny filter utilizing argument functions alpha (ranging from 0.5 to 1.0) and sobel_kernel_size (ranging from 3 to 7). noise: ‘OneOf’ augmentor applies one of the noise functions from the list to each image. It includes argument functions ‘GaussianBlur’ (sigma value ranging from 0 to 1.2), ‘AdditiveGaussianNoise’ (scale value 0.111*255), and ‘AdditiveLaplaceNoise’ (scale value ranging from 0 to 0.111*255). color: This augmentor applies color manipulation with one of the argument functions selected by the ‘OneOf’ list augmentor. This meta augmentor involves functions ‘channel_shuffle’ (1.0), ‘grayscale’ (1.0), ‘MultiplyHueAndSaturation’ (ranging from 0.5 to 1.5 for each pixel in the channel), ‘AddToHueAndSaturation’ (ranging from − 50 to 50 for each pixel in the channel), and ‘KMeansColorQuantization’ (n_colors value ranging from 16). contrast: Meta augmentor ‘OneOf’ applies one of the augmentor functions ‘LinearContrast’ (ranging from 0.75 to 1.5), ‘brightness’ (ranging from 0.35 to 1.65), and ‘brightness_channels’ (ranging from 0.5 to 1.5, per_channel value 0.75) to each image. Table 1 provides the training, validation, and testing data organization following the training data augmentation. The ‘ImageDataGenerator’ function of the Keras module is used in the second stage of augmentation to allow real-time training on randomly transformed augmented training images for each epoch. It includes different argument functions like ‘featurewise_center’ (set as ‘false’), fill_mode (set as ‘nearest’), ‘rotation_range’ (set as ‘10’ degree), ‘horizontal_flip’ (set as ‘True’), ‘vertical_flip’ (set as ‘True’), ‘channel_shift_range’ (20), and ‘brightness_range’ (ranging from 0.6 to 1.4). Table 1Organization of dataset after division and augmentation.ClassTrainValidationTestSuperficial-intermediate (SI)6752526Parabasal (P)5762222koilocytes (K)12784848Metaplastic (M)14585455Dyskeratotic (D)11974545Total5184194196 Table 1Organization of dataset after division and augmentation.ClassTrainValidationTestSuperficial-intermediate (SI)6752526Parabasal (P)5762222koilocytes (K)12784848Metaplastic (M)14585455Dyskeratotic (D)11974545Total5184194196 Organization of dataset after division and augmentation. Methods This section outlines the methodology suggested in this article for examining cervix WSI slides for the detection of malignant samples. Figure 2 depicts the structural layout of the proposed HDNPR model. In contrast to patch-based schemes, the proposed framework processes the Pap smear slides as complete WSIs. Progressive resizing enables the network to first capture global smear organization and cell clustering patterns at lower resolutions, and then refine cytological details at higher resolutions. This strategy preserves contextual relationships between nuclei, cytoplasm, and surrounding tissue artifacts that are typically lost during patch extraction. Fig. 2Structural layout of the proposed model. Structural layout of the proposed model. Feature extraction using DL models with transfer learning and fine tuning Conventional ML methods are generally constrained with limited features and inadequate correlations among those features if the features are extracted manually. It is an expensive exercise to extract attributes from a huge database, and it could induce cognitive biases with a significant impact on the features’ value and subsequently the classification phase. The retrieval of identical features may cause higher incidences of misclassification. The limitations can be severely diminished by the automatic extraction of deep features using artificial neural network (ANN) based CNN classifiers. The potential of DL models to detect and characterize images more swiftly and precisely has been proven. Segmentation, feature extraction, feature selection, and classification are frequently employed assignments in medical image processing, and have seen significant advancements because of DL models. The proposed method utilizes the two prevalent DL models, VGG-1656 and ResNet-15257, to classify the WSI slides of cervical cancer. The feature maps for an image sample of the dataset are shown in Fig. 3 over certain layers of the DL methods VGG-16 and ResNet-152. Fig. 3Feature maps of different layers in VGG-16 and ResNet-152. Feature maps of different layers in VGG-16 and ResNet-152. It requires an enormous amount of computation to perform image classification using the DL paradigm, which necessitates the utilization of a huge amount of data and a dedicated graphics processing unit (GPU) for extended run times during the training phase. This reduces productivity and necessitates the expenditure of significant computing resources. By applying the pre-trained model to a current problem, Transfer Learning expedites the training process, which can lead to greater productivity and overall performance. In Transfer Learning, the knowledge acquired by pre-training the model on a sizable dataset is utilized with the DL model to train on the current problem. The term fine-tuning describes training a new network utilizing the weights of an existing network that has previously been trained. We may, of course, start a DL model with pre-trained parameters and let al.l parameters adapt to new training, but it might affect the learning ability of the model. Fine-tuning addresses this issue by freezing a few initial layers and allowing training for the rest of the layers58. In this experiment, pre-trained parameters on the ImageNet database are utilized alongside the two DL networks VGG-16 and ResNet-152. These transfer learning-based weight knowledge derived by pretraining these DL networks are fine-tuned on augmented training pap-smear WSI data by preserving some of the initial layers frozen while leaving the other levels unfrozen. The initial layers of VGG-16 and ResNet-152 are frozen till the layers named “block4_pool” and “conv4_block18_out,” respectively, and the upper layers are kept unfrozen beyond these. This allows these models to be fine-tuned. Due to the fine-tuning only 7,612,933 and 37,222,405 parameters need to be trained out of total parameters of 15,251,269 and 60,486,533 in VGG-16 and ResNet-152 respectively. Supplementary layers such as global max pooling, batch normalizing, dropout, and dense layers are inserted into the DL models, resulting in the extraction of 1024 features from the individual models. The layered schematic of fine-tuned DL models VGG-16 and ResNet-152 is shown in Figs. 4 and 5 respectively. Both of the DL algorithms have been trained using training data, a learning rate of 10− 4 for 100 iterations, a batch size of 32, and the Adam optimizer. For testing and validation purposes, the batch size is configured to 1. Feature Concatenation Concatenating features is the most efficient way to combine numerous attributes to optimize the method of classification. Here in the proposed method, the feature extracted from the two DL methods i.e. VGG-16 and ResNet-152 is concatenated. Let the feature vectors extracted from the two DL models be x and y having 1×m and 1×n dimensions, respectively, and the feature vector z produced by concatenating the extracted feature vectors have the size of 1×p.1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:z = x \oplus y\:where\:x \in \:R^{m} and\:y \in \:R^{n}$$\end{document}2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:z = [x_{1} ,x_{2} , \ldots \:,x_{m} ]^{T} \oplus [y_{1} ,y_{2} , \ldots \:,y_{n} ]^{T}$$\end{document}3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:z={[{x}_{1},\dots\:,{x}_{m},{y}_{1},,\dots\:,{y}_{n}]}^{T}\in\:{R}^{m+n}$$\end{document}4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{f\left(z\right):R}^{m+n}\to\:{R}^{p}$$\end{document} The individual vectors of the extracted feature have a size of 1 × 1024, and the size of the vector obtained by feature combining is 1 × 2048. These concatenated features are used for the classification of test images of cervix WSI data. Fig. 4Layered schematic of fine-tuned VGG-16 model. Layered schematic of fine-tuned VGG-16 model. Fig. 5Layered schematic of fine-tuned ResNet-152 model. Layered schematic of fine-tuned ResNet-152 model. Classification Finally, a section called the FCN system has been put into place to carry out the classification of test images from cervical cancer WSI pap smear slides. Input data is transformed into a one-dimensional vector by the fully connected layer. The sequential framework of this network includes dropout, batch normalization, and dense layers. The dense layers contain the softmax activation function that computes the score based on the relation:5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:{L}_{softmax}=-\sum\:_{m=1}^{N}\mathrm{log}\frac{{e}^{{W}_{{y}_{m}}^{T}{f}_{m}+{b}_{{y}_{m}}}}{\sum\:_{n=1}^{k}{e}^{{W}_{n}^{T}{f}_{n}+{b}_{n}}}$$\end{document} Here, fm and ym represent the features and actual labels of the mth image. The wn and bn denote weights and bias, respectively, for the nth class. N and k are the number of training instances and classes, respectively. The dataset’s number of classes is employed to decide the output node in the dense layers. The relation between the outcome prediction ‘yj’ and a test sample belonging to class’ j’ among the total classes ‘n’ is as follows:6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:y\: \in \sum {\:_{{i = 1,.,j,.,n}} } y_{i}$$\end{document} The test data is analyzed by setting the epoch to 200, batch size to 1, and a learning rate of 10− 4 with the Adam optimizer. Progressive Resizing Progressive resizing is an extremely efficient method for training the model beginning from scratch or via transfer learning59. In this approach, all images are successively resized so that the CNN networks can be trained on lower to higher-resolution image datasets. In the proposed work, there are three iterations used for separately classifying the test samples. In the first iteration, the classification is done on a feature set obtained by concatenated features extracted from the training of two DL networks on the resolution of the data with 224 × 224 pixels. The second iteration classifies the test data by utilizing the feature set trained on progressively resized training data of 224 × 224 and 512 × 512 pixels via transfer learning. Finally, the third iteration utilizes training on progressively resized data of 224 × 224, 512 × 512, and 1024 × 1024 pixels for the classification of cervical test data. Training parameters and settings To ensure reproducibility, the overall key parameters used in training the proposed HDNPR framework are summarized in Table 2. Table 2Training parameters and model settings.ParameterValue / SettingOptimizerAdamInitial learning rate1 × 10⁻⁴Batch size (training)32Batch size (validation/testing)1Epochs (feature extraction stage)100Epochs (classification stage)200Weight initializationPre-trained ImageNet weightsFrozen layersUp to “block4_pool” (VGG-16) and “conv4_block18_out” (ResNet-152)Dropout rate0.4Batch normalizationApplied after convolutional and dense layersFeature dimension per network1024Concatenated feature dimension2048Progressive resizing sizes224 × 224 → 512 × 512 → 1024 × 1024 Table 2Training parameters and model settings.ParameterValue / SettingOptimizerAdamInitial learning rate1 × 10⁻⁴Batch size (training)32Batch size (validation/testing)1Epochs (feature extraction stage)100Epochs (classification stage)200Weight initializationPre-trained ImageNet weightsFrozen layersUp to “block4_pool” (VGG-16) and “conv4_block18_out” (ResNet-152)Dropout rate0.4Batch normalizationApplied after convolutional and dense layersFeature dimension per network1024Concatenated feature dimension2048Progressive resizing sizes224 × 224 → 512 × 512 → 1024 × 1024 Training parameters and model settings. Results and discussion Experimental environment The proposed network is implemented in Python programming language on a system with Windows installed, an Nvidia Tesla V100 graphics card (GPU), and 16 GB of RAM. Numpy, Keras, Tensorflow, Sklearn, Matplotlib, and OpenCV are some of the Python libraries utilized in the implementation of the proposed network. Performance measures The performance of the proposed network is assessed in terms of Accuracy, Precision, Sensitivity, F-score, and Matthews Correlation Coefficient60. Accuracy (A) is the percentage of instances that were accurately predicted compared to all instances. Precision (P) is the proportion of positively predicted samples to accurately predicted positive samples. Sensitivity (S) depicts the percentage of accurately categorized incidents when a truly positive outcome is expected. To determine the F-score (F), precision and recall are utilized, by computing the harmonic mean of two. The Matthews Correlation Coefficient or MCC (M) is a balanced measure that is generally considered appropriate for use, even when the classes are of significantly varying sizes, and it is used to evaluate the quality of binary and multiclass classifications, accounting for true and false positives and negatives.7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:A=\frac{x}{x+y+z+w}$$\end{document}8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:P=\frac{x}{x+z}$$\end{document}9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:S=\frac{x}{x+w}$$\end{document}10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:F=\frac{2*P*S}{P+S}$$\end{document}11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:M=\frac{\left(x*y\right)-\left(z*w\right)}{\surd\:(x+z)(x+w)(y+z)(y+w)}$$\end{document} Here, x = True Positive, y = True Negative, z = False Positive, w = False Negative. Results The results of the experiment conducted on SIPaKMeD WSI pap smear data for the classification of test images are presented in this section. The proposed model is analyzed here and compared with the individual fine-tuned VGG-16 and ResNet-152 models for iteration 1 (224 × 224), iteration 2 (224 × 224, 512 × 512 Progressive resizing), and iteration 3 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing). Figures 6, 7, 8, 9, 10 and 11 show the confusion matrices for the fine-tuned classifiers i.e. VGG-16 and ResNet-152, and the proposed hybrid network for 5-class and 2-class classification, respectively on SIPaKMeD WSI pap smear data. According to these Figures, the proposed framework predicts the cases with the least error based on the confusion matrix that was produced. Tables 3 and 4 show the performance measures of fine-tuned classifiers i.e. VGG-16 and ResNet-152, and the proposed hybrid network for 5-class and 2-class classification, respectively on SIPaKMeD WSI pap smear data. For 5-class classification, our proposed model gives the highest accuracy of 97.45% compared to others with progressively resized data in iteration 3. This method gives precision, sensitivity, F-score, and MCC of 97.94%, 98.08%, 98.01%, and 0.958 respectively for the 5-class classification of pap-smear SIPaKMeD WSI data. Further, when the model is evaluated for 2-class classification the model performs equally well for both iterations 2 and 3 with an accuracy of 98.58%, precision of 97.85%, sensitivity of 100%, F-score of 98.91%, and MCC value of 0.969. Table 4 concludes that the best results are already obtained for iteration 2 and it does not require to perform further experiments on progressively resized data of size 1024 × 1024 i.e. iteration 3. The accuracy and loss curve for the proposed model is also shown in Figs. 12 and 13, and 14 for iteration 1 (224 × 224), iteration 2 (224 × 224, 512 × 512 Progressive resizing), and iteration 3 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing) for the 5-class classification of WSI slides. Table 5 indicates the FLOPs, Memory Usage, and Inference Time for VGG-16 and ResNet-152. Fig. 6Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-1 (224 × 224) for 5-class classification on SIPaKMeD WSI data. (i), (ii), (iii) Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-1 (224 × 224) for 5-class classification on SIPaKMeD WSI data. (i), (ii), (iii) Fig. 7Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-2 (224 × 224, 512 × 512 Progressive resizing) for 5-class classification on SIPaKMeD WSI data. (iv), (v), (vi) Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-2 (224 × 224, 512 × 512 Progressive resizing) for 5-class classification on SIPaKMeD WSI data. (iv), (v), (vi) Fig. 8Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-3 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing) for 5-class classification on SIPaKMeD WSI data. (vii), (viii), (ix) Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-3 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing) for 5-class classification on SIPaKMeD WSI data. (vii), (viii), (ix) Table 3Performance measures of fine-tuned classifiers and proposed hybrid model for 5-class classification on SIPaKMeD WSI data.DatasetMethodsAccuracy (%)Precision (%)Sensitivity (%)F-Score (%)Iteration-1 224 × 224VGG-1694.3995.4495.0495.24ResNet-15292.8693.494.1793.78Proposed model95.9296.2396.8696.54Iteration-2 (224 × 224, 512 × 512 Progressive resizing)VGG-1694.995.8895.7795.82ResNet-15293.3796.394.4895.38Proposed model96.9497.5897.6597.61Iteration-3 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing)VGG-1695.9296.6996.5496.61ResNet-15293.3796.394.4895.38Proposed model 97.45 97.94 98.08 98.01 Significant values are in bold. Table 3Performance measures of fine-tuned classifiers and proposed hybrid model for 5-class classification on SIPaKMeD WSI data.DatasetMethodsAccuracy (%)Precision (%)Sensitivity (%)F-Score (%)Iteration-1 224 × 224VGG-1694.3995.4495.0495.24ResNet-15292.8693.494.1793.78Proposed model95.9296.2396.8696.54Iteration-2 (224 × 224, 512 × 512 Progressive resizing)VGG-1694.995.8895.7795.82ResNet-15293.3796.394.4895.38Proposed model96.9497.5897.6597.61Iteration-3 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing)VGG-1695.9296.6996.5496.61ResNet-15293.3796.394.4895.38Proposed model 97.45 97.94 98.08 98.01 Significant values are in bold. Performance measures of fine-tuned classifiers and proposed hybrid model for 5-class classification on SIPaKMeD WSI data. Significant values are in bold. Fig. 9Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-1 (224 × 224) for 2-class classification on SIPaKMeD WSI data. (i), (ii), (iii) Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-1 (224 × 224) for 2-class classification on SIPaKMeD WSI data. (i), (ii), (iii) Fig. 10Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-2 (224 × 224, 512 × 512 Progressive resizing) for 2-class classification on SIPaKMeD WSI data. (iv), (v), (vi) Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-2 (224 × 224, 512 × 512 Progressive resizing) for 2-class classification on SIPaKMeD WSI data. (iv), (v), (vi) Fig. 11Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-2 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing) for 2-class classification on SIPaKMeD WSI data. (vii), (viii), (ix) Confusion matrices of (i) VGG-16, (ii) ResNet-152, and (iii) Proposed Model for Iteration-2 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing) for 2-class classification on SIPaKMeD WSI data. (vii), (viii), (ix) Table 4Performance measures of fine-tuned classifiers and proposed hybrid model for 2-class classification on SIPaKMeD WSI data.DatasetMethodsAccuracy (%)Precision (%)Sensitivity (%)F-Score (%)Iteration-1 224 × 224VGG-1696.4594.6210097.24ResNet-15295.0493.5598.8696.13Proposed model97.8796.7710098.36Iteration-2 224 × 224, 512 × 512 (Progressive resizing)VGG-1697.1695.710097.8ResNet-15295.7493.5510096.67Proposed model 98.58 97.85 100 98.91 Iteration-3 224 × 224, 512 × 512, 1024 × 1024 (Progressive resizing)VGG-1697.1695.710097.8ResNet-15295.7493.5510096.67Proposed model 98.58 97.85 100 98.91 Significant values are in bold. Table 4Performance measures of fine-tuned classifiers and proposed hybrid model for 2-class classification on SIPaKMeD WSI data.DatasetMethodsAccuracy (%)Precision (%)Sensitivity (%)F-Score (%)Iteration-1 224 × 224VGG-1696.4594.6210097.24ResNet-15295.0493.5598.8696.13Proposed model97.8796.7710098.36Iteration-2 224 × 224, 512 × 512 (Progressive resizing)VGG-1697.1695.710097.8ResNet-15295.7493.5510096.67Proposed model 98.58 97.85 100 98.91 Iteration-3 224 × 224, 512 × 512, 1024 × 1024 (Progressive resizing)VGG-1697.1695.710097.8ResNet-15295.7493.5510096.67Proposed model 98.58 97.85 100 98.91 Significant values are in bold. Performance measures of fine-tuned classifiers and proposed hybrid model for 2-class classification on SIPaKMeD WSI data. Significant values are in bold. Fig. 12Proposed Model for Iteration-1 (224 × 224) in 5-class classification (a) Accuracy curve and (b) Loss curve. Proposed Model for Iteration-1 (224 × 224) in 5-class classification (a) Accuracy curve and (b) Loss curve. Fig. 13Proposed Model for Iteration-2 (224 × 224, 512 × 512 Progressive resizing) in 5-class classification (a) Accuracy curve and (b) Loss curve. Proposed Model for Iteration-2 (224 × 224, 512 × 512 Progressive resizing) in 5-class classification (a) Accuracy curve and (b) Loss curve. Fig. 14Proposed Model for Iteration-3 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing) in 5-class classification (a) Accuracy curve and (b) Loss curve. Proposed Model for Iteration-3 (224 × 224, 512 × 512, 1024 × 1024 Progressive resizing) in 5-class classification (a) Accuracy curve and (b) Loss curve. Table 5FLOPs, Memory Usage, and Inference Time for VGG-16 and ResNet-152.DatasetInput sizeFLOPs (≈)Inference time / image (≈)GPU memory (≈)VGG-16224 × 224~ 15–16 GFLOPs5–7 ms1.5-2.0 GB512 × 512~ 80–85 GFLOPs16–22 ms3.5–4.5 GB1024 × 1024~ 320–340 GFLOPs55–78 ms9–11 GBResNet-152224 × 224~ 11–12 GFLOPs10–13 ms1.2–1.5 GB512 × 512~ 60–65 GFLOPs38–50 ms3–4 GB1024 × 1024~ 240–260 GFLOPs102–130 ms8–10 GB Table 5FLOPs, Memory Usage, and Inference Time for VGG-16 and ResNet-152.DatasetInput sizeFLOPs (≈)Inference time / image (≈)GPU memory (≈)VGG-16224 × 224~ 15–16 GFLOPs5–7 ms1.5-2.0 GB512 × 512~ 80–85 GFLOPs16–22 ms3.5–4.5 GB1024 × 1024~ 320–340 GFLOPs55–78 ms9–11 GBResNet-152224 × 224~ 11–12 GFLOPs10–13 ms1.2–1.5 GB512 × 512~ 60–65 GFLOPs38–50 ms3–4 GB1024 × 1024~ 240–260 GFLOPs102–130 ms8–10 GB FLOPs, Memory Usage, and Inference Time for VGG-16 and ResNet-152. Ablation analysis of proposed framework An ablation study is conducted to validate the significance of the design elements of the proposed framework. The purpose of the ablation study is to determine the effects of each component of the proposed model on the overall classification performance. To ensure that the comparisons are fair, the same experimental setting is used for all models. The ablation study results for the 5-class and 2-class classification tasks on the SIPaKMeD WSI dataset are presented in Table 6. Table 6Ablation analysis of the proposed HDNPR framework on SIPaKMeD WSI data.Model variantProgressive ResizingFeature concatenation5-class accuracy (%)2-class accuracy (%)VGG-16✗ (224 × 224 only)✗94.3996.45✔ (224 × 224→512 × 512)✗94.997.16✔ (224 × 224→512 × 512→1024 × 1024)✗95.9297.16ResNet-152✗ (224 × 224 only)✗92.8695.04✔ (224 × 224→512 × 512)✗93.3795.74✔ (224 × 224→512 × 512→1024 × 1024)✗93.3795.74Proposed model✗ (224 × 224 only)✔95.9297.87✔ (224 × 224→512 × 512)✔96.94 98.58 ✔ (224 × 224→512 × 512→1024 × 1024)✔ 97.45 98.58 Significant values are in bold. Table 6Ablation analysis of the proposed HDNPR framework on SIPaKMeD WSI data.Model variantProgressive ResizingFeature concatenation5-class accuracy (%)2-class accuracy (%)VGG-16✗ (224 × 224 only)✗94.3996.45✔ (224 × 224→512 × 512)✗94.997.16✔ (224 × 224→512 × 512→1024 × 1024)✗95.9297.16ResNet-152✗ (224 × 224 only)✗92.8695.04✔ (224 × 224→512 × 512)✗93.3795.74✔ (224 × 224→512 × 512→1024 × 1024)✗93.3795.74Proposed model✗ (224 × 224 only)✔95.9297.87✔ (224 × 224→512 × 512)✔96.94 98.58 ✔ (224 × 224→512 × 512→1024 × 1024)✔ 97.45 98.58 Significant values are in bold. Ablation analysis of the proposed HDNPR framework on SIPaKMeD WSI data. Significant values are in bold. Effect of progressive resizing The effect of progressive resizing can be analyzed by comparing the results of Iteration-1, Iteration-2, and Iteration-3. When the training is carried out at a single resolution (Iteration-1), the hybrid model performs relatively poorly. However, as progressively resized images are incorporated (224 × 224 → 512 × 512 → 1024 × 1024), the classification accuracy increases steadily, which confirms that the progressive resizing technique allows the network to learn the smear patterns at a global level at lower resolutions and then focus on the fine cytological details at higher resolutions. Effect of feature concatenation The importance of feature concatenation is highlighted by comparing the hybrid model with the separate backbone models. The hybrid model performs better than the separate VGG-16 and ResNet-152 models for both multiclass and binary classification. This verifies that the combination of deep features from both models is complementary, where VGG-16 provides detailed texture information and ResNet-152 provides contextual information. The combination of both features improves discriminative power over the separate backbone model. Effect of backbone selection The choice of VGG-16 and ResNet-152 as the backbone networks is based on our previous baseline study11, where various fine-tuned deep learning models were compared for the classification task of Pap smear WSI images. The two models have shown consistent and excellent performance in transfer learning scenarios. The ablation study results in this work have also confirmed that the combination of the two models performs better than the individual models, hence the choice of using the two models in the HDNPR framework. In general, the ablation study above confirms that each component of the proposed HDNPR framework makes a positive contribution to the improvement of performance. The combination of hybrid feature concatenation and progressive resizing leads to a positive effect on both 5-class and 2-class classification tasks, thus justifying the choices made in the proposed framework. Comparative assessment The proposed framework is compared to the most recent state-of-the-art techniques for 5-class and 2-class categorization of SIPaKMeD WSI data in Table 7. The accuracy of the proposed HDNPR framework in the categorization of cervical cancer is convincingly demonstrated by this comparison. The present study incorporates several significant enhancements in the WSI detection of cervical cancer by employing progressively resized training data with extensive augmentation to enhance generalization. It prioritizes fine-tuned VGG-16 and ResNet-152 for efficient feature extraction and streamlining the model architecture by reducing the number of DL networks. Consequently, this can lead to faster training and computation times. The new model exhibits an impressive accuracy of 97.45% for a 5-class classification of WSI slides, indicating adaptability. It also attains a high accuracy of 98.58% for the 2-class classification of cervical cancer classification with a slight decrease from the previous model. These advancements demonstrate substantial advancements in training techniques and model effectiveness (Table 7). Table 7Comparative performance of the proposed method with the existing model on the classification of SIPaKMeD WSI data.ArticlesModelsClassification result (%)5-class2-classChauhan [11]VGG-1694.8997.16Kiran [61]Pretrained ResNet-34 + Classification block + PCA96.37–Win [62]Ensemble (LD + SVM+KNN+boosted trees+bagged trees)94.09Wang [63]RCAN+DenseNet-12191.09–Manna [64]Ensemble (Inception-v3 + Xception+DenseNet-169)95.4398.55Hardatin [65]ResNet-5091–Karim Kutty [66]Correlation-based feature selection (WT + CNN) + RF97.01–Yadav [67]Topo-ML model94.21–Khanarsa [68]ResNet152V2-based ensemble method97–Present work Proposed model (HDNPR) 97.45 98.58 Significant values are in bold. Table 7Comparative performance of the proposed method with the existing model on the classification of SIPaKMeD WSI data.ArticlesModelsClassification result (%)5-class2-classChauhan [11]VGG-1694.8997.16Kiran [61]Pretrained ResNet-34 + Classification block + PCA96.37–Win [62]Ensemble (LD + SVM+KNN+boosted trees+bagged trees)94.09Wang [63]RCAN+DenseNet-12191.09–Manna [64]Ensemble (Inception-v3 + Xception+DenseNet-169)95.4398.55Hardatin [65]ResNet-5091–Karim Kutty [66]Correlation-based feature selection (WT + CNN) + RF97.01–Yadav [67]Topo-ML model94.21–Khanarsa [68]ResNet152V2-based ensemble method97–Present work Proposed model (HDNPR) 97.45 98.58 Significant values are in bold. Comparative performance of the proposed method with the existing model on the classification of SIPaKMeD WSI data. Significant values are in bold. McNemar’s statistical test McNemar’s Analysis may be used to examine the prediction outcomes of two models in the framework of statistical or ML models. It is used to assess the p-value for finding the non-parametric statistical decision about the distinctness between proposed and baseline classifiers. The hypotheses considered in this analysis are defined as follows: Null hypothesis (H0): There is no statistically significant difference between the error rates of the compared classifiers. Alternative hypothesis (H1): There exists a statistically significant difference between the error rates of the compared classifiers. The McNemar test is originally designed for paired binary data. In this study, although the classification task considers multiclass settings, the McNemar test is used by transforming the model predictions into paired binary data (correct vs. incorrect predictions) for the same test data. If the p-value is found to be in the bounds of (less than) 0.05 or 5%, then the null hypothesis is said to be false, and the extracted features from the paired models are sufficiently distinct. Here, Table 8 indicates the p-value between the individual baseline models (VGG-16 and ResNet-152) and the proposed classifier. The results indicate that the proposed approach performs substantially distinct from the individual fine-tuned models (Table 8). Table 8McNemar’s statistical test outcome.McNemar’s test comparison between the proposed and fine-tuned model asp-value for the SIPaKMeD dataVGG-160.0012ResNet-1520.0015P < 0.5 indicates that the null hypothesis is untrue or false. Table 8McNemar’s statistical test outcome.McNemar’s test comparison between the proposed and fine-tuned model asp-value for the SIPaKMeD dataVGG-160.0012ResNet-1520.0015P < 0.5 indicates that the null hypothesis is untrue or false. McNemar’s statistical test outcome. P < 0.5 indicates that the null hypothesis is untrue or false. Conclusion and future work The proposed HDNPR model is spurred by the continuous expansion of AI utilization in the early detection and prognosis of cervical cancer. In this study, the WSI data for cervical cancer is analyzed by a hybrid deep network utilizing the two-stage augmentation technique. The proposed HDNPR model utilizes two fine-tuned models, i.e., VGG-16 and ResNet-152, for feature extraction of progressively resized training data of size 224 × 224, 512 × 512, and 1024 × 1024 pixels. The extracted features get concatenated, and classification is performed for binary and multiclass by FCN. This model performs well with an accuracy score of 97.45% and 98.58% for 5-class and 2-class classification on cervical cancer SIPaKMeD WSI test data. In this study, the progressive resizing technique, when used with the hybrid deep network, increases training efficiency, model robustness, and image variability adaptation by gradually increasing image resolution in cervical cancer detection. Experts may examine the cells in the WSI images to a greater extent owing to the precise detection of infectious samples. To improve the model’s efficacy and reduce computational costs, the future perspective incorporates the implementation of more optimal and improved feature selection methods and progressive ensemble approaches employing hybrid methods of ML and DL algorithms. Although the model operates on complete WSIs and preserves global cytological structure through progressive resizing, it does not yet incorporate explicit region-level attention or tile-based histopathology reasoning. Developing a multi-scale WSI attention mechanism will therefore be an important direction for future work. In addition, future investigations will extend validation to additional Pap smear WSI datasets from different clinical environments, enabling its generalizability can be examined in a broader context.