Enhancing land cover object classification in hyperspectral imagery through an efficient spectral-spatial feature learning approach Authors: Masud Ibn Afjal, Md. Nazrul Islam Mondal, Md. Al Mamun Published: 2024 ABSTRACT The classification of land cover objects in hyperspectral imagery (HSI) has significantly advanced due to the development of convolutional neural networks (CNNs). However, challenges such as limited training data and high dimensionality negatively impact classification performance. Traditional CNN-based methods predominantly utilize 2D CNNs for feature extraction, which inadequately exploit the inter-band correlations in HSIs. While 3D CNNs can capture joint spectral-spatial information, they often encounter issues related to network depth and complexity. To address these issues, we propose an innovative land cover object classification approach in HSIs that integrates segmented principal component analysis (Seg-PCA) with hybrid 3D-2D CNNs. Our approach leverages Seg-PCA for effective feature extraction and employs the minimum-redundancy maximum relevance (mRMR) criterion for feature selection. By combining the strengths of both 3D and 2D CNNs, our method efficiently extracts spectral-spatial features. These features are then processed through fully connected dense layers and a softmax layer for classification. Extensive experiments on three widely used HSI datasets demonstrate that our method consistently outperforms existing state-of-the-art techniques in classification performance. These results highlight the efficacy of our approach and its potential to significantly enhance the classification of land cover objects in hyperspectral imagery. 1 Introduction Land cover object classification performs a crucial role in numerous fields, including agriculture, urban planning, environmental monitoring, and resource management [1–3]. Hyperspectral images (HSIs) have significantly advanced this classification task due to their ability to provide detailed spectral information for each pixel. These images offer both 2D spatial information and 1D spectral information, providing rich data that poses significant analytical challenges due to its high dimensionality. HSIs also have diverse applications in multimedia, medicine, atmospheric monitoring, target detection, hydrological assessment, segmentation, anomaly detection, and wind forecasting [4–8]. Traditional classification methods, such as Support Vector Machines (SVM) [9], K-Nearest Neighbor (KNN) [10], and Artificial Neural Networks (ANN) [11], primarily rely on spectral information and utilize feature extraction and dimensionality reduction techniques like Principal Component Analysis (PCA) [12, 13], Linear Discriminant Analysis (LDA) [14, 15], Independent Component Analysis (ICA) [16], and Minimum Noise Fraction (MNF) [17]. PCA, for example, identifies features by capturing variations and transforming them into principal components (PCs). LDA increases class separation by extracting relevant features, while MNF arranges data based on the signal-to-noise ratio (SNR) axis. However, the high data dimensionality present in HSIs can lead to the Hughes phenomenon, where classification accuracy initially rises and then declines [18]. To mitigate this, dimensionality reduction techniques like PCA and LDA have been employed, but they can introduce noise. Feature selection is a critical aspect of machine learning preprocessing, involving the identification of relevant features while eliminating irrelevant or redundant ones from the initial feature subset. Common feature selection strategies include Mutual Information (MI) [19], Normalized Mutual Information (nMI) [20], and Minimum Redundancy-Maximum Relevance (mRMR) [21]. However, a significant challenge with hyperspectral images is the lack of spatial characteristics, limiting their classification using popular machine learning methods like KNN and SVM. Despite advancements in hyperspectral image classification, traditional methods often neglect spatial context, relying solely on spectral information. This oversight results in suboptimal performance, highlighting the need for novel approaches that integrate spectral-spatial features for improved land cover object identification. In computer vision, Deep learning (DL) has emerged as a dominant approach due to its ability to automatically extract complex features. DL has been applied to HSI classification, with various architectures and techniques proposed [22–24]. However, DL methods for HSI classification primarily prioritize spectral information, often overlooking spatial context, which can limit their performance. In response, researchers have employed 2D convolutional neural networks (CNNs) [25–27], which capture spatial patterns and inspire innovative architectures. Other approaches include pixel-pair methods that exploit the similarity between neighboring pixels and small-scale data-driven methods for limited samples [28, 29]. Efforts to enhance HSI classification by combining spectral and spatial features [30–32] have shown that fusion of these two types of information enables algorithms to leverage both spectral diversity and spatial context, achieved through network transformations or double-branch spatial-spectral extraction and fusion. CNNs have strong feature extraction capabilities and have been enhanced to address their limitations [33]. For example, in random weight network [34], dense residual networks where the skip connections ability facilitates smooth gradient flow [35] and have been applied to HSI classification. Deep residual networks with attention mechanisms [36] have also shown promising results. Additionally, GPU development has reduced training time for large parameter networks [37]. Hybrid models that combine 2D-CNN and 3D-CNN layers have been proposed to utilize both spectral and spatial feature maps for maximum accuracy [38–40]. These models address the limitations of using either 2D [25, 26] or 3D CNNs [41, 42], as they may not effectively extract discriminating feature maps from the spectral dimension alone or become computationally complex for classes that have the same texture patterns towards various spectral bands. Recent advancements have introduced hybrid models that extend CNN capabilities. Hybrid-2DNET [43] enhances SpectralNet by integrating feature selection post-factor analysis, identifying high-impact features for improved classification. Similarly, TP-Net [44], a triple-path spectral-spatial network, uses interleaved attention mechanisms to discern important features better. The Deep Spectral Spatial Feature Enhancement (DS2FE) [45] employs a vision-based transformer with a multiscale feature extractor for low-level spectral-spatial features and a regional attention mechanism with a spatially gated module for high-level semantic extraction. S2PNet [46], an interactive learning model, uses multi-stage spectral purification, global-local feature interaction, and shallow-deep integration to enhance classification accuracy by reducing spectral and spatial heterogeneity. Traditional PCA, as a preprocessing step of CNN, seems a single overall model for the complete HSI [38, 47], which may not effectively capture the intricate spatial and spectral variations present in the data. However, segmented PCA [15, 48–51] offers a solution by dividing the data into smaller sub-groups and applying PCA independently to each local sub-group. This localized and adaptive approach leads to improved preservation of spatial-spectral information, consequently enhancing the accuracy of subsequent classification or analysis tasks. This research presents a hybrid spectral-spatial feature extraction technique for effective HSI classification. We first recover spectral properties in local and global domains using segmented PCA. Then, we employ the minimum-redundancy maximum-relevance (mRMR) technique to remove additional duplicate information and select more accurate and meaningful features from the extracted PCs. Subsequently, we utilize a hybrid neural network that incorporates both 3D-CNN and 2D-CNN layers to extract both spectral and spatial features from the selected PCs. The key contributions of this paper are summarized as follows: An effective combination of unsupervised segmented PCA (Seg-PCA)-based feature extraction and supervised mRMR-based feature selection is presented as a preprocessing step for hybrid 3D-2D CNN. An effective combination of unsupervised segmented PCA (Seg-PCA)-based feature extraction and supervised mRMR-based feature selection is presented as a preprocessing step for hybrid 3D-2D CNN. A novel approach is introduced that combines Seg-PCA-mRMR with hybrid spectral-spatial feature extraction using 3D-CNN and 2D-CNN combined. This enables the extraction of enhanced spectral-spatial features from HSIs for enhanced classification. A novel approach is introduced that combines Seg-PCA-mRMR with hybrid spectral-spatial feature extraction using 3D-CNN and 2D-CNN combined. This enables the extraction of enhanced spectral-spatial features from HSIs for enhanced classification. The proposed approach incorporates network inputs and selects features from different sub-regions, leading to enhanced classification accuracy. The approach captures a more complete representation of the HSI data by extracting features from both the spectral and spatial domains. The proposed approach incorporates network inputs and selects features from different sub-regions, leading to enhanced classification accuracy. The approach captures a more complete representation of the HSI data by extracting features from both the spectral and spatial domains. We conducted empirical experiments on three widely used HSI datasets, comparing our proposed framework with state-of-the-art approaches. Our method demonstrated superior performance in terms of classification accuracy, generalization, and robustness. We conducted empirical experiments on three widely used HSI datasets, comparing our proposed framework with state-of-the-art approaches. Our method demonstrated superior performance in terms of classification accuracy, generalization, and robustness. The remaining parts of this article are organized as follows: In-depth discussion on the theoretical basis related to the topic is presented in Section 2. Section 3 outlines the proposed neural network approach and its specific implementation steps. In Section 4, we thoroughly discuss the experimental results obtained and analyze them in detail. Lastly, Section 5 summarizes the conclusions drawn from this study. 2 Related methodology 2.1 PCA-based feature extraction for HSI PCA is a popular feature extraction and dimensionality reduction technique used for analyzing HSI data. Its objective is to retain crucial information while transforming the high-dimensional data into a lower-dimensional space. PCA achieves this by identifying principal components (PCs) that capture maximum variance in the data [52]. To implement PCA, each sample pixel’s spectral vector in the HSI data matrix is denoted as xn = [xn1xn2…xnF]T, with n ∈ [1, S], and S = X × Y representing the spatial dimensions. The data matrix D is used to derive the zero-mean image I, where In = xn − M, and M=1S∑n=1Sxn. The covariance matrix C=1SIIT is computed using eigen decomposition C = V E VT, where E and V are eigenvalues and eigenvectors (PCs), respectively. A subset of q PCs, represented by matrix W, is selected based on descending eigenvalue rankings. Finally, the projection matrix Z of D is obtained as Z = WT × I. Seg-PCA implementation involves dividing the HSI data matrix into L subgroup datasets based on band correlations. Each subgroup captures local characteristics effectively by grouping strongly correlated bands together. Seg-PCA selectively reduces the number of spectral bands while preserving spatial dimensions. In Seg-PCA, the HSI data matrix D is divided into subgroups Dt, where t ∈ [1, 2, ..L]. Eigendecomposition is applied to each Dt to compute covariance matrices. The projection matrices of each subgroup are then combined to obtain the overall projection matrix. 2.2 mRMR based feature selection for HSI mRMR is a widely used feature selection method in the analysis of HSI data. It aims to select the most relevant features related to the target variable while minimizing redundancy among them. The process involves calculating the Mutual Information (MI) between the target variable and each feature. The features are then sorted in descending order based on their MI scores, and they are added one by one to the subset until the desired number is achieved [20, 53]. To perform the MI-based feature selection, two input images, denoted as X and C, are required, along with their respective marginal probability distributions, represented as p(x) and p(c), as well as their joint probability distribution p(x, c). The MI, denoted as I(X, c), is calculated using these probability distributions in the following equation: I(X,C)=∑c=1S∑x=1Sp(x,c)logp(x,c)p(x)p(c) (1) In the context of HSI analysis, MI-based feature selection uses X as the input extracted spectral features and C as the corresponding ground truth image. The aim is to maximize the MI when X and C are equivalent. To avoid redundancy, the greedy approach of MI-based feature selection selects the (k + 1)th feature while taking into account the previously selected Q features, using the following equation: G(Xi,k)=I(Xi,C)-β∑qjϵQI˜(Xi,qj),Xi⊄Q (2) Here, β regulates redundancy’s significance in the mRMR criterion, and I˜ is the normalized Mutual Information defined as: I˜(X,C)=I(X,C)(H(X)H(C)) (3) To ensure adequate feature selection features with I˜(X,C)<ξ are eliminated, where ξ is a threshold representing the minimum level of relevance required for the target data. Also, the condition G˜(Xi,k)>0 is used to verify if the chosen feature is unalike to the already selected ones, ensuring desirable selections. 2.3 Convolutional Neural Network (CNN) Indeed, CNNs are a feedforward neural network that performs remarkably in large-scale image processing tasks. CNNs are specifically designed to handle image data efficiently, using their ability to learn hierarchical features from raw pixel values automatically [54–60]. It leverages the 2D image structure, recognizing pixel correlations in adjacent regions, and utilizes feature sharing through convolution. Thus, it detects local patterns and features in the input images, making them highly effective for tasks such as image classification, object detection, and image segmentation. Due to their ability to handle large-scale image data effectively, CNNs have become the backbone of numerous remotely sensed image applications and have significantly advanced the field of spectral image processing. The CNN architecture, comprising convolutional layers, activation functions (such as ReLU), pooling layers for dimensionality reduction, and fully connected layers, has revolutionized image processing tasks. Convolutional layers apply filters to capture local patterns in the input data, while activation functions introduce nonlinearity for better learning. Pooling layers reduce data size, preventing overfitting. Fully connected layers transmit features to the classifier for making predictions. The backpropagation algorithm optimizes the network’s parameters during training, leading to improved performance and accurate image processing results. CNN-based HSI classification methods encompass 2D and 3D CNNs. For instance, Fast and Compact 3D CNN utilize iPCA and 3D-2D CNNs for feature reduction and classification [41]. SpectralNET leverages factor analysis for preprocessing to reduce data dimension and a wavelet-based architecture for spatial-spectral feature extraction [39]. Similarly, HybridSN incorporates PCA for feature reduction, three 3D CNNs, and a 2D CNN for classification [38]. These represent the foundational, cutting-edge CNN models for HSI classification. However, the field has seen recent advancements in the form of fusion or hybrid models that build upon these aforementioned CNN-based frameworks. One such example is the Hybrid-2DNET [43], which introduces novel modifications to the SpectralNet architecture. By incorporating a multi-resolution mutual information-based feature selection process subsequent to the factor analysis stage, this model enhances classification accuracy. Similarly, there is the triple-path spectral-spatial network, TP-Net [44], which employs an innovative approach to elevate classification performance. TP-Net integrates two distinct interleaved attention mechanisms into its architecture. This strategic incorporation of attention mechanisms contributes to heightened discriminative capabilities, ultimately resulting in improved classification accuracy. These approaches address the challenges of limited training samples and high dimensionality in HSI data, offering effective solutions for accurate HSI classification. 3 Proposed spectral-spatial feature learning approach 3.1 Overview The proposed approach for land cover object identification in HSIs, as illustrated in Fig 1, combines Seg-PCA with mRMR for feature reduction and employs a hybrid 3D-2D CNN architecture for effective spectral-spatial feature extraction. Initially, Seg-PCA is applied to reduce the dimensionality of the hyperspectral data and preserve critical spectral characteristics. Following this, mRMR is used to select the most informative features from the reduced data. The selected features are then processed through a 3D-2D CNN, which captures both spectral and spatial features for improved land cover classification. This network architecture integrates 3D convolutions to extract spectral features and 2D convolutions for spatial features, culminating in a classification layer with dense connections and dropout for robustness. By leveraging this combination of advanced techniques, the proposed method aims to significantly enhance land cover object classification performance in hyperspectral imagery. Proposed spectral-spatial feature learning approach for HSI classification. 3.2 Dataset description In our experiments, we used three publicly available HSI datasets: Indian Pines (IP), University of Pavia (PU), and Salinas Scene (SA). The IP dataset contains images with a spatial dimension of 145 × 145 pixels and 224 spectral bands in the wavelength range of 400 to 2500 nm. We excluded 24 spectral bands corresponding to water absorption regions. The ground truth of the IP dataset provides labels for 16 land cover vegetation object classes [61]. The PU dataset consists of images with a spatial dimension of 610 × 340 pixels and 103 spectral bands in the wavelength range of 430 to 860 nm. The ground truth of the PU dataset classifies nine urban land cover object classes [62]. The SA dataset includes images with a spatial dimension of 512 × 217 pixels and 224 spectral bands in the wavelength range of 360 to 2500 nm. We removed 20 spectral bands affected by water absorption. The SA dataset consists of 16 different land cover object classes [63]. We selected these datasets for their diversity in spatial dimensions, spectral bands, and land-cover classes. This diversity enables us to thoroughly evaluate the performance and generalization capability of our proposed method across various HSI scenarios. 3.3 Seg-PCA-mRMR While PCA is effective, it may overlook subtle information valuable for accurate HSI data analysis, particularly when the original HSI bands exhibit high correlation, resulting in a large covariance matrix and higher computational costs. To address this limitation, Seg-PCA has been proposed [48, 50] and presented [21, 49, 64, 65]. Seg-PCA is specifically designed for handling highly correlated blocks with low inter-correlations. In Seg-PCA implementation, the HSI data matrix is divided into L subgroup datasets (L = 3 in this case) based on band correlations. Within each subgroup, strongly correlated bands are grouped together to capture local characteristics effectively. The sub-grouped data matrices are denoted as Dt, where t ∈ [1, 2, 3], and each Dt contains nt consecutive bands. Eigendecomposition is applied to each Dt to compute covariance matrices. The overall projection matrix is obtained by sequentially combining the projection matrices of each subgroup’s Dt. After applying Seg-PCA and feature extraction to the HSI dataset, we employ the mRMR feature selection process to enhance the feature set by identifying the most informative features. In the mRMR steps, we select the top b ranked PCs from the overall projected data, similar to our previous research with traditional machine learning [15]. The Seg-PCA-mRMR technique specifically reduces the number of spectral bands from F to Q, representing the extracted and selected PC dimensions for the reduced data. However, the spatial dimensions of the data matrix remain unchanged, with S = X × Y, where D = X × Y × F representing the overall input data cube. The modified input data after Seg-PCA-mRMR is represented by ZQ = X × Y × Q, where ZQ corresponds to the modified input data cube after applying Seg-PCA-mRMR. This additional step offers several advantages, such as improved computational efficiency, enhanced classification accuracy, and a deeper understanding of the underlying scientific insights present in the data. The Seg-PCA+mRMR algorithm (Algorithm 1 and 2) is employed as a preprocessing step in the proposed methodology for dimensionality reduction and feature selection. It divides the original hyperspectral data cube D into L subgroup datasets using Seg-PCA and selects the most informative principal components using mRMR. Algorithm 1 Seg-PCA followed by mRMR Feature Selection Require: HSI data cube D with dimensions X × Y × F 1: Divide D into L subgroup datasets based on band correlations 2: for t = 1 to L do 3: Compute covariance matrix Ct for Dt 4: Perform eigendecomposition on Ct to obtain eigenvalues and eigenvectors 5: Select top b eigenvectors from Ct 6: Concatenate selected eigenvectors from each subgroup 7: end for 8: Apply mRMR feature selection on the concatenated eigenvectors (Algorithm 2) 9: return Reduced data ZQ with dimensions X × Y × Q Algorithm 2 mRMR Feature Selection Require: Concatenated eigenvectors Z from Seg-PCA 1: Calculate mutual information between each feature and class labels 2: Calculate redundancy and relevance scores for each feature 3: Rank features based on mRMR score (relevance—redundancy) 4: Select top b features with highest mRMR scores 5: return Selected features 3.4 Neighbourhood extraction process To execute our 3D-2D CNN model, the HSI data cube is divided into small overlapping 3D patches. These patches, denoted as PQ = s × s × Q, are created from the Seg-PCA-mRMR reduced data cube ZQ and are centered at spatial location (i, j). Each patch covers a spatial extent of s × s and contains all Q PCs representing the selected spectral features. The total number of generated 3D patches, denoted as n, from ZQ is determined by (X − s + 1) × (Y − s + 1), where X and Y represent the spatial dimensions of the data cube. As a result, the 3D patch located at (i, j), denoted as PQ, i, j, includes the width from i − (s − 1)/2 to i + (s − 1)/2, height from j − (s − 1)/2 to j + (s − 1)/2, and contains Q selected features of the Seg-PCA-mRMR reduced data cube ZQ. 3.5 Spectral-spatial feature extraction with 3D-2D CNN In our proposed spectral-spatial 3D-2D CNN architecture, we utilize three 3D convolution layers, one 2D convolution layer, and two fully connected layers. The 3D convolution operation involves convolving a 3D kernel with the input data to generate feature maps in the convolution layer. This process captures the spectral information present in multiple contiguous bands of the input layer. In the 3D convolution equation, the activation value at a spatial position (i, j, k) in the f-th feature map of the l-th layer, denoted as xi, j, kl, f, is computed by applying the activation function ϕ to the sum of the bias parameter bl, f and the weighted sum of the input values from the previous layer using the kernel weights wl,f, α. The kernel has a width of 2ξ + 1, a height of 2δ + 1, and a depth of 2η + 1 along with the spectral dimension, where ξ, δ, and η determine the kernel size. The weight parameter wl,f, α represents the weight value for the f-th feature map of the l-th layer and the α-th spectral band. The computation of the activation function is carried out as follows: xl,fi,j,k=ϕ(bl,f+∑α=1λl-1∑β=-ηη∑γ=-ξξ∑σ=-δδwl,f,ασ,γ,β*xl-1,αi+σ,j+γ,k+β) (4) In the case of 2D-CNN, the input data is convolved with 2D kernels. The convolution operation involves computing the sum of the dot product between the input data and the kernel, which is strived over the input data to cover the full spatial dimension. The resulting convolved features are then passed through the activation function to introduce nonlinearity in the model. In the equation for 2D convolution, the activation value at spatial position (i, j) in the f-th feature map of the l-th layer, denoted as xi, jl, f, is computed by applying the activation function ϕ to the sum of the bias parameter bl, f and the weighted sum of the input values from the previous layer using the kernel weights wl,f, α as follows: xl,fi,j=ϕ(bl,f+∑α=1λl-1∑γ=-ξξ∑σ=-δδwl,f,ασ,γ*xl-1,αi+σ,j+γ,) (5) The kernel has a width of 2ξ + 1 and a height of 2δ + 1, and it does not include the spectral dimension as in the 3D convolution equation. Algorithm 3 3D-2D CNN for Spectral-spatial Feature Extraction Require: Reduced data cube ZQ with dimensions X × Y × Q 1: Divide ZQ into overlapping 3D patches PQ of size s × s × Q 2: Initialize CNN architecture with specified parameters 3: Perform 3D convolution with kernel size 3 × 3 × 5 and 8 kernels 4: Perform 3D convolution with kernel size 3 × 3 × 3 and 16 kernels 5: Perform 3D convolution with kernel size 3 × 3 × 3 and 32 kernels 6: Perform 2D convolution with kernel size 3 × 3 and 64 kernels 7: Apply ReLU activation function after each convolution operation 8: Flatten the output tensor to prepare for fully connected layers 9: Add two fully connected layers with ReLU activation functions. 10: Output softmax probabilities for classification 11: return Classification accuracy The 3D-2D CNN algorithm (Algorithm 3) is utilized in the proposed spectral-spatial feature extraction process, where it plays a crucial role in extracting discriminative features from the reduced data cube ZQ. These features are then utilized for subsequent classification tasks. The parameters of the CNN, including bias and kernel weights, are trained using supervised approaches with gradient descent optimization. Conventional 2D CNNs operate on spatial dimensions only, limiting their ability to capture spectral information. On the other hand, 3D CNNs can extract both spatial and spectral features simultaneously, but at the cost of higher computational complexity. To leverage the feature learning capabilities of both 3D and 2D CNNs, our proposed spectral-spatial model combines them to extract different features and then fuses them for a more discriminative representation. The flattening layer is applied to preserve spatial information while reducing spectral dimensionality. 3.6 The model’s architecture and parameter details The model’s architecture and parameter details are summarized in Table 1. The proposed model consists of convolutional layers with different kernel sizes and numbers of kernels, contributing to the extraction of different spatial and spectral features from the HSI data and enhancing the model’s discriminative power. The first 3D CNN has a kernel size of 3 × 3 × 5 and is composed of 8 kernels. The second 3D CNN has a kernel size of 3 × 3 × 3 and contains 16 kernels. The third 3D CNN has the same kernel size of 3 × 3 × 3 but contains 32 kernels. Finally, the 2D CNN has a kernel size of 3 × 3 and includes 64 kernels. The total trainable weight parameters for the proposed model is 4,824,816 for the Indian Pines (IP) dataset when Q = 10. These weights are randomly initialized and trained using the back-propagation algorithm with the Adam optimizer. The network was trained with a learning rate of 0.001, a decay rate of 0.000001, and mini-batch sizes of 256 over 100 epochs, without using batch normalization or data augmentation. The ReLU activation function is applied throughout the network, except in the output layer, as it efficiently captures non-linear relationships and aids in learning complex data representations. The output layer employs softmax activation for multi-class classification, ensuring the predicted probabilities for all classes sum to 1, which allows for accurate predictions. 10.1371/journal.pone.0313473.t001Table 1Layer-wise parameter details for the proposed model architecture with 25 × 25 window size based on IP dataset when Q = 10.Layer TypeOutput ShapeNumber of Parameterinput (InputLayer)25, 25, 10, 10conv3d-1 (Conv3D)23, 23, 6, 8368conv3d-2 (Conv3D)21, 21, 4, 163472conv3d-3 (Conv3D)19, 19, 2, 3213856reshape (Reshape)19, 19, 640conv2d (Conv2D)17, 17, 6436928flatten (Flatten)184960dense-1 (Dense)2564735232dropout-1 (Dropout)2560dense-2 (Dense)12832896dropout-2 (Dropout)1280output (Output)162064Total trainable parameters4824816 Layer-wise parameter details for the proposed model architecture with 25 × 25 window size based on IP dataset when Q = 10. 4 Result and discussion 4.1 Experiment design and parameter setup We divided the labeled samples into training and test subsets using different ratios for each dataset. The ratios used were 5%-95%, 10%-90%, 15%-85%, 20%-80% and 25%-75%. For example, in the case of a 25%-75% ratio, 25% of the data were randomly assigned to the training group, and the remaining 75% was assigned to the testing group. During the CNN training process, 90% of the training samples were used to determine the weights and biases of each neuron, and the remaining 10% were utilized to determine overfitting and inform network design choices. In our proposed model, we first apply Seg-PCA-mRMR for spectral feature reduction by segmenting the HSI datasets into three segments based on average correlation analysis in the segmented subgroups (Table 2). After Seg-PCA, we scale the data using the min-max scaler from the Python sklearn preprocessing library. 10.1371/journal.pone.0313473.t002Table 2Details of datasets segmentation for Seg-PCA.L No.ParameterIPSAPU1Range of bands1-321-361-36Number of bands3236362Range of bands33-7237-10637-72Number of bands4070363Range of bands73-200107-17673-103Number of bands1287031L No. denote the segment number. Details of datasets segmentation for Seg-PCA. L No. denote the segment number. We extracted 3D patches from the input volume with a spatial dimension of 25 × 25 and Q spectral features to ensure a fair comparison, where Q represents the number of input spectral features. In our experiments, we applied Seg-PCA-mRMR for spectral feature reduction and selected the top-ranked 10 features based on mRMR. The order of the selected features for different datasets is shown in Table 3. For the model training, we set the learning rate to 0.001, the decay rate to 0.000001, the batch size to 256, the window size to 25 × 25 and epochs to 100, based on the classification performance. The model were trained with Adam optimizer and categorical cross-entropy loss function. 10.1371/journal.pone.0313473.t003Table 3Order of selected features by mRMR for different dataset.DatasetOrder of selected featuresIP11, 12, 13, 15, 14, 1, 3, 2, 8, 6SA1, 4, 2, 3, 5, 10, 13, 11, 12, 8PU9, 8, 7, 6, 10, 14, 1, 4, 11, 13 Order of selected features by mRMR for different dataset. To assess classification performance, we employed four standard quantitative metrics: Overall Accuracy (OA), Average Accuracy (AA), Kappa Coefficient, and F1 score. OA represents the proportion of correctly classified samples out of the total test samples, while AA calculates the mean accuracy across all classes. The Kappa Coefficient measures agreement between predicted and actual classifications, and the F1 score, as the harmonic mean of precision and recall, provides a balanced indication of a model’s reliability by giving equal weight to both metrics. Additionally, we monitored the training loss and training accuracy during the training process. Finally, we produced classification maps for further visualization of the results and to validate the effectiveness of the proposed method. 4.2 Classification results and analysis The classification results were evaluated using three metrics: OA, AA, and Cohen’s Kappa, as described in the experimental design. The accuracies were measured in percentages for both the proposed method and state-of-the-art methods, including SVM on original data, Segmented PCA + SVM, 2D-CNN, Fast 3D-CNN, SpectralNet, HybridSN, Hybrid-2DNet, TP-Net, S2PNet, and DS2FE. These state-of-the-art experiments are described as follows: 2D-CNN [25]: Applies randomized PCA for dimensionality reduction. It consists of two convolutional layers with filter counts of 30 and 90, respectively, followed by two fully connected layers with dropout rates of 25% and 50%, and an output layer. 2D-CNN [25]: Applies randomized PCA for dimensionality reduction. It consists of two convolutional layers with filter counts of 30 and 90, respectively, followed by two fully connected layers with dropout rates of 25% and 50%, and an output layer. Fast 3D-CNN [41]: Utilizes incremental PCA for dimensionality reduction. The network comprises four convolutional layers with increasing filter counts and three fully connected layers. Fast 3D-CNN [41]: Utilizes incremental PCA for dimensionality reduction. The network comprises four convolutional layers with increasing filter counts and three fully connected layers. SpectralNet [39]: Applies factor analysis for dimensionality reduction and then executes a three-level Wavelet CNN decomposition. SpectralNet [39]: Applies factor analysis for dimensionality reduction and then executes a three-level Wavelet CNN decomposition. HybridSN [38]: Employs PCA for dimensionality reduction. It consists of three 3D convolution layers, one 2D convolution layer, and three fully connected layers. HybridSN [38]: Employs PCA for dimensionality reduction. It consists of three 3D convolution layers, one 2D convolution layer, and three fully connected layers. Hybrid-2DNet [43]: Utilizes factor analysis and mRMR-based feature selection followed by 2D-wavelet CNN in a four-level decomposition for reducing spectral and spatial dimensionalities. Additionally, batch normalization is applied. Hybrid-2DNet [43]: Utilizes factor analysis and mRMR-based feature selection followed by 2D-wavelet CNN in a four-level decomposition for reducing spectral and spatial dimensionalities. Additionally, batch normalization is applied. TP-Net [44]: Proposes a triple-path CNN with an attention mechanism to capture joint features. TP-Net [44]: Proposes a triple-path CNN with an attention mechanism to capture joint features. S2PNet [46]: An interactive learning approach used a multi-stage spectral purification module to reduce noise and spectral heterogeneity, while a global-local mutual guide module improved spatial feature discrimination. S2PNet [46]: An interactive learning approach used a multi-stage spectral purification module to reduce noise and spectral heterogeneity, while a global-local mutual guide module improved spatial feature discrimination. DS2FE [45]: Employs a vision-based transformer module that includes a multiscale feature extractor to capture joint spectral-spatial low-level and shallow features. For high-level semantic feature extraction, it uses a regional attention mechanism combined with a spatially gated module to enhance feature discrimination. DS2FE [45]: Employs a vision-based transformer module that includes a multiscale feature extractor to capture joint spectral-spatial low-level and shallow features. For high-level semantic feature extraction, it uses a regional attention mechanism combined with a spatially gated module to enhance feature discrimination. Table 4 displays the classification outcomes for ten spectral features across five different training-testing separations. The OA of the best-performing methods under various conditions is denoted in bold. The results indicate that our proposed method consistently outperforms other approaches in most scenarios. 10.1371/journal.pone.0313473.t004Table 4Classification results for the proposed and state-of-the-art methods.MethodTraining SamplesIPSAPUOAAAKappaF1OAAAKappaF1OAAAKappaF1Original data +SVM [9]5%45.1725.2533.4538.5469.9865.2266.1068.2662.3438.1745.1054.3510%46.1126.1734.7239.6170.9367.1367.2269.2462.4938.4345.2755.1215%46.4326.3435.2140.0871.4768.0767.8169.8662.6338.4645.5255.3120%46.4426.8935.4140.2771.7468.4668.1370.1662.6238.5345.5655.3525%46.8127.3035.9540.7171.9368.6668.3470.3762.8039.1645.8155.59Seg-PCA+SVM [15, 48, 50]5%64.1253.0658.5161.3291.5894.9690.6991.1086.2075.2981.1883.6610%70.5759.8065.9268.2392.1695.5691.2691.6788.5378.5284.4985.6315%73.6463.5269.5372.1592.5595.5991.6892.0989.8181.4886.2787.4520%75.5567.3671.7874.3692.7196.1391.8792.2190.6183.8287.3988.5625%76.8069.5273.2675.8792.9596.3192.1292.4691.4485.4988.5290.692D-CNN [25]5%64.7765.5560.1462.4697.0698.8396.7396.9095.0994.1993.5294.3110%78.5383.8875.7377.1397.8499.2097.6097.7297.7497.2797.0297.3815%85.0891.6183.1184.1098.2699.3698.0698.1698.9798.5998.6398.8020%87.0493.6085.3586.2098.3799.4498.1998.2899.3899.0799.1899.2825%89.3494.3387.9288.6399.4099.7799.3399.3799.6099.3799.4899.54Fast 3D-CNN [41]5%55.1836.3146.9351.0693.3296.2492.5692.9489.7982.3186.3488.0710%70.5850.9665.9868.2896.7598.3796.3896.5795.1892.3293.6094.3915%62.0245.4055.6858.8599.5499.8299.4899.5197.5796.1096.7797.1720%80.8366.1877.9579.3999.9699.9899.9699.9699.1598.5798.8899.0225%85.4979.0383.4284.4699.8399.7799.8199.8299.0198.1898.6998.85SpectralNet [39]5%31.2016.7422.4626.8399.4899.7599.4399.4697.9196.3497.2497.5910%94.0375.9693.2093.6299.7399.8399.7099.7298.9998.1698.6798.8215%97.6586.1097.3297.4999.9799.97 99.97 99.9599.6299.4099.4999.5620%98.5198.3498.3098.4299.9899.9899.9799.9899.7099.5299.6099.6725%94.5687.3387.5391.1299.9599.9699.9499.9599.7799.82 99.71 99.74HybridSN [38]5%91.6085.19 91.41 91.5199.7599.7499.6999.7299.51 99.07 99.3599.4310%97.59 97.07 97.2597.4299.9299.9199.9299.9299.4898.97 99.48 99.4815%98.8899.2298.7398.8199.9599.8699.9599.9599.6199.5099.4999.5520%98.9498.2798.7998.8799.9899.9899.9899.9899.7599.7099.7399.7425%99.4999.3199.4299.4699.9799.9599.9799.9798.9998.9998.9998.99Hybrid-2DNet [43]5%35.4119.0123.3129.3799.7699.8699.7599.7696.3395.5195.1195.7210%94.0476.1093.2193.6399.9499.9499.9499.9498.9197.9198.4398.6715%97.6797.6597.2997.48 99.97 99.97 99.97 99.9799.6499.4199.5199.5820%98.5888.0798.3898.4999.9999.9899.9899.9999.7299.4799.5799.6525%97.7998.0198.4398.1199.9799.9699.9799.9799.7899.6299.6899.73TP-Net [44]5%37.1123.1327.4232.2999.6199.6599.6099.61 99.53 99.05 99.38 99.4610%86.5179.5881.3783.9599.9399.9399.9399.93 99.55 98.9799.4599.5015%93.5689.7791.8492.7199.9599.9199.9599.9599.6799.6599.6199.6420%98.7198.3698.4498.5799.9899.9699.9799.9899.8599.3499.2399.5425%97.1693.9496.1996.6699.9699.9699.9599.9699.7999.8299.7099.75S2PNet [46]5%92.38 89.07 90.4891.6399.7899.7499.5799.6998.4598.3898.1798.3110%97.7794.7195.8496.2499.8699.8299.6799.7698.7198.6998.4998.6415%98.5996.0296.4597.1699.8899.8499.6699.7999.3299.2898.7198.8520%98.9298.9197.6297.9899.9599.9699.8299.8599.8599.8499.6899.7025%99.4899.4799.0799.2199.9299.9299.8099.8399.2199.2299.0699.17DS2FE [45]5% 92.39 89.0290.41 91.72 99.8199.8099.6399.7298.4098.3398.1598.2810%97.8594.7895.9596.3699.8999.8899.7199.8498.8298.8098.6198.7315%98.6396.1496.5697.2299.9499.9399.8199.8799.4399.3998.8598.9420% 98.97 98.9797.6898.0399.9899.9799.9099.9299.9199.8599.6799.7125%99.4599.4699.0899.2099.9499.9499.8899.9099.3799.3198.9799.15Proposed Method5%92.3688.7690.9791.69 99.78 99.85 99.76 99.7799.1998.4198.2899.7410% 97.85 96.52 97.76 97.82 99.96 99.96 99.96 99.9699.53 98.98 99.3599.4415% 98.96 98.22 98.86 98.92 99.97 99.98 99.9699.97 99.76 99.71 99.68 99.7220% 99.06 98.16 98.93 99.02 100 99.99 100 100 99.94 99.93 99.92 99.9325% 99.61 99.65 99.55 99.59 99.98 99.99 99.98 99.98 99.85 99.89 99.1499.50 Classification results for the proposed and state-of-the-art methods. Among the three datasets, the PU dataset exhibits the lowest number of categories, comprising only nine classes, thereby making it comparatively easier to classify than the other datasets. In contrast, the SA dataset features a larger spatial size and the maximum number of spectral bands, providing more discriminative information for classification. Consequently, the listed methods generally demonstrate higher classification performance on the SA dataset. The IP dataset, although having a relatively small spatial size, encompasses 16 categories, presenting a challenge for accurate classification and resulting in lower accuracy compared to the other datasets. The traditional SVM model applied directly to the original data yields inadequate classification performance. However, upon preprocessing the data using Seg-PCA and integrating it with SVM, there is a notable enhancement in classification accuracy. The 2D-CNN and Fast 3D-CNN methods demonstrate lower classification performances, possibly attributable to their limited utilization of spectral information. Conversely, SpectralNet achieves moderate classification performances by incorporating both spectral and spatial features. HybridSN, which combines several 3D CNNs with a 2D CNN, is a simple and effective approach, yielding discriminative features. The Hybrid-2DNet method demonstrates a notable performance improvement over SpectralNet, highlighting the crucial role of feature selection. However, it does not consistently surpass HybridSN across all scenarios. In contrast, TP-Net consistently outperforms both SpectralNet and HybridSN in most cases, indicating its superior performance. S2PNet performs comparably to TP-Net, while DS2FE surpasses TP-Net in most instances due to its ability to handle joint spectral-spatial low-level and shallow features, along with high-level semantic feature extraction using a regional attention mechanism. Indeed, the proposed approach capitalizes on the combined potential of spatial and spectral features within HSI data. Through the extraction of distinct image features from segmented subgroups’ PCs and their subsequent fusion, the network acquires a rich array of representative and differentiating features. This comprehensive feature set contributes to an enhanced classification performance overall. By leveraging both spatial and spectral information, and harnessing the benefits of feature fusion, the proposed method achieves an advanced level of classification accuracy. Additionally, the class-wise classification accuracies in percentages are shown in Tables 5–7 with top 10 selected features and 20% training samples for all three experimented datasets. The results in Tables 4–7 demonstrate that the proposed methods offer the best OA of 99.61%, 100%, and 99.94% for IP, SA, and PU datasets, respectively. In comparison, the DS2FE, S2PNet, TP-Net, Hybrid-2DNet, HybridSN, and SpectralNet models achieve their optimal results with OAs of 98.97%, 98.92%, 98.71%, 98.58%, 99.49% and 98.51% for IP, 99.98%, 99.95%, 99.98%, 99.99%, 99.98% and 99.98% for SA, and 99.91%, 99.85%, 99.85%, 99.78%, 99.75% and 99.70% for PU datasets under the same spectral features conditions. These results indicate that the proposed method significantly improves the classification accuracy compared to the state-of-the-art models under the same spectral features and training sample conditions. 10.1371/journal.pone.0313473.t005Table 5Class-wise classification accuracies (%) of IP with 10 spectral features and 20% training samples.ClassesOriginal data+SVMSeg-PCA+SVM2D-CNNFast 3D-CNNSpectralNetHybridSNHybrid-2DNETTP-NetS2PNetDS2FEProposed MethodAlfalfa0.0042.5692.5099.7499.8392.5080.43 100 100 100 100 Corn-notill43.2565.3785.7165.80 99.73 99.3698.5096.7796.7797.5098.24Corn-mintill14.5870.6283.4479.3897.2198.9598.3697.9298.3698.95 99.25 Corn0.0057.8493.7585.7199.48 100 97.88 100 97.8898.5098.96Grass-pasture8.1288.7995.2184.99 99.85 99.7498.71 100 98.7199.7499.23Grass-trees47.3787.2087.8179.6799.83 100 99.6694.1999.81 100 98.81Grass-pasture-mowed0.0071.0095.65 100 100 100 100 100 100 100 100 Hay-windrowed31.1290.8790.0990.2899.7499.7499.2299.7199.7499.74 99.78 Oats0.0031.0076.19 100 0.00 100 100 100 100 100 100 Soybean-notill49.5069.5079.3177.5198.7397.7499.3599.5999.7599.59 100 Soybean-mintill39.3768.7589.5381.3897.6198.2598.49 99.20 98.4998.2598.84Soybean-clean0.0067.8780.2869.5298.5298.73 99.57 94.1098.5294.1098.92Wheat0.0091.1295.3593.37 100 100 100 100 100 100 100 Woods72.2594.3397.8298.5699.6199.7097.9797.0799.7098.56 100 Buildings-GTD0.0070.5768.4785.0498.72 100 99.3697.3997.3998.72 100 Stone-Steel-Towers 100 98.0891.3688.8997.1088.7593.59 100 93.5997.1086.75 OA 46.4475.5587.0480.8398.5198.9498.5898.7198.9298.97 99.06 AA 26.8967.3693.6066.1898.3498.2788.0798.3698.91 98.97 98.16 Kappa 35.4171.7885.3577.9598.3098.7998.3898.4497.6297.68 98.93 Class-wise classification accuracies (%) of IP with 10 spectral features and 20% training samples. 10.1371/journal.pone.0313473.t006Table 6Class-wise classification accuracies (%) of SA with 10 spectral features and 20% training samples.ClassesOriginal data+SVMSegPCA+SVM2D-CNNFast 3D-CNNSpectralNetHybridSNHybrid-2DNETTP-NetS2PNetDS2FEProposed MethodBrocoli-green-weeds-191.71 100 100 100 100 100 100 100 100 100 100 Brocoli-green-weeds-290.28 100 99.90 100 100 99.97 100 100 99.75 100 99.98Fallow53.2397.07 100 100 100 100 100 100 100 100 100 Fallow-rough-plow61.5598.7499.6499.9198.9599.07 100 99.9199.0799.91 100 Fallow-smooth60.2199.1499.7899.84 100 100 100 99.81 100 100 100 Stubble96.68 100 99.97 100 99.97 100 100 99.9499.94 100 100 Celery90.16 100 99.9399.87 100 100 100 100 100 100 100 Grapes-untrained57.5678.5298.8699.9198.7499.7999.78 100 100 99.91 100 Soil-vinyard-develop82.3499.58 100 100 100 100 100 100 100 100 100 Corn-senesced-GW60.1096.85 100 99.96 100 100 100 100 100 100 100 Lettuce-romaine-4wk0.0098.6399.7899.8699.9499.98 100 100 99.94 100 100 Lettuce-romaine-5wk83.0398.17 100 100 100 100 100 100 100 100 100 Lettuce-romaine-6wk68.0898.67 100 100 100 100 100 99.8699.67 100 100 Lettuce-romaine-7wk66.4298.14 100 100 100 99.83 100 100 100 100 100 Vinyard-untrained54.5382.5090.5099.9199.16 100 99.6799.97 100 100 100 Vinyard-vertical-trellis82.57 100 100 100 100 100 100 100 100 100 100 OA 71.7492.7198.3799.9699.9899.9899.9999.9899.9599.98 100 AA 68.4696.1399.4499.9899.9899.9899.9899.9699.9699.97 99.99 Kappa 68.1391.8798.1999.9699.9799.9899.9899.9799.8299.90 100 Class-wise classification accuracies (%) of SA with 10 spectral features and 20% training samples. 10.1371/journal.pone.0313473.t007Table 7Class-wise classification accuracies (%) of PU with 10 spectral features and 20% training samples.ClassesOriginal data+SVMSegPCA+SVM2D-CNNFast 3D-CNNSpectralNetHybridSNHybrid-2DNETTP-NetS2PNetDS2FEProposed MethodAsphalt53.9684.7699.7099.0599.9299.7499.4299.9699.9699.96 99.98 Meadows68.4792.5799.9299.6799.7599.5599.9699.7999.9699.96 99.99 Gravel39.5483.4597.5498.9898.78 100 99.1699.3599.1699.35 100 Trees0.0096.1098.6598.6399.5899.5998.9498.8298.8298.94 100 Painted MS94.6710010099.72 100 100 100 100 100 100 100 Bare Soil40.7892.8499.8099.73 100 99.9899.88 100 100 100 100 Bitumen0.0093.0798.7098.0299.1899.6199.81 100 100 100 99.81Self-BB47.1880.5897.4397.1098.8298.9099.3298.8198.9098.82 99.49 Shadows0.00 100 99.4797.75 100 99.7499.7398.7899.73 100 100 OA 62.6290.6199.3899.1599.7099.7599.7299.8599.8599.91 99.94 AA 38.5383.8299.0798.5799.5299.7099.4799.3499.8499.85 99.93 Kappa 45.5687.3999.1898.8899.6099.7399.5799.2399.6899.67 99.92 Class-wise classification accuracies (%) of PU with 10 spectral features and 20% training samples. Fig 2 shows the overall accuracy performances of state-of-the-art methods with the proposed method as the number of training samples increases for different datasets. It is evident from the results in the tables and Fig 2 that the proposed methods consistently exhibit significantly better performance compared to SVM on original data and Seg-PCA + SVM methods for all three datasets. Similarly, our proposed methods clearly outperform the 2D-CNN, Fast 3D-CNN, and SpectralNet methods across all three datasets. Furthermore, in comparison to HybridSN, Hybrid-2DNet, DS2FE, S2PNet, and TP-Net the proposed methods demonstrate notable improvements. Overall, the results consistently indicate that the proposed methods outperform or surpass the performance of several benchmark methods across all three datasets. Moreover, as the number of training samples increases, all models show improved performance, except for when using 25% training samples, which sometimes exhibits lower accuracy compared to the performance with 20% training samples. This observation suggests that a larger training dataset allows the models to learn more representative and discriminative features, thus enhancing classification accuracy. Overall accuracy comparison of state-of-the-art methods with the proposed method as the number of training samples increases for (a) IP, (b) SA, and (c) PU datasets. Fig 3 illustrates the convergence of accuracy and loss for the proposed and state-of-the-art models on the IP dataset, using 20% of training samples over 100 epochs. Our method achieves convergence around the 18th epoch, demonstrating a relatively fast convergence rate compared to other models, except for S2PNet and DS2FE, which converge faster. This highlights the efficiency and effectiveness of our approach in capturing data patterns, resulting in improved classification performance. IP data, (a)-(i) Accuracy, and Loss convergence versus the number of epochs over top 10 selected features and 20% training samples, (a) 2D CNN, (b) Fast 3D-CNN, (c) SpectralNet, (d) HybridSN, (e) Hybrid-2DNet, (f) TP-Net, (g) S2PNet, (h) DS2FE and (i) Proposed Method. 4.3 Classification maps In addition to the quantitative evaluation, a visual assessment of classification maps is performed to validate the effectiveness of the proposed method. Figs 4–6 showcase the classification maps obtained by different methods for various HSIs. These maps are generated using 20% of the training samples for both the proposed and the other studied methods. Upon visual inspection, it becomes evident that the proposed method consistently outperforms the other approaches across all datasets. The classification maps produced by the proposed method exhibit higher clarity and fewer misclassified pixels, demonstrating its superior performance in capturing the complex spatial and spectral information present in hyperspectral imagery. The visually compelling results further validate the efficacy of our proposed method and reinforce its potential for accurate HSI classification tasks. IP HSI (a) ground truth image, (b)-(h) classification maps for different SVM and CNN models with top 10 selected features and 20% training sample. (b) Original data + SVM, (c) Seg-PCA + SVM, (d) 2D CNN, (e) Fast 3D-CNN, (f) SpectralNet, (g) HybridSN, (h) Hybrid-2DNet, (i)TP-Net, (j) S2PNet, (k) DS2FE, and (l) Proposed Method. SA HSI (a) ground truth image, (b)-(h) classification maps for different SVM and CNN models with top 10 selected features and 20% training sample. (b) Original data + SVM, (c) Seg-PCA + SVM, (d) 2D CNN, (e) Fast 3D-CNN, (f) SpectralNet, (g) HybridSN, (h) Hybrid-2DNet, (i)TP-Net, (j) S2PNet, (k) DS2FE, and (l) Proposed Method. PU HSI (a) ground truth image, (b)-(h) classification maps for different SVM and CNN models with top 10 selected features and 20% training sample. (b) Original data + SVM, (c) Seg-PCA + SVM, (d) 2D CNN, (e) Fast 3D-CNN, (f) SpectralNet, (g) HybridSN, (h) Hybrid-2DNet, (i)TP-Net, (j) S2PNet, (k) DS2FE, and (l) Proposed Method. 4.4 Ablation analysis A comprehensive ablation analysis was conducted using the novel integrated approach (Seg-PCA + mRMR + 3D-2D CNN) across five distinct subgroups. The first experiment evaluated dataset segmentation before PCA, comparing the performance of conventional PCA (PCA + 3D-2D CNN) with Seg-PCA (Seg-PCA + 3D-2D CNN), without mRMR-based feature selection. The second experiment assessed the impact of mRMR-based feature selection on both conventional PCA and Seg-PCA. Compared to group 1, the efficacy of mRMR-based feature selection is clearly demonstrated, showing its impact on improving the model’s performance. The third experiment compared the deep 3D-2D model’s performance with traditional machine learning (SVM). In the fourth, the individual performance of 2D and 3D CNNs was evaluated, and the fifth experiment examined the influence of the number of 3D CNN layers in the 3D-2D CNN model. The ablation analysis results, detailed in Table 8, demonstrate that both Seg-PCA and mRMR feature selection significantly improved classification accuracy compared to classical PCA and the absence of feature selection. The findings also determine the optimal number of 3D CNN layers and the advantage of deep 3D-2D CNN over traditional machine learning. Notably, combining Seg-PCA and mRMR produced a synergistic effect, greatly enhancing overall classification performance, underscoring the importance of these techniques both individually and together in the integrated framework. 10.1371/journal.pone.0313473.t008Table 8Classification results for ablation analysis with 10 spectral features.Ablation GroupAblation MethodTraining SamplesIPSAPUOAAAKappaOAAAKappaOAAAKappaGroup 1PCA + 3D-2D CNN (Three 3D CNN)10%97.5496.3097.2299.9199.9099.9199.3198.8499.0320%98.8297.9698.7999.9699.9699.9599.8199.7899.80Seg-PCA + 3D-2D CNN (Three 3D CNN)10%97.6096.3797.2799.9299.9299.9299.3998.9099.1520%98.9598.1298.9599.9899.9799.9899.8899.8699.88Group 2PCA + mRMR + 3D-2D CNN (Three 3D CNN)10%97.6196.3397.2699.9399.9299.9399.3898.9299.1620%98.9698.1198.9599.9899.9899.9899.8999.8599.88Seg-PCA + mRMR + 3D-2D CNN (Three 3D CNN)(Proposed)10%97.8596.5297.7699.9699.9699.9699.5398.9899.3520%99.0698.1698.9310099.9910099.9499.9399.92Group 3Seg-PCA + SVM10%46.1126.1734.7270.9367.1367.2262.4938.4345.2720%46.4426.8935.4171.7468.4668.1362.6238.5345.56Seg-PCA + 3D-2D CNN (Three 3D CNN)10%97.6096.3797.2799.9299.9299.9299.3998.9099.1520%98.9598.1298.9599.9899.9799.9899.8899.8699.88Group 4Seg-PCA + mRMR + 2D CNN (No 3D CNN)10%92.9486.1791.9399.8299.8399.8099.4399.3799.3820%97.6897.6197.3599.8399.8499.8299.7399.7099.71Seg-PCA + mRMR + 3D CNN (No 2D CNN, Three 3D CNN)10%95.7188.8195.1099.7899.7899.7799.4599.3699.3920%98.6198.4698.4199.7999.8499.7799.7899.6899.72Group 5Seg-PCA + mRMR + 3D-2D CNN (One 3D CNN)10%96.2589.5495.7299.8299.8199.7799.4198.8599.0420%98.4397.4898.2199.9699.9699.9599.8999.8499.87Seg-PCA + mRMR + 3D-2D CNN (Two 3D CNN)10%97.1590.2996.1899.8899.8799.8399.4899.0599.2820%98.9598.0198.8099.9799.9799.9699.9299.9199.90Seg-PCA + mRMR + 3D-2D CNN (Three 3D CNN)(Proposed)10%97.8596.5297.7699.9699.9699.9699.5398.9899.3520%99.0698.1698.9310099.9910099.9499.9399.92 Classification results for ablation analysis with 10 spectral features. 4.5 Computation time analysis All experiments were performed on Google Colaboratory using a setup consisting of 2 x vCPUs and a maximum of 25 (mostly 12) GB RAM. In order to ensure a fair and accurate comparison, we applied the same spatial dimension and spectral features extraction technique to obtain 3D patches from various input data. Throughout the experiments, we measured three types of computation times for both the different studies and our proposed method. These computation times are denoted as Pp for preprocessing, Tr for training, and Ts for testing. The results of these time measurements are presented in Table 9, with the time values given in seconds, minutes, and seconds, respectively. It is worth noting that our proposed model demonstrated higher computational efficiency compared to the fast 3D-CNN, SpectralNet, HybridSN, Hybrid-2DNet, TP-Net, DS2FE, and S2PNet, methods when considering the preprocessing, training, and testing times. However, fusion methods DS2FE, S2PNet, Hybrid-2DNet and TP-Net show better classification performances than SpectralNet, and HybridSN but face more computational complexity. Only 2D-CNN offers better computational efficiency than the proposed methods but gives lower classification accuracy. 10.1371/journal.pone.0313473.t009Table 9Different stages computation time, prepossessing (Pp) time in seconds (s), training (Tr) time in minutes (m), testing (Ts) in seconds (s), for the proposed and state-of-the-art methods.MethodTraining SamplesIPSAPUPp(s)Tr(m)Ts(s)Pp(s)Tr(m)Ts(s)Pp(s)Tr(m)Ts(s)2D-CNN [25]10%8.13.85.423.55.98.225.34.66.820%8.17.34.823.510.27.625.38.95.7Fast 3D-CNN [41]10%6.231.558.215.7216.2356.821.6108.5368.620%6.245.451.715.7428.3316.421.6211.6327.4SpectralNet [39]10%25.823.119.675.1121.8103.354.673.161.820%25.843.817.475.1226.191.854.6135.456.7HybridSN [38]10%6.816.232.728.2103.8201.431.151.4103.720%6.830.129.128.2194.7179.031.196.292.2Hybrid-2DNET [43]10%47.822.520.492.3118.1105.659.778.262.320%47.840.716.992.3223.592.459.7138.655.6TP-NET [44]10%6.831.561.629.4197.9187.432.4101.2188.420%6.857.654.729.4357.8155.632.4182.4174.7S2PNet [46]10%67.429.647.1127.5155.1143.892.3108.292.320%67.455.835.2127.5307.5116.792.3198.675.6DS2FE [45]10%38.628.545.493.1142.7137.874.188.7176.820%38.655.433.393.1281.7113.574.1171.4139.9Proposed Method10%7.6 9.8 19.130.2 65.3 135.634.2 37.5 36.920%7.6 13.4 17.030.2 122.5 120.234.4 68.3 32.8 Different stages computation time, prepossessing (Pp) time in seconds (s), training (Tr) time in minutes (m), testing (Ts) in seconds (s), for the proposed and state-of-the-art methods. 4.6 Discussion The experimental results highlight the challenges of limited training samples and high dimensionality in HSI classification, common in real-world applications where acquiring labeled data is difficult and resource-intensive. Despite these challenges, the proposed method effectively overcomes them by combining segmented principal component analysis (Seg-PCA) for feature extraction with 3D-2D CNNs for joint spectral-spatial feature learning. This hybrid approach addresses both the scalability and robustness issues, showing improved performance compared to traditional CNN-based methods. In addressing the limitations of traditional approaches, the integration of Seg-PCA with mRMR-based feature selection enhances the model’s ability to handle high-dimensional HSI data by reducing redundant information and focusing on key features. The incorporation of both 3D and 2D CNN layers balances computational efficiency with feature extraction depth, allowing the model to adapt effectively to variations in dataset characteristics. As a result, the proposed method achieves better classification accuracy and generalization across different datasets, while also mitigating the computational complexity of deep 3D CNNs. This research demonstrating the model’s architecture remains efficient and adaptable as data complexity increases. Additionally, the robustness of the proposed method is evident in its generalizability across three widely used HSI datasets, outperforming state-of-the-art methods such as SVM, 2D-CNN, Fast 3D-CNN, SpectralNet, HybridSN, Hybrid-2DNet, TP-Net, 2PNet, and DS2FE. The results in tables and figures as well as classification maps indicate that the combination of Seg-PCA and 3D-2D CNNs significantly contributes to overcoming the limitations posed by traditional CNNs, particularly in handling high-dimensional data with limited training samples. In respect of computation time the proposed model offer relatively higher computational efficiency compared to the state of the art. While our proposed method demonstrates significant improvements in HSI classification, it does have some limitations that offer avenues for future research. Despite the integration of both 3D and 2D CNN layers to balance computational efficiency, the overall computational cost remains relatively high, particularly when applied to large-scale datasets. The complexity could limit the scalability of the model for such real-time applications where large-scale datasets involve. Although the model generalizes well across the evaluated datasets, further validation on additional, more diverse datasets is necessary to confirm its adaptability to varying environments and spectral conditions. However, expanding the application of the proposed model to diverse fields such as precision agriculture, environmental monitoring, and urban mapping could validate its robustness and scalability across various real-world scenarios. Additionally, the model could be tailored for specific domain requirements, further improving its performance in specialized tasks. 5 Conclusion and future work Land cover object classification is a vital task in remote sensing and has significant implications for fields such as environmental monitoring, urban planning, and agriculture. Despite the advancements in HSI classification, achieving high accuracy remains challenging due to issues like high dimensionality and the limited availability of training samples. Traditional methods often focus on spectral information, which may overlook critical spatial features. To address these limitations, we introduced a novel HSI classification approach that enhances land cover object classification by integrating Seg-PCA with mRMR feature selection and a hybrid 3D-2D CNN architecture. Our proposed method effectively combines Seg-PCA for localized spectral feature extraction with mRMR for feature selection, followed by a 3D-2D CNN framework that captures both spectral and spatial features. This approach not only addresses the challenges of high dimensionality and limited training data but also significantly improves classification performance for land cover object classification tasks. The extensive experiments conducted on three benchmark HSI datasets demonstrate that our method outperforms existing state-of-the-art techniques in terms of OA, AA, and Cohen’s Kappa statistic, proving its effectiveness and robustness. Looking ahead, several avenues for future research could further enhance HSI classification techniques. One potential direction is the exploration of advanced feature fusion strategies that integrate additional data modalities, such as LiDAR or multispectral imagery, to improve classification accuracy and robustness. Another promising area is the development of more efficient network architectures and training algorithms that can handle even larger datasets and more complex classification tasks, potentially leveraging transfer learning or few-shot learning techniques to make better use of limited training data. Finally, our approach sets a new benchmark for land cover object classification in hyperspectral imagery and offers a foundation for future innovations in the field of hyperspectral remote sensing.