Systematic hybrid feature selection using optimal filter classifier pairing and parallel evaluation Authors: Deepak Kumar Rakesh, Shashi Kant Tiwari, Nallakaruppan Kailasanathan, Abhinav Pathak, Aniket S Nagane Published: 2026 ABSTRACT AbstractThere is extensive literature on feature selection algorithms, which are typically categorized into filter, wrapper, and embedded methods. Each category has its own merits and limitations. As a result, researchers have increasingly focused on developing hybrid methods that combine the strengths of these approaches. However, the selection of algorithms in hybrid methods is often made without adequate justification, which limits their applicability and overall effectiveness. We propose a Systematic Hybrid Feature Selection framework with Optimal Filter-Classifier Pairing and Parallel Evaluation (SHFS-OFCP-PE), designed to provide a filter-agnostic and classifier-independent hybrid feature selection strategy. The hybridization process emphasizes selecting features with minimal redundancy and maximum relevance to class labels. It also accounts for improvements in model performance when new features are added. Additionally, we present a parallel framework based on a master-slave architecture to enhance scalability. We conduct extensive experiments on twelve hybrid combinations and compare the results with those of traditional and state-of-the-art methods. The findings confirm the superiority of the proposed hybrid methods across multiple performance metrics and are further validated through a post hoc Analysis of Variance (ANOVA) test.Keywords: Hybrid feature selection, Filter, Wrapper, ClassificationSubject terms: Computational biology and bioinformatics, Engineering, Mathematics and computing Introduction Feature selection (FS) algorithms are the backbone of many real-world applications such as image processing1, big data2, text mining3, intrusion detection4, and other systems5. FS improves data quality by selecting a relevant subset of features, thereby reducing computational time and enhancing the performance and interpretability of machine learning (ML) models6. FS algorithms are broadly categorized into filter, wrapper, and embedded methods. Filter methods are computationally efficient and scalable. However, they often ignore feature dependencies and may not align well with specific learning algorithms. In contrast, wrapper methods consider feature interactions and typically yield higher accuracy, but they are computationally expensive and prone to overfitting. Embedded methods are more integrated. Thus, they also struggle with high-dimensional data due to complexity. Hence, filters are useful in high-dimensional settings because of their lower computational time7,8. It is worth mentioning that filters alone can be unstable as they rely on heuristic criteria. To tackle these limitations, hybrid approaches that combine filter and wrapper methods have received growing interest. These approaches aim to harness the complementary strengths of both paradigms, enabling more robust, scalable, and accurate FS methods across a wide range of applications. Many hybrid methods have been proposed9–12. The author in9 proposed two methods based on information gain (IG) with genetic algorithm (GA) and principal component analysis (PCA) separately. However, they work efficiently on different metrics but suffer from high time complexity, especially for IG-GA. Agarwal et al.10 proposed a hybrid method to select an optimal subset of features for reviews of movies and products. Though their approach can reduce the size of features, the performance of classification was not acceptable. Aghaeipoor and Javidi11 have proposed an mRMR based hybrid method for regression problems. However, most of the proposed methods are based on the particular filter method. Therefore, the demerits of the individual methods are inherited in their hybridization process. Tiwari and Chaturvedi13 proposed an information-theory (IT) hybrid method. This assertion is supported by Bommert et al.7, in which the authors experimented with 22 filter methods and found that none of them is clearly superior to the others. Therefore, a hybridized method developed on a particular filter can not be superior to all other hybrids, and hence the applicability of the method is restricted. Feature selection is a fundamental preprocessing technique in machine learning that aims to identify the most relevant and non-redundant features for predictive modeling. By reducing dimensionality, feature selection improves classification performance, enhances interpretability, mitigates overfitting, and decreases computational cost. The importance of feature selection has been extensively discussed in foundational works such as14, and15, as well as comprehensive surveys including16 and17. In parallel, hybrid physical-deep learning modeling and multichannel feature fusion frameworks demonstrate the effectiveness of integrating complementary information sources while preserving interpretability and computational efficiency18–25. Recent advances in large-scale optimization and intelligent feature selection highlight the growing importance of robust metaheuristic and hybrid frameworks26. For example, Chen et al.27 proposed a particle swarm optimization (PSO) framework with modified global and local search exemplars to enhance large-scale optimization performance, demonstrating improved convergence stability and solution diversity. Similarly, Tu et al.28 introduced a constrained multi-objective evolutionary algorithm based on a dynamic clustering strategy to effectively balance exploration and exploitation in complex search spaces. In applied intelligent systems, Lin and Lin29 utilized multilayer perceptron neural networks for engagement prediction in mechanical systems, illustrating the broader applicability of learning-based optimization techniques. These studies emphasize the increasing demand for scalable, stable, and adaptive hybrid feature selection mechanisms, particularly in high-dimensional classification problems. Despite these advancements, existing hybrid feature selection approaches still face several challenges, including premature convergence in metaheuristic search, computational inefficiency in high-dimensional settings, insufficient stability analysis, and limited cross-domain generalization. Many recent models primarily focus on improving classification accuracy while overlooking the robustness and consistency of the selected feature subsets. Furthermore, several hybrid methods remain tightly coupled with specific filters, classifiers, or optimization strategies, thereby restricting their general applicability. Therefore, there remains a need for a generalized and systematic hybrid feature selection framework that integrates statistical relevance evaluation, redundancy control, and adaptive optimization while ensuring scalability, robustness, and consistent performance across heterogeneous classification problems. Formally, the problem addressed in this work is to design a hybrid feature selection mechanism that (i) systematically determines optimal filter-classifier pairings, (ii) minimizes redundancy among selected features, (iii) maximizes relevance with respect to class labels, and (iv) maintains computational efficiency across diverse datasets. Contributions of the paper To address these challenges, we propose a Systematic Hybrid Feature Selection framework with Optimal Filter–Classifier Pairing and Parallel Evaluation (SHFS-OFCP-PE). The framework is designed to provide a filter-agnostic and classifier-independent hybrid feature selection strategy while ensuring statistical redundancy control and scalable evaluation. Our hybridization does not depend on any particular filter and classifier. It actually guides the selection of filter and classifier method to form the best hybrid combination rather than being dependent on any filter and classifier. This helps overcome the aforesaid limitations. The proposed framework applies the greedy based searching criteria in coordination with the classifier to select the optimal subset of features. The search criterion is based on finding the maximum relevance with the class labels and minimum redundancy with the already selected features and the performance gain in each iteration of the ML model after appending a new feature. The main contributions are summarized as follows: We propose a general hybrid feature selection framework that is not limited to any specific filter or classifier. Unlike existing methods that rely on fixed filter-classifier combinations, our approach systematically explores multiple pairings to identify the best-performing hybrid combination for a given dataset. We propose a general hybrid feature selection framework that is not limited to any specific filter or classifier. Unlike existing methods that rely on fixed filter-classifier combinations, our approach systematically explores multiple pairings to identify the best-performing hybrid combination for a given dataset. The framework introduces a new scoring function that combines feature relevance (using filter scores), redundancy (using feature correlation), and classification performance (through wrapper evaluation). This combined criterion ensures that the selected feature subset is both informative and compact. The framework introduces a new scoring function that combines feature relevance (using filter scores), redundancy (using feature correlation), and classification performance (through wrapper evaluation). This combined criterion ensures that the selected feature subset is both informative and compact. To address the high computational cost of evaluating many filter-classifier combinations, we implement a parallel computing architecture using a master-slave model. This reduces the time complexity from O(nmΓ F) in the serial setting to O(Γ F), making the approach efficient even for high-dimensional datasets. To address the high computational cost of evaluating many filter-classifier combinations, we implement a parallel computing architecture using a master-slave model. This reduces the time complexity from O(nmΓ F) in the serial setting to O(Γ F), making the approach efficient even for high-dimensional datasets. We perform extensive experiments on 12 hybrid filter-classifier configurations across diverse datasets (biomedical, image, and text) and validate filter diversity through a dedicated test. We evaluate the efficacy of the proposed framework against state-of-the-art (SOTA) hybrid feature selection methods using accuracy, AUC, and feature stability, with statistical validation performed using ANOVA and LSD post hoc tests. We also demonstrate computational efficiency through execution-time comparisons between the serial and parallel versions of our framework, as well as with SOTA methods. We perform extensive experiments on 12 hybrid filter-classifier configurations across diverse datasets (biomedical, image, and text) and validate filter diversity through a dedicated test. We evaluate the efficacy of the proposed framework against state-of-the-art (SOTA) hybrid feature selection methods using accuracy, AUC, and feature stability, with statistical validation performed using ANOVA and LSD post hoc tests. We also demonstrate computational efficiency through execution-time comparisons between the serial and parallel versions of our framework, as well as with SOTA methods. Objective of the study: The primary objective of this study is to develop a generalized hybrid feature selection framework that systematically determines optimal filter-classifier pairings while ensuring redundancy control and computational efficiency. The framework is designed to be filter-agnostic, integrating redundancy modeling with classifier-based validation, and incorporating parallel evaluation to enhance scalability and performance across heterogeneous datasets. It is intended for supervised single-label classification tasks involving both low- and high-dimensional data, particularly datasets with redundant or noisy features where predictive performance and efficiency are critical. By combining multiple filter methods with different classifiers, the framework adapts to diverse data characteristics and provides a generalized solution applicable across domains. Computational efficiency is achieved through filter-based ranking combined with a greedy forward selection strategy, which avoids repeated classifier retraining as required in wrapper methods. Furthermore, parallel evaluation reduces execution time, making the framework suitable for large-scale and high-dimensional scenarios. The paper is organized as follows. First, we introduce the related works in section “Research gap”. Then, section “Preliminaries” presents the proposed framework and its serial and parallel computing models. Next, in section “Methodology”, the experimental results are analyzed along with the diversity study of filters. Finally, section “Experimental results and discussion” concludes the paper and puts forward the future work. For the ease of readability, the notations used throughout the paper is summarized in Table 1 concerning a datasets D having instances N, features F, and class labels Y. Table 1Notations used in the article.NotationDescriptionSet of all features of the datasets. represents the class label of instance i.ΓNumber of features to be selected. candidate feature.Selected feature.nNumber of filters.mNumber of classifiers.Set of selected features for filter and classifier.The score value generated by filter method on the feature and the class label Y.The correlation between and .The performance of ML model generated byclassifier onthe set of features and Y.Score of J as per feature selection criteriaon the candidate feature . Notations used in the article. Set of selected features for filter and classifier. The score value generated by filter method on the feature and the class label Y. The performance of ML model generated by classifier onthe set of features and Y. Score of J as per feature selection criteria on the candidate feature . Research gap Feature Selection (FS) aims to identify informative and non-redundant features that improve predictive performance while reducing dimensionality and computational complexity. Existing approaches can be broadly categorized based on their evaluation mechanisms and search strategies. Traditional feature selection methods Many filter, wrapper, and hybrid methods have been proposed in the literature. Filter methods evaluate features independently of any predictive model using statistical, distance-based, or information-theoretic criteria such as ANOVA, Fisher score, ReliefF, and Mutual Information (MI)7,30. These methods are computationally efficient and particularly effective for high-dimensional datasets14. However, filters typically ignore feature interactions and classifier dependency, which may result in suboptimal predictive performance. Moreover, many filter methods do not explicitly model redundancy among selected features. Wrapper methods evaluate subsets using classifier performance as the selection criterion12. While they often produce higher predictive accuracy by accounting for feature interactions, they are computationally expensive and classifier-dependent. Embedded approaches partially address this limitation by incorporating feature selection within model training, yet they remain tied to specific learning architectures and lack flexibility across heterogeneous classifiers. Metaheuristic and optimization-driven hybrid frameworks To balance computational efficiency and predictive accuracy, hybrid methods combine filter ranking with wrapper evaluation. Several studies have proposed hybrid strategies based on specific filters such as MI, Fisher, or ReliefF31–33. Metaheuristic optimization algorithms, including PSO and GA-based variants, have been widely adopted to enhance subset search performance34,35. Although these approaches effectively explore large search spaces, they often introduce increased computational overhead, parameter sensitivity, and convergence instability36. Recent intelligent optimization frameworks further demonstrate the integration of metaheuristics with deep learning architectures. For instance, Logapriya et al.37 proposed Opposition Learning-based Red Panda Optimization for feature selection within a deep learning recommendation framework. Jasmine et al.38 introduced a hybrid deep gated neural model combined with robust feature selection for environmental prediction. Optimization-based adaptive mechanisms have also been applied in large-scale communication and security systems, including energy-aware 6G IoMT clustering39, cooperative caching in 5G networks40, and optimized cloud security frameworks41. While these studies demonstrate the effectiveness of hybrid optimization strategies, most remain domain-specific and tightly coupled with particular optimization algorithms or model architectures. Their feature selection mechanisms are often not statistically grounded in redundancy verification and lack generalizable hybrid integration across multiple classifiers. Limitations of existing approaches and research gap From the above literature, the following limitations can be identified: Strong dependence on specific filters, metaheuristic algorithms, or classifier architectures. Strong dependence on specific filters, metaheuristic algorithms, or classifier architectures. Limited statistical verification of feature–class dependency and inter-feature redundancy. Limited statistical verification of feature–class dependency and inter-feature redundancy. Reduced generalizability across heterogeneous datasets and learning models. Reduced generalizability across heterogeneous datasets and learning models. High computational complexity in optimization-driven wrappers. High computational complexity in optimization-driven wrappers. Insufficient systematic comparison under diverse experimental conditions. Insufficient systematic comparison under diverse experimental conditions. To address these challenges, we propose a generalized hybrid feature selection framework that (i) statistically verifies dependency between candidate features and class labels, (ii) explicitly models redundancy among selected features, (iii) incorporates wrapper-based performance without being restricted to a specific classifier, and (iv) guides the selection of the most effective filter-classifier combination within a unified and adaptable structure. Despite extensive research, existing hybrid feature selection methods lack a systematic mechanism to evaluate and identify optimal filter-classifier pairings across datasets with varying characteristics. Most approaches are tied to specific filters or optimization strategies, limiting their generalizability and reproducibility. This limitation motivates the proposed framework. Unlike many domain-specific optimization-driven methods, the proposed framework is filter-agnostic, classifier-independent, and designed for robust applicability across diverse classification problems. Preliminaries Let us first define the concepts of feature relevancy, redundancy and importance for the two candidate features and on the datasets D (refer Table 1) as, Feature relevancy: If the feature is ranked higher than the feature using any filter method then feature is more relevant than with respect to Y. In other words, the filter score for is higher than for . Feature redundancy: If the correlation between and the selected feature is greater than the correlation between and then the feature is more redundant than . Feature importance: If the performance measured after adding with the selected feature is greater than that of the same with using any classifier then the feature is considered as more important than the feature . Methodology In this section, we propose a Systematic Hybrid Feature Selection framework with Optimal Filter-classifier Pairing and Parallel Evaluation (SHFS-OFCP-PE). The framework is designed to provide a filter-agnostic and classifier-independent hybrid feature selection strategy while ensuring statistical redundancy control and scalable evaluation. Feature selection (FS) is suitable for a dataset if it produces stable performance with reasonable computation time. Hybrid FS methods are one of the approaches that justify using the FS method by harnessing the strength of traditional FS methods. Therefore, our proposed framework generates the ranking of features based on filter methods. This ranking gives information about the dependency of features with the class labels. In addition to this, we find a correlation between every pair of features. This helps avoid selecting features that are highly correlated with those already chosen. Next, the framework applies a greedy-based sequential search algorithm to find the optimal subset of features in coordination with the classifier. It is worth mentioning that although greedy forward selection does not guarantee global optimality in an NP-hard problem such as feature selection, exhaustive search becomes computationally infeasible in practice42. Greedy strategies are therefore widely adopted due to their strong balance between computational efficiency and empirical performance14. Furthermore, the proposed framework mitigates the risk of poor local optima by integrating relevance, redundancy, and wrapper performance within a unified objective function in an iterative manner. The iterative classifier feedback further reduces myopic decisions by reassessing the predictive contribution at each step. Empirically, the stability and competitive performance observed across datasets indicate that the greedy strategy does not significantly degrade solution quality. Further, our framework searches for the best performing hybrid combination based on different filters and classifiers used. Figure 1 illustrates the architecture of the proposed hybrid feature selection framework, where n filter methods and m classifiers are emlpoyed in the hybridization process. In this framework, each dataset is first processed through all n filters to generate feature ranking based on various statistical or information-theoretic criteria. These rankings reflect the relevance of each feature with respect to the target class labels. Subsequently, each of the m classifiers is applied to evaluate and select the optimal subset of features based on their classification performance, guided by the rankings produced in the filtering phase. Since every filter is paired with each classifier, the framework explores a total of mxn distinct filter-classifier combinations. For each such combination, it allows the framework to identify the most effective hybrid configurations, ensuring that the selected features are not only relevant but also enhance the performance of the corresponding machine learning model. The hybridization process of the proposed framework effectively integrates filter and wrapper approach based on relevancy, redundancy, and predictive performance of the classifiers. Architecture of the proposed framework. Feature selection process of the proposed framework The earlier proposed hybrid methods of feature selection was strongly related to the particular filter method used. Thus, the applicability of these methods is restricted to that filter method. Hence, we present the proposed hybrid framework, whose feature selection criteria are independent of the filter and the classifier used (Table 2). The goal of our method is to reduce feature redundancy and increase relevancy to class labels, improving ML model performance. We consider these three cases to decide the selection for a candidate feature as follows: On the application of a filter method, say on two candidate features and , if the score generated on is greater than that of , i.e., then is selected. On the application of a filter method, say on two candidate features and , if the score generated on is greater than that of , i.e., then is selected. On the application of a correlation method, say ρ on and with the selected feature , if the correlation coefficient generated with is less than , i.e., , then is selected. On the application of a correlation method, say ρ on and with the selected feature , if the correlation coefficient generated with is less than , i.e., , then is selected. If the performance of an ML model measured after adding with the selected features is greater than the performance of the same model measured after adding , i.e., , then is preferred to , If the performance of an ML model measured after adding with the selected features is greater than the performance of the same model measured after adding , i.e., , then is preferred to , Given the aforesaid information, we formulate the criterion for the proposed framework for selecting the feature subset as i.e., where, S is the selected subset of features and shows the final score generated over the candidate feature after considering the three criteria over the candidate feature . The framework selects a subset of Γ features. Table 2Representative sensitivity analysis of three parameters , and γ on two datasets.α (Relevance)β (Redundancy)γ (Performance)BCA (%)WAV (%)0.330.330.3496.84 ± 0.7284.62 ± 0.950.400.300.3096.41 ± 0.8183.95 ± 1.120.300.300.4097.12 ± 0.6585.08 ± 0.880.500.250.2595.77 ± 0.9382.31 ± 1.340.250.250.5097.26 ± 0.5885.34 ± 0.760.600.200.2094.92 ± 1.0581.74 ± 1.41 Representative sensitivity analysis of three parameters , and γ on two datasets. Weight sensitivity of the objective function: To evaluate the robustness of the proposed objective function, we conducted a sensitivity analysis by introducing weighting parameters into Eq. (2). The generalized formulation is defined as: where and Experiments were performed on two representative datasets (Breast Cancer and Waveform) using HybridMI with a Linear SVM classifier, selecting 15% of features under a 70:30 train–test split. Each configuration was repeated five times with different random seeds. Table X reports the mean classification accuracy and standard deviation. The results indicate that moderate variations around the equal-weight configuration () do not significantly affect predictive performance. However, extreme dominance of a single component (e.g., α =60) leads to noticeable degradation in both datasets. Increasing the wrapper contribution (γ ) slightly improves performance, suggesting that classification feedback plays an important role in refining the selected subset. Overall, the equal-weight setting provides a stable and well-balanced default configuration without introducing additional hyperparameter tuning complexity. This analysis also provides a component-wise evaluation of the hybrid objective function, illustrating the relative contribution of relevance, redundancy, and wrapper performance to final predictive accuracy. In the proposed framework, feature redundancy is estimated using Pearson correlation due to its computational efficiency and scalability in high-dimensional settings, where redundancy must be repeatedly computed during greedy forward selection. Pearson correlation has been widely used in filter-based feature selection methods, including correlation-based feature selection (CFS), due to its simplicity and effectiveness in reducing multicollinearity43. Although Pearson correlation primarily captures linear dependencies, the wrapper performance component incorporated in the objective function provides implicit modeling of complex and potentially non-linear feature interactions through classifier feedback. As discussed in hybrid feature selection literature14, wrapper methods can compensate for limitations of filter criteria by directly optimizing predictive performance. Alternative redundancy measures such as Mutual Information-based criteria, Maximal Information Coefficient (MIC), and distance correlation have been proposed to capture nonlinear dependencies15. While these measures may better characterize complex feature relationships, they typically introduce higher computational overhead, which may limit scalability when embedded within iterative hybrid feature selection frameworks. Next, we present the serial and parallel computation of the proposed framework to generate nm sets of selected features based on various combinations of filters and classifiers. Serial computation Figure 2 shows the flowchart of the serial algorithm. As shown in this figure, the process is executed serially for each combination of filter and classifier . The first selected feature is the one which is ranked higher using a filter and the set of Γ features are selected consecutively based on criterion 2. The pseudo-code for the serial algorithm is presented as Algorithm 1. Here, each feature is a single-dimensional vector with training samples, . The class label Y is also a single-dimensional vector with vector size so that we can calculate the feature relevancy and the feature redundancy among features and the class label. Maxs is a variable that stores the maximum value of the objective function for obtaining the most informative candidate feature. The classifier measures the model’s performance on the selected features plus one candidate feature in each iteration. For each hybrid combination (lines 1–2), the proposed framework initializes the already-selected features for filter and classifier , the threshold Γ and Maxs (lines 3–5). Second, we calculate the feature relevancy between the class labels and features (lines 6–8). Third, according to criterion 2, the framework selects features in each iteration until γ is equal to Γ (lines 9–23). Note that first, we select the feature with the largest relevance with class labels (line 11). Flowchart of serial algorithm. In the aforesaid algorithm, the feature ranking through filter takes O(F) (lines 6–8), and the selection of a set of features of size Γ in co-ordination with the classifier takes O (Γ F) (lines 9–23). As a result, the time complexity for a single hybrid combination is O (Γ F). However, based on the filter and classifier used, our goal is to find the best hybrid combination out of all the possible combinations. Hence, for nm hybrid combinations (Fig. 1), the time complexity is O (nmΓ F). Serial hybrid method. Parallel hybrid method Parallel computation The serial version of the hybrid method, as described in the previous section, requires O(nm Γ F) time. Here, we present a parallel computing model of the proposed framework and the corresponding algorithm, which runs in O(Γ F) time only. Fig. 3 demonstrates the parallel computing model based on master-slave architecture. For ease of understanding, both the serial and parallel methods are shown side by side in this figure. While the serial employs one computing node that sequentially generates the feature subset for each hybrid combination, the Parallel method adopts the same hybridization process as that of serial computing. However, it distributes the working of different hybrid combinations over master-slave nodes. The master nodes are created corresponding to filter methods to generate feature ranking. The slave nodes perform the wrapper process in-coordination with the classifier. Further, each master-slave combination performs the complete hybridization process to generate the feature subset in parallel with each other. The corresponding flowchart and pseudo code for the parallel method are presented in Fig. 4 and Algorithm 2 respectively. Parallel architecture of the proposed framework. Flowchart of parallel algorithm (refer Figure 2). Parallel algorithm The flowchart for the parallel algorithm (see Fig. 4) shows that the feature selection process for each combination of and are executed in parallel to generate the set of Γ features simultaneously. The Algorithm 2 shows that n filters are distributed over the n master nodes (line 1). Next, each master node generates the ranking of features (lines 2–4). Finally, according to criterion 2 each slave node generates the subset of features of threshold (Γ ) (lines 5–9). In parallel computing, each hybridization process takes place on each master-slave pair that runs in parallel. So the time complexity is reduced to O(Γ F). Unlike existing hybrid feature selection methods, which are often designed around fixed filter-classifier combinations, the proposed framework is filter- and classifier-independent. It systematically evaluates multiple filter-classifier pairs to identify the optimal hybrid combination for a given dataset, making it highly adaptive and generalizable. Additionally, it incorporates relevance, redundancy, and model performance into a unified scoring function and introduces serial and parallel computing architectures to improve scalability. These features make our framework adaptable across diverse datasets and problem domains, and thus represent a significant advancement over prior work. Optimal filter-classifier pairing and parallel evaluation Traditional hybrid feature selection methods typically rely on predefined filter-classifier combinations, which may introduce bias toward specific learning models or evaluation criteria. In contrast, the proposed framework systematically evaluates all possible filter-classifier pairs within a unified structure. Each pair is assessed using the hybrid objective function that integrates feature relevance, redundancy control, and classifier performance. The optimal combination is determined based on aggregated cross-validation performance, ensuring a data-driven and statistically justified pairing strategy. This systematic pairing enhances generalizability by avoiding dependence on a specific filter or classifier. Unlike metaheuristic-based hybrid approaches that embed a single optimization strategy, the proposed method explores multiple hybrid configurations in a controlled and comparable manner. To address computational overhead, the evaluation of independent filter-classifier combinations is executed in parallel using a master–slave architecture. Each slave process computes the hybrid objective for a specific pair, while the master node aggregates results and identifies the optimal configuration. Since each hybrid combination is independent, parallelization significantly reduces practical runtime without altering theoretical complexity. This design improves scalability, particularly for high-dimensional datasets where multiple combinations must be evaluated. Overall, the proposed pairing and parallel evaluation strategy ensures systematic exploration, improved robustness, and computational efficiency. Computational complexity and theoretical analysis For a dataset with n samples and d features, classical filter methods typically require computations, as they evaluate each feature independently with respect to the class labels. Wrapper methods incur higher complexity, generally , where k denotes the selected subset size and C represents the classifier training cost. Metaheuristic-based feature selection approaches (e.g., PSO or GA) further increase computational burden due to population-based iterative search mechanisms, often resulting in complexity, where T and P denote the number of iterations and population size, respectively. The proposed SHFS-OFCP-PE framework adopts a deterministic greedy forward selection strategy combined with statistical redundancy verification, yielding scoring complexity per classifier without requiring stochastic population updates. Although multiple filter-classifier pairings are evaluated, the parallel evaluation strategy distributes these computations across processing units, thereby reducing practical runtime while preserving theoretical efficiency. This design achieves a controlled trade-off between computational cost and selection robustness. The computational efficiency of the proposed framework arises from two factors: (i) filter-based ranking reduces the search space, and (ii) parallel evaluation enables simultaneous processing of multiple filter-classifier combinations. As a result, the effective number of sequential classifier evaluations is reduced, leading to improved scalability compared to conventional wrapper-based approaches. In addition to computational complexity, several theoretical aspects of the proposed framework are noteworthy. First, with respect to the curse of dimensionality, high-dimensional datasets often degrade model performance and increase computational cost. The proposed framework mitigates this issue through filter-based ranking, which reduces the effective feature space before classification, thereby improving scalability and generalization. Second, the optimal subset search problem is addressed using a greedy forward selection strategy. While exhaustive search guarantees optimality with exponential complexity, the proposed approach provides a near-optimal solution with significantly reduced computational overhead, making it suitable for large-scale applications. Third, in comparison to traditional feature selection paradigms, filter methods are computationally efficient but lack interaction with classifiers, wrapper methods offer improved accuracy at the expense of high computational cost, and embedded methods are model-dependent. The proposed hybrid framework effectively combines the strengths of these approaches by integrating filter-based ranking with classifier-based validation in a systematic and flexible manner. Finally, redundancy handling is achieved through statistical verification and stability analysis, which reduce the selection of highly correlated or redundant features. This leads to improved robustness and consistency of the selected feature subsets across different datasets. Overall, the proposed framework provides a balanced trade-off between computational efficiency, scalability, and selection performance. Experimental results and discussion Experimental setup To evaluate the effectiveness of the proposed framework, we construct 12 hybrid models by pairing four filter methods-ANOVA, Fisher Score, ReliefF, and Mutual Information (MI)-with three classifiers: Naïve Bayes, k-nearest neighbors (k = 3), and Support Vector Machines (SVM). These combinations are tested on multiple benchmark datasets with diverse characteristics, including high dimensionality and class imbalance. Statistical validation is performed using ANOVA followed by post hoc tests to assess the significance of the observed improvements. The experiments were conducted on a personal computer running Windows 10 with an Intel Core i3-5005U CPU and 8 GB of RAM. We also utilized Google Colab1 and the Scikit-learn Python package for implementation. The above configuration was used to generate the results presented in Tables 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 and 16. The datasets used in the experiments were obtained from the UCI Machine Learning Repository and the literature15, as shown in Table 3. These datasets span various domains, including biological data (e.g., Lung_discrete), image data (e.g., ORL), and other application areas. The selected datasets represent diverse characteristics, including variations in dimensionality, feature redundancy, and class distribution. This enables evaluation of the proposed framework under different conditions, particularly in high-dimensional and noisy environments where feature selection is critical. We have applied 5-fold cross-validation strategy is employed within the proposed framework. In each fold, feature selection is performed using only the training portion, and the objective function J as shown in Eq. 2 is computed by integrating relevance (ψ), redundancy (ρ), and wrapper performance (). The wrapper performance term is evaluated within each training fold, and the resulting objective values are averaged across the five folds to determine feature selection. The validation fold remains completely unseen during feature selection. Table 3Details of datasets used in experiments.DatasetsShort nameInstancesFeaturesClassesImbalance ratioTypesBreast_cancer44BCA6991021.86DiscreteLung_discrete15LDI7332574.2DiscreteColon15COL62200021.82DiscreteWarpPIE10P15WAR2102420101.0DiscreteWaveform45WAV5002131.02ContinuousSpambase46SPA46015721.54ContinuousMice Protein47MPR10808271.43ContinuousLung15LUN2032312523.17ContinuousORL15ORL4001024401.0Continuous Details of datasets used in experiments. Table 4ρ value of Spearman’s rank correlation coefficient.DatasetsFSAnovaFisherReliefFMIWaveformAnova1.0000− 0.17400.06490.0363Fisher− 0.17401.00000.4779− 0.0597ReliefF0.06490.47791.00000.3610MI0.0363− 0.05970.36101.0000SCADIAnova1.00000.07930.18390.1037Fisher0.07931.00000.03120.0345ReliefF0.18390.03121.00000.043MI0.10370.03450.0431.000 ρ value of Spearman’s rank correlation coefficient. Table 5ρ value of Kendall’s rank correlation coefficient.DatasetsFSAnovaFisherReliefFMIWaveformAnova1.0000− 0.15230.02850.0095Fisher− 0.15231.00000.4− 0.0571ReliefF0.02850.41.00000.3523MI0.0095− 0.05710.35231.0000SCADIAnova1.00000.05480.12770.0740Fisher0.05481.00000.02350.0312ReliefF0.12770.02351.00000.0249MI0.07400.03120.02491.000 ρ value of Kendall’s rank correlation coefficient. Table 6Accuracy comparison of different filter-classifier combinations on low-dimensional datasets. The table evaluates the effectiveness of hybrid feature selection against baseline filter methods. Benchmark methods are as defined in section “Benchmark implementation and verification”.ClassifierThresholdValueAnova/HybridAFisher/HybridFMI/HybridMIReliefF/HybridR5.A Breast cancerNB10 %91.61/95.5090.06/95.1992.76/95.2386.09/90.1525 %94.66/96.0795.04/96.1695.42/96.4390.66/93.7550 %95.42/97.6795.42/97.5694.82/97.8594.09/95.00k-NN10 %89.14/88.5388.57/93.0789.33/96.3983.23/88.2925 %93.71/93.0795.23/97.1492.76/95.2390.47/96.7550 %95.61/97.6796/97.6794.85/97.8594.06/95SVM10 %92.38/94.6490.47/94.1093.32/95.0084.95/92.1425 %95.61/94.7895.80/95.8994.85/96.6090.66/96.8250 %96.38/97.1495.61/94.3295.23/97.3295.04/94.28Average93.83/95.0093.24/95.6793.70/96.4389.91/93.575.B WaveformNB10 %64.77/67.5064.50/68.1263.62/68.5065.73/67.6225 %69.84/73.3569.92/72.7271.81/80.2380.05/74.5550 %78.08/83.1077.98/82.8277.84/84.0180.05/84.92k-NN10 %60.32/67.9062.08/69.1060.32/67.0065.68/68.4025 %70.13/73.1267.20/71.3571.52/75.2080.40/78.2050 %79.52/82.5078.48/82.1078.02/81.9080.40/84.35SVM10 %64.61/69.3765.10/70.5067.54/71.9066.90/71.9025 %74.53/78.2572.89/73.3777.01/80.3275.76/75.4550 %79.45/84.0878.30/84.8680.90/85.9280.37/85.38Average71.25/75.4670.71/74.9972.06/77.2275.03/76.75 Accuracy comparison of different filter-classifier combinations on low-dimensional datasets. The table evaluates the effectiveness of hybrid feature selection against baseline filter methods. Benchmark methods are as defined in section “Benchmark implementation and verification”. Table 7Execution time comparison of SVM-RFE, Tree-based feature selection, and the proposed HybridMI method using SVM.DatasetsSVM-RFETree-based FSSVM-HybridMIAccuracy/featuresAccuracy/featuresAccuracy/featuresExecution timecomparisonSPA92.32/2791.97/2194.30/1427/35/38MPR84.35/4181.10/3387.89/3025/32/43The results highlight computational efficiency relative to baseline and hybrid approaches. Benchmark methods are as defined in section “Benchmark implementation and verification”. Execution time comparison of SVM-RFE, Tree-based feature selection, and the proposed HybridMI method using SVM. The results highlight computational efficiency relative to baseline and hybrid approaches. Benchmark methods are as defined in section “Benchmark implementation and verification”. Table 8Average classification accuracy (mean ± std.) with statistical significance using Naïve Bayes (NB). FSLDICOLWARLUNORLAvg.Anova51.01 ± 3.2471.19 ± 7.5476.42 ± 6.7269.26 ± 10.6924.65 ± 9.3858.70Fisher55.23 ± 4.7374.12 ± 3.5071.03 ± 5.6168.75 ± 9.7027.89 ± 8.6759.40MI58.43 ± 4.5680.04 ± 6.3167.11 ± 5.8972.73 ± 8.5229.93 ± 4.5261.64ReliefF59.47 ± 7.9670.51 ± 4.7373.81 ± 5.7470.56 ± 7.7730.11 ± 7.3260.89JMIM63.51 ± 3.7880.27 ± 7.6479.28 ± 4.5772.92 ± 8.3935.95 ± 5.4867.18DCSF72.41 ± 4.7879.03 ± 5.8980.82 ± 2.2078.45 ± 7.5638.68 ± 6.6970.79HybridA74.90 ± 3.0677.21 ± 6.2586.25 ± 4.8372.19 ± 5.0531.42 ± 5.8173.23HybridF69.06 ± 4.1481.20 ± 3.5980.85 ± 5.1176.29 ± 6.2737.75 ± 3.1471.89HybridMI73.54 ± 5.0889.73 ± 4.2181.95 ± 3.4387.98 ± 6.5339.07 ± 5.9477.92HybridR71.35 ± 3.7579.24 ± 5.7484.00 ± 4.8586.51 ± 7.8642.32 ± 4.2172.48The table compares hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) across multiple datasets. Benchmark methods are as defined in section “Benchmark implementation and verification”.Significant values are in bold. Average classification accuracy (mean ± std.) with statistical significance using Naïve Bayes (NB). The table compares hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) across multiple datasets. Benchmark methods are as defined in section “Benchmark implementation and verification”. Significant values are in bold. Table 9Average classification accuracy (mean ± std.) with statistical significance using kNN (k=3).FSLDICOLWARLUNORLAvg.Anova62.20 ± 4.7570.97 ± 5.6367.00 ± 3.1179.27 ± 6.8462.20 ± 7.3568.32Fisher67.21 ± 2.2572.09 ± 3.1768.91 ± 4.3581.74 ± 4.3860.69 ± 6.1870.12MI70.50 ±4.1279.10 ± 2.1573.51 ± 4.2387.98 ± 6.1563.08 ± 8.1774.13ReliefF64.03 ± 5.4271.64 ± 4.5369.39 ± 3.1383.69 ± 5.4159.85 ± 7.0669.72JMIM62.29 ± 4.7575.09 ± 2.4874.66 ± 5.0989.61 ± 3.7769.65 ± 8.2174.26DCSF71.82 ± 2.8970.83 ± 5.5283.84 ± 3.6390.91 ± 4.6465.95 ± 7.2776.47HybridA72.94 ± 5.3079.99 ± 4.3971.20 ± 6.8482.26 ± 5.7569.78 ± 7.3275.23HybridF75.61 ± 2.4581.40 ± 2.3274.46 ± 3.8086.53 ± 2.3379.34 ± 5.2479.37HybridMI78.81 ± 3.9589.94 ± 4.8392.67 ± 2.7795.97 ± 3.5277.69 ± 7.0787.02HybridR77.63 ± 4.3082.72 ± 3.0276.91 ± 4.2788.25 ± 4.4171.67 ± 6.1479.86The table compares hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) across multiple datasets. Benchmark methods are as defined in section “Benchmark implementation and verification”.Significant values are in bold. Average classification accuracy (mean ± std.) with statistical significance using kNN (k=3). The table compares hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) across multiple datasets. Benchmark methods are as defined in section “Benchmark implementation and verification”. Significant values are in bold. Table 10Average classification accuracy (mean ± std.) with statistical significance using SVM.FSLDICOLWARLUNORLAvg.Anova58.39 ± 7.2970.02 ± 8.8470.73 ± 6.9183.06 ± 8.7764.42 ± 5.2969.32Fisher59.32 ± 4.5072.23 ± 4.6771.38 ± 5.3080.45 ± 6.5065.27 ± 4.5469.93MI67.83 ± 6.9077.13 ± 5.9669.62 ± 5.6689.46 ± 3.7967.29 ± 5.3774.26ReliefF61.49 ± 5.3471.38 ± 6.3773.79 ± 6.3181.89 ± 4.0461.87 ± 6.6769.68JMIM71.42 ± 4.7274.25 ± 4.4578.72 ± 3.5679.57 ± 5.4568.42 ± 2.4876.47DCSF67.54 ± 5.3076.06 ± 6.1576.18 ± 3.9487.30 ± 5.8172.53 ± 5.9075.92HybridA72.36 ± 3.4679.49 ± 6.3891.09 ± 2.7487.60 ± 6.9867.80 ± 3.5079.66HybridF70.27 ± 2.2685.30 ± 4.9084.97 ± 2.6985.32 ± 4.3079.34 ± 2.2779.75HybridMI76.01 ± 4.9685.73 ± 5.5287.27 ± 2.6094.62 ± 3.5681.12 ± 4.4883.55HybridR72.42 ± 3.6283.50 ± 3.4586.60 ± 4.1691.04 ± 2.6871.66 ± 3.8881.84The table compares hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) across multiple datasets. Benchmark methods are as defined in section “Benchmark implementation and verification”.Significant values are in bold. Average classification accuracy (mean ± std.) with statistical significance using SVM. The table compares hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) across multiple datasets. Benchmark methods are as defined in section “Benchmark implementation and verification”. Significant values are in bold. Table 11The comparison of hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) using all features with the highest average accuracy over three classifiers.FSLDICOLWARLUNORLAnova57.8973.0676.2485.5657.20Fisher63.1476.2474.8082.9458.47MI69.3683.3074.6188.5059.15ReliefF67.3374.9576.5383.4556.70JMIM69.9880.8781.2985.7862.65DCSF74.2380.4182.7890.2664.26HybridA76.9583.6887.0885.1260.97HybridF74.1785.3282.9286.5468.26HybridMI79.2292.4389.1196.5471.25HybridR77.4985.3886.1192.8465.96All_features74.0674.6091.9790.2577.83Benchmark methods are as defined in section “Benchmark implementation and verification”.Significant values are in bold. The comparison of hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) using all features with the highest average accuracy over three classifiers. Benchmark methods are as defined in section “Benchmark implementation and verification”. Significant values are in bold. Table 12Output of ANOVA test on accuracy of FS methods.Source of variationSum of squaredfMean squareF- statisticSignificance valueF- criticalBetweengroups20306.36792256.26316.475.0001.9158Withingroups39716.481290136.953Total60022.848299 Output of ANOVA test on accuracy of FS methods. Between groups Within groups Table 13LSD post hoc test for hybrid methods (HybridA).Between methodsMean differenceSignificance value95% Confidence interval for mean differenceLower boundUpper boundHybridA-Anova11.820.0005.8717.77HybridA-Fisher9.124.0033.1815.07HybridA-MI8.394.0062.4514.34HybridA-ReliefF8.172.0072.2314.12HybridA-JMIM6.155.043.2112.10HybridA-DCSF8.235.0072.2914.18HybridA-HybridF− 3.984.188− 9.931.96HybridA-HybridMI− 14.454.000− 20.40− 8.51HybridA-HybridR− 7.367.015− 13.31− 1.42 LSD post hoc test for hybrid methods (HybridA). Table 14LSD post hoc test for hybrid methods (HybridF).Between methodsMean differenceSignificance value95% Confidence interval for mean differenceLower boundUpper boundHybridF-Anova15.805.0009.8421.75HybridF-Fisher13.108.0007.1619.06HybridF-MI12.378.0006.4318.33HybridF-ReliefF12.157.0006.2118.10HybridF-JMIM10.139.0014.1916.09HybridF-DCSF12.220.0006.2718.17HybridF-HybridA3.984.188− 1.969.93HybridF-HybridMI− 10.469.001− 16.42− 4.52HybridF-HybridR− 3.383.264− 9.332.56 LSD post hoc test for hybrid methods (HybridF). Table 15LSD post hoc test for hybrid methods (HybridMI).Between methodsMean differenceSignificance value95% Confidence interval formean differenceLower boundUpper boundHybridMI-Anova26.274.00020.3332.22HybridMI-Fisher23.578.00017.6329.52HybridMI-MI22.848.00016.9028.79HybridMI-ReliefF22.626.00016.6828.57HybridMI-JMIM20.608.00014.6628.64HybridMI-DCSF22.689.00016.7428.64HybridMI-HybridA14.454.0008.5120.40HybridMI-HybridF10.469.0014.5216.42HybridMI-HybridR7.086.0201.1413.03 LSD post hoc test for hybrid methods (HybridMI). HybridMI- Anova Table 16LSD post hoc test for hybrid methods (HybridR).Between methodsMean differenceSignificance value95% Confidence interval for mean differenceLower boundUpper boundHybridR- Anova19.188.00013.2425.13HybridR-Fisher16.491.00010.5422.44HybridR-MI15.761.0009.8121.71HybridR-ReliefF15.540.0009.5921.49HybridR-JMIM13.522.0007.5719.47HybridR-DCSF15.603.0009.6621.55HybridR-HybridA7.367.0151.4213.31HybridR-HybridF3.383.264− 2.569.33HybridR-HybridMI− 7.086.020− 13.03− 1.14 LSD post hoc test for hybrid methods (HybridR). To robustly evaluate the proposed feature selection (FS) framework, we selected a diverse set of datasets that vary in number of features (dimensionality), number of classes, number of instances, class imbalance ratio, and data types (discrete and continuous). For datasets containing continuous attributes, discretization was applied as a standard preprocessing step to ensure stable entropy-based feature scoring and compatibility with information-theoretic filters. Prior studies have shown that supervised entropy-based discretization can improve the stability of mutual information estimation and enhance classification performance, particularly for probabilistic classifiers such as Naïve Bayes48. However, discretization may also influence statistical measures such as ANOVA by altering variance structure and potentially introducing a bias–variance trade-off. Furthermore, recent work on feature transformation and stochastic selection mechanisms highlights that preprocessing decisions can directly affect model robustness and generalization49. In this study, discretization is treated uniformly as preprocessing rather than as a comparative experimental factor, as the primary objective is to evaluate the proposed hybrid feature selection framework. The class imbalance ratio is calculated as the number of instances in the most frequent class divided by the number of instances in the least frequent class. This diversity ensures a comprehensive assessment of the robustness and scalability of the proposed method. For example, Breast_cancer is a low-dimensional binary classification dataset, while Lung_discrete and Colon are high-dimensional, small-sample medical datasets typical in bioinformatics. WarpPIE10P, with 2420 features and 10 classes, evaluates performance in multi-class, high-dimensional settings. Waveform and Spambase represent moderate-dimensional, continuous datasets for signal and text classification tasks. Mice Protein and Lung offer biomedical data with varied class distributions and feature counts. Finally, the ORL dataset presents a high-dimensional image recognition task with 40 classes. Collectively, these datasets address key challenges in FS, such as high dimensionality, class imbalance, and domain variability, thereby enabling a thorough and generalizable validation of our approach. To analyze the behavior of the hybrid methods in detail, we selected two representative datasets (Breast_cancer and Waveform) and conducted experiments using three different FS thresholds. For the high-dimensional datasets, a fixed threshold was used. Missing values, such as those in the Mice Protein dataset, were imputed using the mean of the corresponding column. Continuous features were discretized into five equal-width bins, and nominal features were encoded as integers. Finally, we applied the 10-fold cross-validation to evaluate the performance of the final model. Herein, for each outer fold, feature selection and model training were performed strictly within the training partition, and performance was computed on the corresponding validation fold. Final performance metrics represent the average across all outer folds. Statistical comparisons were performed on aggregated performance measures rather than individual fold-level results. Although cross-validation introduces partial dependence between folds due to overlapping training data, this evaluation protocol follows established best practices in machine learning research50. In the next section, we discuss the diversity study of the filter methods used. Diversity study In order to show that the feature ranking produced by different filter methods are significantly different, we conducted a diversity study on the four filters based on Spearman’s rank and Kendall’s rank correlation coefficient on two datasets Scadi and Waveform. The diversity analysis is chosen to reflect contrasting characteristics in dimensionality and class distribution. The purpose of this analysis is to illustrate the behavioral tendencies of the proposed hybrid framework rather than to establish universal statistical generalization across all datasets. Since the diversity measure depends primarily on the interaction between feature selection mechanisms and classifier behavior, similar trends are expected for datasets with comparable structural properties. The results are shown in Tables 4 and 5 respectively in which the ρ values are in the range of [-1, 1], where 1 indicates that the corresponding filter methods are equal. It can be observed that the ρ values are far away from 1 for two different filter methods, but equals to 1 when the two same filters are compared. This can be seen in the diagonals of the tables. This simple experiment shows that the filter methods are chosen in this study generate different feature rankings, thereby ensuring enough diversity. Similar experiments have been done in earlier work51 but with different filters. Results and discussion Benchmark implementation and verification To ensure a fair and reproducible comparison, all baseline feature selection methods, ANOVA14, Fisher Score52, ReliefF53, Mutual Information (MI)54, JMIM55, and DCSF56 were implemented using standard Scikit-learn library functions. For all comparative experiments, identical preprocessing steps, discretization strategy, cross-validation protocol, feature selection thresholds, and classifier parameter settings were maintained across methods. This controlled experimental setup ensures that observed performance differences arise from the feature selection mechanisms themselves rather than from inconsistencies in implementation or evaluation. Furthermore, each computational table explicitly specifies the corresponding comparison group, ensuring clarity in the evaluation of baseline, hybrid, and state-of-the-art methods. Comparative performance against baseline filters First, we performed experiments on two representational datasets (Breast cancer and Waveform) with three different feature threshold values, i.e., 10% , 25% , and 50% of the original set of features. The thresholds (10%, 25%, and 50%) are selected to evaluate the robustness of the proposed framework across varying subset sizes rather than to determine a single optimal cutoff. Reporting performance under multiple selection ratios enables analysis of stability and generalization behavior. While adaptive thresholding approaches based on data complexity measures as used in57 may provide a principled mechanism for selecting γ, the present study focuses on evaluating the hybrid feature integration mechanism independently of a specific subset-size optimization strategy. We compare the resulted accuracy of basic filter methods (i.e., Anova, Fisher, ReliefF, and Mutual information) as well as their hybrid combinations with three classifiers (i.e., NB, k-NN, and SVM) referred as HybridA, HybridF, HybridR, and HybridMI. The comparisons are shown shown in Table 6. For different threshold values, the results of the best performing ML models are shown in bold. It can be observed from the table that the hybrid combinations outperform the basic filter methods on most of the cases for both the datasets. However, the HybridMI performs best among all the hybrid methods on almost all threshold values. We propose hybrid methods to harness the strength of filter and wrapper methods. Hence, it is crucial to explicitly compare the time complexity of our method with the existing basic methods. For this, we choose a wrapper SVM-RFE and embedded Tree-based FS method on the datasets with moderate dimensions. We choose these datasets to make the comparison fair as wrapper and embedded methods suffer from high time complexity on high-dimensional datasets58. Table 7 manifests the computational time of SVM-RFE, Tree-based FS, and HybridMI with SVM on two datasets, spambase and Mice protein. Table 7 shows the accuracy of the datasets with minimum features. The last column of the table refers to the execution time comparison concerning the number of wins in terms of execution time. It can be observed from the table that HybridMI wins 38 number of times as compared to SVM-RFE and Tree-based FS, whose wins are 27 and 35 respectively on Spambase datasets. In the case of Mice protein datasets, HybridMI wins 43 number of times compared to SVM-RFE (25), and Tree-based FS (32). It is found that the hybrid method generate better average accuracy with less number of features and low computational time. High-dimensional datasets with three classifiers Next, we discuss the results on high-dimensional datasets. We compare the hybrid methods with basic filters (Anova, Fisher, ReliefF, and MI) as well as two recently proposed MI-based FS methods: JMIM55 and DCSF56. We choose the MI-based methods for comparison because HybridMI has demonstrated superior performance over other hybrid approaches, as discussed earlier. We choose to select the first 30 features (Γ) to calculate the average accuracy and the highest accuracy. Earlier literature59 has used a similar threshold value. The reasons are as follows. The threshold should not be too small or large. The smaller threshold will have fewer feature alternatives and hence diversity of selected features can be compromised. At the same time, a larger threshold value will increase the computational time of the model. Tables 8, 9 and 10 show the average accuracy and the standard deviations over three classifiers. Rows show the feature selection methods and columns show the datasets. The last column shows the average accuracy over five datasets on the corresponding feature selection method. We can observe that in most cases, hybrid methods perform better than traditional methods. Among the hybrid methods, HybridMI obtains the highest average accuracy as 77.92%, 87.02%, and 83.55% on kNN, NB, and SVM, respectively. In Table 8, HybridMI outperforms Anova, Fisher, ReliefF, MI, JMIM, DCSF, HybridF, HybridA, and HybridF by up to 24.66%, 23.76%, 21.85%, 20.89%, 13.78%, 9.15%, 7.73%, 6.98% and 6.01% respectively, in terms of average accuracy. HybridMI performs better than eight, nine, seven, nine, and eight other methods out of ten methods on Lung_discrete, Colon, WarPIE10P, Lung, and ORL datasets, respectively. In Table 9, HybridR obtains the second-best accuracy followed by HybridF, DCSF, HybridA, JMIM, MI, Fisher, ReliefF, and Anova. In Table 10, HybridMI performs better than nine, eight, eight, nine, and nine other methods out of ten methods on Lung_discrete, Colon, WarPIE10P, Lung, and ORL datasets, respectively. HybridMI outperforms Anova, ReliefF, Fisher, MI, DCSF, JMIM, HybridA, HybridF, and HybridR by up to 17.03%, 16.60%, 16.33%, 11.11%, 9.13%, 8.47%, 4.65%, 4.53%, and 2.04% respectively, in terms of average accuracy. Moreover, it is evident from the results shown in Tables 8, 9 and 10 that there are notable differences between the three classifiers. For instance, on the datasets ORL, the accuracy of HybridF is higher than the other nine methods on the classifier kNN. However, on the classifier NB, the accuracy of HybridF is higher than Anova, Fisher, MI, ReliefF, JMIM, and HybridA, while lower than HybridR and HybridMI and comparable to DCSF. Similarly, on the same datasets, HybridR outperforms nine other methods on the classifier NB and perform comparable to HybridA on the classifier SVM. In contrast, on the classifier kNN, it has accuracy lower than HybridMI, comparable to HybridA and JMIM and outperforms the rest five methods. The performance of the classification is strongly related to the datasets and the classifier60. This shows that the classifiers used for experiments are significantly different. Finally, Table 11 shows the comparison of ten methods using all features with highest average accuracy over three classifiers. Figure 5 shows the Areas Under ROC Curves (AUCs) of three different classifiers for twelve methods. The results show that hybrid methods outperform the comparing eight methods. In the context of hybrid methods, HybridMI shows better performance than HybridA, HybridF, and HybridR. Lastly, we compare the feature stability of the ranking of features generated through these FS methods. However, various stability measures are proposed to capture the variations of feature preferences generated by FS methods. In this article, we use relative weighted consistency for measuring feature stability which effectively recognizes randomness in FS and ensures a reliable comparison of stability61. Note that the same stability measure has also been used in62. Although alternative stability metrics such as the Kuncheva index63 and Nogueira’s64 stability measures have been proposed in the literature, relative weighted consistency was selected because it accommodates ranking-based comparisons and varying subset sizes, which align with the hybrid nature of the proposed framework. A comparative analysis of multiple stability measures is an interesting direction for future work. Figure 6 shows the average stability 2 of twelve methods on three classifiers. The X-axis shows the feature threshold values for measuring the stability ranging from the first 10 to 60 features. The Y-axis shows the corresponding average stability over three classifiers. It can be observed from the figure that the hybrid methods show better stability as compared to the traditional FS methods. AUC comparison across all benchmark datasets. The table compares hybrid feature selection methods with baseline filter approaches and recently proposed MI methods (JMIM and DCSF) across multiple datasets. Benchmark methods are as defined in section “Benchmark implementation and verification”. Average stability of ten methods using three classifiers on different feature thresholds. Alignment of objectives with experimental evaluation The experimental design is structured to directly validate the objectives of the proposed framework. First, the identification of optimal filter-classifier pairings is evaluated through comparative performance analysis across multiple combinations, as presented in Tables 6, 8, 9, 10, and 11, along with AUC analysis in Fig. 5 and comparison with state-of-the-art methods in Table 18. These results demonstrate that different combinations yield varying performance, highlighting the effectiveness of the proposed framework in selecting optimal pairings. Second, the adaptability of the framework across datasets with diverse characteristics is validated using experiments on both low- and high-dimensional datasets, as shown in Table 6, 8, 9 and 10. The consistency of performance across datasets, further supported by Fig. 5, confirms the robustness of the framework under varying feature distributions. Third, redundancy control and feature stability are assessed through stability analysis in Fig. 6 and rank correlation measures reported in Tables 4 and 5. These results indicate that the proposed method maintains consistent feature selection while effectively handling redundant information. Finally, computational efficiency is evaluated using execution time comparisons presented in Tables 7 and 19. The results demonstrate that the proposed framework achieves reduced computational cost, supported by its parallel evaluation strategy. Overall, these experiments collectively confirm that the proposed framework satisfies its objectives in terms of optimal pairing, adaptability, redundancy control, and computational efficiency. Table 18Comparison with state-of-the-art feature selection methods using SVM.LDICOLWARLUNORLHybridA72.36 ± 3.4679.49 ± 6.3891.09 ± 2.7487.60 ± 6.9867.80 ± 3.50HybridF70.27 ± 2.2685.30 ± 4.9084.97 ± 2.6985.32 ± 4.3079.34 ± 2.27HybridMI76.01 ± 4.9685.73 ± 5.5287.27 ± 2.6094.62 ± 3.5681.12 ± 4.48HybridR72.42 ± 3.6283.50 ± 3.4586.60 ± 4.1691.04 ± 2.6871.66 ± 3.886571.25 ± 11.5366.19 ± 11.0676.67 ± 26.9893.10 ± 3.9981.25 ± 5.39MBDE6680.54 ± 10.5486.14 ± 16.2692.86 ± 8.8494.00 ± 2.4565.75 ± 3.88The table demonstrates the effectiveness of the proposed hybrid framework relative to existing approaches. Benchmark methods are as defined in section “Benchmark implementation and verification”.Significant values are in bold. Comparison with state-of-the-art feature selection methods using SVM. The table demonstrates the effectiveness of the proposed hybrid framework relative to existing approaches. Benchmark methods are as defined in section “Benchmark implementation and verification”. Significant values are in bold. Table 19Execution time comparison in seconds.LDICOLWARLUNORLSerial3.127.1838.9428.6765.94Parallel8.7411.5925.9621.8838.93656.356.0725.0021.4875.92MBDE6627.52153.44274.64382.55184.98 Execution time comparison in seconds. Statistical test: analysis of variance (ANOVA) We statistically verified the significance of experimental results by means of hypothesis testing. For this purpose, we used a statistical method named ANOVA which compare the differences between the means of two or more groups. Based on the outcomes of ANOVA, it decides that a null hypothesis can be rejected or not. The null hypothesis states that the means of the methods are same. In our case, the null hypothesis is defined as follows: In contrast, the alternative hypothesis is as follows: To test the null hypothesis, we take 30 samples for each method at a significance level of α = 0.05. The null hypothesis () is rejected based on two conditions from the ANOVA test: (i) the p-value (significance value) must be lower than the significance level, and (ii) the F-statistic must be greater than the F-critical value. Table 12 shows that both conditions are satisfied with respect to the accuracy of the methods. Therefore, we reject the null hypothesis, which means there is a statistically significance difference in accuracy among the methods. The ANOVA tells us that at least one method performs differently, however, it fails to identify which specific methods differ. To find that out, we conduct a Fisher Least Significant Difference (LSD) post hoc test. This test compares the average accuracy between each pair of methods. If the difference between two methods is larger than a certain threshold, that is LSD, we conclude that their performance is significantly different. We calculate the LSD at a significance level α as where is the critical value from the t-distribution, MSE is the mean square error from ANOVA, and , are the sample sizes of the two groups being compared. In this study, since each method uses 30 samples, both and equal 30. A statistically significant difference is identified when the 95% confidence interval for the mean difference does not include zero and the p-value is less than 0.05. This allows us to pinpoint which hybrid methods perform better than other. Here, Tables 13, 14, 15, and 16 show the post hoc analysis for HybridA, HybridF, HybridMI and HybridR, respectively. It can be observed from the tables that each hybrid method shows a positive mean difference with the traditional methods, as well as JMIM and DCSF. Moreover, their confidence interval does not contain zero and having significance value lower than 0.05. Hence, we can conclude that hybrid methods performed significantly better than other comparing methods. Among hybrids, HybridMI (refer 15) shows the same behaviour with respect to HybridA, HybridF and HybridR which shows its supremacy over other hybrid methods (Table 17). Table 17Mapping of research objectives to experimental validation.ObjectiveExperimental evidenceOptimal filter-classifier pairingPerformance comparison across hybrid combinations (Tables 6, 8, 9, 10, 11, 18) and AUC analysis (Fig. 5)Adaptability across low- and high-dimensional datasetsEvaluation across multiple datasets and classifiers (Tables 6, 8, 9, 10) demonstrating consistent performanceRedundancy control and feature stabilityStability analysis (Fig. 6) and rank correlation using Spearman’s and Kendall’s coefficients (Tables 4 and 5)Computational efficiency and scalabilityExecution time comparison demonstrating reduced execution time and scalability (Tables 7 and 19) Mapping of research objectives to experimental validation. Comparison with state-of-the-art technique Table 18 presents a comparison of state-of-the-art (SOTA) methods65,66 proposed in 2024. For consistency, the SOTA methods were implemented using an SVM classifier and compared against the results generated by the proposed framework, also using SVM. From the table, it can be observed that the method in66 outperforms others on the LDI dataset. It shows comparable performance to HybridMI on the COL and LUN datasets, and to HybridA on the WAR dataset. However, it is outperformed by HybridMI on the ORL dataset. It is important to note that the method in66 is a metaheuristic-based optimization feature selection technique, which makes it computationally expensive. Additionally, it is classifier-dependent and, therefore, lacks generalization capability. In contrast, the proposed framework guides the selection of the best hybrid combination by identifying which filter and classifier will perform best for a given dataset. Execution time comparison The execution time comparison between the proposed framework (in both serial and parallel modes) and state-of-the-art (SOTA) methods is shown in Table 19. The execution times were recorded on a Windows 10 system with a 64-bit processor: Intel i7-10750H CPU @ 2.60GHz, 2592 MHz, 6 cores, 24 GB RAM, and a 4 GB GPU. It is worth mentioning that the aforesaid configuration is specifically used to compare the execution time as shown in Table 19. The runtime comparisons are valid only within experiments conducted on the same hardware environment. As observed in the table, the parallel framework demonstrates lower execution time compared to the serial framework on the WAR, LUN, and ORL datasets. However, for the LDI and COL datasets, the parallel version shows higher execution time. This is primarily due to the overhead associated with initializing and managing parallel processes, such as data copying and process synchronization, which outweigh the performance benefits in lightweight computations as compared to serial. Further, when compared with SOTA methods65,66, the proposed parallel framework is only outperformed by the method in65 on smaller datasets. In contrast, it outperforms the method in66 across all five datasets. Theoretical analysis of parallel efficiency: The practical speedup achieved by the parallel implementation is influenced by system-level overhead, including communication and synchronization costs associated with the master–slave architecture. According to Amdahl’s Law67, the maximum achievable speedup is limited by the non-parallelizable fraction of the algorithm. For smaller datasets, where computational workload per worker is limited, overhead may dominate, resulting in marginal or negative speedup. However, as dataset dimensionality and feature evaluation complexity increase, the parallelizable fraction grows, leading to improved acceleration. This behavior aligns with Gustafson’s Law68, which indicates that speedup scales with problem size. Thus, the break-even point is primarily governed by feature dimensionality, subset size (τ), and classifier training cost. Limitations and scope for future work While the proposed hybrid feature selection framework demonstrates strong improvements over traditional and recent methods, several limitations should be acknowledged. First, the framework has been evaluated only on single-label classification tasks, which limits its applicability to more complex real-world scenarios such as multi-label or multi-output learning. In addition, although cross-validation is a standard evaluation strategy, the folds are not strictly independent, and therefore some statistical assumptions may not be fully satisfied. From an optimization perspective, the use of greedy forward selection provides computational efficiency for the NP-hard feature selection problem; however, it does not guarantee globally optimal feature subsets. More advanced search strategies, such as metaheuristic or submodular optimization methods, may further improve solution quality. From a modeling standpoint, the scoring function relies on fixed weights for relevance, redundancy, and classifier performance. Although sensitivity analysis indicates robustness, adaptive or learnable weighting mechanisms could improve generalization across diverse datasets. Furthermore, redundancy estimation is currently based on linear correlation, which may not capture complex nonlinear dependencies among features. Incorporating nonlinear dependency measures would enhance the representational capability of the framework. Additionally, the feature subset size is determined using predefined thresholds rather than adaptive, data-driven criteria. From a scalability perspective, despite the use of parallel evaluation, the framework may still incur computational overhead for smaller datasets and may face challenges when applied to extremely high-dimensional data (e.g., large-scale omics datasets). Advanced acceleration strategies, such as GPU-based or distributed implementations, could address these limitations. Finally, the framework evaluates filter-classifier combinations exhaustively rather than learning to predict optimal combinations for a given dataset. Future work may explore meta-learning approaches to recommend effective combinations, thereby improving scalability, adaptability, and practical deployment. Conclusion This article has proposed a general framework for the hybrid feature selection method, which is independent of a particular filter or classifier method. In order to reduce time complexity, we have also presented a parallel architecture of the proposed framework. We conducted diversity test on the filters and performed experiments with 12 combinations of hybrid methods and compared them with traditional as well as the two state-of-the-art methods. We have found that the hybrid methods outperform others with respect to accuracy, AUC, and feature stability. Specifically, we have shown that the MI-based hybrid method exhibits 24.66% and 2.04% a maximum and a minimum improvement respectively for high dimensional datasets in terms of average accuracy. Although MI-based hybrid methods show strong performance in several datasets, the proposed framework provides a systematic mechanism to identify optimal filter-classifier combinations across diverse data characteristics, ensuring general applicability rather than reliance on a single method. The results confirm that the proposed framework effectively adapts to diverse feature characteristics and task objectives by selecting optimal feature–classifier pairings across datasets. Finally, as the research finds MI based hybrid method as the best model, our future effort will be made to work on the same model for multi-label datasets incorporating label-specific information shared by the features. Abbreviations Analysis of variance Dynamic change of selected feature Feature selection Hybrid method using ANOVA Hybrid method using fisher score Hybrid method using mutual information Hybrid method using ReliefF Joint mutual information maximisation k-Nearest neighbors Least significant difference Machine learning Mutual information Naive Bayes Support vector machine Author contributions DKR was involved in conceptualization, methodology, software, SKT involved with writing, and editing. MKN provided resources and revision of the article. AP provided visualization ad data curation and ASN provided supervision, project management and funding Funding Open access funding provided by Symbiosis International (Deemed University). This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Data availability Inquiries about data availability should be directed to the Table 2. Dataset is taken from references23–26. Declarations Competing interests The authors declare no competing interests. Footnotes colab.research.google.com. https://github.com/DKRakesh/Stabilitymeasure/blob/main/feature_selection_stability2.py. Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement Inquiries about data availability should be directed to the Table 2. Dataset is taken from references23–26.