KGroups: A Versatile Univariate Max-Relevance Min-Redundancy Feature Selection Algorithm for High-dimensional Biological Data

Malick Ebiele, Malika Bendechache, Rob Brennan · 2026

Read KGroups: A Versatile Univariate Max-Relevance Min-Redundancy Feature Selection Algorithm for High-dimensional Biological Data by Malick Ebiele, Malika Bendechache, Rob Brennan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28417.txt.

Introduction

Ebiele, Bendechache, and Brennan propose KGroups, a fast feature selection method for high-dimensional biology data. Most filter methods differ in how they score relevance and redundancy, not in how they pick. KGroups instead picks by clustering, balancing top relevance against low overlap in a single-variable way. On 14 biological benchmarks it matched the strong multivariate mRMR method in accuracy. It ran up to 821 times faster and can be tuned further, unlike its rivals.

Catalog record

CREATORMalick Ebiele, Malika Bendechache, Rob Brennan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61133
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28417

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

KGroups: A Versatile Univariate Max-Relevance Min-Redundancy Feature Selection Algorithm for High-dimensional Biological Data Malick Ebiele, Malika Bendechache, Rob Brennan Abstract This paper proposes a new univariate filter feature selection (FFS) algorithm called KGroups. The majority of work in the literature focuses on investigating the relevance or redundancy estimations of feature selection (FS) methods. This has shown promising results and a real improvement of FFS methods’ predictive performance. However, limited efforts have been made to investigate alternative FFS algorithms. This raises the following question: how much of the FFS methods’ predictive performance depends on the selection algorithm rather than the relevance or the redundancy estimations? The majority of FFS methods fall into two categories: relevance maximisation (Max-Rel, also known as KBest) or simultaneous relevance maximisation and redundancy minimisation (mRMR). KBest is a univariate FFS algorithm that employs sorting (descending) for selection. mRMR is a multivariate FFS algorithm that employs an incremental search algorithm for selection. In this paper, we propose a new univariate mRMR called KGroups that employs clustering for selection. Extensive experiments on 14 high-dimensional biological benchmark datasets showed that KGroups achieves similar predictive performance compared to multivariate mRMR while being up to 821 times faster. KGroups is parameterisable, which leaves room for further predictive performance improvement through hyperparameter finetuning, unlike mRMR and KBest. KGroups outperforms KBest. JEL Classification]C02, C38, C45, C55 MSC Classification]68T01 , 68T20 keywords Feature Selection, Clustering-based Filter Method, Univariate Max-Relevance Min-Redundancy (mRMR), Data Mining, Knowledge Discovery pacs [ pacs [ 1 Introduction Feature selection (FS) methods in machine learning (ML) and statistical analysis have existed since at least the 1990s with an ever increasing interest 47 . They define ways of selecting a subset of features from the original dataset while maintaining a competitive predictive performance. Feature selection methods are important in tackling computational cost (in terms of training time) and resource requirements 19 . Today, they are also important for addressing the energy consumption, efficiency, monetary cost of training, and the sustainability of Artificial Intelligence (AI) models 48 ; 35 ; 12 . There are three primary type of FS methods: filters, wrappers, and embedded. Wrappers are considered the most predictively performant, while filters are considered the most computationally efficient. Filters are also considered the most generalisable to other models (models they were not initially trained for; see Table 1 in section 2 below) 19 ; 23 ; 47 . With the rise of interest in green AI initiatives, filter feature selection (FFS) methods are receiving more attention from researchers and practitioners because they align with sustainable AI objectives 37 ; 46 ; 5 ; 40 ; 22 ; 50 . This can also be explained by the fact that FFS methods have only one drawback to address (lag in predictive performance) compared to two drawbacks for wrappers (computational inefficiency and failure to generalise to other models). FFS methods have been widely applied to feature selection and dimensionality reduction tasks in heterogeneous domains including but not limited to biology, healthcare, marketing, information retrieval, human activity recognition 8 ; 14 ; 15 ; 57 ; 1 ; 4 ; 38 ; 31 ; 43 ; 4 ; 20 . FS methods can be classified into two groups: value-based and non-value-based. Value-based FS methods first assign a relevance or a redundancy value to each feature then perform the selection using either or both of the assigned relevance and redundancy values. Max-Rel, mRMR, and KGroups are all value-based FFS. Non-value-based FS techniques perform the selection task without assigning any values to the features; they are in majority wrappers (see Table 1 in the background section). This paper focuses on value-based FFS methods because they are computationally efficient, have only one drawback, and can be integrated with explainable AI 11 . Value-based FFS estimate features relevance using a similarity measure from i) information theory (e.g. mutual information also known as information gain), ii) statistics (e.g. F-value also known as F-statistics), or iii) machine learning models’ built-in feature ranking methods also known as embedded FS techniques. Once a relevance value is assigned to each of the feature, the features are sorted in descending order of relevance and the top k are selected (thus the names Max-Rel, SelectKBest or KBest). As to multivariate FFS methods, a redundancy estimator is needed in addition to the relevance estimator. They consist of maximising the difference between or the quotient of the relevance \text{Rel}(F_{i}) by the redundancy \text{Red}(F_{i};\mathcal{S}) of the candidate feature F_{i} with respect to the already selected features \mathcal{S} using an incremental search algorithm (see section 3 , particularly Eqs. ( 5 ) and ( 7 )). These methods are referred to as mRMR meaning Min-Redundancy Max-Relevance. This paper proposed a new versatile univariate algorithm, called KGroups, that simultaneously maximises the relevance and minimises the redundancy using clustering. In other words, KGroups is a univariate mRMR. First, a relevance value is assigned to each feature. Then, the features are clustered into k groups using binning. Finally, the feature with the maximum relevance value from each cluster is selected (for more details, see section 4 below). There are been previous attempts to use binning-based clustering or discretisation for FS and dimensionality reduction 15 ; 29 ; 34 ; 41 ; 56 . KGroups extends the Supervised Cosine Similarity Information Gain (SCSIG) proposed by 15 . SCSIG was originally designed for cosine similarity-based binning and mutual information-based selection (see section 4 for more details). The paper addresses the following research question: To what extent can a univariate Max-Relevance Min-Redundancy filter feature selection algorithm improve the performance of feature selection methods in terms of classification accuracy and computational cost? To answer the research question above, we performed extensive experimental analysis of KGroups with competing algorithms (KBest, and multivariate mRMR) on 14 high-dimensional biological benchmark datasets and reported their respective average 5-fold cross validation accuracy score and execution time (see section 6 ). This paper has the following contributions: (1) Design and implementation of KGroups, a versatile univariate mRMR algorithm that competes with the state of the art multivariate mRMR FFS algorithm while being up to 821 times faster. (2) Extension of the SCSIG 15 FFS method into a flexible algorithm that support any type of relevance estimator. KGroups is parameterisable and introduces the concept of tie-breaking; unlike SCSIG. (3) Comprehensive Analysis of KGroups, highlighting its strengths, weaknesses, and potentials. The remainder of the paper is as follows. The next section defines the cores concepts of FFS methods. In section 3 , we present the related work. Section 4 and 5 describes our proposed method and the experimental setting, respectively. The experimental results and discussion are presented in section 6 . The limitations and potentials of the KGroups algorithm is presented in section 7 . Finally, the concluding remarks and future work is highlighted in section 8 . 2 Background In this section, we provide the definitions of the core concepts of the Feature Selection (FS) domain. 2.1 Feature Selection Methods There are three primary types of FS methods: filters, wrappers, and embedded 19 ; 23 ; 33 ; 47 . Table 1 below gives a comparative analysis of the three primary types of FS in terms of relative predictive performance, computational cost, and generalisation [of the

Read the complete text