FSEVAL: Feature Selection Evaluation Toolbox and Dashboard

Muhammad Rajabinasab [email protected], Arthur Zimek [email protected] · 2026

Read FSEVAL: Feature Selection Evaluation Toolbox and Dashboard by Muhammad Rajabinasab [email protected], Arthur Zimek [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18227.txt.

Introduction

Rajabinasab and Zimek present FSEVAL, a toolbox plus dashboard for judging feature-selection methods. Feature selection trims inputs that add little while keeping models easy to explain, unlike methods that compress inputs into opaque mixes. The toolbox gives one shared way to test methods with or without labels, using shared scores. Researchers can run wide tests with ease instead of piecing together one-off checks.

Catalog record

CREATORMuhammad Rajabinasab [email protected], Arthur Zimek [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19748
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.18227

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FSEVAL: Feature Selection Evaluation Toolbox and Dashboard Muhammad Rajabinasab [email protected], Arthur Zimek [email protected] Abstract Feature selection is a fundamental machine learning and data mining task, involved with discriminating redundant features from informative ones. It is an attempt to address the curse of dimensionality by removing the redundant features, while unlike dimensionality reduction methods, preserving explainability. Feature selection is conducted in both supervised and unsupervised settings, with different evaluation metrics employed to determine which feature selection algorithm is the best. In this paper, we propose FSEVAL, a feature selection evaluation toolbox accompanied with a visualization dashboard, with the goal to make it easy to comprehensively evaluate feature selection algorithms. FSEVAL aims to provide a standardized, unified, evaluation and visualization toolbox to help the researchers working in the field, conduct extensive and comprehensive evaluation of feature selection algorithms with ease. Author One and Author Two † † heading: 23 2022 1- 1/21; Revised 5/22 9/22 21-0000 † † shortheadings: FSEVAL / FSEVAL † † firstpage: 1 keywords Feature Selection, Evaluation, Visualization, toolbox, Dashboard 1 Introduction Feature selection is a fundamental machine learning and data mining task. The objective of feature selection is to identify the most relevant features for representing y while eliminating redundant or irrelevant dimensions, thereby improving model generalization and reducing computational costs by overcoming the curse of dimensionality ( 5 ; 8 ) . Compared to dimensionality reduction methods, feature selection offers a huge advantage; keeping the explainability properties of the original feature space despite reducing the dimensionality. This is why, despite the existence of successful methods for dimensionality reduction, such as Principal Component Analysis (PCA) ( 13 ) , Autoencoders ( 1 ) , t-distributed Stochastic Neighbor Embedding (t-SNE) ( 16 ) , and Uniform Manifold Approximation and Projection (UMAP) ( 10 ) , feature selection methods are still important, and in many cases preferred, to tackle the curse of dimensionality ( 5 ; 4 ; 2 ; 18 ; 6 ) . The evaluation of feature selection is most often conducted by a downstream task. In supervised learning, it is often measured using classification accuracy (ACC) and Area Under the Curve (AUC), reflecting the model’s predictive power using the selected subset. For unsupervised scenarios, algorithms are often evaluated using k -Means clustering ( 9 ) , using metrics such as Clustering Accuracy (CLSACC) and Normalized Mutual Information (NMI), which assess the alignment between the discovered clusters and the underlying data structure ( 14 ; 11 ) . Recent advancements have introduced more comprehensive frameworks to address the limitations of these standard metrics. The Feature Selection Dynamic Evaluation Metric (FSDEM) offers a dynamic assessment by tracking performance across varying subset sizes ( 14 ) , while the Average Angle Difference (AAD) provides a model-independent evaluation based on the alignment of principal components between the original and reduced feature spaces ( 15 ) . The Baseline Fitness Improvement (BFI) normalizes performance against the full feature set, ensuring that any gain is attributed to the selection quality rather than dataset simplicity ( 11 ) . Furthermore, the stability of these selections is essential for ensuring robustness; foundational indices by 7 and the more rigorous consistency measures by 12 quantify how sensitive the selection process is to data perturbations. And more importantly, the stability from an informational perspective is also offered by the FSDEM metric ( 14 ) . In this paper, we propose FSEVAL 1 1 1 Python source code provided at: https://github.com/mrajabinasab/FSEVAL , a comprehensive toolbox for the extensive evaluation of feature selection algorithms. Programmed purely using Python, FSEVAL provides all the necessary tools to evaluate feature selection algorithms using supervised and unsupervised tasks and metrics, model-independent AAD metric ( 15 ) , calculate FSDEM score ( 14 ) , and conducting comprehensive runtime analysis. FSEVAL comes with a visualization dashboard, which can easily load the evaluation output and offer visualizations for different metrics, runtime analysis, and additionally, valuable insights such as rank analysis and Critical Difference (CD) diagrams ( 3 ) . 2 FSEVAL Toolbox and Dashboard FSEVAL addresses a critical gap in machine learning research: the lack of a standardized, rigorous framework for the evaluation and comparison of the feature selection algorithms. While many feature selection methods exist, evaluating them often involves fragmented scripts that fail to account for stochastic variance or how a method scales as the number of features increases. FSEval provides a unified pipeline that automates the evaluation and comparison, allowing researchers to evaluate their feature selection methods of interest, using their desired datasets, in a robust and comprehensive evaluation setup. The FSEVAL dashboard helps with easy and interactive visualization of the evaluation outputs. 2.1 FSEVAL Toolbox FSEVAL is built in a modular format that treats feature selection as a dynamic process rather than a static result. Instead of conducting a single-point evaluation, which often hides many aspects of the feature selection process, FSEval evaluates performance across a spectrum of feature sizes—often referred to as “feature ratio grids”. This reveals the “elbow point” of an algorithm, identifying the exact moment where removing more features begins to degrade model accuracy, as well as yielding insights into the overall feature selection process, which can be easily reflected by feeding the evaluation output of FSEVAL to FSDEM ( 14 ) . FSEVAL conducts experiments on the feature selection process by default on the first 10%, by a step size of 0.5%, and the entire range, by a step size of 5%, to test the feature selection performance both in the entire process, and also in extensive and extreme feature selection scenarios. There are different evaluation metrics built in to the FSEVAL toolbox: • Supervised Evaluation: Assesses the predictive power of a feature subset via cross-validated classification performance, utilizing metrics such as Accuracy (ACC) and the Area Under the ROC Curve (AUC) ( 8 ) . • Unsupervised Evaluation: Measures the accordance of the dimensionality-reduced data structure to the external labels. This is quantified through clustering-based metrics, including Clustering Accuracy (CLSACC) and Normalized Mutual Information (NMI) ( 8 ) . • Model-Agnostic Evaluation: Quantifies the quality of the feature selection process by calculating AAD ( 15 ) . • Stability: Uses FSDEM ( 14 ) to measure the informational stability of the feature selection algorithm. This process requires the evaluation results of at least one metric of interest. • Custom Evaluation: FSEVAL supports easy integration of custom metrics, which can be easily defined as a function and fed to the evaluation procedure. Besides the evaluation metrics, FSEVAL also facilitates conducting Runtime Analysis , by stress-testing feature selection algorithms using data with different number of instances and features, highlighting to which a method is more sensitive. 2.2 FSEVAL Dashboard The FSEVAL dashboard is designed to provide easy, comprehensive, and interactive visualization for the evaluation results produced by the FSEVAL toolbox. An already online version of this dashboard is accessible at: https://fseval.imada.sdu.dk . This dashboard can automatically process the evaluation results from FSEVAL toolbox, and create visualizations and tables with valuable insights. All the datasets used in the experiments behind the dashboard are selected from the scikit-feature repository ( 8 ) . The primary capabilities provided by the dashboard include: • Multi-Metric Performance Profiling: Generates interactive curves that visualize how

Read the complete text