StackFeat: a convergent algorithm for optimal predictor selection in genomic data

\firstname Akbar \lastname Yermekov \fnsep, \firstname D. A. \lastname Herrera-Martí · 2026

Read StackFeat: a convergent algorithm for optimal predictor selection in genomic data by \firstname Akbar \lastname Yermekov \fnsep, \firstname D. A. \lastname Herrera-Martí (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22887.txt.

Introduction

Yermekov and Herrera-Martí present StackFeat, a feature-selection algorithm for genomic data where many more features than samples make selection unstable. It repeats cross-validation and keeps only features that score highly on both signed coefficients and selection frequency. On a COVID-19 miRNA data set, it finds a stable five-marker signature from 332 features, beating a nine-gene benchmark set. The signature includes a marker linked to both COVID-19 survival and Dengue infection. The dual rule gives convergence guarantees single-rule methods lack.

Catalog record

CREATOR\firstname Akbar \lastname Yermekov \fnsep, \firstname D. A. \lastname Herrera-Martí
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19646
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22887

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

StackFeat: a convergent algorithm for optimal predictor selection in genomic data \firstname Akbar \lastname Yermekov \fnsep, \firstname D. A. \lastname Herrera-Martí Abstract In high-dimensional genomic data, the curse of dimensionality ( d ≫ n d\gg n ) and limited sampling make feature selection inherently unstable—a critical barrier to biomarker discovery. We introduce StackFeat, an iterative algorithm that accumulates two statistics across repeated cross-validation: signed coefficients (measuring effect strength and direction) and selection frequencies (estimating selection probability). Only features ranking highly by both criteria are retained. On a COVID-19 miRNA dataset (GSE240888), StackFeat identified a stable 5-miRNA signature from 332 features (98.5% reduction), achieving AUC 0.922, significantly outperforming the benchmark 9-gene set (AUC 0.907, p = 0.0016). The signature includes hsa-miR-150-5p, a marker implicated in both COVID-19 survival and Dengue infection. This dual-criterion approach provides convergence guarantees absent in single-criterion methods, enabling discovery of known biomarkers, novel candidates, and previously unknown relationships. Keywords: marker selection, feature selection, bioinformatics, dimensionality reduction, robust algorithm, stacking, miRNA, COVID-19 1 Introduction The identification of reliable biomarkers from high-dimensional genomic data is a primary challenge in bioinformatics, defined by the ( d>n ) problem. Datasets in this domain often contain 30000+ features (for typical transcriptomics datasets) or 300+ features (for miRNA data) for fewer than 100-200 samples, as in this study’s case. A key failing of many feature selection methods in this context is instability: the selected feature sets are not reproducible and vary wildly with small data perturbations. This severely limits their clinical and biological reliability. This instability arises from the curse of dimensionality: when d\gg n , many feature subsets achieve similar predictive performance, making the solution highly sensitive to small perturbations in the data. Correlated features exacerbate this, as the regularization path can arbitrarily select one feature over an equally predictive alternative. The two dominant approaches to this problem present a trade-off. The Lasso algorithm tibshirani1996 provides minimal (sparse) solutions but can be unstable when predictors are highly correlated - its L1 penalty tends to select one feature from a correlated group and zero out the others, and the choice is sensitive to minor changes in the data. The Elastic Net algorithm zou2005 was designed to solve this by grouping correlated variables, resulting in high stability but at the cost of minimalism. Identifying robust biomarkers is particularly important when they have demonstrated clinical relevance — for example, markers predictive of survival in critically ill patients. Such markers may otherwise be missed by unstable feature selection methods. The goal is to achieve both minimalism and robustness. We present StackFeat, a novel iterative algorithm that cumulatively aggregates feature scores to find a stable, minimal, and highly predictive set of markers. This method adapts the concept of “Stacked Generalization” wolpert1992 ; instead of stacking predictions , it stacks feature importance signals (score and frequency) from multiple methods to build a robust consensus. We validate this method against a recent 2023 study by Gao et al., demonstrating our method’s superiority on the same dataset (GSE240888) gao2023 . 2 Materials and Methods 2.1 Dataset and Benchmark We utilized the public miRNA expression dataset GSE240888 from the NCBI Gene Expression Omnibus (GEO). This dataset, first analyzed by Gao et al., contains profiles from 122 patients (COVID-19 vs. healthy controls). After preprocessing, the feature space consists of 332 microRNA genes (miRNAs) , presenting a significant dimensionality challenge (332 features vs 122 samples). We benchmarked our algorithm against the 9-marker set from Gao et al., which achieved a 0.907 AUC. 2.2 The StackFeat Algorithm We designed a fully automated algorithm to identify a minimal, stable set of predictive features. The method is an iterative process where gene scores are cumulatively aggregated until the classifier’s aggregate performance score converges. Figure 1: StackFeat algorithm workflow. The algorithm’s logic is depicted in Figure 1 and proceeds in the following steps: 1. Initialization: Set iteration counter t=1 . Initialize empty dictionaries w and c to accumulate gene-level statistics across all iterations, where w_{j} stores cumulative signed coefficients and c_{j} stores selection counts for each gene j . 2. Reshuffling: Using seed s_{0}+t , randomly reshuffle the dataset to create new fold assignments for iteration t . 3. Nested Cross-Validation: • Outer CV Loop: Perform k -fold stratified cross-validation ( k=10 ) on the dataset • Inner CV Loop: Within each outer fold f , run ElasticNetCV to obtain coefficients \hat{\beta}^{(t,f)} • Model Training: Train an ensemble estimator on the selected features and evaluate performance 4. Score Aggregation: After completing all k folds, update cumulative statistics: • Signed coefficients: w_{j}\leftarrow w_{j}+\sum_{f=1}^{k}\hat{\beta}_{j}^{(t,f)} • Selection counts: c_{j}\leftarrow c_{j}+\sum_{f=1}^{k}\mathbf{1}[\hat{\beta}_{j}^{(t,f)}\neq 0] 5. Dynamic Thresholding: Set candidate set size: m=\left\lfloor\frac{\text{mean}(\text{genes\_per\_fold})}{4}\right\rfloor 6. Dual-Criterion Set Intersection: Generate the current feature set: S^{(t)}=\underbrace{\{j:|w_{j}|\text{ in top }m\}}_{S_{w}^{(t)}}\cap\underbrace{\{j:c_{j}\text{ in top }m\}}_{S_{c}^{(t)}} 7. Convergence Check: Convergence requires two consecutive iteration differences below tolerance \varepsilon : |AUC^{(t)}-AUC^{(t-1)}|<\varepsilon\quad\text{and}\quad|AUC^{(t-1)}-AUC^{(t-2)}|<\varepsilon 8. Iteration: • If converged: Output final feature set S^{(t)} • If not converged: Set t\leftarrow t+1 , return to Step 2 We set \varepsilon=0.02 empirically for this dataset. A more stringent threshold (e.g., 0.01) was unnecessary because the two-consecutive-iterations requirement already guards against false convergence. 2.3 Dual-Criterion Selection Accumulated Statistics. Over T iterations with k folds each, we accumulate two statistics for each feature j : Cumulative signed coefficient: w_{j}^{(T)}=\sum_{t=1}^{T}\sum_{f=1}^{k}\hat{\beta}_{j}^{(t,f)} (1) Selection count: c_{j}^{(T)}=\sum_{t=1}^{T}\sum_{f=1}^{k}\mathbf{1}[\hat{\beta}_{j}^{(t,f)}\neq 0] (2) Here w_{j}^{(t)} denotes the cumulative value of w_{j} after iteration t , and similarly for c_{j}^{(t)} . Population Quantities. Define the expected coefficient and selection probability: \bar{\mu}_{j}=\mathbb{E}\left[\hat{\beta}_{j}^{(t,f)}\right],\quad p_{j}=P\left(|\hat{\beta}_{j}^{(t,f)}|>0\right) (3) where t indexes iterations and f\in\{1,\ldots,k\} indexes folds. Convergence. Let \tilde{\beta}_{j}^{(t)}=\sum_{f=1}^{k}\hat{\beta}_{j}^{(t,f)} denote the iteration-level sum. Since iterations are independent, by the law of large numbers: \frac{w_{j}^{(T)}}{T}\to\mathbb{E}[\tilde{\beta}_{j}^{(t)}]=k\cdot\bar{\mu}_{j} (4) where the last equality follows from linearity of expectation. Dividing by k : \frac{w_{j}^{(T)}}{T\cdot k}\to\bar{\mu}_{j},\qquad\frac{c_{j}^{(T)}}{T\cdot k}\to p_{j} (5) Repeated cross-validation thus provides a resampling-based estimate of population-level feature importance. Selection Rule. At iteration t , using the cumulative statistics w_{j}^{(t)} and c_{j}^{(t)} accumulated so far, we rank features by |w_{j}^{(t)}| and by c_{j}^{(t)} , selecting the top m under each criterion: S_{w}^{(t)}=\{j:|w_{j}^{(t)}|\text{ ranks in top }m\} (6) S_{c}^{(t)}=\{j:c_{j}^{(t)}\text{ ranks in top }m\} (7) The final selected set is their intersection: S^{(t)}=S_{w}^{(t)}\cap S_{c}^{(t)} (8) This dual requirement guards against two failure modes that single-criterion methods miss: Feature type |w_{j}| c_{j} Outcome True signal high high Selected Noise low low Rejected Sign-inconsistent low high Rejected by S_{w} Infrequent but consistent high low Rejected by S_{c} Table 1: Failure modes addressed by dual-criterion selection. Sign-inconsistent: A feature selected frequently but with inconsistent coefficient sign across folds—contributions cancel, yielding low |w_{j}| . Infrequent but consistent: A feature overshadowed by correlated alternatives, selected rarely but with consistent direction when selected. Comparison to Stability Selection. Unlike stability selection meinshausen2010 , which thresholds on selection frequency alone, StackFeat requires both coefficient consistency and selection frequency. A feature selected in every fold but with alternating sign would pass stability selection but be rejected by StackFeat. Additionally, StackFeat uses convergence-based stopping rather than a fixed iteration count B , allowing the algorithm to adapt to dataset complexity. 3 Results 3.1 Benchmarking and Dimensionality Reduction The most significant result of our methodology is the extreme dimensionality reduction. The algorithm successfully identified a minimal 5-feature set from the initial 332 miRNAs, a >98% reduction in features . This 5-marker

Read the complete text