Hierarchical Variable Importance with Statistical Control for Medical Data-Based Prediction

Joseph Paillard, Antoine Collas, Denis A. Engemann, Bertrand Thirion · 2025

Read Hierarchical Variable Importance with Statistical Control for Medical Data-Based Prediction by Joseph Paillard, Antoine Collas, Denis A. Engemann, Bertrand Thirion (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08724.txt.

Introduction

Joseph Paillard and three co-authors introduce a way to judge which brain features matter in medical prediction, with error control attached. Their method appeared in twenty twenty-five. Flexible models predict well from scans, but naming the regions that drive a forecast is hard when features are tightly correlated.

Their approach, Hierarchical-CPI, tests groups of variables arranged in a tree, from broad clusters down to fine subgroups. Testing groups keeps the search affordable while the error rate across the whole family stays capped. A tree-based allocation stops importance from vanishing among correlated neighbors. On dementia scans and brain-wave data, it flagged biologically sensible features. Rivals were beaten in the head-to-head benchmarks.

Catalog record

CREATORJoseph Paillard, Antoine Collas, Denis A. Engemann, Bertrand Thirion
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-08724

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hierarchical Variable Importance with Statistical Control for Medical Data-Based Prediction Joseph Paillard, Antoine Collas, Denis A. Engemann, Bertrand Thirion Abstract Recent advances in machine learning have greatly expanded the repertoire of predictive methods for medical imaging. However, the interpretability of complex models remains a challenge, which limits their utility in medical applications. Recently, model-agnostic methods have been proposed to measure conditional variable importance and accommodate complex non-linear models. However, they often lack power when dealing with highly correlated data, a common problem in medical imaging. We introduce Hierarchical-CPI, a model-agnostic variable importance measure that frames the inference problem as the discovery of groups of variables that are jointly predictive of the outcome. By exploring subgroups along a hierarchical tree, it remains computationally tractable, yet also enjoys explicit family-wise error rate control. Moreover, we address the issue of vanishing conditional importance under high correlation with a tree-based importance allocation mechanism. We benchmarked Hierarchical-CPI against state-of-the-art variable importance methods. Its effectiveness is demonstrated in two neuroimaging datasets: classifying dementia diagnoses from MRI data (ADNI dataset) and analyzing the Berger effect on EEG data (TDBRAIN dataset), identifying biologically plausible variables. 1 Introduction Within the field of medcial imaging, machine learning holds great promise to facilitate prediction of clinical outcomes, see e.g. [ 1 , 2 , 3 , 4 , 5 , 6 ] . However, these advances have also opened major interpretability challenges. A key issue is how to infer the importance of features from prediction models going beyond ordinary least squares to accommodate a large number of predictors and represent non-linear associations between features and outcomes. Therefore, developing methods to measure variable importance in a model-agnostic manner is critical in order to obtain clinical insights and develop biomarkers, for instance, using brain images for the diagnosis of Alzheimer Disease (AD) based on existing cohorts, or data from clinical trials [ 7 , 8 , 2 , 6 ] . However, to develop trustworthy methods, it is essential to understand their theoretical guarantees, particularly concerning the risk of making false discoveries, which can be captured by the Family-Wise Error Rate (FWER) (see e.g. [ 9 , 10 ] ). Only few variable importance methods give access to such guarantees. Moreover, we focus here on conditional importance , meaning the importance measure whether a variable is directly predictive of the outcome, without being explained away by other variables [ 11 , 12 ] . Such conditional importance is needed to establish that a marker carries independent information about the outcome, rather than merely reflecting distributed factors that are also present in other variables. Conditional importance analysis is particularly difficult in datasets that exhibit strong correlation structures such as image- or genomics-based biomarkers, or health data that reflect common latent factors [ 13 ] . We assess the face validity of the approach with two tasks that have been extensively studied in the literature—the effect of AD on structural MRI and the Berger effect on electroencephalography (EEG)—to allow for a form of confirmation, addressing the challenge posed by the absence of ground truth in variable importance methods. 1.1 Related Work This work focuses global variable importance, as opposed to local variable importance methods such as LIME [ 14 ] or SHAP [ 15 ] . Global variable importance is estimated using methods such as global sensitivity analysis [ 11 ] or the popular Leave One Covariate Out (LOCO) approach [ 16 , 17 ] . These methods can accommodate different types of learners, taking advantage of advances in machine learning to measure importance in complex and nonlinear models [ 17 , 18 ] . Similarly, conditional permutation approaches have been shown to estimate a quantity equivalent to LOCO at a cheaper computational cost and with a faster convergence rate [ 19 ] . These methods have in common to provide a good control of the type-1 error rate, that is considering a null variable (or group) as important. However, all approaches suffer from an inherent limitation: conditional importance decreases as correlation increases. For instance, considering two random normal variables X_{1},X_{2} with correlation \rho in a simple linear model y=\beta_{1}X_{1}+\beta_{2}X_{2} the importance of X_{1} decreases proportionally to (1-\rho^{2}) . To mitigate this issue, methods based on variables grouping have been proposed to identify groups of highly-correlated, hence indistinguishable variables that predict the outcome [ 12 , 13 ] . Variable grouping can be performed based on prior knowledge about the data or by using clustering techniques. While this effectively increases the statistical power by averaging correlated variables, it also reduces the precision in the sense that error control only holds at the group level. When performing the grouping in a data-driven way, choosing the clustering scheme and parameters has a critical impact yet has no obvious solution. A line of work relying on linear models and agglomerative clustering has been proposed in that direction with applications to genomics data [ 9 , 20 , 10 ] . Agglomerative clustering offers a compelling solution as it naturally explores different groupings at various resolutions along the hierarchical tree learned from the data. However, this approach relied on Lasso regression and would consequently limit the user in the choice of the model used to predict the outcome of interest from the variables. Another popular model explainability approach is Shapley Additive Global importancE ( SAGE ) [ 21 ] . Based on Shapley values, this approach estimates an importance score for a given variable by conditioning on all subgroups that include this variable. This procedure provides a more nuanced view than strict conditional importance, because it decomposes additively the variance explained by the model into variable importance. However, it suffers from two main limitations. First, as an aggregated statistic, it obcures the role of variables in the prediction function [ 18 ] , and is unable to distinguish between a predictive variable and another, non-predictive variable yet correlated with a predictive one. Second, the exploration of all submodels comes with an exponential explosion of computation cost, making this approach intractable. While implementations rely on Monte-Carlo sampling instead of exhausting the full combinatorial sum, the number of sampling steps needed to obtain accurate estimates still leads to intractable computation costs [ 18 ] . This two limitations are clearly visible in our experiments in Figure 2 . Our contributions are i) to introduce Hierarchical-CPI, a model-agnostic variable importance measure that improves FWER control; it explores subgroups in a tree-guided manner, using agglomerative clustering to provide more information than variable-level importance while remaining tractable; ii) to present an approach that enforces importance conservation through downstream importance allocation strategy, addressing the issue of vanishing importance under high correlation. 2 Methods Notations: We denote X as the variables, y as the outcome, and \mu as the predictive model. X_{G} represents the set of variables belonging to a group G , and X_{-G} denotes the set of variables in the complement of G . The importance of a group is denoted as \psi_{G} . We use S^{*} to denote the support (or set of active variables) and S_{0} for the set of null variables. In the hierarchical tree defined by the clustering, P refers to a parent node, and L and R refer to

Read the complete text