Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies
Kaito Baba, Yoshihiko Ozaki, Shuhei Watanabe · 2026
Read Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies by Kaito Baba, Yoshihiko Ozaki, Shuhei Watanabe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20800.txt.
Introduction
Kaito Baba, Yoshihiko Ozaki, and Shuhei Watanabe tackle a bookkeeping problem in machine learning tuning. Their 2026 paper concerns hyperparameter importance: which settings of a model matter most. Tools like PED-ANOVA estimate that importance fast, inside the best regions of the search space. But they assume every setting is always present. Real search spaces are conditional.
In a conditional space, one setting can switch another on or off. It can also change the range a setting may take. The original tool cannot handle such cases in a sound way. The authors propose conditional PED-ANOVA to fill that gap. It estimates importance while respecting which settings are active. The work helps tuners spend effort on the knobs that truly drive results.
Catalog record
| CREATOR | Kaito Baba, Yoshihiko Ozaki, Shuhei Watanabe |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20800
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Machine learning approaches CCS: Mathematics of computing Mathematical analysis Kaito Baba, Yoshihiko Ozaki, Shuhei Watanabe Abstract . We propose conditional PED-ANOVA (condPED-ANOVA) , a principled framework for estimating hyperparameter importance (HPI) in conditional search spaces, where the presence or domain of a hyperparameter can depend on other hyperparameters. Although the original PED-ANOVA provides a fast and efficient way to estimate HPI within the top-performing regions of the search space, it assumes a fixed, unconditional search space and therefore cannot properly handle conditional hyperparameters. To address this, we introduce a conditional HPI for top-performing regions and derive a closed-form estimator that accurately reflects conditional activation and domain changes. Experiments show that naive adaptations of existing HPI estimators yield misleading or uninterpretable importance estimates in conditional settings, whereas condPED-ANOVA consistently provides meaningful importances that reflect the underlying conditional structure. 1. Introduction Analysis of parameter importance ( 35 ; 36 ; 13 ; 14 ) is a central tool for understanding and designing optimization procedures. Over the past decade, techniques for measuring the importance of parameters have attracted considerable attention in the automated machine learning (AutoML) community ( 16 ; 5 ; 41 ) , particularly for improving the efficiency of hyperparameter optimization (HPO) ( 2 ; 3 ; 34 ; 22 ) . Given a hyperparameter search space and a set of evaluated configurations, methods for hyperparameter importance (or HPI methods for short) aim to quantify how strongly each hyperparameter influences the observed performance. Such information is useful for interpreting optimization runs, identifying critical design choices, detecting irrelevant or redundant parameters, and guiding subsequent search or manual tuning ( 27 ; 12 ) . As a result, HPI is widely used across various settings, including HPO of machine learning models and HPO algorithm design, where understanding hyperparameter sensitivity is often essential ( 41 ; 44 ; 29 ; 39 ) . Classic HPI methods such as functional analysis of variance (f-ANOVA) ( 16 ) quantify HPI by measuring how much variance in the objective can be attributed to each hyperparameter. However, their existing local variants mix contributions from unpromising regions, leading to inaccurate estimates of HPI in top-performing subspaces ( 16 ; 6 ) . PED-ANOVA ( 42 ) mitigates this by defining local HPI within a top-performing subset of the search space and efficiently approximating it using the Pearson divergence between one-dimensional distributions of each hyperparameter. Moreover, PED-ANOVA computes both global and local HPIs substantially faster than f-ANOVA while maintaining accurate HPI estimates. Thanks to this combination of speed and reliability, it has been adopted for HPI analysis in practical HPO frameworks, such as Optuna ( 1 ) , demonstrating its utility in real-world workflows. However, we argue that PED-ANOVA has a practical limitation: it cannot properly handle hierarchical / dynamic search spaces, where the presence/domain of a hyperparameter depends on other hyperparameters (hereafter referred to as conditional search spaces). In practice, many real-world problems involve such conditional search spaces ( 17 ) . Typical examples include model selection, where hyperparameters are dependent on a chosen algorithm ( 40 ; 20 ; 11 ; 19 ) , and neural network architecture design, where active parameters depend on the chosen layers ( 4 ; 38 ) . Unfortunately, PED-ANOVA cannot be directly applied to such settings, since it assumes a fixed, unconditional search space and defines distributions over a single, fixed domain for each hyperparameter. We observe that naive extensions of PED-ANOVA, such as filtering inactive configurations, imputing missing values, or artificially expanding parameter domains, do not resolve this issue: these heuristics often violate the conditional semantics of the search space and yield unnatural or uninterpretable HPI estimates (cf. Section 5.4 ). To address this limitation, we propose conditional PED-ANOVA (condPED-ANOVA) , a principled framework for estimating HPI in conditional search spaces. We identify a critical flaw in the original definition of local HPI in PED-ANOVA when applied to conditional search spaces: the importance of hyperparameters that induce conditionality is spuriously attributed to the hyperparameters whose domains or activation depend on them, leading to systematically misleading importance estimates. Consequently, merely constructing a method that correctly computes local HPI in conditional search spaces is still insufficient. We introduce conditional local HPI , which measures importance within top-performing regions without confounding from conditional activation or domain changes. Building on this definition, we derive a closed-form PED-ANOVA-style estimator that remains efficient and yields an interpretable HPI in conditional settings. Experiments confirm that condPED-ANOVA reliably produces a meaningful HPI that reflects the underlying conditional structure, while naive extensions of existing methods fail to do so. Our contributions are summarized as follows: • We identify a fundamental pitfall of original local HPI in conditional search spaces: the contribution of conditioning hyperparameters is spuriously attributed to conditionally active (or domain-shifted) hyperparameters, yielding misleading importance estimates. • We introduce conditional local HPI , a definition that measures importance within top-performing regions while accounting for conditional activation and domain changes. • We derive a closed-form PED-ANOVA-style estimator for conditional local HPI, retaining the efficiency of the original PED-ANOVA via Pearson divergence of one-dimensional distributions estimated with KDE. • We empirically demonstrate that condPED-ANOVA consistently produces meaningful importances in conditional settings, whereas naive extensions of existing methods, including PED-ANOVA, produce unnatural results. 2. Related Work Here, we review prior work most relevant to ours. Additional related work is provided in Appendix A . 2.1. Variance-Based HPI and f-ANOVA HPI aims to quantify how strongly each hyperparameter (and their interactions) influences performance, typically to support search-space design, debugging, and interpretation of HPO runs. Early work on parameter importance analysis builds on variance-based sensitivity analysis such as Sobol indices ( 35 ; 36 ) and ANOVA decompositions ( 13 ; 14 ) , which attribute the global variance of an objective to individual coordinates and their interactions. In the context of HPO, f-ANOVA ( 16 ) applies this idea to HPO objectives by fitting a surrogate model such as a random forest over the hyperparameter space and then estimating main and interaction effects by Monte Carlo integration over the surrogate. This yields importance values that reflect how much of the global variation in performance across the entire search space can be explained by each hyperparameter. f-ANOVA includes efficient marginal computation over random-forest predictions and produces a global ranking of important hyperparameters, making it a standard post-hoc analysis tool in AutoML and algorithm configuration. 2.2. PED-ANOVA PED-ANOVA ( 42 ) extends HPI to settings where the region of interest is not an entire search space. It defines local HPI as the variance contribution of each hyperparameter within the top- \gamma subset of the search space, thereby measuring importance directly in the most relevant region. It further shows that this local HPI can be approximated in closed form using the Pearson divergence between the one-dimensional distributions of each hyperparameter of two top-performing subsets, estimated via one-dimensional KDEs. Empirically, this leads to much faster HPI computation than f-ANOVA while maintaining accurate importance estimates in top-performing subspaces. The details of PED-ANOVA