Hyperparameter Optimisation with Practical Interpretability and Explanation Methods in Probabilistic Curriculum Learning
Llewyn Salt, Marcus Gallagher · 2025
Read Hyperparameter Optimisation with Practical Interpretability and Explanation Methods in Probabilistic Curriculum Learning by Llewyn Salt, Marcus Gallagher (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06683.txt.
Introduction
Llewyn Salt and Marcus Gallagher open the black box of tuning a curriculum learning algorithm, showing which knobs matter and how they interact. Their 2025 paper studies probabilistic curriculum learning, which orders a reinforcement learner's lessons to boost performance, inside tasks like point-maze navigation and motor control. The learner's settings are notoriously touchy and expensive to tune.
Using an optimization framework with a standard estimator, they map how refining the search space speeds the hunt. Their novelty is a SHAP-based explainer adapted to hyperparameters, ranking each setting's push on performance and exposing pair interactions. The output is practical: guidelines and visual tools that make tuning cheaper and legible. The authors pitch interpretability as an efficiency tool, not a luxury, for reinforcement learning work.
Catalog record
| CREATOR | Llewyn Salt, Marcus Gallagher |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06683
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hyperparameter Optimisation with Practical Interpretability and Explanation Methods in Probabilistic Curriculum Learning Llewyn Salt, Marcus Gallagher Abstract Hyperparameter optimisation (HPO) is crucial for achieving strong performance in reinforcement learning (RL), as RL algorithms are inherently sensitive to hyperparameter settings. Probabilistic Curriculum Learning (PCL) is a curriculum learning strategy designed to improve RL performance by structuring the agent’s learning process, yet effective hyperparameter tuning remains challenging and computationally demanding. In this paper, we provide an empirical analysis of hyperparameter interactions and their effects on the performance of a PCL algorithm within standard RL tasks, including point-maze navigation and DC motor control. Using the AlgOS framework integrated with Optuna’s Tree-Structured Parzen Estimator (TPE), we present strategies to refine hyperparameter search spaces, enhancing optimisation efficiency. Additionally, we introduce a novel SHAP-based interpretability approach tailored specifically for analysing hyperparameter impacts, offering clear insights into how individual hyperparameters and their interactions influence RL performance. Our work contributes practical guidelines and interpretability tools that significantly improve the effectiveness and computational feasibility of hyperparameter optimisation in reinforcement learning. 1 Introduction Hyperparameter optimisation (HPO) is a crucial component of the machine learning pipeline due to the significant impact hyperparameters have on model performance. Machine learning models possess numerous configurable settings, called hyperparameters, such as learning rates, hidden layer sizes, optimisation algorithms and their settings, regularisation parameters, and more. Unlike model parameters, hyperparameters are not directly learned during training but rather set prior to optimisation. These hyperparameters are often codependent, numerous, and complex to tune [ 1 ] . Thus, effective hyperparameter optimisation significantly enhances model performance and generalisation [ 2 , 3 , 4 , 5 , 6 ] . The optimisation of hyperparameters typically involves navigating high-dimensional, multimodal spaces that are prone to local minima. Common methods for hyperparameter optimisation include random search, grid search [ 7 ] , Bayesian optimisation [ 8 ] , and evolutionary algorithms [ 9 ] . However, these approaches often require considerable computational resources and extensive evaluation budgets. Additionally, hyperparameter spaces frequently comprise mixtures of discrete and continuous variables, prohibiting gradient-based optimisation. Reinforcement learning (RL), a subfield focused on training agents to make decisions by maximising cumulative rewards, poses particular challenges for hyperparameter optimisation due to the inherent complexity and sensitivity of RL algorithms to hyperparameters such as discount factors, exploration rates, and neural network architectures [ 10 ] . Successful RL algorithms, including AlphaGo [ 11 ] and AlphaStar [ 12 ] , illustrate the crucial role of HPO, yet these successes often rely heavily on vast computational resources unavailable to many practitioners. Furthermore, existing literature frequently discusses HPO superficially, lacking detailed methodological insights into interpreting hyperparameter interactions and effects, often relying on automated frameworks like AutoML [ 13 ] without deeper exploration. In this paper, we focus on hyperparameter optimisation and analysis within probabilistic curriculum learning (PCL) [ 14 ] , as it is a reinforcement learning algorithm which are known for their sensitivity to hyperparameters [ 15 , 16 ] . We present empirical results conducted on RL benchmark tasks, such as point-maze navigation and DC motor control, using the AlgOS framework [ 17 ] . AlgOS provides valuable HPO support, including unified interfaces for hyperparameter bounding, structured logging of experimental metadata for post-hoc analysis, and direct integration with the Optuna hyperparameter optimisation framework [ 18 ] . Our primary contributions include: 1. A thorough empirical analysis of hyperparameter impacts and interactions within the PCL algorithm, including demonstrating empirically hyperparameters have the described effects. 2. Practical guidelines and strategies to refine hyperparameter bounds, improving both optimisation efficiency and model performance. 3. Development and demonstration of a novel SHAP-based interpretability approach specifically designed for analysing hyperparameter importance in reinforcement learning tasks. Through these contributions, we aim to provide clear, actionable insights into hyperparameter optimisation and interpretation, addressing a crucial gap in current reinforcement learning and hyperparameter optimisation research. 2 Background 2.1 Probabilistic Curriculum Learning Probabilistic Curriculum Learning (PCL) is a curriculum learning strategy that structures the learning process of reinforcement learning agents by introducing tasks of increasing complexity. PCL employs a probabilistic approach to sample tasks from a distribution, allowing the agent to learn from simpler tasks before progressing to more complex ones. This method has been shown to improve the efficiency and effectiveness of the learning process in continuous environments [ 14 ] . PCL uses a deep mixture density network [ 19 , 20 , 21 ] (MDN) to model the likelihood of reaching a goal state given the current state and action. The MDN is trained to predict the distribution of possible next states, allowing the agent to sample from this distribution to select actions that are more likely to lead to successful outcomes. This probabilistic approach enables the agent to adaptively adjust its learning strategy based on the complexity of the tasks it encounters. 2.2 Hyperparameter Optimisation via Tree-Structured Parzen Estimators Bayesian optimisation techniques, such as the Tree-Structured Parzen Estimator (TPE), have become popular for hyperparameter optimisation due to their efficiency and ability to handle complex hyperparameter spaces [ 22 ] . TPE iteratively builds probabilistic models of the objective function based on previously observed hyperparameter configurations, efficiently guiding the search toward promising regions of the parameter space. The TPE algorithm starts by sampling initial hyperparameter configurations, often via random or Latin hypercube sampling, to evaluate the objective function. It then constructs two probabilistic models: one density function l(x)l(x), estimating configurations with low observed loss, and another density function g(x)g(x), estimating configurations associated with high observed loss. The algorithm uses the Expected Improvement (EI) criterion defined as: \text{EI}(x)=\frac{l(x)}{g(x)} (1) This criterion preferentially selects hyperparameter configurations predicted to yield better (lower loss) results. At each iteration, the TPE algorithm maximises the EI to determine the next hyperparameter configuration to evaluate. The process continues until a predefined stopping criterion is met. TPE’s primary advantages include scalability, flexibility, and its non-parametric nature. However, TPE can become computationally intensive for very high-dimensional parameter spaces, and performance may be sensitive to the choice of initial sampling methods and hyperparameter space definitions [ 22 ] . In this work, we specifically utilise TPE as implemented by Optuna due to its robustness, scalability, and effective integration within the AlgOS experimental framework. 2.3 Interpretability with SHAP (SHapley Additive exPlanations) SHapley Additive exPlanations (SHAP) is a popular interpretability method that uses Shapley values from cooperative game theory to allocate contributions to model predictions among individual input features [ 23 ] . Shapley values ensure a fair distribution of importance among features by averaging marginal contributions across all possible feature coalitions. Unlike traditional sensitivity analyses, which often fail to capture nonlinear interactions or complex dependencies among variables, SHAP provides a rigorous theoretical basis for interpreting the contributions of each feature. SHAP has been effectively employed in diverse fields including healthcare diagnostics, environmental modelling, concrete strength estimation, diesel yield optimisation, and polymer characteristic predictions [ 24 , 25 , 26 , 27 , 28 ] . In the context of hyperparameter optimisation, SHAP offers a powerful way to elucidate the relative importance and interactions among hyperparameters on model performance, something often neglected or superficially addressed in reinforcement learning literature. Our work presents a novel SHAP-based analysis methodology explicitly tailored to evaluating hyperparameter contributions in reinforcement learning,