HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data

Jannis Maier [email protected], Lennart Purucker [email protected] · 2026

Read HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data by Jannis Maier [email protected], Lennart Purucker [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10582.txt.

Introduction

Jannis Maier and Lennart Purucker introduce HAPEns, a post-hoc ensembling method for tabular data that weighs accuracy against hardware cost. Bigger ensembles predict better but demand more from the machine, and no hardware-aware baseline existed. Borrowing from multi-objective and quality-diversity ideas, HAPEns builds a spread of ensembles along the Pareto front of performance and resource use. On 83 tabular datasets it beat baselines at finding good trade-offs, and memory use proved an especially telling cost measure.

Catalog record

CREATORJannis Maier [email protected], Lennart Purucker [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34916
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.10582

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data Jannis Maier [email protected], Lennart Purucker [email protected] Abstract Ensembling is commonly used in machine learning on tabular data to boost predictive performance and robustness, but larger ensembles often lead to increased hardware demand. We introduce HAPEns, a post-hoc ensembling method that explicitly balances accuracy against hardware efficiency. Inspired by multi-objective and quality diversity optimization, HAPEns constructs a diverse set of ensembles along the Pareto front of predictive performance and resource usage. Existing hardware - aware post - hoc ensembling baselines are not available, highlighting the novelty of our approach. Experiments on 83 tabular classification datasets show that HAPEns significantly outperforms baselines, finding superior trade-offs for ensemble performance and deployment cost. Ablation studies also reveal that memory usage is a particularly effective objective metric. Further, we show that even a greedy ensembling algorithm can be significantly improved in this task with a static multi-objective weighting scheme. Figure 1 : Illustration of three ensemble selection strategies: a standard method ignoring hardware constraints, a naive hardware-aware variant that sacrifices accuracy, and an advanced hardware-aware method that balances accuracy and efficiency. Box size reflects model resource usage; the red dashed line indicates the hardware resource constraint. 1 Introduction Ensembling is a central technique in machine learning, used to improve predictive performance, stability, and robustness across a wide range of applications. From boosting and bagging in classical supervised learning to stacking in modern deep learning workflows, ensembles are frequently adopted to combine the strengths of diverse models. In many practical scenarios, models produced during training or exploratory analysis are later combined into ensembles in a post-hoc fashion to substantially improve performance ( 8 ; 1 ) . This workflow has been further popularized by automated machine learning (AutoML) systems for tabular data ( 17 ; 13 ; 7 ) , where greedy ensemble selection (GES) by 5 has emerged as a widely used method to automatically build strong ensembles from model libraries. While post-hoc ensembling generally improves predictive performance, larger ensembles lead to increased hardware demands at inference time. Each additional model increases prediction latency and resource consumption, inducing higher costs. While this matters greatly in production settings, it is ignored by standard post-hoc ensembling methods. As machine learning is increasingly deployed in environments with tight resource constraints, the gap between high predictive accuracy and hardware feasibility has become more pronounced. We address this challenge by introducing HAPEns, a post-hoc ensembling method that explicitly balances predictive performance against hardware costs. It improves on existing baselines by constructing Pareto fronts of ensembles that more effectively balance competing objectives. Thus, practitioners can select better models that satisfy both performance and deployment requirements under their specific hardware constraints. Drawing inspiration from multi-objective optimization ( 12 ) and quality diversity optimization ( 16 ) , HAPEns maintains a diverse population of ensembles that vary in both hardware cost and predictive behavior, while optimizing for predictive performance. The result is a set of candidate ensembles that offer distinct trade-offs between both objectives. To evaluate HAPEns, we performed experiments on 83 tabular classification datasets of varying size and complexity. All cost metrics are concrete measurements taken on the same system, held fixed across methods for fair comparison. We compare ensembles constructed by our method to those selected by baselines like GES and a novel multi-objective baseline. Our findings reveal that optimizing for memory footprint is a particularly effective metric for deployment cost and that our method significantly outperforms competitors in balancing hardware costs and predictive performance. To our knowledge, this is the first systematic study of hardware-aware post-hoc ensemble selection. Prior hardware-aware work focuses on model search or NAS, not ensemble construction from fixed model libraries. Our Contributions. In this work, we: (i) Propose a novel post-hoc ensembling algorithm that explicitly incorporates hardware cost into the selection process; (ii) Demonstrate through extensive benchmarking that our method achieves superior accuracy–cost trade-offs compared to existing baselines; (iii) Show that memory-awareness yields substantial gains even in inference-time efficiency; (iv) Limitations, including the dependence on a single hardware configuration, are discussed, with directions for extending the method to heterogeneous devices. (v) Ensure reproducibility by open-sourcing all code 1 1 1 All code used for this publication is available at: https://anonymous.4open.science/r/C07F , results, and integration with popular ensembling frameworks. 2 Related Work Figure 2 : Overview of the main research areas. HW-NAS (red) is shown as a parallel area, while the others (orange) directly influence HAPEns. This work focuses solely on tabular data (blue). Ensembling—combining multiple pre-trained models—is an effective approach to improve predictive performance and robustness. Common strategies include bagging, stacking, and ensemble selection (ES). Bagging and stacking are typically integrated into the training process, whereas ES can be applied post hoc, that is, after model training has completed. ES might then also be referred to as post hoc ensembling. Figure 2 provides an overview of the research fields discussed in the following paragraphs. ES as introduced by 5 is a forward selection algorithm that greedily constructs an ensemble by iteratively adding the model that improves the predictive performance of the ensemble the most. The resulting ensemble is defined by a weight vector derived from this superset of selected models. In this work, we adopt a broader interpretation of ensemble selection: Any algorithm that produces such a weight vector from a pool of trained models qualifies as ES. To distinguish the classical algorithm of Caruana et al., we refer to it as greedy ensemble selection (GES). Post-hoc ensembling is a widely adopted component in automated machine learning (AutoML) systems, particularly for tabular data ( 7 ; 10 ; 17 ) . It enables the reuse of models generated during training without retraining, making it a computationally attractive final optimization step. Although ensemble selection can theoretically be used during training, we reserve the term ES for its post-hoc usage in this work. The term blending also appears in this context, but it specifically refers to ensemble selection applied to a holdout validation set distinct from the training data. Recent years have seen the integration of multi-objective optimization (MOO) into various stages of the machine learning and AutoML pipelines, including neural architecture search (NAS) ( 2 ; 3 ) . These methods optimize for trade-offs such as accuracy versus latency, energy consumption, or memory usage. However, the use of MOO techniques in post-hoc ensemble selection remains largely unexplored. 20 introduced DivBO, a diversity-aware Bayesian optimization framework that incorporates ensemble selection during candidate evaluation to promote both accuracy and diversity. Although their approach targets the model search stage rather than post-hoc optimization, it highlights the potential of multi-objective formulations to improve ensemble composition. Nevertheless, to the best of our knowledge, no prior work systematically investigates the construction of Pareto-optimal ensembles that explicitly account for hardware constraints such as inference time or memory usage. Modern implementations of GES—still the de facto standard in AutoML frameworks such as Auto-sklearn ( 9 ) and AutoGluon ( 7 ) —typically optimize only for predictive performance and remain agnostic to deployment cost. Consequently, they may produce ensembles that are unnecessarily large or infeasible for deployment due to hardware requirements. Our work addresses this identified gap by introducing a hardware-aware approach to ES, explicitly targeting the trade-off between

Read the complete text