ParamBoost: Gradient Boosted Piecewise Cubic Polynomials
Nicolas Salvadé, Tim Hillel · 2026
Read ParamBoost: Gradient Boosted Piecewise Cubic Polynomials by Nicolas Salvadé, Tim Hillel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18864.txt.
Introduction
Salvadé and Hillel present ParamBoost, a generalized additive model whose shapes are learned by gradient boosting that fits cubic polynomials at leaf nodes. Glass box interpretability alone does not let modellers add expert knowledge, so it adds constraints on continuity, monotonicity, convexity, interactions, and specification. The unconstrained model beats leading GAMs on several real world data sets. They can also impose chosen constraints at a small cost to predictive performance.
Catalog record
| CREATOR | Nicolas Salvadé, Tim Hillel |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43166 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.18864
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ParamBoost: Gradient Boosted Piecewise Cubic Polynomials Nicolas Salvadé, Tim Hillel Abstract Generalized Additive Models (GAMs) can be used to create non-linear glass-box (i.e. explicitly interpretable) models, where the predictive function is fully observable over the complete input space. However, glass-box interpretability itself does not allow for the incorporation of expert knowledge from the modeller. In this paper, we present ParamBoost , a novel GAM whose shape functions (i.e. mappings from individual input features to the output) are learnt using a Gradient Boosting algorithm that fits cubic polynomial functions at leaf nodes. ParamBoost incorporates several constraints commonly used in parametric analysis to ensure well-refined shape functions. These constraints include: (i) continuity of the shape functions and their derivatives (up to C 2 C^{2} ); (ii) monotonicity; (iii) convexity; (iv) feature interaction constraints; and (v) model specification constraints. Empirical results show that the unconstrained ParamBoost model consistently outperforms state-of-the-art GAMs across several real-world datasets. We further demonstrate that modellers can selectively impose required constraints at a modest trade-off in predictive performance, allowing the model to be fully tailored to application-specific interpretability and parametric-analysis requirements. [orcid=0009-0000-0961-9844] [orcid=0000-0001-6872-2235] † † highlights: ParamBoost is a new GAM that uses Gradient Boosting with cubic polynomials to learn shape functions. Existing glass-box models lack support for incorporating expert knowledge. It supports constraints like monotonicity, convexity, and smooth continuity. The unconstrained model outperforms state-of-the-art GAMs on real-world datasets. Constraints can be selectively applied with only a modest trade-off in accuracy. keywords Machine Learning ,Gradient Boosting ,Generalized Additive Models ,Interpretability ,Shape Function Constraints † † credit: Writing – original draft, Visualization, Software, Methodology, Conceptualization, Formal analysis, Investigation † † credit: Writing – review & editing, Supervision, Methodology, Conceptualization, Project administration † † corresponding: Corresponding author 1 Introduction Machine Learning (ML) methods (and Deep Neural Networks (DNNs) in particular) offer improved predictive capabilities over traditional parametric models. However, many applications still require parameter-based interpretations, where practitioners rely on gradients (i.e., marginal effects or elasticities ) to quantify how a feature (or exogenous variable ) influences the output. Such analysis is crucial in domains including: (i) transportation modelling ( 49 ; 41 ) ; (ii) econometrics ( 56 ) ; (iii) health ( 54 ) ; and (iv) structural equation modelling and energy demand ( 15 ) . Recent years have seen a growing focus on interpretable ML, leading to the establishment of the field of eXplainable Artificial Intelligence (XAI). XAI techniques largely fall into two different approaches: (i) post-hoc techniques that approximate a black-box model; and (ii) intrinsically interpretable glass- (or white-) box models. Among post-hoc methods, permutation-based techniques such as LIME ( 39 ) and SHAP ( 27 ) , have been most widely used, with applications in healthcare ( 13 ; 55 ; 28 ) , finance ( 16 ; 25 ; 2 ) , road safety ( 51 ; 33 ) and travel behaviour ( 43 ; 10 ; 30 ; 38 ; 17 ; 40 ) . However, as these methods rely on local approximations of the model predictive functions, they only provide valid explanations at observed data points and offer no guarantee of validity over the full input space. On the other hand, recent work in intrinsically interpretable techniques has focused on Generalised Additive Models (GAMs) ( 19 ) , where each feature is associated with a non-linear shape function learnt from the data; commonly using splines or Gaussian processes (GPs) ( 50 ) , and more recently incorporating structural constraints ( 22 ) ). Other ML algorithms have also been adapted to GAMs, including scalable tree-based methods ( 8 ; 24 ; 9 ; 21 ; 14 ; 41 ) , NNs ( 1 ; 36 ; 52 ) , piecewise-linear models ( 48 ) and polynomial-based models ( 11 ; 12 ) , often achieving performance close to black-box models. Despite these advances, glass-box interpretability itself does not enable the incorporation of domain knowledge. We identify four essential constraints that, when applied to a GAM, allow for the integration of expert knowledge: (i) continuous differentiability of shape functions over the full input space, enabling computation of marginal effects and related indicators; (ii) restrictable feature interactions , enabling each feature’s effect on the predictive function to be represented and visualised with a single exact shape function; (iii) monotonicity or curvature (i.e. convex/concave) control of shape functions; and (iv) output-specific feature assignment in multi-output problems (e.g. multi-class and multi-label classification, multi-target regression, and other multitask problems), such that only relevant features influence each predictive function. For example, for disease prediction in blood tests, indicators related to a specific disease should only appear in the corresponding function (e.g. a white blood cell count should influence the blood cancer class only). While some existing methods satisfy some individual constraints above, none provide all four at the same time, limiting their usage in domains such as health, transport, and policy evaluation. Table 1 summarises the interpretability capabilities of state-of-the-art GAMs. Table 1 : Comparison of shape function constraint capabilities of state-of-the-art GAMs. Diff.: continuous differentiability of shape functions; FI: restrictable feature interactions; Mono./curv. monotonicity and/or curvature control of shape functions; OSF: output-specific feature assignment in multi-output problems Model Modelling tasks Typical base learner Diff. FI Mono./curv. OSF EBM ( 8 ; 9 ) Regr. and class. Trees ✗ ✓ \sim * ✗ EGBM ( 24 ) Regr. Randomised trees ✗ ✗ ✗ ✗ Stump Forests ( 14 ) Regr. and class. Trees (stumps) ✗ ✓ ✗ ✗ GRAND-SLAMIN ( 21 ) Regr. and class. Soft trees C^{0} ✓ ✗ ✗ APLR ( 48 ) Regr. and class. Linear trees C^{0} ✓ \sim * ✗ NAM ( 1 ) Regr. and class. Neural networks ✗ ✓ ✗ ✗ GAMI-Net ( 52 ) Regr. and class.** Neural networks ✗ ✓ ✗*** ✗ NBM ( 36 ) Regr. and class. Neural basis ✗ ✓ ✗ ✗ SPAM ( 11 ) Regr. and class. Polynomials ✗ ✗**** ✗ ✗ CAT ( 12 ) Regr. and class. Polynomials ✗ ✗ ✗ ✗ MGCV ( 50 ) Regr. and class. Splines C^{2} ***** ✓ ✗ ✓ ELAAN ( 22 ) Regr. Splines C^{2} ***** ✓ ✗ ✗ ParamBoost (our approach) Regr. and class. Cubic trees C^{2} ✓ ✓ ✓ *only monotonicity constraints **only binary classification ***based on the paper ****if interactions are turned off, the model falls back to linear/logistic regression *****with cubic splines To address these limitations, we propose ParamBoost , a novel parametric gradient boosting GAM that fits cubic polynomial functions at leaf nodes. This allows each shape function to be expressed as a polynomial up to cubic degree, enabling flexible non-linear effects while maintaining continuous gradients for computing marginal effects and confidence intervals. ParamBoost introduces four tunable model components: 1. Degree D of the polynomial trees O^{D} : up to the cubic degree ( O^{3} ), where D=0 reduces to classical Gradient Boosting Decision Trees (GBDT); 2. Smoothness S of shape functions C^{S} : , from discontinuous ( S=-1 ) to C^{2} (i.e. continuous second derivative), subject to S\leq D-1 ; 3. Monotonicity and curvature constraints: restricting each feature’s shape function to incorporate domain knowledge; and 4. Output-specific feature assignment: restricting which features impact which predictive functions, again incorporating domain knowledge