Semi-parametric bulk and tail regression using spline-based neural networks

Reetam Majumder · 2025

Read Semi-parametric bulk and tail regression using spline-based neural networks by Reetam Majumder (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19994.txt.

Introduction

Reetam Majumder and co-authors build a regression method that models both the bulk of data and its extreme tail, in a paper from 2025. Many real data sets, from wildfire areas to finance, have heavy tails where rare huge values matter most. A flexible spline-based method fit the bulk well but failed in the tail and broke known extreme-value rules.

The team creates a blended distribution that covers the full range of data. It glides smoothly into an exact extreme-value tail, with no need to pick a cutoff by hand. Joined to the spline method, it gives reliable estimates far beyond the observed range. Scores that rate each input's role can be read apart for the bulk and the tail. Tests use simulations and United States wildfire areas over three decades.

Catalog record

CREATORReetam Majumder
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-19994

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Semi-parametric bulk and tail regression using spline-based neural networks Reetam Majumder Abstract Semi-parametric quantile regression (SPQR) is a flexible approach to density regression that learns a spline-based representation of conditional density functions using neural networks. As it makes no parametric assumptions about the underlying density, SPQR performs well for in-sample testing and interpolation. However, it can perform poorly when modelling heavy-tailed data or when asked to extrapolate beyond the range of observations, as it fails to satisfy any of the asymptotic guarantees provided by extreme value theory (EVT). To build semi-parametric density regression models that can be used for reliable tail extrapolation, we create the blended generalised Pareto (GP) distribution, which i) provides a model for the entire range of data and, via a smooth and continuous transition, ii) benefits from exact GP upper-tails without the need for intermediate threshold selection. We combine SPQR with our blended GP to create semi-parametric quantile regression for extremes (SPQRx), which provides a flexible semi-parametric approach to density regression that is compliant with traditional EVT. We handle interpretability of SPQRx through the use of model-agnostic variable importance scores, which provide the relative importance of a covariate for separately determining the bulk and tail of the conditional density. The efficacy of SPQRx is illustrated on simulated data, and an application to U.S. wildfire burnt areas from 1990–2020. Keywords: blended generalised Pareto distribution, conditional density estimation, deep extreme quantile regression, explainable AI, extreme value theory 1 Introduction Extreme quantile regression models provide a useful tool for risk assessment in a wide range of applied fields, such as econometrics ( Chavez-Demoulin and Guillou, 2018 ; Daouia et al., 2019 ; Daouia et al., 2023 ) , insurance ( Daouia et al., 2024 ) , natural hazard modelling ( Richards and Huser, 2026b ; Yadav et al., 2023 ) , chemometrics ( Gardes and Stupfler, 2019 ) , flood risk assessment ( Jóhannesson et al., 2022 ; Pasche and Engelke, 2024 ) , and downscaling ( Wang et al., 2012 ) . A common approach to extreme quantile regression follows via a parametric assumption on the upper-tails of the conditional distribution of a response variable Y\in\mathbb{R} given a set of realised covariates \mathbf{x}\in\mathbb{R}^{p} of a random vector \mathbf{X} . The de-facto standard assumption is that excesses of Y above a high threshold u(\mathbf{x}) follow a generalised Pareto (GP) distribution (see, e.g., Coles (2001) for discussion) with covariate-dependent shape and scale parameters, \xi(\mathbf{x})\in\mathbb{R} and \sigma_{u}(\mathbf{x})>0 , respectively. That is, one assumes the parametric GP regression model: \bigg(Y-u(\mathbf{x})\bigg)\>|\;\bigg(Y>u(\mathbf{x}),\mathbf{X}=\mathbf{x}\bigg)\sim\mbox{GP}\bigg(\sigma_{u}(\mathbf{x}),\xi(\mathbf{x})\bigg), where the scale \sigma_{u}(\mathbf{x}) is dependent on the threshold u(\mathbf{x}) . A regression model of this form benefits from the asymptotic guarantees of the GP distribution ( Coles, 2001 , e.g., threshold stability; see) , and allows for reliable estimation of extreme conditional \tau -quantiles (for \tau\in(0,1) close to one); see Davison and Smith (1990) for details. The choice of the intermediate threshold u(\mathbf{x}) is an ubiquitous problem in extreme value theory (EVT) ( Murphy et al., 2025 , see, e.g.,) . In a regression setting, the threshold u(\mathbf{x}) can be modelled as a function of covariates. Choosing and estimating an appropriate model for u(\mathbf{x}) introduces uncertainty into the modelling procedure. Moreover, the additional complication that the scale \sigma_{u}(\mathbf{x}) depends on u(\mathbf{x}) makes interpretability of model estimates difficult ( Richards et al., 2023 ) . To overcome the problem of threshold choice, one can replace the parametric GP model (for threshold excesses) with a model for the entire support of data that retains (exact or approximate) GP upper-tails, without requiring threshold estimation. Many such univariate models have been proposed in the literature ( Scarrott and MacDonald, 2012 , see review by) . Examples include splicing or mixing of density functions ( Frigessi et al., 2002 ; Behrens et al., 2004 ; de Melo Mendes and Lopes, 2004 ; Carreau and Bengio, 2007 ; Reynkens et al., 2017 ; Castro-Camilo et al., 2019 , with at least one constituent density being the GP density; see, e.g.,) , blending of distribution functions ( Castro-Camilo et al., 2022 ; Krakauer, 2024 ) , or parametric generalisations of the GP distribution ( Papastathopoulos and Tawn, 2013 ; Naveau et al., 2016 ; Stein, 2021a ; Stein, 2021b ; de Carvalho et al., 2022 ) . Although these all-in-one models are compliant with traditional EVT, and can thus perform reliable extrapolation into the upper-tail, they often suffer in terms of flexibility as both the bulk and tail of the distribution are parametric. Bulk and tail models with non-parametric bodies have been proposed ( Tancredi et al., 2006 ; MacDonald et al., 2011 ; Tencaliec et al., 2020 , e.g.,) , but these are yet to be extended to the regression setting. After estimating u(\mathbf{x}) , a further modelling choice for GP regression is the functional forms of \sigma_{u}(\mathbf{x}) and \xi(\mathbf{x}) . Although classical GP regression models used linear models ( Davison and Smith, 1990 ; Wang et al., 2012 , e.g.,) , there has been growing interest in more flexible and computationally-scalable alternatives, including additive models ( Chavez-Demoulin and Davison, 2005 ; Youngman, 2019 ) , local kernel smoothing ( Daouia et al., 2011 ; Velthoen et al., 2019 ; Gardes and Stupfler, 2019 ) , decision trees ( Farkas et al., 2021 ; Farkas et al., 2024 ) , random forests ( Gnecco et al., 2024 ) , gradient boosting ( Koh, 2023 ; Velthoen et al., 2023 ) , and deep neural networks ( Wilson et al., 2022 ; Richards et al., 2023 ; Cisneros et al., 2024 ; Pasche and Engelke, 2024 ; Mackay et al., 2025 ) . In particular, deep parametric GP regression methods ( de Carvalho et al., 2026 , i.e., those that exploit neural networks for modelling \sigma_{u}(\mathbf{x}) and \xi(\mathbf{x}) ; see overview by Richards and Huser (2026a) , Ch. 21 of) are particularly appealing as they are highly flexible, scalable to high covariate dimension p , and can be estimated quickly using easily-accessible software packages ( Richards, 2022 , see, e.g.,) . However, existing deep GP regression models suffer from the same two limitations of all parametric GP regression models: i) they require modelling and estimation of the intermediate threshold u(\mathbf{x}) , and ii) they do not provide a model for the bulk (or body) and lower-tails of the conditional distribution, that is, Y\>|\;(Y<u(\mathbf{x}),\mathbf{X}=\mathbf{x}) . To create flexible, semi-parametric regression models that are EVT-compliant and capable of modelling the entire range of the response data, we introduce the blended GP (bGP) distribution. Similarly to the construction of the blended generalised extreme value distribution of Castro-Camilo et al. (2022) , we blend the GP distribution with an appropriately chosen distributional model for the bulk of the data. The bGP distribution can model the entire range of data, and has exact GP upper-tails without the need for threshold selection. We utilise the blended GP for regression by combining a deep GP regression model, for the upper-tails, with a deep semi-parametric regression model for the bulk of the distribution. For the latter, we employ the semi-parametric quantile regression (SPQR) model of Xu and Reich (2021) , which has been used previously for inference with spatial extremal processes ( Majumder and Reich, 2023

Read the complete text