Risk Bounds For Distributional Regression

Carlos Misael Madrid Padilla, Oscar Hernan Madrid Padilla, Sabyasachi Chatterjee · 2025

Read Risk Bounds For Distributional Regression by Carlos Misael Madrid Padilla, Oscar Hernan Madrid Padilla, Sabyasachi Chatterjee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-09075.txt.

Introduction

Carlos Misael Madrid Padilla, Oscar Hernan Madrid Padilla, and Sabyasachi Chatterjee bound the risk of methods that predict a whole probability distribution instead of a single number. Distributional regression serves forecasts where the spread matters as much as the center. Writing in twenty twenty-five, the team proves general upper bounds for convex-constrained estimators. Two yardsticks are used: a proper scoring rule for distributions, and the worst-case squared error across the domain. The theory specializes neatly. Isotonic and trend-filtering versions inherit convergence rates matching those of ordinary mean estimation.

A separate bound handles non-convex constraints, with neural network estimators as the showcase application. Simulations and real data back the theory in practice. The paper gives distribution forecasters the same kind of guarantees that point forecasters have long enjoyed.

Catalog record

CREATORCarlos Misael Madrid Padilla, Oscar Hernan Madrid Padilla, Sabyasachi Chatterjee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE124 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-09075

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Risk Bounds For Distributional Regression Carlos Misael Madrid Padilla, Oscar Hernan Madrid Padilla, Sabyasachi Chatterjee Abstract This work examines risk bounds for nonparametric distributional regression estimators. For convex-constrained distributional regression, general upper bounds are established for the continuous ranked probability score (CRPS) and the worst-case mean squared error (MSE) across the domain. These theoretical results are applied to isotonic and trend filtering distributional regression, yielding convergence rates consistent with those for mean estimation. Furthermore, a general upper bound is derived for distributional regression under non-convex constraints, with a specific application to neural network-based estimators. Comprehensive experiments on both simulated and real data validate the theoretical contributions, demonstrating their practical effectiveness. 1 Introduction While regression methods are widely popular across statistics and machine learning, it is well recognized that the conditional mean alone often fails to capture the full relationship between a response variable and a set of covariates. As noted by Shaked and Shanthikumar 2007 , “the ultimate goal of regression analysis is to obtain information about the conditional distribution of a response given a set of explanatory variables.” A common framework is quantile regression, which estimates conditional quantiles to provide a more detailed view of the response distribution [ 34 ] . A more direct approach is distributional regression, which estimates the conditional distribution of the response given the covariates. Distributional regression has found applications in diverse areas, including electricity spot price analysis [ 32 ] , understanding income determinants [ 33 ] , modeling weather data [ 62 ] , and improving precipitation forecasts [ 27 , 54 ] . The estimation of distributions of random variables under structural constraints is a fundamental problem in many statistical and machine learning tasks, including nonparametric regression, density estimation, and probabilistic forecasting [ 26 , 61 , 23 ] . Consider the sequence distributional model, in which we observe independent random variables y_{1},\ldots,y_{n}\in\mathbb{R} , each drawn from an unknown distribution F_{1}^{*},\ldots,F_{n}^{*} , respectively. The objective of this paper is to estimate the vector F^{*}(t)=(F_{1}^{*}(t),\ldots,F_{n}^{*}(t))^{\top} for t\in\mathbb{R} . Here, F_{i}^{*}(t)=\mathbb{P}(y_{i}\leq t) represents the cumulative distribution function (CDF) at a specified t for each observation y_{i} . In this paper, we will explore different structural constraints to estimate F^{*}(t) . These constraints not only ensure interpretable and robust estimators but also prevent overfitting, making them essential in domains such as signal processing, medical diagnostics, and probabilistic weather forecasting. For example, in survival analysis, monotonicity reflects the cumulative nature of survival probabilities, while in genomics, smoothness helps capture gradual trends in gene expression data [ 3 , 61 ] . To rigorously evaluate the quality of the estimators, we employ the continuous ranked probability Score (CRPS), a widely used metric for assessing the accuracy of probabilistic forecasts [ 22 ] . By quantifying the distance between the estimated and true CDFs, CRPS provides an interpretable and robust framework for comparing estimators under various structural constraints. 1.1 Summary of Results We now provide a brief summary of the contributions in this paper. Unified Framework for Estimation: We study a unified framework for estimating F^{*}(t) by minimizing a quadratic loss over a convex set K_{t}\subset\mathbb{R} . The convex set K_{t}\subset\mathbb{R} enforces structural constraints on the parameter F^{*}(t) . For the resulting estimator, we provide rigorous theoretical guarantees, including non-asymptotic bounds on both the mean squared error (MSE) and the CRPS. Applications to Monotonicity and Bounded Variation: We demonstrate the applicability of our framework to convex constraints arising in monotonicity [ 9 , 2 , 12 ] and bounded total variation [ 41 , 61 , 23 ] . These examples illustrate the flexibility and practical utility of the proposed approach for structured parameter estimation. General Theory for Distributional Regression: We establish a general theory for distributional regression with constraints encoded by arbitrary sets. The main result provides a uniform bound on the empirical \ell_{2} error of F^{*}(t) across t , with the upper bound explicitly dependent on the approximation error and the complexity of the sets K_{t} . Convergence Rates for Neural Networks: Exploiting the general results for arbitrary sets, we derive convergence rates for distributional regression using dense neural networks. This extends the framework of Kohler and Langer 2021 to the context of distributional regression. 1.2 Other Related Work Distributional regression involves modeling the cumulative distribution function (CDF) of a random variable, whereas quantile regression focuses on estimating the inverse CDF. Koenker et al. 2013 provides a comprehensive review of the distinctions between these two approaches. For instance, if the outcome variable is income and the covariate is a binary variable representing educational attainment, distributional regression would model the probability that income falls below a certain threshold for each educational group. In contrast, quantile regression would estimate income differences between individuals ranked at the same quantile within the two groups. For further discussion on these differences, see also Peracchi 2002 . The approach we adopt in this paper is based on modeling the mean of the random variables of the form 1_{\{y_{i}\leq t\}} , which represent the indicator of the events \{y_{i}\leq t\} for i=1,\ldots,n . This idea was first introduced in [ 20 ] and has since been explored in various contexts, including Firpo and Sunao 2023 , Rothe 2012 , Rothe and Wied 2013 , and Chernozhukov et al. 2013 . More recently, distributional regression has been studied in diverse settings, such as isotonic regression [ 27 ] , random forests [ 10 ] , and neural networks [ 57 , 30 ] . These works highlight the versatility and applicability of distributional regression across a range of methodologies and problem domains. Finally, other nonparametric approaches to distributional regression include Dunson et al. 2007 and Hall et al. 1999 . 2 Notation For an event A , we denote by 1_{A} its indicator, which takes value 1 if A holds and zero otherwise. Througout, for a vector v\in\mathbb{R}^{n} , we denote by \|v\| and \|v\|_{1} the \ell_{2} and \ell_{1} norms, respectively. Thus, \|v\|=\sqrt{\sum_{i=1}^{n}v_{i}^{2}} and \|v\|_{1}\,=\,\sum_{i=1}^{n}|v_{i}| . Furthermore, given two functions G,H\,:\,\mathbb{R}\rightarrow\mathbb{R} , we define \mathrm{CRPS}(G,H)=\int_{-\infty}^{\infty}(G(t)-H(t))^{2}dt. Also, for for \eta>0 and v\in\mathbb{R}^{n} , we write B_{\eta}(v)\,:=\,\left\{u\in\mathbb{R}^{n}\,:\,\|v-u\|\leq\eta\right\}. For a metric space (\mathcal{X},d) , let K be a subset of \mathcal{X} , and r>0 be a positive number. Let B_{r}(x,d) be the ball of radius r with center x\in\mathcal{X} . We say that a subset C\subset\mathcal{X} is an r -external covering of K if K\,\subset\,\cup_{x\in C}B_{r}(x,d). Then external covering number of K , written as N(r,K,d) , is defined as the minimum cardinality of any r -external covering of K . Furthermore, for a function f\,:\,\mathcal{X}\rightarrow\mathbb{R} , we define its \ell_{\infty} norm as \|f\|_{\infty}\,:=\,\sup_{x\in\mathcal{X}}|f(x)| . Also, for two sequences a_{n} and b_{n} , we write a_{n}\lesssim b_{n} if a_{n}\leq cb_{n} for a positive constant c , and if a_{n}\lesssim b_{n} and b_{n}\lesssim a_{n} , then we write a_{n}\asymp b_{n} . The indicator function of a set A is denoted as 1_{A}(t) , which takes value 1 if t\in A and 0 otherwise. Finally, the indicator of an event A is 1_{A} which takes value 1 if A holds and 0 otherwise. 2.1 Outline The paper is organized as follows: Section 3 introduces the methodology for the convex case. In particular, Section 3.1 establishes a unified framework

Read the complete text