Neural Generative Distributional Regression

Jinhang Chai, Jianqing Fan, Yihong Gu · 2026

Read Neural Generative Distributional Regression by Jinhang Chai, Jianqing Fan, Yihong Gu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02062.txt.

Introduction

Jinhang Chai, Jianqing Fan, and Yihong Gu estimate a full conditional distribution with a neural generative method. Any such distribution can be produced by transforming a known noise variable, such as a uniform or normal draw, through a function of the inputs. The authors estimate that transform by minimising the energy distance between the true outcome distribution and the generated one.

Neural networks implement the estimator and can adapt to low-dimensional structure in the data on their own. The estimated distribution then serves tasks such as conditional moments, prediction intervals, and density estimation. Theory gives an oracle inequality with adaptive optimal rates for nonparametric problems. Simulations and a real data analysis show the method works in practice.

Catalog record

CREATORJinhang Chai, Jianqing Fan, Yihong Gu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE152 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.02062

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Neural Generative Distributional Regression Jinhang Chai, Jianqing Fan, Yihong Gu Abstract Any continuous conditional distribution of Y Y given X X can be generated from a transform of a known noise distribution U U such as the uniform or normal distribution via Y = g ⁡ ( X , U ) Y=g(X,U) . This paper provides an estimator of such a generative transformation g g by minimizing the empirical energy distance between distributions of Y Y and g ⁡ ( X , U ) g(X,U) , and implements it via neural networks. The estimated distribution can then be readily applied to downstream tasks such as conditional moment estimation, predictive interval construction, and conditional density estimation. By leveraging the representation power of neural networks, the estimator can adaptively exploit low-dimensional structures in a purely algorithmic manner. Theoretically, we establish an oracle inequality attaining the adaptive optimal nonparametric rates. Numerical simulations and real data analysis further demonstrate the practical effectiveness of the proposed method. Keywords : Energy Statistics, Neural Networks, Distributional Regression, Generative Model 1 Introduction Deep learning has achieved tremendous success across a wide range of domains, including computer vision ( 74 ) , game playing ( 64 ) , and natural language processing ( 7 ) . The success of deep neural network models is often attributed to their remarkable ability to represent and approximate complex, high-dimensional functions. From a theoretical standpoint, this perspective aligns naturally with statistical learning theory, where deep networks have been shown to possess powerful approximation and generalization capabilities. Notably, 60 demonstrated that neural networks, with carefully designed architectures, can achieve the minimax-optimal rate for nonparametric regression while adaptively exploiting compositional structures in the target function. Traditional nonparametric regression ( 19 ; 29 ; 71 ) focuses on estimating the conditional mean function. However, the conditional mean captures only a limited view of the relationship between covariates and responses ( 61 ; 32 ) . In many applications, richer aspects of the conditional distribution are of primary interest. For instance, conditional quantiles are crucial in heavy-tailed settings or risk-sensitive decision-making ( 76 ; 25 ) ; prediction intervals are indispensable for uncertainty quantification in fields such as economics ( 14 ) and meteorology ( 72 ) ; and conditional score functions or higher-order moments play key roles in semi-parametric inference ( 13 ) and financial risk modeling ( 31 ) . Generative distributional regression directly estimates the conditional distribution of the response Y given the covariate X instead of a fixed point estimate. There is a considerable literature proposing parametric or semi-parametric methods for distributional regression ( 57 ; 36 ) . Although the effects of the covariates on the conditional distribution can be modeled nonparametrically, for example, using neural networks ( 40 ; 42 ) , these methods do assume the conditional distribution belongs to certain parametric families. This may risk model misspecification and lack scalability under complex data-generating processes. With the development of modern machine learning models, including neural networks and tree-based methods, there is also a considerable literature on realizing distributional regression using these nonparametric techniques and a regression-type loss that shares a similar spirit to L_{2} and L_{1} losses in that the computation is easy and thus the implementation is similar to running conditional mean or conditional quantile regression, both in optimization stability and computation time. Two main types of loss are under consideration: the Continuous Ranked Probability Score (CRPS) ( 34 ; 15 ; 53 ) and the conditional Maximum Mean Discrepancy (MMD) loss ( 55 ; 37 ; 8 ; 10 ) . However, it is still unclear whether these losses can be realized for both sample-efficient and computation-efficient distributional regression in theory. There is some work establishing the statistical rate of convergence for the estimators using CRPS loss, including the isotonic regression ( 34 ) , and kernel estimators ( 54 ) ; the latter suffers from the curse-of-dimensionality in the covariate dimension d . 53 shows that a variant of CRPS loss, integrated by neural networks, can efficiently adapt to the low-dimensional structure in the conditional CDF function. However, only the in-sample error of CDF, i.e., \frac{1}{n}\sum_{i=1}^{n}\int\{\widehat{F}(t|X=X_{i})-F^{\star}(t|X=X_{i})\}^{2}dt , is established, and it is unclear how to simulate or estimate the conditional distribution of Y for new observed x and the correspondings theoretical guarantees. Furthermore, this estimation procedure is not computation-efficient, given that one needs to run the loss for different quantiles t , and thus is conceptually similar to running quantile regressions with \Omega(\mathrm{poly}(n)) cumulative probabilities. 10 establishes a convergence rate for a neural network using a conditional MMD loss. However, it explicitly uses the nearest neighbour in the estimation procedure, which slows the computation and inevitably leads to a statistical rate that suffers from the curse of dimensionality in d . In this paper, we fill the above gap by showing that a neural generative distributional regression method, which uses regression-type loss, with similar computation and implementation as regression, can fully leverage the neural network’s sample efficiency in modeling conditional distributions and further leads to accurate downstream conditional quantities estimation. The core idea, which dates back to the conditional generative adversarial network ( 48 ) , is to represent the conditional law of Y|X by transforming an exogenous noise variable through a deep neural network, i.e., Y=g(X,U) with the covariate X\in\mathbb{R}^{d} and some noise U . This implicit formulation unifies various estimation objectives – conditional means, quantiles, moments, or densities – into a single modeling framework. Once trained, downstream quantities can be efficiently estimated via Monte Carlo sampling from the fitted conditional simulator. While we consider the general kernel function in Appendix D , we focus on the specific instance where the estimator is an empirical risk minimizer of a regression-type loss – energy distance ( 69 ) loss. This loss is a realization of the average MMD loss ( 37 ) with first-order Sobolev kernel, or constrained kernel K(x,y)=-|x-y| , and is also an unbiased estimate of the CRPS loss. The estimator is conceptually the same as the Engression ( 63 ) when the response Y is one-dimensional. We provide a thorough and sharp non-asymptotic analysis of the proposed method. We summarize the key findings below. • We establish the non-asymptotic oracle inequalities akin to regression for the proposed estimator. Moreover, in a similar spirit to the conditional moment regression, we show that as long as the conditional quantile function admits the form of a hierarchical composition model ( 3 ; 60 ; 21 ) , the estimator can get an optimal, no curse-of-dimension rate of convergence in the L_{2} distance of the induced CDF. • Our error bound offers an explicit dependence on the number of auxiliary noises U . We show that a constant number of noise samples per batch, in particular, two, is sufficient for an optimal rate of convergence. This distinguishes significantly from prior distributional estimation methods that require a growing number of noise/quantile values to attain consistency. • Although the conditional distribution simulator \widehat{g} only has error guarantees of L_{2} error in conditional CDFs functions, which is a relatively weak distance compared with, for example, TV distance, we show the simulator can successfully result in (1) optimal error rates in downstream

Read the complete text