Generative Quantile Bayesian Prediction

Maria Nareklishvili, Nicholas Polson, Vadim Sokolov · 2025

Read Generative Quantile Bayesian Prediction by Maria Nareklishvili, Nicholas Polson, Vadim Sokolov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21784.txt.

Introduction

Maria Nareklishvili, Nick Polson, and Vadim Sokolov build a prediction method that learns quantiles rather than full odds curves. A quantile marks a level that future values should fall below with a chosen chance. Predicting quantiles therefore gives a direct read on risk. That helps with large forecasting problems.

Their method is called Generative Quantile Bayesian Prediction. It uses generative modeling to build maps from data to quantiles. The authors set it beside rival methods such as conformal prediction and fiducial prediction. They argue it has gains in both theory and practice. Worked examples cover the classic normal-mean learning setup. A causal inference problem is covered too. The paper ends by pointing to open directions for more research.

Catalog record

CREATORMaria Nareklishvili, Nicholas Polson, Vadim Sokolov
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21784

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generative Quantile Bayesian Prediction Maria Nareklishvili, Nicholas Polson, Vadim Sokolov Abstract Prediction is a central task of machine learning. Our goal is to solve large scale prediction problems using Generative Bayesian Prediction (GBP). By directly learning predictive quantiles rather than densities we achieve a number of theoretical and practical advantages. We contrast our approach with state-of-the-art methods including conformal prediction, fiducial prediction and marginal likelihood. Our distinguishing feature of our method is the use of generative methods for predictive quantile maps. We illustrate our methodology for normal-normal learning and causal inference. Finally, we conclude with directions for future research. Keywords: Generative methods, Bayesian Prediction, Deep Learning, Conformal Prediction, Quantile Neural Networks, Uncertainty Quantification 1 Introduction Prediction is a key task for modern day machine learning. Our goal is to provide a large scale predictive framework. Efron 2020 describes how this is a challenge for modern statistical methods and how pure black-box ( Breiman 2001 ) methods such as deep learning, neural networks and random forests can outperform traditional statistical regression methods. Hill 1968 proposed Bayesian inference approach that does not require specification of a prior, later Lei and Wasserman 2014 extended it to the case of regression. We focus on generative Bayesian methods that directly model the predictive quantile function and hence are likelihood- and -prior-free. An important property in high-dimensional problems as was recently pointed out by Ritov 2025 . Our approach builds on the quantile inference framework developed by Parzen (2004, 2009) which we describe in detail. Our approach provides an alternative to conformal prediction methods used extensively in machine learning. The basic insight of conditional generative modeling is that is can be performed by non-parametric quantile regression White 1992 . Specifically, we wish to find a family of quantile functions to describe the predictive distribution of an output variable Y given X given by F_{Y|X}^{-1}(\tau|x):(\tau,x)\rightarrow\inf\{t\in\mathbb{R}^{d}:F_{Y|X}(t|x)\geq\tau\} Our work also builds on the implicit quantile NN literature Dabney et al. 2018 and generative Bayesian modeling framework of Polson and Sokolov 2023 . We wish to directly find the prediction rule Breiman 2001 . We will circumvent the use of parameters. There are three ways of writing a conditional predictive expectation: (1) using densities, (2) using survival functions, and (3) using quantiles. \displaystyle\hat{Y}(x)= \displaystyle E(Y\mid X)=\int_{-\infty}^{\infty}yf(y\mid x)dy \displaystyle= \displaystyle\int_{0}^{\infty}(1-F_{Y\mid X}(x))dx\text{ if }Y>0 \displaystyle= \displaystyle\int_{0}^{1}F^{-1}_{Y\mid X}(x)dx In Bayes framework, the predictive density calculation requires evaluating the marginal density via integration f_{Y\mid X}(x)=\int f_{Y\mid\theta}(y\mid\theta)(\theta\mid x)d\theta. It can be done via MCMC. In Generative AI Polson and Sokolov 2023 we directly find the quantile function map F^{-1}_{Y|X}(\tau) as a neural network. While our approach is prior-free and likelihood-free in the sense that we do not explicitly specify these quantities, it is important to recognize that implicit assumptions are embedded in the methodology. The choice of neural network architecture, loss function, and regularization implicitly defines a class of conditional distributions we can represent. Understanding the properties of this implicit class remains an important area for theoretical investigation. The connection to Wang’s distortion functions, illustrated in our normal-normal learning example, provides valuable theoretical insight. This shows that our quantile updates can be interpreted as applying a distortion function to transform prior distributions into posterior distributions. Characterizing the class of distortion functions that can be learned by neural networks would provide theoretical guidance for architecture design. Consider a pure prediction problem. The adjective “pure” is justified by an algorithm’s focus on prediction and the neglect of estimation and attribution Efron 2020 . The basic strategy is simple: to go directly for high predictive accuracy and not worry about surface plus noise models. This has some striking advantages and some drawbacks, too. Specifically, suppose that we have input-output pairs D=(X_{i},Y_{i}) . Pure prediction algorithms can be very different from each other. The least intricate and easiest to describe is random forests Breiman 2001 . Recent research has focused on conformal prediction and prediction-powered inference. The basic prediction problem can be described as follows. One has training data Z_{i}=(Y_{i},X_{i})_{i=1}^{n} . The variables are assumed to be generated in a stochastic fashion and the joint distribution assumed to be exchangeable. The future observations consist of unobserved outcomes Z_{i}=(\cdot,X_{i})_{i=n}^{N} and observed covariance. We are interested in the case where N\geq n . The question is how do you predict the missing Y ’s? The goal is to find a distribution-free prediction rule together with uncertainty quantification. We model i.i.d data Z_{1},\ldots,Z_{n}\sim P where Z_{i}=(X_{i},Y_{i})\in\mathbb{R}^{d}\times\mathbb{R} comprises a response and a d -dim vector of features (aka. predictors). Let the regression surface be \mu(x)=E(Y|X=x),x\in\mathbb{R}^{d} The goal is to predict Y_{new} at a new set of feature x_{new} with no assumptions on \mu,P ! There have been a number of approaches to this problem. Recently, Zrnic and Candès 2024 propose a prediction-powered inference framework. This is based on conformal prediction intervals which take the form \mathbb{P}\left(Y_{new}\in C(x_{new})\right)\geq 1-\alpha In turn this is very similar to Cox 1975 fiducial prediction interval idea. Barndorff-Nielsen and Cox 1996 provide asymptotic for such methods and a general discussion. In classification, we have an output rule f(x,d) that, for any predictor vector x , yields a prediction rule \hat{y}=f(x,d) . The hope is that the apparent error rate of the rule, for classification problems the proportion of cases where err=\#(\hat{y}_{i}\neq y)/n is small. More crucially, we hope that the true error rate is small, Err=E(f(X,d)\neq Y)\;\;{\rm where}\;\;(X,Y)\sim P Now a central tenet of Bayesian inference is coherence: the requirement that all probabilistic statements and predictions should be internally consistent and derived from a single joint probability model. This coherence is achieved by specifying a prior distribution over the parameters, which is then updated via Bayes’ theorem in light of observed data. We build on the statistical framework of Parzen 2004 ; Parzen 2009 who shows that Bayes rule for quantiles is composite (superposition) of functions. As such we can replace the prediction problem by one of non-parametric distribution matching Lu and Wong 2025 using quantile neural networks Polson et al. 2024 ; Polson and Sokolov 2023 . Quantile Bayesian Predictive This has been widely used in distributional decision-making Dabney et al. 2017 ; Polson et al. 2024 and econometrics Chernozhukov et al. 2010 ; Chernozhukov et al. 2021 . Although classical quantile methods were developed for univariate variable Koenker 2005 , recently there were several methods proposed for multivariate case Carlier et al. 2016 ; Kim et al. 2025 . They require the use of neural networks to train a transport map from estimated. The posterior distribution for parameters \pi_{F}(\theta|D_{obs}) (where D_{obs} is the observed data) can be used to form a predictive distribution, denoted by p(y_{new}|x_{new},D_{obs}) , for a new observation Y_{new} given x_{new} : Let \theta be an unobserved latent variable (a.k.a. parameter) F_{Y_{new}|x_{new},D_{obs}}(y)=E_{\theta|x_{new},D_{obs}}(F_{Y_{new}|\theta}(y))=\int_{\Theta}F_{Y_{new}|\theta}(y)dF_{\theta|x_{new},D_{obs}}(\theta), where F_{Z_{new}|Z} is the conditional predictive distribution. Bayesian Predictive Density The posterior distribution for parameters \pi_{F}(\theta|D_{obs}) (where D_{obs} is the observed data) can be used to form a predictive distribution, denoted by p(y_{new}|x_{new},D_{obs}) , for a new observation Y_{new} given x_{new} : p_{F}(y_{new}|x_{new},D_{obs})=\int p(y_{new}|\theta,x_{new})\pi_{F}(\theta|D_{obs})d\theta where p(y_{new}|\theta) is the model’s likelihood for a new observation given the parameter \theta . From this predictive distribution, one can construct prediction intervals, for instance, by taking the (\alpha/2) and (1-\alpha/2) quantiles. From computational point of view,

Read the complete text