New (and old) predictive schemes with a.c.i.d. sequences

Marco Battiston, Lorenzo Cappello · 2025

Read New (and old) predictive schemes with a.c.i.d. sequences by Marco Battiston, Lorenzo Cappello (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21874.txt.

Introduction

Marco Battiston and Lorenzo Cappello widen a statistical method in a 2025 paper on Bayesian prediction. The approach draws conclusions from a chain of forecasts alone, skipping the usual model, prior, and heavy computation. Which forecast chains are allowed was an open question, since the standard ones obey a strict fairness rule.

The pair define a looser class where forecasts act as near-fair chains, with a dial that sets how fast dependence between future observations fades. Under mild conditions the observations behave, in the long run, as if exchangeable, which licenses the predictive method. The class covers classic tools such as kernel estimators and newer ones such as the parametric bootstrap. The paper is a theory step for leaner statistical inference.

Catalog record

CREATORMarco Battiston, Lorenzo Cappello
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE117 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-21874

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

New (and old) predictive schemes with a.c.i.d. sequences Marco Battiston, Lorenzo Cappello Abstract There is a growing interest in procedures for Bayesian inference that bypass the need to specify a model and prior but simply rely on a predictive rule that describes how we learn on future observations given the available ones. At the heart of the idea is a bootstrap-type scheme that allows us to move from the realm of prediction to that of inference. Which conditions the predictive rule needs to satisfy to produce valid inference is a key question. In this work, we substantially relax previous assumptions building on a generalization of martingales, opening up the possibility of employing a much wider range of predictive rules that were previously ruled out. These include “old” ideas in Statistics and Learning Theory, such as kernel estimators, and more novel ones, such as the parametric Bayesian bootstrap or copula-based algorithms. Our aim is not to advocate in favor of one predictive rule versus the other ones, but rather to showcase the benefits of working with this larger class of predictive rules. Keywords— Almost Supermartingales, Conditional identical distribution, Kernel Methods, Martingale Posterior, Parametric Bootstrap, Predictive Resampling. 1 Introduction 1.1 Motivation Prediction lies at the foundation of Bayesian Statistics. Traditionally, much research has focused on the relationship between a specified statistical model and prior, and the resulting predictive distribution of future observations given current information; see Fortini and Petrone 2025 for a recent review. However, when the primary goal is inference, a (Bayesian) statistician usually specifies a statistical model and a prior, and then, given a sample, computes the corresponding posterior distribution. Predictive distributions do not play a role in this process; possibly except for the necessary checks of model fit ( Gelman et al. 1996 , e.g. predictive checks,) . A recent stream of literature is taking a new stance, suggesting that it is possible to do Bayesian inference by simply specifying a predictive rule - how to update the prediction as new information becomes available - fully bypassing the need for models and priors ( Fortini et al. 2020 ; Berti et al. 2021 ; Fong et al. 2023 ; Holmes and Walker 2023 ; Fortini and Petrone 2023 ) . The peculiarity of this approach is that prediction is done with inference in mind rather than forecasting. This emerging area, often referred to as Bayesian predictive inference or modeling , is the focus of the paper. Several reasons support the popularity of this growing research area. First, it bypasses the need to specify a statistical model and subjective probabilities on latent parameters, which may lack interpretability or be high-dimensional. Instead, it relies on a probabilistic learning rule on the observables - an assumption that can be formally assessed, for example, via predictive checks ( Gelman et al. 1996 ) . Second, inference can be performed using any available tools, including recent methods for handling model misspecification. Third, it is computationally appealing: predictive resampling can avoid MCMC methods and be parallelized, greatly speeding up inference ( Fong et al. 2023 ) . These gains include predictive rules developed as fast approximations to Bayesian models, which fit naturally within this framework ( Fortini et al. 2020 ) . Lastly, it has the potential to bridge the two cultures - statistical inference and prediction ( Breiman 2001b ) , offering the potential of using popular machine learning algorithms for inference. The current framework falls short in exploiting this connection. This paper proposes a step toward closing that gap. Let \theta be an estimand and x_{1:n} a sample. The key observation underlying predictive inference is that the uncertainty described by a posterior distribution \mathbb{P}(\theta|x_{1:n}) can be equivalently described through the predictive distribution of the yet-to-be-seen observations \mathbb{P}(X_{n+1:\infty}|x_{1:n}) , or equivalently a sequence of one-step-ahead distributions (\mathbb{P}(X_{n+m}|x_{1:n+m-1}))_{m\geq 1} . The scheme to move from prediction to inference is called predictive resampling , which is a generalization of the Bayesian boostrap ( Rubin 1981 ) , where one samples many synthetic data from \mathbb{P}(X_{n+1:n+M}|x_{1:n}) for some large M , and then for each synthetic sample computes an estimate \hat{\theta}=f(x_{1:n+M}) , where f is an appropriately chosen estimator which crucially uses both the original data and the synthetic one. The resulting empirical distribution of \hat{\theta} is an approximation to the martingale posterior ( Fong et al. 2023 ) . Crucially, under suitable conditions, this posterior is full equivalent to the one obtained with the usual application of Bayes rule starting from a model and a prior. What conditions the predictive rule needs to satisfy to get a valid martingale posterior is one of the key research questions. Given a filtration \mathcal{G}=(\mathcal{G}_{n})_{n\geq 1} (for example, the natural filtration of (X_{n})_{n\geq 1} ), Fong et al. 2023 assume that (\mathbb{P}(X_{n+1}\in\cdot|\mathcal{G}_{n}))_{n\geq 0} is a martingale, for all Borel set A , i.e. , \mathbb{E}(\mathbb{P}(X_{n+2}\in A|\mathcal{G}_{n+1})|\mathcal{G}_{n})-\mathbb{P}(X_{n+1}\in A|\mathcal{G}_{n})=0. The same condition appears also in Fortini et al. 2020 . The martingale assumption has many desirable implications. It represents a departure from the assumption of exchangeability on (X_{n})_{n\geq 1} to the more general class of conditionally distributed sequences (c.i.d.) ( Kallenberg 1988 ; Berti et al. 2004 ) . This enlarges the class of predictive distributions and models that can be considered, moving away from exchangeable Bayesian predictive schemes ( Fortini et al. 2020 , e.g.) . Such a departure comes at no substantial cost: while the convergence of predictive distribution for exchangeable sequences is given by de Finetti’s representation theorem, here, the random sequence (\mathbb{P}(X_{n+1}\in\cdot|\mathcal{G}_{n}))_{n\geq 0} converges weakly almost surely (a.s.) by martingale properties ( Aldous 1985 , see) . Crucially, (X_{n})_{n\geq 1} is also asymptotically exchangeable , meaning that it behaves in the limit as an exchangeable sequence ( Aldous 1985 ) . This property justifies the existence of a statistical model and a prior distribution via the representation theorem, and makes the martingale posterior a valid approximation for \mathbb{P}(\theta|x_{1:n}) . What’s appealing is that there is no need to specify a model and prior or even assume exchangeability. The martingale assumption has other desirable features. It is an elegant framework that links Bayesian predictive inference to applications of martingales developed by Doob 1949 . In addition, Fong et al. 2023 use the preservation of expectation of martingales to justify the coherence of the procedure ( Robins and Wasserman 2000 , in the sense of) . Nevertheless, it seems clear that what matters is the almost sure weak convergence of (\mathbb{P}(X_{n+1}\in\cdot|\mathcal{G}_{n}))_{n\geq 0} and the resulting asymptotic exchangeability of (X_{n})_{n\geq 1} . This emerges in subsequent works and is stated clearly by Garelli et al. 2024 , who argue that martingales are mostly useful for the mathematical argument. It is becoming clear that the martingale assumption could be limiting. For example, consider the following: Example 1 . Let \text{N}(\mu,\sigma^{2}) denote a Gaussian distribution with mean \mu and variance \sigma^{2} , and sample recursively X_{n+1}|x_{1:n}\sim\text{N}(\bar{x}_{n},\sigma^{2}) , where \bar{x}_{n}=\frac{1}{n}\sum_{i=1}^{n}x_{i} denotes the sample mean and \bar{x}_{0}=0 . This update corresponds to sampling X_{n+1} from the parametric model \text{N}(\hat{\mu}_{n},\sigma^{2}) , in which the MLE \hat{\mu}_{n}=\bar{x}_{n} is recursively updated and plugged in, to define the next one-step-ahead predictive distribution. Here, the resulting sequence (\mathbb{P}(X_{n+1}\in\cdot|\mathcal{G}_{n}))_{n\geq 0} is not a martingale. The literature includes now many examples that

Read the complete text