Statistical Inference via T-Posterior Randomised Estimators
Yannick BARAUD · 2026
Read Statistical Inference via T-Posterior Randomised Estimators by Yannick BARAUD (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03674.txt.
Introduction
Yannick Baraud and Gabriel Romon propose a new method that yields randomized estimators for the unknown distribution behind an observed random variable. Rather than a single fitted distribution, their T-posterior approach randomizes. They study how well it performs in theory. Randomized procedures can offer robustness that fixed estimators lack.
They prove performance bounds that hold for each fixed sample size, not just in the limit. They also show robustness to model misspecification: the method degrades gracefully when the assumed model is wrong. Notably, the proofs avoid concentration inequalities and empirical process theory, the usual heavy machinery for such bounds. An illustration applies the approach to estimating the intensity of a Poisson process.
Catalog record
| CREATOR | Yannick BARAUD |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 141 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03674
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Statistical Inference via T-Posterior Randomised Estimators Yannick BARAUD Abstract . Given a statistical model, we propose a novel estimation method that yields randomised estimators for the unknown distribution of an observed random variable. We establish non-asymptotic bounds for the performance of these estimators and demonstrate their robustness to potential model misspecification. Notably, these properties are established by circumventing the use of concentration inequalities and empirical process theory. We provide an illustration of this approach to the problem of estimating the intensity of a Poisson process. 1991 Mathematics Subject Classification Primary 62G05, 62G35, 62F35, 62F15 1. Introduction The aim of this paper is to propose a new approach for estimating the distribution {\mathbf{P}}^{\star} of an observed random variable {\boldsymbol{X}} . Given a parameterised family {\mathscrbf{M}}=\left\{{{\mathbf{P}}_{\theta},\;\theta\in\Theta}\right\} of candidate probability distributions for {\mathbf{P}}^{\star} , our aim is to propose an estimation strategy that yields an estimator \widehat{\theta} whose values belong to \Theta and for which {\mathbf{P}}^{\star} and {\mathbf{P}}_{\smash{\widehat{\theta}}} are as close as possible, in a sense that we shall specify later. Unlike the classical frequentist approach, in which the estimator is a deterministic function of the data {\boldsymbol{X}} , ours is drawn over the parameter space \Theta with a distribution from {\boldsymbol{\pi}}_{{\boldsymbol{X}}} that depends on {\boldsymbol{X}} . From this perspective, our approach can be interpreted as a randomised estimation procedure, or a Bayes-like one, and we shall see that it offers certain advantages over the frequentist approach. To design our random estimator, or equivalently the distribution {\boldsymbol{\pi}}_{{\boldsymbol{X}}} that generates it, we need three main elements. Firstly, we need a loss function \ell on the parameter space \Theta as well as a test statistic {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) between \theta and \theta^{\prime} that allows us to compare two candidate parameters \theta,\theta^{\prime} in \Theta . When {\mathbf{P}}^{\star}={\mathbf{P}}_{\theta^{\star}} belongs to our model {\mathscrbf{M}} and \theta is much closer to \theta^{\star} than to \theta^{\prime} (relatively to our loss \ell ), we expect our test statistic {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) to take very negative values. Since we are considering test statistics that satisfy {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime})=-{\mathbf{T}}({\boldsymbol{X}},\theta^{\prime},\theta) for all (\theta,\theta^{\prime})\in\Theta^{2} , {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) is automatically large in the opposite situation where \theta^{\prime} is much closer to \theta^{\star} than to \theta . When these parameters are either both close to or both far from \theta^{\star} , the choice between one or the other becomes unimportant, as does the value of {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) . The statistic {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) can be regarded as an estimator of the difference \ell(\theta^{\star},\theta)-\ell(\theta^{\star},\theta^{\prime}) and we shall sometimes refer to this interpretation when describing certain heuristics. Our second ingredient is a prior distribution over the parameter space \Theta . This prior (for short) will prove to be a practical tool for assessing the complexity of the parameter space, and more specifically, of an element \theta within it. We shall use it to replace other notions of complexity such as VC-dimensions, entropies, Rademacher complexities, amongst others. Even though the latter notions are commonly used in the frequentist paradigm, they may also be difficult to evaluate in general. To explain the heuristic that underlies our approach, let us assume for a moment that our test statistic {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) is the ideal quantity \ell(\theta^{\star},\theta)-\ell(\theta^{\star},\theta^{\prime}) . For \theta\in\Theta , let us introduce {\mathbf{T}}({\boldsymbol{X}},\theta)=\int_{\Theta}{\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime})\;d\widetilde{\boldsymbol{\pi}}_{{\boldsymbol{X}},\theta}(\theta^{\prime})\quad\text{where}\quad\frac{d\widetilde{\boldsymbol{\pi}}_{{\boldsymbol{X}},\theta}}{d{\boldsymbol{\pi}}}(\theta^{\prime})=\frac{\exp\left({\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime})\right)}{\int_{\Theta}\exp\left({\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime})\right)\cdot d{\boldsymbol{\pi}}(\theta^{\prime})} and define the distribution {\boldsymbol{\pi}}_{{\boldsymbol{X}}} as (1) \frac{d{\boldsymbol{\pi}}_{{\boldsymbol{X}}}}{d{\boldsymbol{\pi}}}(\theta)=\frac{\exp\left[{-{\mathbf{T}}({\boldsymbol{X}},\theta)}\right]}{\int_{\Theta}\exp\left[{-{\mathbf{T}}({\boldsymbol{X}},\theta)}\right]d{\boldsymbol{\pi}}(\theta)}. For \theta\in\Theta , we note that \widetilde{\boldsymbol{\pi}}_{{\boldsymbol{X}},\theta} is a Gibbs measure associated with the function \theta^{\prime}\mapsto{\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) . Our interest in this Gibbs measure lies in the fact that it concentrates its mass around the values of \theta^{\prime} for which {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) is large. Since in our case {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime})=\ell(\theta^{\star},\theta)-\ell(\theta^{\star},\theta^{\prime}) , \widetilde{\boldsymbol{\pi}}_{{\boldsymbol{X}},\theta} concentrates its mass in a neighbourhood of \theta^{\star} . Furthermore, for this specific test statistic, the density of \widetilde{\boldsymbol{\pi}}_{{\boldsymbol{X}},\theta} takes the form \frac{d\widetilde{\boldsymbol{\pi}}_{{\boldsymbol{X}},\theta}}{d{\boldsymbol{\pi}}}(\theta^{\prime})=\frac{\exp\left[{-\ell(\theta^{\star},\theta^{\prime})}\right]}{\int_{\Theta}\exp\left[{-\ell(\theta^{\star},\theta^{\prime})}\right]d{\boldsymbol{\pi}}(\theta^{\prime})} and is therefore independent of \theta . As a consequence, the statistic \displaystyle{\mathbf{T}}({\boldsymbol{X}},\theta)=\int_{\Theta}{\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime})d\widetilde{\boldsymbol{\pi}}_{{\boldsymbol{X}},\theta}(\theta^{\prime})=\int_{\Theta}\left({\ell(\theta^{\star},\theta)-\ell\left({\theta^{\star},\theta^{\prime}}\right)}\right)d\widetilde{\boldsymbol{\pi}}_{{\boldsymbol{X}},\theta}(\theta^{\prime}) is of the form {\mathbf{T}}({\boldsymbol{X}},\theta)=\ell(\theta^{\star},\theta)+C for some constant C\in{\mathbb{R}} and we observe that {\boldsymbol{\pi}}_{{\boldsymbol{X}}} is also a Gibbs measure, but associated with -{\mathbf{T}}({\boldsymbol{X}},\theta)=-\ell(\theta^{\star},\theta)-C since its density takes the form \frac{d{\boldsymbol{\pi}}_{{\boldsymbol{X}}}}{d{\boldsymbol{\pi}}}(\theta)=\frac{\exp\left[{-\ell(\theta^{\star},\theta)-C}\right]}{\int_{\Theta}\exp\left[{-\ell(\theta^{\star},\theta^{\prime})-C}\right]d{\boldsymbol{\pi}}(\theta^{\prime})}=\frac{\exp\left[{-\ell(\theta^{\star},\theta)}\right]}{\int_{\Theta}\exp\left[{-\ell(\theta^{\star},\theta^{\prime})}\right]d{\boldsymbol{\pi}}(\theta^{\prime})}. The distribution {\boldsymbol{\pi}}_{{\boldsymbol{X}}} therefore concentrates its mass around these \theta for which \ell(\theta^{\star},\theta) is small and a random variable \widehat{\theta} with distribution {\boldsymbol{\pi}}_{{\boldsymbol{X}}} is likely to lie close to \theta^{\star} . In order to be more precise and derive a risk bound, at least in the simple situation where the parameter space \Theta is finite or countable, let us first observe that (2) \int_{\Theta}\exp\left[{\ell(\theta^{\star},\theta)}\right]d{\boldsymbol{\pi}}_{{\boldsymbol{X}}}(\theta)=\frac{{\boldsymbol{\pi}}(\Theta)}{\int_{\Theta}\exp\left[{-\ell(\theta^{\star},\theta^{\prime})}\right]d{\boldsymbol{\pi}}(\theta^{\prime})}=\frac{1}{\int_{\Theta}\exp\left[{-\ell(\theta^{\star},\theta^{\prime})}\right]d{\boldsymbol{\pi}}(\theta^{\prime})}. Using ( 2 ) and the concavity property of the logarithm, we obtain that \displaystyle\int_{\Theta}\ell(\theta^{\star},\theta)d{\boldsymbol{\pi}}_{{\boldsymbol{X}}}(\theta) \displaystyle=\int_{\Theta}\log\exp\left[{\ell(\theta^{\star},\theta)}\right]d{\boldsymbol{\pi}}_{{\boldsymbol{X}}}(\theta)\leqslant\log\int_{\Theta}\exp\left[{\ell(\theta^{\star},\theta)}\right]d{\boldsymbol{\pi}}_{{\boldsymbol{X}}}(\theta) \displaystyle=-\log\int_{\Theta}\exp\left[{-\ell(\theta^{\star},\theta)}\right]d{\boldsymbol{\pi}}(\theta)=-\log\left[{\sum_{\theta\in\Theta}\exp\left[{-\ell(\theta^{\star},\theta)}\right]{\boldsymbol{\pi}}(\theta)}\right] \displaystyle\leqslant\inf_{\theta\in\Theta}\left[{\ell(\theta^{\star},\theta)+\log\left({\frac{1}{{\boldsymbol{\pi}}(\theta)}}\right)}\right]. This bound can be interpreted as {\mathbb{E}}\left[{\ell(\theta^{\star},\widehat{\theta})|{\boldsymbol{X}}}\right]\leqslant\inf_{\theta\in\Theta}\left[{\ell(\theta^{\star},\theta)+\log\frac{1}{{\boldsymbol{\pi}}(\theta)}}\right] when \widehat{\theta} has a distribution {\boldsymbol{\pi}}_{{\boldsymbol{X}}} conditionally on {\boldsymbol{X}} . Taking the expectation on both sides with respect to {\boldsymbol{X}} , we obtain the risk bound (3) {\mathbb{E}}\left[{\ell(\theta^{\star},\widehat{\theta})}\right]\leqslant\inf_{\theta\in\Theta}\left[{\ell(\theta^{\star},\theta)+\log\left({\frac{1}{{\boldsymbol{\pi}}(\theta)}}\right)}\right]. We observe that it decomposes into two terms. The second term \log\left({1/{\boldsymbol{\pi}}(\theta)}\right) depends solely on our choice of the prior distribution. If the parameter space \Theta is finite and has cardinality N , we can take {\boldsymbol{\pi}}(\theta)=1/N , in which case the risk depends logarithmically on N and the quantity \log N can be interpreted as the complexity of our parameter space. Nevertheless, it may sometimes be more sensible to choose a prior distribution {\boldsymbol{\pi}} that favours certain parameters over others, and \log\left({1/{\boldsymbol{\pi}}(\theta)}\right) then corresponds to a complexity value associated with the parameter \theta . The first term on the right-hand side of ( 3 ) is an approximation term. It measures the quality of the approximation of \theta^{\star} by an element with complexity \log\left({1/{\boldsymbol{\pi}}(\theta)}\right) . The risk bound provides the best compromise between approximation and complexity amongst the elements of the parameter space. Since \theta^{\star} is unknown, it is unfortunately impossible to take {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime})=\ell(\theta^{\star},\theta)-\ell(\theta^{\star},\theta^{\prime}) as we did in the heuristic argument that we presented above. The main idea is therefore to replace \ell(\theta^{\star},\theta)-\ell(\theta^{\star},\theta^{\prime}) with a suitable estimator of it. In this paper, we shall see how it is possible to establish risk bounds under an appropriate control of the Laplace transform of a certain linear combinaison of the test statistics {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) , namely that of the random variable \Delta_{\kappa}({\boldsymbol{X}},\theta,\theta^{\prime},\theta^{\prime\prime})=\kappa{\mathbf{T}}({\boldsymbol{X}},\theta^{\prime\prime},\theta^{\prime})-{\mathbf{T}}({\boldsymbol{X}},\theta^{\prime\prime},\theta) where \kappa denotes of positive number. Interestingly, it is sufficient to establish this control for fixed values of \theta,\theta^{\prime},\theta^{\prime\prime} in \Theta . In contrast, a frequentist approach based on minimising a criterion {\rm crit}({\boldsymbol{X}},\cdot) over the parameter space would generally require a uniform control of the difference {\rm crit}({\boldsymbol{X}},\theta)-{\mathbb{E}}\left[{{\rm crit}({\boldsymbol{X}},\theta)}\right] over \Theta to establish such risk bounds. These controls generally rely on concentration inequalities, which can be quite difficult to establish when the data are not independent. We refer the reader to Koltchinskii ( Koltchinskii, 2011 ) and Massart ( Massart, 2000 ) , amongst other references, for the crucial roles that play concentration inequalities in statistical estimation and statistical learning in the frequentist paradigm. Our approach entirely circumvents these difficulties. The test statistics we consider here have the property of yielding robust tests between two probabilities. This means that the initial assumption that {\mathbf{P}}^{\star}={\mathbf{P}}_{\theta^{\star}} belongs to {\mathscrbf{M}} can be relaxed. The value of {\mathbf{T}}({\boldsymbol{X}},\theta,\theta^{\prime}) will in fact depend on the proximity (in a sense to be specified) of {\mathbf{P}}^{\star} relative to {\mathbf{P}}_{\theta} and {\mathbf{P}}_{\theta^{\prime}} . The robustness of these statistics, combined with the fact that we only need to control the Laplace transform of \Delta_{\kappa}({\boldsymbol{X}},\theta,\theta^{\prime},\theta^{\prime\prime}) , allows us to establish these risk bounds under weak assumptions on {\mathbf{P}}^{\star} , in particular not only in situations where {\boldsymbol{X}}=(X_{1},\ldots,X_{n}) consists