Bayes Risk for Goodness of Fit Tests

Nicholas G. Polson, Vadim Sokolov, Daniel Zantedeschi · 2026

Read Bayes Risk for Goodness of Fit Tests by Nicholas G. Polson, Vadim Sokolov, Daniel Zantedeschi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-15297.txt.

Introduction

Polson and colleagues develop a setup for fit checks through Bayes risk. Old methods are tuned either at fixed strictness, the CLT scale, or through fast powers, the LDP scale. They show Bayes-risk best tuning works on the middle-shift scale, giving square-root-of-log-n growth of refusal limits and fading Type I error.

First, they state the Rubin-Sethuraman plan for KS-type stats as a risk-tuning theorem with rules on beliefs and tools. Second, they develop the link between Bayes-risk series and Sanov info limits, showing how log-n-order cuts arise when risk, not powers, is the rule. Third, they provide uses for place checks under Laplace groups, shape checks via Bayes parts, and Fisher info. The rule is that data size enters Bayes-optimal limits through the middle-shift scale, joining KS and Sanov views under one rule.

Catalog record

CREATORNicholas G. Polson, Vadim Sokolov, Daniel Zantedeschi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-15297

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bayes Risk for Goodness of Fit Tests Nicholas G. Polson, Vadim Sokolov, Daniel Zantedeschi Abstract We develop a unified framework for goodness-of-fit (GOF) testing through the lens of Bayes risk. Classical GOF procedures are commonly calibrated either at fixed significance level (CLT scale) or through exponential error exponents (LDP scale). We establish that Bayes-risk optimal calibration operates on the moderate-deviation (MDP) scale, producing canonical log ⁡ n \sqrt{\log n} inflation of rejection thresholds and polynomially decaying Type I error. Our main contributions are: (i) we formalise the Rubin–Sethuraman program for KS-type statistics as a risk-calibration theorem with explicit regularity conditions on priors and empirical-process functionals; (ii) we develop the precise connection between Bayes-risk expansions and Sanov information asymptotics, showing how log ⁡ n \log n -order truncations arise naturally when risk, rather than pure exponents, is the evaluation criterion; (iii) we provide detailed applications to location testing under Laplace families, shape testing via Bayes factors, and connections to Fisher information geometry. The organizing principle throughout is that sample size enters Bayes-optimal GOF cutoffs through the MDP scale, unifying KS-based and Sanov-based perspectives under a single risk criterion. Keywords: Bayes risk; goodness-of-fit; moderate deviations; Kolmogorov–Smirnov; Sanov theorem; Bayes factors; Fisher information. 1 Introduction Goodness-of-fit (GOF) testing occupies a central position in statistical methodology, serving as the primary mechanism for model criticism, specification search, and validation of probabilistic assumptions. Classical GOF procedures, including Kolmogorov–Smirnov (KS) statistics, likelihood-ratio tests, chi-squared tests, and entropy-based criteria, have been studied extensively under two dominant asymptotic frameworks: fixed- \alpha calibration (CLT scale), where critical values control Type I error at a constant level, and exponential error exponents (LDP scale), where performance is characterised by large-deviation rate functions and Chernoff bounds ( Chernoff 1952 ; Dembo and Zeitouni 1998 ) . Neither framework is appropriate when the inferential objective is Bayes risk minimisation ( Berger 1985 ; Le Cam 1986 , cf.) . Fixed- \alpha calibration ignores the fact that, as sample size grows, an increasing number of alternatives become statistically distinguishable and must be jointly serviced. Pure large-deviation calibration enforces overly stringent rejection thresholds that sacrifice power against local and moderate alternatives, leading to suboptimal Bayesian risk even when Type I error is extremely small. The central message of this paper is that Bayesian calibration of GOF tests naturally operates on the moderate-deviation (MDP) scale. This intermediate regime, lying strictly between CLT and LDP scales, arises because the number of alternatives that a sample of size n can distinguish from the null grows polynomially, and a Bayes-risk criterion must service all of them. Concretely, at sample size n the experiment resolves distributions at Fisher distance \sqrt{\log n/n} from the null; any prior that is absolutely continuous near \mathcal{F}_{0} places polynomial mass in this shell. Balancing false rejections against the cumulative missed-detection cost over this growing set of distinguishable alternatives forces thresholds to inflate as \sqrt{\log n} , Type I error to decay polynomially, and Bayes risk to concentrate on the boundary shell. The information-theoretic lineage of this perspective is extensive. Sanov 1957 established that empirical measures satisfy a large-deviation principle with Kullback–Leibler rate function. Bahadur 1960 and Bahadur and Rao 1960 showed that polynomial prefactors in tail probabilities affect statistical efficiency at moderate scales. Rubin and Sethuraman 1965a ; Rubin and Sethuraman 1965b analysed Bayes risk for empirical-process statistics and derived moderate-deviation optimality for KS-type procedures. Good 1955 emphasised weight-of-evidence interpretations of log-likelihood ratios, while Pearson’s \chi^{2} test and entropy-deficit criteria ( Jaynes 1957 ) were later shown to share identical large-deviation rate functions. What has been missing is a single generative mechanism that explains why these distinct threads converge at the MDP scale, and a template (Lemma 2.8 ) that makes each instantiation a two-line corollary. Our main contributions are as follows. We state an explicit Bayes-Risk MDP Principle (Section 2 ) and prove it as a theorem under regularity conditions that cover KS-type statistics, Sanov-based tests, and standard parametric GOF problems. We formalise the Rubin–Sethuraman program for KS statistics (Section 3 ) as a risk-calibration theorem with sharp constants, showing that the Bayes-optimal threshold satisfies t_{n}^{*}=\sqrt{(\kappa/4)\log n}+O(\sqrt{\log\log n}) . We develop the connection between Bayes risk and Sanov information asymptotics (Section 4 ), showing that the effective KL exponent is truncated to O(\log n/n) under Bayes-risk optimisation, and providing the information-theoretic interpretation of MDP scaling. Finally, we apply the theory to location testing under Laplace families, shape testing via Bayes factors, multinomial GOF, and Fisher information geometry (Sections 5 – 6 ). Rubin–Sethuraman identify the MDP calibration in the KS setting; our contribution is to isolate the underlying (\rho,\kappa) mechanism and show it applies uniformly across GOF statistics, including empirical-process, KL/Sanov, multinomial, and parametric geometric instances. Recent work ( Datta et al. 2026 ) develops a unified theory of Bayesian hypothesis testing via moderate deviation asymptotics, focusing on Bayes factors and integrated risk expansions. While both approaches identify the moderate deviation regime as central for Bayesian calibration, the present paper isolates a distinct mechanism: a generic risk decomposition template based solely on (i) Gaussian-type null tails and (ii) local prior mass exponents. Our results do not depend on Bayes factors or likelihood-ratio structure per se. Instead, Theorem 2.6 and Lemma 2.8 provide a reusable calibration principle applicable to a broad class of statistics, including goodness-of-fit tests, empirical processes, and divergence-based procedures. In this sense, the moderate deviation scaling emerges here as a structural consequence of risk balancing, rather than as a property specific to Bayesian hypothesis testing. Our references to Dawid’s decomposition and prequential results follow the published accounts in Dawid 2011 ; Dawid 1992 . The paper is organised as follows. Section 2 formalises the GOF testing problem, defines Bayes risk, and states the organizing principle. Section 3 establishes Bayes-risk optimality of MDP calibration for KS-type statistics. Section 4 develops the Sanov/information-asymptotic perspective. Section 5 gives applications. Section 6 connects to Fisher information geometry. Section 7 provides numerical verification of the calibration template. Section 8 discusses implications and extensions. Appendix B provides a compact triangulation linking Bayes factors (Good), large deviations (Hoeffding), entropy geometry (Jaynes), and Wilks’ chi-square limit through KL curvature, clarifying the geometric foundation of the moderate-deviation boundary derived here. 2 Setup and the Bayes-Risk GOF Principle 2.1 The Testing Problem Let X_{1},\ldots,X_{n} be i.i.d. with distribution F on a measurable space (\mathcal{X},\mathcal{A}) . The goodness-of-fit testing problem is H_{0}:F\in\mathcal{F}_{0}\qquad\text{vs.}\qquad H_{1}:F\in\mathcal{F}_{1}, (1) where \mathcal{F}_{0} and \mathcal{F}_{1} are disjoint classes of distributions. We allow several standard configurations: a simple null \mathcal{F}_{0}=\{F_{0}\} ; a parametric null \mathcal{F}_{0}=\{F_{\theta}:\theta\in\Theta_{0}\} (see Kiefer and Wolfowitz 1956 , for consistency under growing parameter spaces) ; and composite or nonparametric alternatives. A (possibly randomized) test is a measurable map \delta_{n}:\mathcal{X}^{n}\to[0,1] , where \delta_{n}(x^{n}) is the probability of rejecting H_{0} given data x^{n} . 2.2 Empirical Measures and GOF Statistics Many GOF procedures are functionals of the empirical measure \hat{\pi}_{n}\;\mathrel{\mathop{:}}=\;\frac{1}{n}\sum_{i=1}^{n}\delta_{X_{i}}, (2) or equivalently the empirical distribution function F_{n}(t)=n^{-1}\sum_{i=1}^{n}\mathbf{1}\{X_{i}\leq t\} . GOF statistics can be written as T_{n}=\mathsf{T}(\hat{\pi}_{n};F_{0}) for a discrepancy functional \mathsf{T} . Representative examples include the Kolmogorov–Smirnov statistic S_{n}=\sup_{t}|F_{n}(t)-F_{0}(t)| , Cramér–von Mises, Anderson–Darling, and likelihood-ratio statistics. 2.3 Bayes Risk Formulation We embed GOF testing in Bayesian decision theory by placing priors on \mathcal{F}_{0} and \mathcal{F}_{1} and assigning costs to Type

Read the complete text