Efficient Bayes Factor Sensitivity Analysis via Posterior Density Ratios

František Bartoš*, Eric-Jan Wagenmakers, Maarten Marsman, Don van den Bergh · 2026

Read Efficient Bayes Factor Sensitivity Analysis via Posterior Density Ratios by František Bartoš*, Eric-Jan Wagenmakers, Maarten Marsman, Don van den Bergh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21596.txt.

Introduction

František Bartoš and three colleagues recover Bayes factor sensitivity curves from one model fit, instead of refitting per value. Each Bayes factor splits into an anchor value and a Savage-Dickey ratio from an extended hyper-prior model. Each value follows from the anchor and posterior density ordinates, estimated by importance weighting. Because the parameter enters only through the prior, the likelihood cancels, leaving prior evaluations on MCMC draws. The estimator is fast, accurate on samples, beats kernel density estimation, and extends to multiple parameters and model averaging.

Catalog record

CREATORFrantišek Bartoš*, Eric-Jan Wagenmakers, Maarten Marsman, Don van den Bergh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61599
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.21596

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient Bayes Factor Sensitivity Analysis via Posterior Density Ratios František Bartoš*, Eric-Jan Wagenmakers, Maarten Marsman, Don van den Bergh Abstract Bayes factor sensitivity analysis examines how the evidence for one hypothesis over another depends on the prior distribution. In complex models, the standard approach refits the model at each hyper-parameter value, and the total computational cost scales linearly in the grid size. We propose a method that recovers the entire sensitivity curve from a single additional model fit. The key identity decomposes the Bayes factor at any hyper-parameter value γ x \gamma_{x} into an “anchor” Bayes factor at a fixed reference γ 0 \gamma_{0} and a Savage–Dickey density ratio in an extended model that places a hyper-prior on γ \gamma . Once this extended model is fit, the Bayes factor at any γ x \gamma_{x} follows from the anchor value and a ratio of two posterior density ordinates. To approximate this ratio, we employ the importance-weighted marginal density estimator (IWMDE). Because the sensitivity parameter enters the model only through the prior distribution on the model parameters, the data likelihood cancels in the IWMDE, reducing it to a simple ratio of prior density evaluations on the MCMC draws, without any additional likelihood computation. The resulting estimator is fast, remains accurate even with small MCMC samples, and substantially outperforms kernel density estimation across the full sensitivity range. The method extends naturally to simultaneous sensitivity over multiple hyper-parameters and to Bayesian model averaging. We illustrate it on a univariate Bayesian t t -test with exact Bayes factors for validation, a bivariate informed t t -test, and a Bayesian model-averaged meta-analysis, obtaining accurate sensitivity curves at a fraction of the brute-force cost. *Correspondence concerning this article should be addressed to František Bartoš at [email protected]. Keywords Savage–Dickey density ratio, prior distribution, robustness analysis, informed inference 1 Introduction Bayes factors quantify the evidence that the data provide for one statistical model over another ( Jeffreys 1939 ; Kass & Raftery 1995 , e.g.,) . For two competing hypotheses \mathcal{H}_{0} and \mathcal{H}_{1} , the Bayes factor is defined as the ratio of the marginal likelihoods: \textsc{BF}_{10}=\frac{p(y\mid\mathcal{H}_{1})}{p(y\mid\mathcal{H}_{0})}, (1) where p(y\mid\mathcal{H}_{1})=\int p(y\mid\boldsymbol{\xi},\mathcal{H}_{1})\,p(\boldsymbol{\xi}\mid\mathcal{H}_{1})\,{\rm d}\boldsymbol{\xi} denotes the marginal likelihood obtained by integrating the likelihood over the prior distribution of all model parameters \boldsymbol{\xi} under \mathcal{H}_{1} (and analogously for \mathcal{H}_{0} ). The Bayes factor is therefore a ratio of prior predictives, which makes the choice of prior distribution crucial: different prior distributions encode different hypotheses and formalize different theoretical commitments about the parameters. The prior distribution should be considered an integral part of the model; if a different question is asked by specifying a different prior distribution, a different answer ought to follow ( Vanpaemel 2010 ) . Sources of information that can be used to construct informative prior distributions include theory, logic, previous data, and expert elicitation ( Lee & Vanpaemel 2018 ) . At the same time, it is important to ascertain that modest quantitative perturbations of the prior distribution do not lead to qualitatively different conclusions. One might also wish to know how much the hypothesis would need to be modified before arriving at a qualitatively different conclusion, or how the evidence depends on a specific prior choice selected from among several reasonable alternatives ( Dickey 1973 ; Kass & Raftery 1995 ; Goodman 1999 , e.g.,) . These questions are answered through Bayes factor sensitivity analysis (or robustness analysis): a range of prior distributions is specified, usually by varying the hyper-parameters of a prior distribution on one or more model parameters, and the Bayes factor is examined as a function of those hyper-parameters ( Berger 1990 ; Berger 1982 ; Box & Tiao 1962 ; Sinharay & Stern 2002 , e.g.,) . In the robust Bayesian tradition, the general problem of prior sensitivity is formalized by specifying a class \Gamma of plausible prior distributions and determining the range of posterior quantities, including Bayes factors, as the prior ranges over \Gamma ( Berger 1990 ) . The approach proposed here can be seen as an efficient implementation of this program for the common case where the class of prior distributions is a parametric family indexed by a hyper-parameter \gamma . Bayes factor sensitivity analysis is regularly emphasized in reporting guidelines for Bayesian analyses ( Andraszewicz et al. 2015 ; van Doorn et al. 2020 ; Depaoli & Van de Schoot 2017 ; Kruschke 2021 ) and has been historically approached from several angles. Franck & Gramacy 2020 proposed visualizing the “Bayes factor surface,” the Bayes factor plotted as a function of user-specified hyper-parameters, and used Gaussian process surrogate models to approximate these surfaces when direct computation is expensive. Johnson et al. 2023 introduced “Bayes factor functions” that express Bayes factors from common test statistics as functions of standardized effect sizes. In the context of particle physics, Fowlie 2024 proposed to plot contours of the Bayes factor over phenomenological parameters, computed through the Savage–Dickey density ratio ( Dickey 1971 ; Wagenmakers et al. 2010 , SDDR;) . Support intervals ( Wagenmakers et al. 2022 ) , evidential calibration ( Pawel et al. 2024 ) , and reverse Bayes approaches ( Held et al. 2022 ) further develop the idea of exploring the Bayes factor as a function of the prior distribution on the model parameters. In simple models, sensitivity analyses are fast. For instance, the Bayesian t -test of Jeffreys 1948 (also see Rouder et al. 2009 ) involves an alternative hypothesis \mathcal{H}_{1} under which the standardized effect size \delta is assigned a Cauchy prior distribution with median 0 and scale parameter r ; consequently, the associated Bayes factor has a one-dimensional integral representation that allows rapid evaluation across a grid of r values. Alternatively, approximate Bayes factors for nested hypotheses can use Savage–Dickey normal approximation ( Spiegelhalter et al. 2004 ; Bartoš & Wagenmakers 2023 ; Dienes 2014 , e.g.,) which makes sensitivity analysis for a focal parameter nearly instantaneous given only a maximum likelihood estimate and its standard error (plus the prior ordinate for the test-relevant parameter at the value specified under \mathcal{H}_{0} ). For more complex models, however, Bayes factor sensitivity analysis can be prohibitively expensive. The reason is straightforward: each value of the hyper-parameter requires fitting the complete model anew. If the sensitivity analysis spans K grid points and each point requires new posterior samples, whether obtained via the SDDR, bridge sampling ( Meng & Wong 1996 ; Gronau et al. 2017 ) , or nested sampling ( Skilling 2006 ) , the total computational cost scales as K times the cost of a single fit. For models where a single fit can take from hours to days, such as model-averaged meta-analyses, structural equation models, or network models, a sensitivity analysis spanning dozens of hyper-parameter values quickly becomes infeasible ( Sekulovski et al. 2024 , see e.g.,) . In this manuscript, we combine two ideas to obtain an efficient, general-purpose method for Bayes factor sensitivity analysis that recovers the entire sensitivity curve from a single additional model fit. The first idea is to place a hyper-prior on the sensitivity parameter \gamma (e.g., the scale of a prior distribution) and exploit Bayes factor transitivity to decompose the target Bayes factor at any value \gamma_{x} into two components:

Read the complete text