Posterior and Likelihood Sensitivity in Bayesian Distributionally Robust Optimization

Jun-ya Gotoh † \dagger, Michael Jong Kim ‡ \ddagger, Andrew E.B. Lim ∗ * · 2026

Read Posterior and Likelihood Sensitivity in Bayesian Distributionally Robust Optimization by Jun-ya Gotoh † \dagger, Michael Jong Kim ‡ \ddagger, Andrew E.B. Lim ∗ * (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31306.txt.

Introduction

Gotoh, Lim and Kim measure toughness in Bayesian robust optimisation. They introduce worst-case posterior and likelihood sensitivity. These gauge how expected cost responds to worst-case shifts in each. Choice makers get two hard robustness gauges. The first gauge fades as its spread shrinks to zero with learning, but learning does not kill the second. A robust design trades payoff against both gauges near Pareto-best, the authors show for deviation-based doubt sets.

Catalog record

CREATORJun-ya Gotoh † \dagger, Michael Jong Kim ‡ \ddagger, Andrew E.B. Lim ∗ *
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40198
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.31306

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Posterior and Likelihood Sensitivity in Bayesian Distributionally Robust Optimization Jun-ya Gotoh † \dagger, Michael Jong Kim ‡ \ddagger, Andrew E.B. Lim ∗ * Abstract . We introduce the notion of worst-case posterior and worst-case likelihood sensitivity. These measure, respectively, the sensitivity of the expected cost to worst-case perturbations of the posterior distribution and worst-case perturbations of the likelihood of a Bayesian model. Each defines a quantitative measure of robustness. A decision maker concerned about the sensitivity of the out-of-sample expected cost to deviations from her assumptions will want a decision for which both sensitivities are small. We derive posterior and likelihood sensitivities for uncertainty sets defined in terms of deviation measures. Posterior sensitivity vanishes when the posterior variance shrinks to zero, which occurs when parameter uncertainty is eliminated from learning. Parameter learning does not eliminate likelihood sensitivity. A distributionally robust formulation of a Bayesian optimization problem makes a near-Pareto-optimal tradeoff between performance (expected cost) and robustness (posterior and likelihood sensitivity). Key words: Bayesian distributionally robust optimization, worst-case sensitivity, model uncertainty, posterior sensitivity, likelihood sensitivity, robustness–performance tradeoff. 1. Introduction Bayesian models provide a natural framework for learning from data. A Bayesian model requires two inputs from the decision maker (DM), a likelihood function L^{\theta}(\cdot)\equiv L(\cdot|\theta) and a prior distribution \rho for the uncertain parameter \theta . Together they characterize the joint distribution of the uncertain parameter \theta and the random variable Y . The prior on \theta is updated using observed samples of Y . The resulting posterior induces the predictive distribution which is used to compute the expected cost under a decision x . A DM concerned about misspecification would like a small expected cost that is insensitive to deviations from the predictive distribution. In this paper, we introduce the notions of posterior and likelihood sensitivity. Posterior sensitivity is the sensitivity of the expected cost to worst-case deviations from the posterior while likelihood sensitivity defines an analogous notion for the likelihood function. They can be interpreted as quantitative measures of robustness. A DM concerned about the impact of misspecification on the out-of-sample performance would like a decision with small expected cost and low posterior and likelihood sensitivity. Adapting ideas from [ 8 , 9 , 10 ] for robust empirical optimization problems, we show that a broad class of distributionally robust Bayesian optimization problems can be approximated by a “regularized” Bayesian problem where the regularizer is a weighted sum of posterior and likelihood sensitivities. It follows that approximately Pareto optimal tradeoffs between performance (expected cost) and robustness (posterior and likelihood sensitivity) can be obtained by solving a Bayesian Distributionally Robust Optimization (DRO) problem. We derive expressions for posterior and likelihood sensitivity, which depend on the uncertainty sets for posterior and likelihood, and show that posterior sensitivity vanishes when the posterior distribution degenerates to a point mass, but not so for the likelihood. In other words, improvements in parameter estimates from updating the prior with data can reduce posterior, but not likelihood, sensitivity. As illustrated in [ 10 ] for empirical optimization, sensitivity measures can guide the selection of uncertainty sets, identify when the “price of robustness” is high, and guide system redesign to reduce this cost. The outline of the paper is as follows. We briefly review the literature in Section 2 . We introduce the nominal model (the predictive distribution resulting from the posterior and likelihood) in Section 3 and worst-case posterior and worst-case likelihood sensitivity in Section 4 . Examples where uncertainty sets for the posterior and likelihood are smooth \phi -divergence and formed using different \phi -divergences are considered. We show in Section 5.1 that Bayesian DRO problems map out a nearly Pareto-optimal tradeoff between expected cost, posterior sensitivity and likelihood sensitivity. We consider a simple pricing application in Section 6 . We adopt the “constrained” formulation of the Bayesian DRO problem, where alternative posterior and likelihood distributions are determined by a constraint on the deviation from the nominal. A generalization to the penalty formulation of the Bayesian model can be found in the Appendix. 2. Literature review This paper is related to three streams of work: approximations of DRO as a regularized nominal problem, robustness measures induced by worst-case sensitivity, and Bayesian DRO. Several papers [ 2 , 3 , 4 , 5 , 7 , 6 , 12 ] show that DRO can be approximated by a regularized empirical optimization problem. [ 10 ] takes this as the starting point, shows that the regularizer can be interpreted as a quantitative measure of robustness (worst-case sensitivity), and explores the implications for robust modeling including uncertainty set selection (family and size) and system design. The present paper extends this perspective to single stage Bayesian problems. A Bayesian model consists of a likelihood function and a prior distribution; the predictive distribution (formed from the posterior and likelihood) is the nominal model. Generalizing ideas from [ 10 ] , we introduce worst-case posterior and worst-case likelihood sensitivity so that a Bayesian decision maker can quantify the fragility of her out-of-sample expected cost to the components of the predictive distribution. We show how each sensitivity measure depends on the uncertainty sets for the posterior and likelihood and that a distributionally robust Bayesian problem maps out a tradeoff between performance (expected cost) and robustness (posterior and likelihood sensitivities). Our results show that posterior sensitivity vanishes when there is no parameter uncertainty, but not so for the likelihood. [ 15 ] develops Bayesian DRO and, among other results, derive worst-case likelihood sensitivity with a Kullback–Leibler uncertainty set. We complement this by introducing posterior sensitivity, and considering uncertainty sets for deviation measures beyond Kullback–Leibler divergence. There is a related body of work in Bayesian statistics that studies the sensitivity of posterior-based inferences to perturbations of the (subjective) prior and likelihood (see for example [ 1 , 11 , 13 ] ). A key difference is the focus on decision making and its role in our model (i.e., we explicitly include the objective function); in particular, how decisions can be chosen when posterior or likelihood sensitivity is large. By showing that the DRO objective can be decomposed into expected cost plue a weighted sum of posterior sensitivity and likelihood sensitivity, we provide a systematic approach to choosing decisions that balance performance and sensitivity to misspecification. 3. Nominal model Let (\theta,Y) be random variables with a joint distribution {\mathbb{P}}({\rm d}\theta,{\rm d}y) . We denote the marginal distribution {\mathbb{P}}({\rm d}\theta)=\rho({\rm d}\theta) and the conditional distribution {\mathbb{P}}({\rm d}y|\theta)=L^{\theta}({\rm d}y) so \displaystyle{\mathbb{P}}({\rm d}\theta,{\rm d}y)=\rho({\rm d}\theta)L^{\theta}({\rm d}y). Bayesian learning models belong to this class: {\mathcal{L}}=\{L^{\theta}|\theta\in\Theta\} is a parameterized family of likelihoods and \rho is the posterior distribution of the uncertain parameter \theta . The posterior is obtained by updating prior \rho_{0}(d\theta) using data Y_{1},\cdots,Y_{m} and Bayes’ rule. The likelihood function L^{\theta} is a modeling choice, while the posterior depends on the user-specified prior \rho_{0}(d\theta) and likelihood L^{\theta}(dy) . We are concerned about the sensitivity of the expected cost to deviations from {\mathbb{P}}({\rm d}\theta,{\rm d}y) . Another important class is mixture models with populations indexed by \theta\in\{\theta_{1},\cdots,\theta_{n}\} and population distribution {\mathbb{P}}({\rm d}y|\theta=\theta_{i})=L^{\theta_{i}}({\rm d}y) . For concreteness, we refer to the marginal distribution \rho(d\theta) as the posterior and the conditional distribution L^{\theta}({\rm d}y) as the likelihood, fully recognizing that it is

Read the complete text