Robust model selection using likelihood as data Jongwoo Choi, Neil A. Spencer, Jeffrey W. Miller Abstract Model selection is a central task in statistics, but standard methods are not robust in misspecified settings where the true data-generating process (DGP) is not in the set of candidate models. The key limitation is that existing methods—including information criteria and Bayesian posteriors—do not quantify uncertainty about how well each candidate model approximates the true DGP. In this paper, we introduce a novel approach to model selection based on modeling the likelihood values themselves. Specifically, given K K candidate models and n n observations, we view the n × K n\times K matrix of negative log-likelihood values as a random data matrix and observe that the expectation of each row is equal to the vector of Kullback–Leibler divergences between the K K models and the true DGP, up to an additive constant. We use a multivariate normal model to estimate and quantify uncertainty in this expectation, providing calibrated inferences for robust model selection under misspecification. The procedure is easy to compute, interpretable, and comes with theoretical guarantees, including consistency. Keywords— Bayesian inference; Likelihood as Data; Model misspecification; Robustness 1 Introduction A good statistical model should be flexible enough to capture the essential structure of the data, yet simple enough to interpret, communicate, and use in practice. Model selection helps balance these goals by choosing among competing models of varying complexity and explanatory power ( 5 ) . In most applications, the true data-generating process (DGP) is unknown and all of the candidate models are misspecified ( 4 ) . In such cases, it is desirable to find a model that is as close as possible to the true DGP while remaining sufficiently simple to interpret and use. Selection criteria such as the Akaike Information Criterion (AIC; 1 ) and Bayesian Information Criterion (BIC; 20 ) are based on the log-likelihood, which can be viewed as estimating the relative closeness of each model to the true DGP. However, they do not quantify uncertainty in these estimates or in the choice of model. Meanwhile, Bayesian posteriors quantify uncertainty in the choice of model ( 24 ) , but posterior probabilities do not measure how far each model is from the truth; a model may receive a high probability simply because the alternatives are worse. When none of the candidate models are correct, these information criteria and Bayesian posteriors asymptotically concentrate on the model with the smallest Kullback–Leibler (KL) divergence from the true DGP, even if there is a much simpler model that is nearly as good ( 3 ) . Further, these methods tend to be highly unstable when more than one model attains the minimal KL divergence, even approximately ( 19 ) . Due to these issues, the standard leading methods do not provide well-calibrated inferences about the utility of each model in a way that is robust to misspecification. Figure 1 : Limitations of existing model selection methods. (Left) Histogram of the Shapley galaxy velocities ( \times 10^{3} km/s). We trim the top 0.5\% of data points to remove the extremely long right tail extending beyond 45{,}000 km/s. (Middle) Boxplots summarizing draws from the Bayesian posterior on k for a Gaussian mixture model with k components, for sample sizes n\in\{40,120,400,1200,4000\} . As n grows, the posterior shifts toward larger k , a typical phenomenon under misspecification. (Right) Values of k selected by AIC (red) and BIC (blue) for each n . These results illustrate lack of robustness and inability to quantify uncertainty in how well a k -component Gaussian mixture can approximate the true DGP. To illustrate these limitations, consider the problem of choosing the number of components k in a finite mixture of Gaussians. Figure 1 (left) shows a histogram of radial velocity measurements of 4215 galaxies in the Shapley supercluster, a large concentration of gravitationally interacting galaxies ( 6 ) . We apply AIC, BIC, and a Bayesian model with a prior on the number of components to increasing subsets of this data set. In Figure 1 (middle), we see that the posterior distribution concentrates on larger and larger values of k as the sample size n increases, and does not provide any insight into how close each model is to the true DGP. Similarly, Figure 1 (right) shows that AIC and BIC select increasing values of k as n grows, and furthermore, they do not provide any uncertainty quantification in the choice of k . See Section S3.1 for details. In this paper, we introduce a new framework for model selection that overcomes these issues by jointly quantifying uncertainty in the expected log-likelihoods of the candidate models. More precisely, let Z_{ik} denote the negative log-likelihood for model k on observation i , assuming for the moment that there are no unknown parameters. Then \mu_{k}=E(Z_{ik}) equals the KL divergence between the true DGP and model k , plus the (unknown) entropy of the true DGP. We propose viewing Z_{ik} as data—hence the term “Likelihood as Data”—and using a Bayesian multivariate normal model to compute the posterior distribution of \mu=(\mu_{1},\ldots,\mu_{K}) . This makes it straightforward to perform Bayesian inference for a range of questions, including not only which model is closest to the true DGP, but whether a given model is closer than another model, and which is the simplest model within a specified tolerance of being closest to the true DGP. A key feature of our approach is that it is fully robust to misspecification of the candidate models since it is based inferring the \mu_{k} ’s, which are well defined and interpretable without requiring any assumptions of model correctness. We prove that the method is consistent and remains stable under ties – that is, when multiple models are the same distance from the true DGP. We propose a procedure for finding the simplest class of models within a specified tolerance of being closest to the true DGP, and choosing the best-fitting model (or models) within this class. In a range of simulations and real-data applications, we find that the proposed approach yields reliable and informative inferences for model selection in the presence of misspecification. The paper is organized as follows. Section 2 introduces our proposed methodology, and Section 3 provides algorithms for computing the method. In Section 4 , we provide theoretical results. Section 5 contains empirical demonstrations on Gaussian mixture models, sparse multivariate normal models, microbial ecology models, and population structure admixture models in genetics. Section 6 concludes with a brief discussion. 2 Methodology In this section, we introduce the Likelihood as Data framework ( Section 2.1 ), describe our robust model selection criterion ( Section 2.2 ), discuss an interpretation of the selection threshold ( Section 2.3 ), extend the method to parametrized models ( Section 2.4 ), and provide an adjustment for overfitting bias in parametrized models ( Section 2.5 ). Suppose X_{1},\ldots,X_{n}\in\mathcal{X} are independent and identically distributed (i.i.d.) observations from some unknown distribution P_{0} , where \mathcal{X} denotes the sample space. Assume P_{0} admits a density f_{0} with respect to a sigma-finite measure \lambda , for example, Lebesgue measure for continuous observations or counting measure for discrete observations. Consider a finite collection of candidate models P_{1},\ldots,P_{K} with associated densities f_{1},\ldots,f_{K} (also with respect to \lambda ), respectively. To clearly explain the main idea of our proposed method, we assume for now that there are no unknown parameters to be estimated in any of the candidate models; however, in Section 2.4 , we extend the method to handle unknown parameters. We do not assume P_{0} coincides with one of the candidate models, that is, all K models may be misspecified. 2.1 Likelihood as Data (LaD) framework Let Z_{ik}:=-\log(f_{k}(X_{i})) be the negative log-likelihood value for each observation i\in\{1,\ldots,n\} and each model k\in\{1,\ldots,K\} , and collect these values into a matrix Z=[Z_{ik}]\in\mathbb{R}^{n\times K} . The key idea is to treat each vector Z_{i}:=(Z_{i1},\dots,Z_{iK})^{\texttt{T}}\in\mathbb{R}^{K} as a data point, and analyze the resulting data set, Z_{1},\ldots,Z_{n} . In this way, we regard the likelihood values as data in their own right, which is the foundation of the LaD approach. As with any data set, it is natural to estimate the mean \mu^{0}:=\mathbb{E}(Z_{1})\in\mathbb{R}^{K} and quantify uncertainty in it. In the LaD setting, \mu^{0} has a special interpretation since \mu^{0}_{k}=\mathbb{E}(Z_{ik})=\mathbb{E}(-\log(f_{k}(X_{i})))=D(f_{0}\|f_{k})+H(f_{0}) where H(f_{0}):=-\int_{\mathcal{X}}f_{0}(x)\log f_{0}(x)\lambda(dx) is the entropy of the true DGP and D(f_{0}\|f_{k}) is the Kullback–Leibler (KL) divergence between the true DGP and model k , D(f_{0}\|f_{k}):=\int_{\mathcal{X}}f_{0}(x)\Big(\log\frac{f_{0}(x)}{f_{k}(x)}\Big)\lambda(dx). (1) Thus, \mu^{0}=(\mu^{0}_{1},\ldots,\mu^{0}_{K})^{\mathtt{T}} is a vector containing the KL divergences between the true DGP and the K models, up to a common additive constant H(f_{0}) . Estimation and inference for \mu^{0} are straightforward. Let \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}:=\frac{1}{n}\sum_{i=1}^{n}Z_{i} and observe that since Z_{1},\ldots,Z_{n} are i.i.d., we have \begin{split}\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}&\xrightarrow[n\to\infty]{\mathrm{a.s.}}\mu^{0},\\ \sqrt{n}(\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}-\mu^{0})&\xrightarrow[n\to\infty]{\mathrm{d}}\mathcal{N}(0,\Sigma^{0})\end{split} (2) where \Sigma^{0}=\mathrm{{\mathrm{Cov}}}(Z_{1})\in\mathbb{R}^{K\times K} , by the strong law of large numbers and the central limit theorem (CLT), assuming \Sigma^{0} exists and is positive definite ( 2.1 ). Thus, one could estimate \mu^{0} by the sample mean \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n} and construct Wald-type confidence sets for \mu^{0} based on an estimate of \Sigma^{0} such as the sample covariance matrix. Instead, however, we use a Bayesian multivariate normal model since it facilitates inference for many questions of interest and tends to be more stable at small sample sizes. Specifically, we employ a Bayesian model where \begin{split}Z_{1},\ldots,Z_{n}\mid\mu,\Sigma\;&\overset{\mathrm{iid}}{\sim}\;\mathcal{N}(\mu,\Sigma),\\ (\mu,\Sigma)&\sim\pi(\mu,\Sigma),\end{split} (3) and \pi(\mu,\Sigma) is a prior on the unknown mean \mu and covariance matrix \Sigma ; see Section 3.1 . We write \mu , \Sigma for the parameters of this Bayesian model, and \mu^{0} , \Sigma^{0} for the true values. Uncertainty quantification for \mu enables statistical inference for the relative distances from the true DGP. Note that looking only at marginal uncertainties in each entry \mu_{k} alone is not particularly useful, for at least two reasons: first, H(f_{0}) is unknown and can be very difficult to estimate; second, when the log-likelihood values for models j and k are strongly correlated, it can happen that there is very little uncertainty in the difference \mu_{j}-\mu_{k} even if there is high uncertainty in \mu_{j} and \mu_{k} . Hence, multivariate inference for \mu=(\mu_{1},\ldots,\mu_{K})^{\texttt{T}} is far more informative than univariate inference for each \mu_{k} . The LaD approach does not require any assumptions regarding the form of the models or the true DGP, except for the moment conditions needed for the CLT to apply. While the Bayesian model does assume the negative log-likelihoods Z_{1},\ldots,Z_{n} are Gaussian, which is unlikely to hold, this discrepancy has minimal impact on inferences for \mu since the posterior only depends on Z_{1},\ldots,Z_{n} through the sufficient statistics, namely the sample mean and sample covariance of the Z_{i} ’s. Consequently, we find that LaD provides correctly calibrated inferences for model selection, despite the incorrectness of the Gaussian model. 2.2 Robust model selection using LaD Using the LaD framework in Section 2.1 , one can perform frequentist or Bayesian inference for various hypotheses, such as whether model k minimizes the KL divergence to the true DGP (that is, \mu^{0}_{k}=\min_{j}\mu^{0}_{j} ) or whether model k is closer than model j to the true DGP (that is, \mu^{0}_{k}<\mu^{0}_{j} ). To perform robust model selection, we aim to identify the simplest model that is “close enough” to having minimal KL divergence. Specifically, we consider any model whose KL divergence is within a tolerance \delta>0 of the minimum to be sufficiently close. This enables more parsimonious models to be favored when their fit is roughly comparable to that of more complex models. Definition 1 . Given \delta>0 , under the setup above, we say model k is \delta -optimal if D(f_{0}\|f_{k})\leq\min_{j}D(f_{0}\|f_{j})+\delta. (4) Note that Equation 4 is equivalent to \mu^{0}_{k}\leq\min_{j}\mu^{0}_{j}+\delta . While our focus in this paper is on KL divergence, Definition 1 extends naturally to other measures of discrepancy between distributions, such as Wasserstein distance or Bregman divergence. 2.2.1 Best minimal-complexity \delta -optimal model Let M_{\delta}(\mu) denote the set of \delta -optimal models as a function of \mu , that is, M_{\delta}(\mu):=\Big\{k:\mu_{k}\leq\min_{j}\mu_{j}+\delta\Big\}. (5) Suppose c:\{1,\ldots,K\}\to[0,\infty) assigns a complexity c(k) to model k , with smaller values corresponding to simpler models. We refer to a set of models with the same c(k) as a complexity class. Among \delta -optimal models, the minimal complexity attained is c_{\delta}^{*}(\mu):=\min\{c(k):k\in M_{\delta}(\mu)\}. (6) Since there may be multiple \delta -optimal models that attain this minimal complexity c_{\delta}^{*}(\mu) , among these we prefer the ones with best fit, that is, with minimal KL from the true DGP. Thus, we define the best minimal-complexity \delta -optimal model(s) as M^{*}_{\delta}(\mu):=\argmin_{k\,:\,c(k)=c_{\delta}^{*}(\mu)}\mu_{k}. (7) The set M^{*}_{\delta}(\mu) contains those models that (i) have the lowest possible complexity attainable by a \delta -optimal model, and (ii) have the smallest KL divergence within that complexity class. Typically, M^{*}_{\delta}(\mu) will consist of a single model, but it can contain multiple models if there are ties \mu_{j}=\mu_{k} . For robust model selection, our goal is to quantify uncertainty about which model or models belong to M^{*}_{\delta}(\mu^{0}) for the true \mu^{0} . 2.2.2 Reproducible inference for M^{*}_{\delta}(\mu) A natural way to infer the true set M^{*}_{\delta}(\mu^{0}) would be to consider the posterior probability that each model k is in M^{*}_{\delta}(\mu) , that is, P(k\in M^{*}_{\delta}(\mu)\mid Z_{1:n}) , where \mu\mid Z_{1:n} is distributed according to the posterior under a Bayesian model as in Equation 3 . However, if we use a continuous prior on \mu and the true M^{*}_{\delta}(\mu^{0}) contains multiple models, then P(k\in M^{*}_{\delta}(\mu)\mid Z_{1:n}) is not an accurate representation of uncertainty. For example, when there is correlation among models, some models that belong to M^{*}_{\delta}(\mu^{0}) may receive arbitrarily small posterior probability; see Section 4 . One solution would be to use a prior that allows \mu_{j}=\mu_{k} with positive probability for any j,k , but this would make posterior inference more computationally burdensome and complicate theoretical analysis. Instead, we propose a computationally and theoretically attractive alternative based on a smooth selection function that relaxes the hard minimum in Equation 7 . For each model k and sample size n , we define a within-class soft-selection score, \displaystyle r_{nk}(\mu):=\exp\!\big(-\alpha_{n}(\mu_{k}-\mu_{\min,c(k)})\big), (8) where \mu_{\min,c(k)}=\min_{j\,:\,c(j)=c(k)}\mu_{j} and \alpha_{n}>0 is a temperature parameter such that \alpha_{n}\to\infty and \alpha_{n}=o(\sqrt{n}) . In all of the examples in this paper, we use \alpha_{n}=n^{0.45} . Note that r_{nk}(\mu)\in(0,1] , with r_{nk}(\mu)=1 for models k having minimal \mu_{k} within their complexity class c(k) . We now introduce our proposed model selection score. Definition 2 . The smooth LaD criterion (SLC) score is w_{\delta}(k\mid Z_{1:n})=P\big(c_{\delta}^{*}(\mu)=c(k)\mid Z_{1:n}\big)\,\mathbb{E}\big(r_{nk}(\mu)\mid Z_{1:n}\big) (9) where \mu\mid Z_{1:n} is distributed according to posterior of a model as in Equation 3 . An estimator of M_{\delta}^{*}(\mu^{0}) can be constructed as \hat{M}^{*}_{\delta}=\{k:w_{\delta}(k\mid Z_{1:n})>\omega\} , where \omega is a threshold between 0 and 1 . In Section 4 , we justify the SLC score by showing that it is asymptotically consistent, that is, w_{\delta}(k\mid Z_{1:n}) concentrates on the true target set M^{*}_{\delta}(\mu^{0}) as n grows. The SLC score w_{\delta}(k\mid Z_{1:n}) is defined as the product of (i) a between-class selection factor and (ii) a within-class selection factor. The first factor represents the probability that model k attains the minimal complexity among all \delta -optimal models. The second factor quantifies model k ’s relative performance within its complexity class. We compute each of the two factors with a Monte Carlo approximation using posterior samples from our Bayesian model; see Algorithm 2 for details. For interpretation, if there is only one model per complexity class, then r_{nk}(\mu)=1 for all k , so Equation 9 simplifies to w_{\delta}(k\mid Z_{1:n})=P\big(\argmin_{j\in M_{\delta}(\mu)}c(j)=\{k\}\mid Z_{1:n}\big) , because c_{\delta}^{*}(\mu)=c(k) if and only if \argmin_{j\in M_{\delta}(\mu)}c(j)=\{k\} . More generally, whenever |M^{*}_{\delta}(\mu^{0})|=1 , the SLC score w_{\delta}(k\mid Z_{1:n}) behaves similarly to the posterior probability P(k\in M^{*}_{\delta}(\mu)\mid Z_{1:n}) . Meanwhile, when |M^{*}_{\delta}(\mu^{0})|>1 , the SLC score w_{\delta}(k\mid Z_{1:n}) remains stable since it does not concentrate on a single model. This is because, under the posterior, \mu_{k}-\mu_{\min,c(k)}=O_{p}(n^{-1/2}) for the best fitting models k with complexity c(k) , and we choose \alpha_{n}=o(\sqrt{n}) , which makes \mathbb{E}\big(r_{nk}(\mu)\mid Z_{1:n}\big)\to 1 as n\to\infty . 2.3 Interpreting the tolerance \delta The tolerance \delta makes the selection procedure robust to model misspecification, since it allows one to select models that have a KL divergence within \delta of being minimal. Smaller values of \delta enforce stricter optimality but may only be satisfied by complex models, whereas larger values of \delta encourage parsimony at the cost of fit. Rather than choosing one value of \delta , we generally recommend computing the SLC score for a range of \delta values, as demonstrated in the examples in Section 5 . Since the units of \delta may not be intuitively clear, in this section we provide a technique for putting \delta on an interpretable scale. Motivated by an idea presented by Pescador-Barrios et al. 2025 (Section 4.3), we augment the set of candidate models with a deliberately misspecified “noise” model M_{\mathrm{noise}} with density f_{\mathrm{noise}} , chosen to represent a baseline that does not capture any meaningful signal in the data. Under the noise model, the expected negative log-likelihood is \mu^{0}_{\mathrm{noise}}:=\mathbb{E}(-\log f_{\mathrm{noise}}(X_{i}))=D(f_{0}\|f_{\mathrm{noise}})+H(f_{0}). This noise model should satisfy: (i) full support over \mathcal{X} , so that D(f_{0}\|f_{\mathrm{noise}})<\infty , (ii) stability under estimation, so that \mu^{0}_{\mathrm{noise}} can be estimated with little uncertainty, and (iii) minimal signal (for example, an intercept-only or null model). Now, we define \tau:=\delta/(\mu^{0}_{\mathrm{noise}}-\mu^{0}_{\mathrm{min}}) (10) where \mu^{0}_{\mathrm{min}}=\min_{k}\mu^{0}_{k} . Note that, equivalently, \tau=\delta/(D_{\mathrm{noise}}-D_{\mathrm{min}}) , where D_{\mathrm{noise}}:=D(f_{0}\|f_{\mathrm{noise}}) and D_{\mathrm{min}}:=\min_{k}D(f_{0}\|f_{k}) . Thus, \tau is a rescaled version of \delta that puts it in interpretable units: Specifically, \tau represents the proportion of the explainable information that we are willing to sacrifice in return for model simplicity. A value of \tau>1 signifies that \delta is excessively large, such that even it tolerates the noise model. By the definition of the \delta -optimal set M_{\delta}(\mu^{0}) ( Equation 5 ), for any model k\in M_{\delta}(\mu^{0}) , D(f_{0}\|f_{k})\leq D_{\mathrm{min}}+\tau\,(D_{\mathrm{noise}}-D_{\mathrm{min}}). Rearranging this expression, we obtain \frac{D_{\mathrm{noise}}-D(f_{0}\|f_{k})}{D_{\mathrm{noise}}-D_{\mathrm{min}}}\geq 1-\tau. This yields a natural interpretation: A model is \delta -optimal if it recovers at least 100(1-\tau)\% of the total possible improvement from the noise model to the best available model, in terms of KL divergence. Thus, the choice of \delta becomes interpretable as a relative tolerance expressed on this information-theoretic scale. Suitable choices for the noise model depend on the context. For example, for multivariate Gaussian models, we use f_{\mathrm{noise}}=\mathcal{N}(0,I) . For generalized linear models, we use an intercept-only model (no covariates). For mixture models, a suitable noise baseline could be a uniform density over the observed data range or a one-component mixture. 2.4 Extending to unknown parameters and other loss functions Up to now, we have assumed that each model consists of a single distribution with known parameters, but in practice, a model will usually have unknown parameters that need to be estimated. We now show how to handle such models in our framework. Suppose model k is a parametric family with densities \{f_{k}(x;\,\theta_{k}):\,\theta_{k}\in\Theta_{k}\} , where \Theta_{k}\subseteq\mathbb{R}^{d_{k}} and d_{k} is the dimension of the parameter vector \theta_{k} for model k . Furthermore, we generalize from the negative log-likelihood to an arbitrary loss function \ell_{k} for model k . As before, suppose X,X_{1},\ldots,X_{n}\in\mathcal{X} i.i.d. \sim P_{0} , where P_{0} has density f_{0} with respect to \lambda . For each k\in\{1,\ldots,K\} , let \ell_{k}:\mathcal{X}\times\Theta_{k}\to\mathbb{R} be measurable, and define \displaystyle\theta_{k}^{*}\in\argmin_{\theta_{k}\in\Theta_{k}}\mathbb{E}\big(\ell_{k}(X;\theta_{k})\big). (11) Define the random vector \ell(X;\theta^{*}):=[\ell_{1}(X;\theta_{1}^{*}),\ldots,\ell_{K}(X;\theta_{K}^{*})]^{\texttt{T}}\in\mathbb{R}^{K} , and set \begin{split}\mu^{0}&:=\mathbb{E}\big(\ell(X;\theta^{*})\big)\in\mathbb{R}^{K},\\ \Sigma^{0}&:={\mathrm{Cov}}\big(\ell(X;\theta^{*})\big)\in\mathbb{R}^{K\times K}.\end{split} (12) In Equations 11 and 12 , the expectations are with respect to X\sim P_{0} . Condition 2.1 . Assume: (i) \mathbb{E}\big(\ell_{k}(X;\theta_{k}^{*})^{2}\big)<\infty , that is, \ell(X;\theta^{*}) has finite second moments, and (ii) \Sigma^{0} is a positive definite matrix. To estimate the parameter vector \theta_{k} for each model k , we use the M-estimator, \hat{\theta}_{k}:=\argmin_{\theta_{k}\in\Theta_{k}}\Big(\frac{1}{n}\sum_{i=1}^{n}\ell_{k}(X_{i};\theta_{k})\Big). In the examples in this article, we focus on the negative log-likelihood loss, in which case the M-estimator is simply the maximum likelihood estimator (MLE), but the method extends to any loss leading to a consistent M-estimator. Plugging in the estimators \hat{\theta}_{1},\ldots,\hat{\theta}_{K} for models 1,\ldots,K , respectively, the vector of loss values for X_{i} becomes Z_{i}(\hat{\theta}):=\ell(X_{i};\hat{\theta})=[\ell_{1}(X_{i};\hat{\theta}_{1}),\ldots,\ell_{K}(X_{i};\hat{\theta}_{K})]^{\texttt{T}}\in\mathbb{R}^{K}. Then, the sample mean and covariance based on Z_{1}(\hat{\theta}),\ldots,Z_{n}(\hat{\theta})\in\mathbb{R}^{K} are \begin{split}\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}(\hat{\theta})&:=\frac{1}{n}\sum_{i=1}^{n}\ell(X_{i};\hat{\theta}),\\ S_{n}(\hat{\theta})&:=\frac{1}{n}\sum_{i=1}^{n}\big(\ell(X_{i};\hat{\theta})-\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}(\hat{\theta})\big)\big(\ell(X_{i};\hat{\theta})-\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}(\hat{\theta})\big)^{\texttt{T}}.\end{split} (13) Under regularity conditions ( S2.1 ), including that \hat{\theta}_{k} is a consistent estimator of \theta_{k}^{*} for each k , the limits in Equation 2 also hold for \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}(\hat{\theta}) (see Theorem S2.3 ) generalizing from the no-parameter case to the setting of estimated parameters. Thus, frequentist inference for \mu^{0} could again be used in this more general setting; but as mentioned in Section 2.1 , we instead recommend a Bayesian approach ( Section 3 ). 2.5 Adjustment for overfitting bias For large sample sizes n , plugging in a consistent estimator \hat{\theta}_{k} as a proxy for \theta^{*} leads to negligible error. However, for smaller sample sizes, \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}(\hat{\theta}) tends to be downward biased as an estimator of \mu^{0} , because the parameters are fit on the same data that is used to estimate \mu^{0} . To adjust for this overfitting, we propose a simple bias correction based on the asymptotic distribution of the likelihood-ratio test statistic. To derive the correction, assume the loss is the negative log-likelihood \ell_{k}(x;\theta_{k})=-\log f_{k}(x;\theta_{k}) . Then the likelihood-ratio test statistic is \Lambda_{k}=2\Big(\sum_{i=1}^{n}\ell_{k}(X_{i};\theta_{k}^{*})-\sum_{i=1}^{n}\ell_{k}(X_{i};\hat{\theta}_{k})\Big). Rearranging and taking the expectation under P_{0} yields that \mu^{0}_{k}=\mathbb{E}\Big(\frac{1}{n}\sum_{i=1}^{n}\ell_{k}(X_{i};\theta_{k}^{*})\Big)=\mathbb{E}\Big(\frac{1}{n}\sum_{i=1}^{n}\ell_{k}(X_{i};\hat{\theta}_{k})\Big)+\frac{1}{2n}\mathbb{E}\big(\Lambda_{k}\big). Hence, the bias of \frac{1}{n}\sum_{i=1}^{n}\ell_{k}(X_{i};\hat{\theta}_{k}) as an estimator of \mu^{0}_{k} is -(2n)^{-1}\mathbb{E}(\Lambda_{k}) . By Wilks’ theorem, if model k were correctly specified, then \Lambda_{k} would be approximately \chi^{2} -distributed with d_{k}=\mathrm{dim}(\Theta_{k}) degrees of freedom, under regularity conditions. Since the mean of \chi^{2}(d_{k}) is d_{k} , this suggests using the bias-corrected estimator \hat{\mu}_{k}^{\mathrm{bc}}:=\frac{1}{n}\sum_{i=1}^{n}\ell_{k}(X_{i};\hat{\theta}_{k})+d_{k}/(2n) . Since LaD operates on vectors Z_{1},\ldots,Z_{n}\in\mathbb{R}^{K} , we apply this adjustment entrywise to define bias-corrected LaD vectors \displaystyle Z^{\mathrm{bc}}_{i}(\hat{\theta}):=Z_{i}(\hat{\theta})+\frac{d}{2n} (14) where d=(d_{1},\ldots,d_{K})^{\texttt{T}}\in\mathbb{R}^{K} . In practice, we expect all of the models to be misspecified, so Wilks’ theorem does not directly apply; nonetheless, empirically we find that the approximation is sufficiently good that the bias correction in Equation 14 works well. 3 Computation In this section, we describe our Bayesian model for the LaD framework ( Section 3.1 ) and provide a step-by-step algorithm for our proposed methodology ( Section 3.2 ). 3.1 Bayesian inference in the LaD framework Let Z_{1},\ldots,Z_{n}\in\mathbb{R}^{K} denote the LaD vectors, Z_{i}=(Z_{i1},\ldots,Z_{iK})^{\texttt{T}} ; these may comprise negative log-likelihood values Z_{ik}=-\log(f_{k}(X_{i})) (as in Section 2.1 ), loss values Z_{ik}=\ell_{k}(X_{i};\hat{\theta}_{k}) evaluated at estimated parameters \hat{\theta}_{k} (as in Section 2.4 ), or bias-corrected versions of such values Z_{ik}=\ell_{k}(X_{i};\hat{\theta}_{k})+d_{k}/(2n) (as in Equation 14 ). To perform Bayesian inference, we model Z_{1},\ldots,Z_{n} as i.i.d. \sim\mathcal{N}(\mu,\Sigma) and place a Normal-Inverse-Wishart (NIW) prior on (\mu,\Sigma) . Specifically, the prior density of (\mu,\Sigma) is \mathrm{NIW}(\mu,\Sigma\mid\mu_{0},\lambda_{0},\Psi_{0},\nu_{0})=\mathcal{N}(\mu\mid\mu_{0},\Sigma/\lambda_{0})\,\mathrm{InverseWishart}(\Sigma\mid\Psi_{0},\nu_{0}) where \mu_{0}\in\mathbb{R}^{K} is the prior mean, \lambda_{0}>0 is a scaling factor, \nu_{0}>K-1 is the degrees of freedom, and \Psi_{0} is the (positive definite) inverse scale matrix. The posterior distribution of \mu,\Sigma\mid Z_{1:n} is NIW with updated parameters, specifically, \displaystyle\mu,\Sigma\mid Z_{1:n}\;\sim\;\mathrm{NIW}(\mu_{n},\lambda_{n},\Psi_{n},\nu_{n}) (15) where \lambda_{n}=\lambda_{0}+n , \mu_{n}=(\lambda_{0}\mu_{0}+n\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n})/\lambda_{n} , \Psi_{n}=\Psi_{0}+nS_{n}+(\lambda_{0}n/\lambda_{n})(\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}-\mu_{0})(\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}-\mu_{0})^{\texttt{T}} , and \nu_{n}=\nu_{0}+n , where \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}=\frac{1}{n}\sum_{i=1}^{n}Z_{i} and S_{n}=\frac{1}{n}\sum_{i=1}^{n}\big(Z_{i}-\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}\big)\big(Z_{i}-\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}\big)^{\texttt{T}} . To generate a posterior sample of (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) , all that needs to be done is to draw \Sigma^{\prime}\sim\mathrm{InverseWishart}(\Psi_{n},\nu_{n}) and then draw \mu^{\prime}\sim\mathcal{N}(\mu_{n},\Sigma^{\prime}/\lambda_{n}) . To perform posterior inference, we generate multiple posterior samples and use Monte Carlo approximations. For the hyperparameter settings, we recommend \mu_{0}=0 , \nu_{0}=K+2 , \lambda_{0}=0.01 , and \Psi_{0}=I_{K} (the K\times K identity matrix), since these yield a weakly informative prior that allows the data to mostly dominate. 3.2 Algorithm for LaD model selection Algorithm 1 provides a step-by-step description of the workflow for our proposed method. Algorithm 1 Workflow for LaD model selection Input: Observations X_{1},\ldots,X_{n} , parametrized models k=1,\ldots,K and their corresponding complexities c(1),\ldots,c(K)\geq 0 , number of Monte Carlo samples T>0 , and tolerance \delta\geq 0 . 1. Compute parameter estimates \hat{\theta}_{k} for each model k . 2. Compute the bias-corrected LaD values Z_{ik}=\ell_{k}(X_{i};\hat{\theta}_{k})+d_{k}/(2n) for all i and k . 3. Sample \mu^{(t)},\Sigma^{(t)} from the NIW posterior ( Equation 15 ) for t=1,\ldots,T . 4. Compute the SLC scores \hat{w}_{\delta}(k) for each k using Algorithm 2 . 5. (Optional) Compute the rescaled tolerance \hat{\tau}=\delta/(\hat{\mu}_{\mathrm{noise}}-\hat{\mu}_{\mathrm{min}}) where \hat{\mu}_{\mathrm{noise}} is an estimate of \mu^{0}_{\mathrm{noise}} ( Section 2.3 ), \hat{\mu}_{\mathrm{min}}=\min_{k}\hat{\mu}_{k} , and \hat{\mu}_{k}=\frac{1}{n}\sum_{i=1}^{n}Z_{ik} . Output: Return the samples \mu^{(t)},\Sigma^{(t)} , SLC scores \hat{w}_{\delta}(k) , and rescaled tolerance \hat{\tau} . Recall that our SLC score w_{\delta}(k\mid Z_{1:n}) ( Equation 9 ) is defined as the product of a between-class selection factor P\big(c_{\delta}^{*}(\mu)=c(k)\mid Z_{1:n}\big) and a within-class selection factor \mathbb{E}\big(r_{nk}(\mu)\mid Z_{1:n}\big) . To compute w_{\delta}(k\mid Z_{1:n}) , we generate posterior draws \mu^{(1)},\ldots,\mu^{(T)} given Z_{1:n} as described in Section 3.1 , then compute each of these two factors using a simple Monte Carlo approximation, and take their product. See Algorithm 2 for details. Algorithm 2 Computing the smooth LaD criterion (SLC) scores Input: Posterior samples \mu^{(1)},\ldots,\mu^{(T)}\in\mathbb{R}^{K} , complexities c(1),\dots,c(K)\geq 0 , tolerance \delta\geq 0 , and temperature \alpha_{n}>0 . for t=1,\dots,T do 1. Between-class selection: Compute the set of \delta -optimal models, M_{\delta}(\mu^{(t)})=\big\{k:\mu_{k}^{(t)}\leq\min_{j}\mu_{j}^{(t)}+\delta\big\}. Then, compute the minimal complexity among \delta -optimal models, c_{\delta}^{*}(\mu^{(t)})=\min\big\{c(k):k\in M_{\delta}(\mu^{(t)})\big\}. 2. Within-class selection: For each model k=1,\ldots,K , compute the within-class minimum \mu_{\min,c(k)}^{(t)}=\min\big\{\mu_{j}^{(t)}:c(j)=c(k)\big\} , and compute the soft-selection score r_{nk}(\mu^{(t)})=\exp\big(-\alpha_{n}(\mu_{k}^{(t)}-\mu_{\min,c(k)}^{(t)})\big). for k=1,\dots,K do \hat{p}_{\delta}(k):=\frac{1}{T}\sum_{t=1}^{T}\mathds{1}\big(c_{\delta}^{*}(\mu^{(t)})=c(k)\big)\hskip 9.24994pt\text{and}\hskip 9.24994pt\hat{r}(k):=\frac{1}{T}\sum_{t=1}^{T}r_{nk}(\mu^{(t)}). Output: Return the estimated SLC scores \hat{w}_{\delta}(k):=\hat{p}_{\delta}(k)\,\hat{r}(k) for k=1,\ldots,K . 4 Theory In this section, we prove that our proposed method is asymptotically consistent ( Section 4.1 ) and we show that alternative approaches based on a plug-in posterior or using a hard minimum exhibit instability ( Section 4.2 ). 4.1 Consistency of the SLC score Our first result establishes that the SLC score w_{\delta}(k\mid Z_{1:n}) concentrates on the true target set M_{\delta}^{*}(\mu^{0}) , asymptotically. Theorem 4.1 . Assume Conditions 2.1 and S2.1 , and suppose \delta\neq\mu^{0}_{k}-\mu^{0}_{\mathrm{min}} for all k=1,\ldots,K . Let \alpha_{n}\geq 0 such that \alpha_{n}\to\infty with \alpha_{n}=o(\sqrt{n}) . Define w_{\delta}(k\mid Z_{1:n}) as in Equation 9 based on the NIW posterior in Section 3.1 . Then, for all k\in\{1,\dots,K\} , w_{\delta}(k\mid Z_{1:n})\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(k\in M^{*}_{\delta}(\mu^{0})\big). All proofs are collected in Section S2 . The next result shows that the Monte Carlo estimates \hat{w}_{\delta}(k) produced by Algorithm 2 also concentrate on M_{\delta}^{*}(\mu^{0}) . Theorem 4.2 . Under the same assumptions as Theorem 4.1 , for all k\in\{1,\dots,K\} , \hat{w}_{\delta}(k)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(k\in M^{*}_{\delta}(\mu^{0})\big). Theorem 4.3 . Under the same assumptions as Theorem 4.1 , for any fixed \omega\in(0,1) , P\big(\hat{M}^{*}_{\delta}=M^{*}_{\delta}(\mu^{0})\big)\xrightarrow[n\to\infty]{}1, where \hat{M}^{*}_{\delta}:=\{k:w_{\delta}(k\mid Z_{1:n})>\omega\} . Theorem 4.3 also holds when using \hat{w}_{\delta}(k) instead of w_{\delta}(k\mid Z_{1:n}) . 4.2 Instability of alternative approaches at ties The results in this section provide the rationale for using our soft-selection score for within-class selection ( Equation 8 ), rather than the Bayesian plug-in posterior P(k\in M^{*}_{\delta}(\mu)\mid Z_{1:n}) or a hard-minimum selection score. When multiple models of equal complexity are tied for having minimal KL divergence, these naive alternatives randomly allocate probability mass among tied models, and not necessarily in a uniform way. To understand why the issue arises, suppose there are K=3 models of equal complexity and all three are equally close to the true DGP, that is, \mu^{0}=(\mu^{0}_{1},\mu^{0}_{2},\mu^{0}_{3})^{\texttt{T}} where \mu^{0}_{1}=\mu^{0}_{2}=\mu^{0}_{3} . Furthermore, suppose models 1 and 2 are strongly negatively correlated and have much higher variance than model 3 , for example, Σ^0 = [ 1 -0.99 0 -0.99 1 0 0 0 0.01 ] . Under the posterior, suppose for simplicity that \mu\sim\mathcal{N}(\mu^{0},\Sigma^{0}/n) given Z_{1:n} , which is the expected asymptotic behavior. Then with fairly high probability, \min\{\mu_{1},\mu_{2}\}<\mu_{3} because \mu_{3} has much smaller variance than the other two, and the strong negative correlation makes \mu_{1}\approx-\mu_{2} . Specifically, P(\mu_{k}=\min_{j}\mu_{j}\mid Z_{1:n}) is \approx 0.48 for k\in\{1,2\} and \approx 0.04 for k=3 . Thus, model 3 is rarely selected in this scenario, even though all three models are equally good. Figure 2 visualizes this pattern. Figure 2 : Instability under ties with K=3 . The figure shows 1,000 posterior draws from \mu\sim\mathcal{N}(\mu^{0},\Sigma^{0}/n) with n=500 , \mu^{0}=(0,0,0)^{\mathtt{T}} , and \Sigma^{0} as specified in the text. The left panel shows the strong negative correlation between \mu_{1} and \mu_{2} , the middle panel shows that \mu_{3} is much more concentrated than \mu_{1} and \mu_{2} , and the right panel makes clear that \mu_{3} is rarely the minimum. We make this phenomenon precise in the following results. For x\in\mathbb{R}^{K} , define \displaystyle\pi_{k}(x)=\left\{\begin{array}[]{ll}P\Big(W_{j}-W_{k}\leq x_{j}-x_{k}\text{ for all }j\in M_{\delta}^{*}(\mu^{0})\setminus\{k\}\Big)&\mbox{if }k\in M_{\delta}^{*}(\mu^{0})\\ 0&\mbox{if }k\not\in M_{\delta}^{*}(\mu^{0})\end{array}\right. where W\sim\mathcal{N}(0,\Sigma^{0}) . In Theorem 4.4 , parts (a) and (b) show the instability of the plug-in posterior and the hard-minimum rule, respectively. When |M^{*}_{\delta}(\mu^{0})|>1 , the choice among tied models is randomly distributed in a way that depends on the covariance of the limiting Gaussian. Theorem 4.4 . Assume Conditions 2.1 and S2.1 , let \mu,\Sigma\mid Z_{1:n} be distributed according to Equation 15 , and suppose \delta\neq\mu^{0}_{k}-\mu^{0}_{\mathrm{min}} for all k=1,\ldots,K . Let W\sim\mathcal{N}(0,\Sigma^{0}) . (a) \displaystyle\big(P(k\in M^{*}_{\delta}(\mu)\mid Z_{1:n})\big)_{k=1}^{K}\xrightarrow[n\to\infty]{\mathrm{d}}\big(\pi_{k}(W)\big)_{k=1}^{K} . (b) Consider a modified score \tilde{w}_{\delta}(k\mid Z_{1:n}) defined as in Equation 9 but with \tilde{r}_{nk}(\mu)=\mathds{1}(\mu_{k}=\mu_{\min,c(k)}) in place of r_{nk}(\mu) . Then \displaystyle\big(\tilde{w}_{\delta}(k\mid Z_{1:n})\big)_{k=1}^{K}\xrightarrow[n\to\infty]{\mathrm{d}}\big(\pi_{k}(W)\big)_{k=1}^{K} . Interestingly, the limiting behavior of \tilde{w}_{\delta}(k\mid Z_{1:n}) coincides with the asymptotic BayesBag posterior with M=N ( 19 , Theorem 3.2) . Corollary 4.5 shows that in the case of two tied models, the scores of the two models are asymptotically distributed as (U,1-U) where U\sim\mathrm{Uniform}(0,1) . Corollary 4.5 . Assume the conditions of Theorem 4.4 . If M_{\delta}^{*}(\mu^{0})=\{k_{1},k_{2}\} then \big(\tilde{w}_{\delta}(k_{1}\mid Z_{1:n}),\tilde{w}_{\delta}(k_{2}\mid Z_{1:n})\big)\xrightarrow[n\to\infty]{\mathrm{d}}(U,1-U) where U\sim\mathrm{Uniform}(0,1) . 5 Examples 5.1 Gaussian mixture models We revisit the motivating example from Section 1 using finite Gaussian mixture models on the Shapley galaxy data. We implement the LaD workflow in Algorithm 1 as follows. First, we fit a k -component Gaussian mixture for each k=1,\ldots,10 using the expectation-maximization (EM) algorithm to estimate the maximum a posteriori (MAP) solution ( 14 ) . For this, we employ the mclust package in R ( 37 ; 32 ) with random initialization, using a moderately informative normal-inverse-gamma prior on the mean and variance of each mixture component, independently. For each k , we run the algorithm on 50 independent restarts and keep the run with the highest observed log-likelihood. See Section S3.1 for model details. We then compute the bias-corrected LaD values Z_{ik}=\ell_{k}(X_{i};\hat{\theta}_{k})+d_{k}/(2n) ( Equation 14 ) where \hat{\theta}_{k} is the MAP estimate and d_{k}=3k-1 since a univariate k -component mixture with unconstrained means, variances, and mixture weights has a total of k+k+(k-1)=3k-1 parameters. While mixtures do not perfectly satisfy the LRT conditions, we find that this bias correction is an effective approximation in practice. We draw T=1000 samples of \mu\mid Z_{1:n} from the NIW posterior ( Equation 15 ), and then run Algorithm 2 to obtain the SLC scores \hat{w}_{\delta}(k) for a range of values of the tolerance \delta>0 . We use \alpha_{n}=n^{0.45} for the temperature and c(k)=k for the model complexities. To compute the rescaled tolerance \hat{\tau} as described in Algorithm 1 , we let {\hat{\mu}}_{k}=\frac{1}{n}\sum_{i=1}^{n}Z_{ik} be the mean of the bias-adjusted LaD values, and we define the noise baseline to be the uniform distribution over the observed range [\min_{i}x_{i},\max_{i}x_{i}] , which yields {\hat{\mu}}_{\mathrm{noise}}=\log(\max_{i}x_{i}-\min_{i}x_{i}) . This puts \delta on an interpretable scale, namely, \hat{\tau}\in[0,1] is the fraction of explainable KL improvement one is willing to give up for model simplicity. Figure 3 : Gaussian mixture model results on the Shapley galaxy dataset. (Top row) Usual Bayesian posterior on k (bars) with AIC (red dashed) and BIC (blue dot-dash) overlaid; as n increases, all three trend toward larger k . (Second row) LaD posteriors on \mu_{k} for each k show the fit to the true DGP as a function k , and concentrate as n grows. (Third row) SLC score \hat{w}_{\delta}(k) for tolerances \delta\in\{0.3,0.12,0.06\} . Larger \delta selects simpler models; increasing n concentrates the scores onto a single k for each \delta . (Bottom row) Heatmap of “posterior path” of SLC score \hat{w}_{\delta}(k) shows the transition from selecting more complex to simpler models as a function of the rescaled tolerance \hat{\tau}=\delta/(\hat{\mu}_{\mathrm{noise}}-\hat{\mu}_{\mathrm{min}})\in[0,1] . Figure 3 displays the results. For comparison, the top row shows the usual Bayesian posterior on k , with AIC and BIC overlaid; all three increasingly favor larger k as n grows, exhibiting unstable drift toward complexity under misspecification. The second row shows boxplots of the LaD posterior on \mu_{k} for each k , which provide estimates (with uncertainty quantification) of the KL divergence between the true DGP and each model k , up to a common additive constant. As n increases, we see that the uncertainty in these estimates decreases. As k increases, the estimated KL decreases and gradually levels off, indicating a diminishing improvement in fit as the model complexity grows. The third row plots the SLC scores \hat{w}_{\delta}(k) for \delta\in\{0.30,0.12,0.06\} . Larger \delta values are more tolerant, resulting in selection of simpler models; meanwhile, smaller \delta values enforce a tighter tolerance, requiring more complex models in order to achieve a KL within \delta of being minimal. As n increases, the \hat{w}_{\delta}(k) values concentrate on a single k for each \delta , illustrating the large-sample concentration guaranteed by our theory when no two models are tied in terms of both complexity and KL. Notably, in contrast with AIC, BIC, and the usual Bayesian posterior results in Figure 1 , the LaD results stabilize as n increases. This is illustrated further in the bottom row, which shows the posterior path of \hat{w}_{\delta}(k) as \hat{\tau} varies from 0 to 1 . Here, we see a continuous transition from selecting more complex to simpler models as the proportion of explainable KL that we are willing to sacrifice increases from 0 to 1 . For instance, while the usual Bayesian posterior selects values of k between 10 and 25 when n=4000 ( Figure 1 ), the LaD posterior path shows that k>8 is only needed for very tight tolerances of \hat{\tau}<0.01 , that is, <1\% of explainable KL sacrificed. This plot encapsulates the key features of LaD: It operates without assuming any candidate model is correctly specified, quantifies uncertainty in their relative KL divergences from the true DGP, and places these divergences on an interpretable scale. 5.2 Sparse multivariate normal mean models In this simulation, we consider multivariate normal (MVN) models with sparse mean vectors. We generate n i.i.d. samples x_{1},\ldots,x_{n}\sim\mathcal{N}(\theta_{0},I) , where the true mean vector is \theta_{0}=(1,1,0.5,0.5,0.4,0)^{\texttt{T}} , and I is the 6\times 6 identity matrix. We compare K=7 candidate models, each of which is \mathcal{N}(\theta,I) but with a different sparsity pattern imposed on the mean vector \theta , and we define the complexity c(k) to be the number of nonzero entries of \theta . The models vary in complexity and KL divergence from the truth, \min_{\theta_{k}}D\big(\mathcal{N}(\theta_{0},I)\,\|\,\mathcal{N}(\theta_{k},I)\big) subject to the sparsity pattern in Table 1 ; see Section S3.2 . Table 1 : Candidate models for sparse multivariate normal simulation example. k \theta_{k} c(k) \min D 1 (\theta_{1},0,0,\theta_{4},0,0)^{\texttt{T}} 2 0.705 2 (\theta_{1},\theta_{2},0,0,0,0)^{\texttt{T}} 2 0.33 3 (\theta_{1},\theta_{2},0,0,\theta_{5},0)^{\texttt{T}} 3 0.25 4 (\theta_{1},\theta_{2},0,\theta_{4},0,0)^{\texttt{T}} 3 0.205 5 (\theta_{1},\theta_{2},\theta_{3},0,0,0)^{\texttt{T}} 3 0.205 6 (\theta_{1},\theta_{2},\theta_{3},\theta_{4},\theta_{5},0)^{\texttt{T}} 5 0 7 (\theta_{1},\theta_{2},\theta_{3},\theta_{4},\theta_{5},\theta_{6})^{\texttt{T}} 6 0 Models 1 and 2 are the most parsimonious, but differ greatly in their distance from the truth. Models 4 and 5 are equally close to the truth and have the same complexity, while model 3 is slightly farther in KL divergence but has the same complexity as 4 and 5 . Models 6 and 7 both recover the true DGP exactly, but model 6 is simpler. We implement the LaD approach in Algorithm 1 as follows. We use the MLE to estimate the parameters for model k , which is the vector \hat{\theta}_{k}\in\mathbb{R}^{6} containing the sample means in the coordinates that are free in that model, and zeros elsewhere. That is, for coordinate j\in\{1,\ldots,6\} , \hat{\theta}_{kj}=\bar{x}_{j}\mathds{1}(j\in J_{k}) where \bar{x}_{j}=\frac{1}{n}\sum_{i=1}^{n}x_{ij} and J_{k}\subseteq\{1,\ldots,6\} is the set where model k allows \theta_{kj} to be nonzero. The bias-corrected LaD values are then Z_{ik}=\ell_{k}(x_{i};\hat{\theta}_{k})+d_{k}/(2n) where \ell_{k}(x_{i};\hat{\theta}_{k})=\frac{6}{2}\log(2\pi)+\frac{1}{2}\|x_{i}-\hat{\theta}_{k}\|^{2} and d_{k}=|J_{k}| . We draw T=1000 samples of \mu\mid Z_{1:n} ( Equation 15 ) and compute the SLC scores \hat{w}_{\delta}(k) using Algorithm 2 with a temperature of \alpha_{n}=n^{0.45} . We consider three tolerance levels: (i) \delta=0.75 is large enough to tolerate all of the models, (ii) \delta=0.26 excludes models 1 and 2 but still permits models 3–7, and (iii) \delta=0.05 is small enough that only models 6 and 7 are within \delta of the true DGP; see Table 1 . To interpret the value of \delta , we compute the rescaled tolerances \tau as in Algorithm 1 , using \mathcal{N}(0,I) for the noise model. For instance, the \delta values of 0.75 , 0.26 , and 0.05 correspond to \tau values of approximately 55.3\% , 18.8\% , and 3.7\% , respectively. Figure 4 : Sparse multivariate normal model results. (Top) SLC scores \hat{w}_{\delta}(k) for sample sizes n\in\{50,500,5000\} (columns) and tolerances \delta\in\{0.75,0.26,0.05\} (rows). (Bottom) Boxplots of the LaD posteriors on \mu_{k} for k\in\{1,\ldots,7\} . Figure 4 summarizes the LaD results across the \delta settings, for sample sizes n\in\{50,500,5000\} . Here, we show results for one representative data set of each size. In the first setting ( \delta=0.75 ), both models 1 and 2 are given a nonnegligible SLC score when n is small, but the score of model 1 quickly goes to zero and the score of model 2 goes to 1 as n grows. This is because all candidate models are \delta -optimal at this tolerance level, but models 3–7 have greater complexity, and model 1 has worse fit than model 2. The SLC score concentrates at model 2 since it is the best minimal-complexity \delta -optimal model at this tolerance; see Table 1 . In the second setting ( \delta=0.26 ), the SLC scores are spread more evenly across models when n is small, due to posterior uncertainty. As n increases, the SLC scores concentrate on both models 3 and 4, which have equal complexity and equal KL divergence from the true DGP. This occurs because models 1 and 2, while simpler, are not \delta -optimal at this tolerance level; models 6 and 7 have higher complexity; and model 3 has larger KL from the true DGP than models 4 and 5. Notably, the SLC score is stable in the sense that it does not randomly select one of models 4 and 5, but instead gives them roughly equal score. Thus, the method correctly treats these two models as equally good with respect to this true DGP. In the third setting ( \delta=0.05 ), the SLC scores concentrate on model 6. This is because models 1–5 are not \delta -optimal at this tolerance level, and model 7 is more complex than model 6. Thus, the method correctly chooses model 6 at this value of \delta . 5.2.1 Performance comparison with other methods Up to this point, we have considered qualitative aspects of performance in our comparisons with existing methods. In this section, we quantitatively compare model selection performance with several alternative approaches. For these comparisons, we consider the following methods, including three versions of LaD: (i) (LaD-soft) - Our proposed LaD method (see Algorithm 1 ). (ii) (LaD-hard) - A variant of LaD using hard-minimum selection (see Theorem 4.4 (b)). (iii) (LaD-diag) - A variant of LaD using diagonal covariance \Sigma (see Section S4 ). (iv) (c-posterior) - Coarsened posterior with powers \alpha\in\{10,100\} (see Section S3.2 ). (v) (Bayes) - Usual Bayesian posterior probabilities on models. (vi) (AIC, BIC) - Standard model selection criteria that select a single model. For each n\in\{50,500,5000\} , we generate 50 simulated data sets from \mathcal{N}(\theta_{0},I) as above and, for each data set, we apply each method to select from the set of candidate models in Table 1 . To quantify performance for each \delta , consider the Brier loss S_{\mathrm{B}}:=\sum_{k=1}^{K}(w_{k}-w_{k}^{0})^{2} where w_{k} is the weight assigned to model k by a given method and w_{k}^{0} is an indicator of whether model k is in the true target set, that is, w_{k}^{0}=\mathds{1}(k\in M_{\delta}^{*}(\mu^{0})) . Figure 5 : Performance comparisons using Brier score. Points show the mean Brier loss, and lines show \pm standard error across 50 datasets for sample sizes n\in\{50,500,5000\} (x-axis; vertical dashed lines separate values of n ). Each panel corresponds to tolerance settings: left \delta=0.75 , middle \delta=0.26 , and right \delta=0.05 . Colors indicate the competing methods. Figure 5 displays the means and standard errors of the Brier loss S_{\mathrm{B}} for each tolerance setting, \delta\in\{0.75,0.26,0.05\} . LaD-soft attains the smallest Brier loss on average across all n and \delta . LaD-hard performs comparably to LaD-soft except when there are ties, such as between models 4 and 5 in the case of \delta=0.26 , in which case LaD-soft performs better. LaD-diag works reasonably well but is generally worse than LaD-soft, particularly when \delta=0.05 , demonstrating that modeling the full covariance matrix is important. The c-posterior with \alpha=10 is competitive at tolerance \delta=0.75 , but performs poorly at smaller \delta . Meanwhile, the c-posterior with \alpha=100 performs well at \delta=0.05 , but poorly at larger \delta . This illustrates that the choice of \alpha for the c-posterior needs to be selected in a \delta -dependent way, but this is challenging because \alpha does not have a direct interpretation in terms of distance from the true DGP. Finally, Bayes, AIC, and BIC are only competitive at \delta=0.05 since M_{\delta}^{*}(\mu^{0}) only contains the true DGP in this case. Overall, LaD-soft reliably provides excellent performance across all settings. 5.3 Thermal performance curves in microbial ecology Thermal performance curves (TPCs)—unimodal functions relating performance (such as growth rate) to temperature—are widely used in microbial ecology for understanding how populations respond to environmental conditions ( 22 ) . For example, Prorocentrum minimum is a mixotrophic dinoflagellate that can form harmful blooms; studying its temperature–growth relationship helps predict these events and mitigate their impact on marine ecosystems and human health ( 11 ) . There are many mechanistic and phenomenological TPC models, none of which is universally optimal across traits or taxa ( 24 ; 13 ; 2 ) . We analyze the P. minimum data of Grzebyk and Berland 1996 consisting of n=148 growth-rate observations at 10 temperatures from 10^{\circ} C to 31^{\circ} C. We fit K=9 widely used TPC models of varying complexity: three-parameter models (Gaussian, Mitchell–Angilletta, Brière I, Eubank); four-parameter models (Weibull, Modified Gaussian); a five-parameter model (extended Brière); a six-parameter model (Poly-5); and a seven-parameter model (Sharpe–Schoolfield). These represent a diverse mix of TPC models that are commonly recommended for fitting microbial growth data ( 24 ; 10 ; 23 ) ; see Section S3.3 for their functional forms. Figure 6 : (A) NLS fits (blue lines) of the K=9 candidate TPC models to the n=148 growth-rate observations (black points) across temperatures 10^{\circ}\mathrm{C} – 31^{\circ}\mathrm{C} . Growth rates are reported as divisions per day (\mathrm{div\,d^{-1}}) , converted from per-second rates (\mathrm{s^{-1}}) via \mathrm{div\,d^{-1}}=86{,}400/\ln{2}\times\mathrm{s^{-1}} , as in Grzebyk and Berland 1996 . (B) Posterior distributions of \mu_{k}-\min_{j}\mu_{j} for each model k . Horizontal lines indicate selected \delta thresholds (orange: \delta=0.187 , purple: \delta=0.075 , green: \delta=0.037 , and blue: \delta=0.001 ). (C) SLC scores \hat{w}_{\delta}(k) across models k for each \delta , using the same color for each \delta as in (B). The data consist of pairs (y_{i},t_{i}) where y_{i} is the outcome (growth rate) and t_{i} is the temperature, for observations i=1,\ldots,n . We implement LaD as follows ( Algorithm 1 ). For each candidate model k with parameter \theta_{k} and mean curve g_{k}(t;\theta_{k}) , we fit \hat{\theta}_{k} by nonlinear least squares (NLS), modeling y_{i}\sim\mathcal{N}(g_{k}(t_{i};\theta_{k}),\,\sigma_{k}^{2}) , a standard approach for these models implemented in R ( 15 ) . Figure 6 (A) shows the fitted curve for each model, overlaid on a scatterplot of the data. The bias-corrected LaD values are Z_{ik}=\ell_{k}((y_{i},t_{i});\hat{\theta}_{k})+d_{k}/(2n) where \ell_{k}((y_{i},t_{i});\hat{\theta}_{k})=\frac{1}{2}\log(2\pi\hat{\sigma}_{k}^{2})+\big(y_{i}-g_{k}(t_{i};\hat{\theta}_{k})\big)^{2}/(2\hat{\sigma}_{k}^{2}) , \hat{\sigma}_{k}^{2}=\frac{1}{n}\sum_{i}\big(y_{i}-m_{k}(T_{i};\hat{\theta}_{k})\big)^{2} , and d_{k} is the dimension of \theta_{k} plus 1 for the variance. We draw T=1000 samples of \mu\mid Z_{1:n} ( Equation 15 ) and compute the SLC scores \hat{w}_{\delta}(k) using Algorithm 2 with complexities c(k)=d_{k} and temperature \alpha_{n}=n^{0.45} . To define the rescaled tolerances \hat{\tau} , we introduce both a noise model and a flexible nonparametric model to represent upper and lower bounds, respectively, on the KL divergences from the true DGP. The noise model is defined as y_{i}\sim\mathcal{N}(\varphi,\sigma^{2}_{\mathrm{noise}}) , and the flexible model is y_{i}\sim\mathcal{N}(\beta_{t_{i}},\sigma^{2}_{\mathrm{flex}}) where \beta=(\beta_{t}:t\in\mathcal{T}) is defined at the set of 10 observed temperatures, \mathcal{T} . For the noise model and the flexible model, we compute the MLEs of the parameters and take the average negative log-likelihood to obtain \hat{\mu}_{\mathrm{noise}} and \hat{\mu}_{\mathrm{flex}} , respectively; see Section S3.3 for details. We set \hat{\tau}=\delta/({\hat{\mu}}_{\mathrm{noise}}-{\hat{\mu}}_{\mathrm{flex}}) . Figure 6 (B) displays the distribution of \mu_{k}-\min_{j}\mu_{j} for each k under the posterior \mu\mid Z_{1:n} , and we overlay lines indicating the \delta tolerance values used in panel (C). In Figure 6 (C), we show the SLC scores \hat{w}_{\delta}(k) for \delta=0.187 (orange), 0.075 (purple), 0.037 (green), and 0.001 (blue). The SLC scores behave as follows: (i) small tolerance ( \delta=0.001 , \hat{\tau}=0.1\% ) put higher weight on the most flexible models such as Sharpe-Schoolfield and Poly-5; (ii) moderate tolerance ( \delta\in\{0.037,\,0.075\} , \hat{\tau}\in\{5\%,10\%\} ) shifts selection to simpler 4–5 parameter models; (iii) large tolerance ( \delta=0.187 , \hat{\tau}=25\% ) shifts to the 3-parameter models, with Brière I receiving the highest score. Notably, the Eubank and Gaussian models receive relatively low weight in panel (C) across all \delta values considered, indicating that these two models are not appropriate for this data set, regardless of the tolerance level. 5.4 Population structure admixture models Inferring latent population structure from multilocus genotype data is a central task in population genetics ( 31 ; 19 ) . In many realistic settings, individuals are admixed — that is, their genomes originate from multiple ancestral populations. Modeling this admixture requires decomposing each individual’s genotype into proportions contributed by k unknown source populations, while estimating allele frequencies in each population. Bayesian admixture models provide a formal probabilistic framework for this task. A widely used software program for admixture models is STRUCTURE ( 31 ) , which performs posterior inference via Markov chain Monte Carlo. In the model, each individual is associated with a vector of ancestry proportions over k populations, however, STRUCTURE requires the user to specify k in advance. This introduces a model selection problem: The true number of populations is unknown, and must be chosen to balance fit and complexity. If k is too small, genetically distinct groups may be merged, obscuring meaningful structure. If k is too large, the model may chase noise and infer spurious populations. In practice, users often choose k by comparing in-sample log-likelihoods or by applying ad hoc heuristics such as Evanno’s method ( 8 ) , but these procedures lack theoretical justification. We address this problem by applying the LaD framework to perform principled, robust model selection in admixture models. The aim is to capture the explainable structure while avoiding unnecessary complexity. We apply our methodology to the brook trout data set of Erdman et al. 2022 , consisting of multilocus genotypes for n=8454 fish sampled from multiple streams in the Eastern United States. Each individual is genotyped at L=5 loci with two allele copies per locus. Specifically, the genotype for individual i is x_{i}=[(x_{il1},x_{il2})]_{l=1}^{L} where x_{ila}\in\{1,\ldots,V_{l}\} is the allele observed at locus l , allele copy a\in\{1,2\} . Following Pritchard et al. 2000 , the admixture model assumes alleles are drawn from one of k latent populations with allele frequencies \phi_{jl}=(\phi_{jl1},\ldots,\phi_{jlV_{l}})\sim\mathrm{Dirichlet}(1,\ldots,1) for j=1,\ldots,k , according to ancestry proportions q_{i}=(q_{i1},\ldots,q_{ik})\sim\mathrm{Dirichlet}(\psi) , where \psi=(\psi_{1},\ldots,\psi_{k}) is given a hyperprior, \psi_{j}\sim\mathrm{Uniform}(0,10) independently. Specifically, \displaystyle s_{ila} \displaystyle\sim\mathrm{Categorical}(q_{i}), \displaystyle x_{ila}\mid s_{ila}=j \displaystyle\sim\mathrm{Categorical}(\phi_{jl}). The global parameters are \theta:=(\phi,\psi) , whereas q_{i} and s_{ila} are individual-specific. We implement the LaD approach in Algorithm 1 as follows. For each k=1,\ldots,10 , we do the following. For a given k , we run STRUCTURE 20 times and keep the run with the largest estimated log probability; see Section S3.4 . We take the posterior means \hat{\phi}_{jl} and \hat{\psi} from that run, and compute per-individual negative log-likelihoods \ell_{k}(x_{i};\hat{\theta})=-\log p(x_{i}\mid\hat{\phi},\hat{\psi}) by integrating out q_{i}|\hat{\psi} and s_{ila}|q_{i} using Monte Carlo; see Section S3.4 for details. It is necessary to integrate out q_{i} and s_{ila} when defining the negative log-likelihood values because these are observation-specific latent variables. The bias-corrected LaD values are then Z_{ik}=\ell_{k}(x_{i};\hat{\theta})+d_{k}/(2n) where d_{k}=k\sum_{l=1}^{L}(V_{l}-1)+k , since each \phi_{jl} contributes V_{l}-1 degrees of freedom and \psi contributes k . We then apply Algorithm 2 with \alpha_{n}=n^{0.45} and complexity c(k)=k to compute the SLC scores \hat{w}_{\delta}(k) . To compute the rescaled tolerances \hat{\tau} , we take the noise model to be discrete uniform at each locus: x_{ila}\sim\mathrm{Uniform}(\{1,\ldots,V_{l}\}) independently for all i,l,a , which yields {\hat{\mu}}_{\mathrm{noise}}=\frac{1}{n}\sum_{i=1}^{n}\sum_{l=1}^{L}N_{il}\log V_{l} , where N_{il}\in\{0,1,2\} is the number of observed copies at locus l for individual i . Figure 7 : (Left) Boxplot of the posterior of \mu_{k} for each candidate number of populations k=1,\ldots,10 . (Right) Posterior path showing the SLC scores for k=1,\ldots,10 (y-axis) versus the rescaled tolerance \hat{\tau}\in[0,1] (x-axis), where \hat{\tau}=\delta/({\hat{\mu}}_{\mathrm{noise}}-\min_{k}{\hat{\mu}}_{k}) . Darker red indicates a higher SLC score. Figure 7 shows the LaD results for the brook trout data. The boxplots (left) show the posterior distribution of \mu_{k} for each candidate number of populations k , and the heatmap (right) shows the SLC score \hat{w}_{\delta}(k) as a function of the rescaled tolerance \hat{\tau} . As we increase \hat{\tau} from 0 to 1, the selected number of populations k gradually transitions from higher (more complex models) to lower (simpler models). This provides clear insights into the tradeoff between model complexity and the cost in terms of KL divergence. For instance, the k=4 population model can capture 65% of the improvement in fit (that is, \hat{\tau}=0.35 ) from the noise model to the most complex candidate model. For comparison, Figure 8 shows the results for Evanno’s method ( 8 ) , which is widely used to select k for STRUCTURE. Evanno’s method is based on finding a sharp elbow in the mean log-likelihood L(k) (see Section S3.4 ), or more precisely, a large value of \Delta k , defined as the second-order difference of L(k) divided by the standard deviation of L(k) across runs. Evanno’s method strongly suggests that k=2 should be selected on these data; see Figure 8 . Interestingly, however, our LaD results in Figure 7 show that k=2 is not a particularly useful choice. Using k=2 captures around 40% of the possible improvement in fit ( \hat{\tau}=0.6 ), but Figure 7 shows that using k=1 is nearly as good as k=2 on these data in terms of fit. It is necessary to jump up to k=3 or 4 to obtain more substantial gains in fit to the true DGP. Figure 8 : Evanno’s method. (Left) Mean L(k) increases with k ; error bars show standard deviation across replicates. (Middle) First difference L^{\prime}(k) . (Right) \Delta k=\mathbb{E}|L^{\prime\prime}(k)|/\text{sd}\big(L(k)\big) peaks at k=2 . Here, |L^{\prime\prime}(k)| is the absolute second difference. 6 Discussion The Likelihood as Data (LaD) framework is a novel approach to comparing models based on treating per-observation negative log-likelihoods as data to be analyzed. LaD quantifies uncertainty in the KL divergence between the true DGP and each candidate model, enabling robust model selection under misspecification by quantitatively characterizing the tradeoff between simplicity and fit. The LaD approach is simple to compute, asymptotically consistent, and stable in the presence of ties. Although LaD is a versatile method, its performance hinges on several assumptions. These include independent observations, finite second moments of the per-observation negative log-likelihoods, and accurate point estimators for the model parameters. If these assumptions are not satisfied, LaD may be unstable or exhibit overconfidence. Furthermore, when there are a large number of candidate models, LaD may encounter difficulties due to the computational burden of exhaustively evaluating all models and estimating the full K\times K covariance \Sigma . There are several interesting directions for future work. Placing a prior on \mu_{1},\ldots,\mu_{K} that allows \mu_{k}=\mu_{j} with positive probability may improve the stability of using the plug-in posterior P(k\in M^{*}_{\delta}(\mu)\mid Z_{1:n}) rather than our SLC score. Extending LaD to general loss functions beyond negative log-likelihood and developing a frequentist analogue based on sampling distributions are also promising directions. Large K could be handled by clustering the columns of Z and working at the cluster level rather than over all K . Finally, the LaD idea can be used more broadly beyond model selection. For example, one could explore the relationships among models and among data points via dimensionality reduction of the Z matrix. Supplementary material for “Robust model selection using likelihood as data” S1 Previous work To the best of our knowledge, no prior work proposes LaD in the form we use or develops the theoretical motivation we rely on. This section briefly summarizes the most closely related previous work. The core idea behind LaD is related to Vuong’s test ( 44 ) , which compares two parametric models using the average per-observation log-likelihood difference and tests the null that the models are equally close to the true DGP, where “closeness” is defined in terms of KL divergence. Similarly, Diebold and Mariano 1995 propose a test for equal predictive accuracy of two competing forecasts, again yielding a pairwise testing tool rather than a joint analysis of multiple models. Several extensions of Vuong’s test and other related model selection methods have been proposed (see, for example, 39 ; 34 ; 33 ; 7 ; 1 ; 9 ; 36 ; 35 ; 5 ), but none of these are close to the LaD method, which extends to more than two models and jointly quantifies uncertainty in how well each model approximates the DGP. Several methods have been proposed for finding a parsimonious model whose fit is nearly as good as the best available model. Widely used heuristics such as the one-standard-error rule ( 18 ; 6 ) select the most parsimonious model whose cross-validated error is within one estimated standard error of the minimum. Hansen et al. 2011 introduce the model confidence set (MCS), which compares multiple candidate models via sequential testing and elimination: clearly inferior models are removed until the remaining set contains the best model(s) with a chosen confidence level. In Bayesian work, Goutis and Robert 1998 and Dupuis and Robert 2003 quantify the loss of explanatory power incurred by KL projection from a full model to a submodel and retain only those submodels whose estimated loss stays below a pre-specified threshold; see also Vehtari and Lampinen 2004 and Lindsay and Liu 2009 . Our LaD method also provides a way of tackling this goal, but operates very differently: it works directly with expected log-likelihood (equivalently, KL distance), incorporates model complexity in a transparent way, and yields a Bayesian posterior over the full vector of KL distances. Another line of work addresses misspecification by modifying the posterior update itself. In contrast with classical robust Bayesian analysis, which focuses on sensitivity to the prior ( 2 ) , these more recent approaches robustify likelihood-based inference by modifying the likelihood, for instance, using the coarsened posterior ( 28 ) or generalized Bayes updates based on loss functions ( 4 ) . There are numerous advances in this direction for parametric inference ( 22 ; 13 ; 20 ; 15 ; 8 ) , but not primarily for model selection. Meanwhile, for robust model selection, there have also been recent advances in Bayesian methodology. Shao et al. 2019 introduce a method that replaces the log marginal likelihood with the Hyvärinen score, a proper scoring rule to obtain Bayes-factor-like comparisons that remain well defined under diffuse or improper priors. Li and Dunson 2020 propose D-probabilities that compare parametric models to a flexible nonparametric reference via KL divergence, yielding complexity-penalized weights that are less sensitive to prior specification than standard posterior model probabilities. Huggins and Miller 2023 consider BayesBag, a robust Bayesian model selection method that applies bagging to the posterior by averaging posterior model probabilities over bootstrap-resampled versions of the data. More recently, Shirvaikar et al. 2024 propose a probabilistic framework for model uncertainty based on forward sampling of missing observations using one-step-ahead prediction. There is a mature literature on predictive Bayesian approaches to model assessment, selection, and averaging—including cross-validation and predictive aggregation methods ( 43 ; 45 ; 41 ; 27 ) . These approaches are appealing for predictive performance, but their target is typically a predictive utility rather than uncertainty quantification for the KL divergence between the true DGP and each candidate model. S2 Proofs In this section, we provide proofs of the results in Section 4 . For x\in\mathbb{R}^{d} , \|x\| denotes the Euclidean norm and for A\in\mathbb{R}^{d\times d} , \|A\| denotes the Frobenius norm. For r>0 and x_{0}\in\mathbb{R}^{d} , define the open ball B_{r}(x_{0}):=\{x\in\mathbb{R}^{d}:\,\|x-x_{0}\|0 such that B_{r}(\theta^{*}_{k})\subseteq\Theta_{k} . (a) \mathbb{E}\big(|\ell_{k}(X;\theta_{k})|\big)<\infty for all \theta_{k}\in B_{r}(\theta^{*}_{k}) . (b) For all x\in\mathcal{X} , the map \theta_{k}\mapsto\ell_{k}(x;\theta_{k}) is twice continuously differentiable on B_{r}(\theta^{*}_{k}) . (c) There exists a function H:\,\mathcal{X}\to[0,\infty) such that \sup_{\theta_{k}\in B_{r}(\theta^{*}_{k})}\|\nabla^{2}_{\theta_{k}}\ell_{k}(x;\theta_{k})\|\leq H(x) for all x\in\mathcal{X} , and \mathbb{E}\big(H(X)\big)<\infty . (d) \mathbb{E}\Big(\big\|\nabla_{\theta_{k}}\ell_{k}(X;\theta^{*}_{k})\big\|^{2}\Big)<\infty . (e) There exists an estimator \hat{\theta}_{k}=\hat{\theta}_{k}(X_{1},\ldots,X_{n})\in\Theta_{k} such that \|\hat{\theta}_{k}-\theta^{*}_{k}\|=O_{p}(1/\sqrt{n}) . We use the following lemma to combine local bounds with high probability events. Lemma S2.2 . Let A_{1},A_{2},\ldots be events such that P(A_{n})\to 1 as n\to\infty . Suppose X_{n} and Y_{n} are random variables such that |X_{n}|\leq Y_{n} on A_{n} , and Y_{n}\geq 0 almost surely. If Y_{n}=o_{p}(1) , then X_{n}=o_{p}(1) . If Y_{n}=O_{p}(1) , then X_{n}=O_{p}(1) . Proof of Lemma S2.2 . First, assume Y_{n}=o_{p}(1) . Let \epsilon>0 . Then P\big(|X_{n}|>\epsilon\big)\leq P(A_{n}^{c})+P\big(|X_{n}|>\epsilon,\,A_{n}\big)\leq P(A_{n}^{c})+P(Y_{n}>\epsilon)\longrightarrow 0 (18) because Y_{n}=o_{p}(1) and P(A_{n})\to 1 . Since \epsilon>0 is arbitrary, X_{n}=o_{p}(1) . Next, assume Y_{n}=O_{p}(1) . Let \delta>0 . Since Y_{n}=O_{p}(1) , there exists M<\infty such that P\big(|Y_{n}|>M)<\delta/2 for all sufficiently large n . Since also P(A_{n}^{c})\to 0 , then for all sufficiently large n , P\big(|X_{n}|>M\big)\leq P(A_{n}^{c})+P\big(|X_{n}|>M,\,A_{n}\big)\leq P(A_{n}^{c})+P\big(|Y_{n}|>M\big)<\delta. (19) This shows that X_{n}=O_{p}(1) . ∎ S2.1 Plug-in approximation error Theorem S2.3 establishes that replacing the unknown optimal parameters \theta_{k}^{*} with their estimates \hat{\theta}_{k} introduces error O_{p}(1/n) in the average log-likelihood, which is negligible compared to the empirical approximation error of order O_{p}(1/\sqrt{n}) . Theorem S2.3 . Assume S2.1 holds for each k=1,\ldots,K . Define \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.02228pt$Z$\kern 0.0pt}}_{nk}(\theta_{k})=\frac{1}{n}\sum_{i=1}^{n}\ell_{k}(X_{i};\theta_{k}), for \theta_{k}\in\Theta_{k} . Let \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.02228pt$Z$\kern 0.0pt}}_{n}(\theta)=\big(\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.02228pt$Z$\kern 0.0pt}}_{n1}(\theta_{1}),\ldots,\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.02228pt$Z$\kern 0.0pt}}_{nK}(\theta_{K})\big)^{\mathtt{T}} for \theta=(\theta_{1},\ldots,\theta_{K}) . Then \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.02228pt$Z$\kern 0.0pt}}_{n}(\hat{\theta})=\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.02228pt$Z$\kern 0.0pt}}_{n}(\theta^{*})+O_{p}(1/n). (20) Proof of Theorem S2.3 . We fix k and show Equation 20 holds in the k th component. For notational clarity, we suppress the subscript k in the proof, and we define the function R_{n}(\theta)=\tfrac{1}{n}\sum_{i=1}^{n}\ell(X_{i};\theta) to represent \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{nk}(\theta_{k}) . Define the event A_{n}=\{\|\hat{\theta}-\theta^{*}\|0 , P\big(\max_{1\leq i\leq n}H(X_{i})>\epsilon n)\leq nP\big(H(X)>\epsilon n\big)\leq\frac{\mathbb{E}\big(H(X)\mathds{1}(H(X)>\epsilon n)\big)}{\epsilon}\xrightarrow[n\to\infty]{}0. (50) Thus, \frac{1}{n}\sum_{i=1}^{n}H(X_{i})^{2}\leq\Big(\max_{1\leq i\leq n}H(X_{i})\Big)\Big(\frac{1}{n}\sum_{i=1}^{n}H(X_{i})\Big)=o_{p}(n). (51) Combining with \|\hat{\theta}-\theta^{*}\|^{4}=O_{p}(1/n^{2}) shows the second term of Equation 49 is o_{p}(1/n) . Hence, combining Equations 46 and 49 , there exists a random variable U_{n} such that (i) U_{n}=O_{p}(1/n) and (ii) \frac{1}{n}\sum_{i=1}^{n}\|Y_{i}-Y_{i}(\theta^{*})\|^{2}\leq U_{n} on A_{n} . For the second factor of Equation 45 , note that \frac{1}{n}\sum_{i=1}^{n}\big(\|Y_{i}\|+\|Y_{i}(\theta^{*})\|\big)^{2}\leq\frac{2}{n}\sum_{i=1}^{n}\|Y_{i}\|^{2}+\frac{2}{n}\sum_{i=1}^{n}\|Y_{i}(\theta^{*})\|^{2}. (52) The second term in Equation 52 is O_{p}(1) since S_{n}(\theta^{*})\xrightarrow{\mathrm{a.s.}}\Sigma^{0} implies \frac{1}{n}\sum_{i=1}^{n}\|Y_{i}(\theta^{*})\|^{2}=\mathrm{tr}\big(S_{n}(\theta^{*})\big)=O_{p}(1) . Meanwhile, for the first term in Equation 52 , \frac{1}{n}\sum_{i=1}^{n}\|Y_{i}\|^{2}\leq\frac{2}{n}\sum_{i=1}^{n}\|Y_{i}(\theta^{*})\|^{2}+\frac{2}{n}\sum_{i=1}^{n}\|Y_{i}-Y_{i}(\theta^{*})\|^{2} (53) and we just showed that \frac{1}{n}\sum_{i=1}^{n}\|Y_{i}-Y_{i}(\theta^{*})\|^{2}\leq U_{n} on A_{n} , where U_{n}=O_{p}(1/n) . Hence, there exists V_{n} such that (i) V_{n}=O_{p}(1) and (ii) \frac{1}{n}\sum_{i=1}^{n}\big(\|Y_{i}\|+\|Y_{i}(\theta^{*})\|\big)^{2}\leq V_{n} on A_{n} . From the bounds above, the right-hand side of Equation 45 is upper bounded by U_{n}^{1/2}V_{n}^{1/2} on A_{n} , and U_{n}^{1/2}V_{n}^{1/2}=O_{p}(1/\sqrt{n})\,O_{p}(1)=o_{p}(1) . Since P(A_{n})\to 1 by S2.1 (e), Lemma S2.2 implies \|S_{n}-S_{n}(\theta^{*})\|=o_{p}(1) . Therefore, \|S_{n}-S_{n}(\theta^{*})\|\xrightarrow{\mathrm{p}}0 . Along with S_{n}(\theta^{*})\xrightarrow{\mathrm{a.s.}}\Sigma^{0} , this shows that S_{n}\xrightarrow{\mathrm{p}}\Sigma^{0} as n\to\infty . ∎ Theorems S2.7 and S2.8 pertain to the marginal distributions of \mu^{\prime} and \Sigma^{\prime} , integrating out the data; this is sometimes referred to as the data-averaged posterior. In other words, we consider the marginal distributions of \mu^{\prime} and \Sigma^{\prime} when Z_{1},\ldots,Z_{n} are distributed according to the true data distribution and (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) according to the Bayesian model. See the beginning of Section S2.3 for a summary of the notation. Theorem S2.7 . Assume Conditions 2.1 and S2.1 hold, and (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 . Then \Sigma^{\prime}\xrightarrow[n\to\infty]{\mathrm{p}}\Sigma^{0}. Proof of Theorem S2.7 . By Theorem S2.6 , we have S_{n}\xrightarrow{p}\Sigma^{0} as n\to\infty . Thus, since \Psi_{0} is fixed, \lambda_{0}/\lambda_{n}\to 0 , and \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}=O_{p}(1) by Theorem S2.3 and the law of large numbers, \frac{1}{n}\Psi_{n}=\frac{1}{n}\Psi_{0}+S_{n}+\frac{\lambda_{0}}{\lambda_{n}}(\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}-\mu_{0})(\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}-\mu_{0})^{\texttt{T}}\xrightarrow[n\to\infty]{\mathrm{p}}\Sigma^{0}, (54) recalling the definitions of \Psi_{0} and \Psi_{n} from Equation 15 . Since \nu_{n}=\nu_{0}+n , for n large enough that \nu_{n}>K+1 , standard results on the moment properties of the inverse-Wishart distribution ( 30 ) give that \mathbb{E}(\Sigma^{\prime}\mid\Psi_{n},\nu_{n})=\frac{\Psi_{n}}{\nu_{n}-K-1}=\frac{\Psi_{n}}{n}\,\frac{n}{\nu_{0}+n-K-1}\xrightarrow[n\to\infty]{\mathrm{p}}\Sigma^{0}. (55) For n large enough that \nu_{n}>K+3 , the variance of the (i,j) -th entry is {\mathrm{Var}}\big((\Sigma^{\prime})_{ij}\mid\Psi_{n},\nu_{n}\big)=\frac{(\nu_{n}-K+1)(\Psi_{n})^{2}_{ij}+(\nu_{n}-K-1)(\Psi_{n})_{ii}(\Psi_{n})_{jj}}{(\nu_{n}-K)(\nu_{n}-K-1)^{2}(\nu_{n}-K-3)}. (56) Recall that \Psi_{n}/n\xrightarrow{\mathrm{p}}\Sigma^{0} by Equation 54 . Fix any M>\max_{a,b}\big|(\Sigma^{0})_{ab}\big| . Then, P\left(\max_{a,b}\big|(\Psi_{n})_{ab}\big|\leq Mn\right)\xrightarrow[n\to\infty]{}1. (57) On the event \max_{a,b}\big|(\Psi_{n})_{ab}\big|\leq Mn , the numerator of Equation 56 is O(n^{3}) and denominator is \Omega(n^{4}) , so there exists a constant C_{ij}(M)>0 such that, on this event, {\mathrm{Var}}\big((\Sigma^{\prime})_{ij}\mid\Psi_{n},\nu_{n}\big)\leq\frac{C_{ij}(M)}{n}. (58) Then, for all \epsilon>0 , by the law of total expectation and Chebyshev’s inequality, \displaystyle P\Big( \displaystyle\big|(\Sigma^{\prime})_{ij}-\mathbb{E}\big((\Sigma^{\prime})_{ij}\mid\Psi_{n},\nu_{n}\big)\big|>\epsilon\Big) (59) \displaystyle=\mathbb{E}\Big(P\Big(\big|(\Sigma^{\prime})_{ij}-\mathbb{E}\big((\Sigma^{\prime})_{ij}\mid\Psi_{n},\nu_{n}\big)\big|>\epsilon\;\Big|\;\Psi_{n},\nu_{n}\Big)\mathds{1}\big(\max_{a,b}\big|(\Psi_{n})_{ab}\big|\leq Mn\big)\Big)+ (60) \displaystyle~~~~\mathbb{E}\Big(P\Big(\big|(\Sigma^{\prime})_{ij}-\mathbb{E}\big((\Sigma^{\prime})_{ij}\mid\Psi_{n},\nu_{n}\big)\big|>\epsilon\;\Big|\;\Psi_{n},\nu_{n}\Big)\mathds{1}\big(\max_{a,b}\big|(\Psi_{n})_{ab}\big|>Mn\big)\Big) \displaystyle\leq\frac{C_{ij}(M)}{n\epsilon^{2}}+P\Big(\max_{a,b}\big|(\Psi_{n})_{ab}\big|>Mn\Big)\xrightarrow[n\to\infty]{}0. (61) Now, we control the whole matrix using the max norm and a union bound: P\Big(\big\|\Sigma^{\prime}-\mathbb{E}\big(\Sigma^{\prime}\mid\Psi_{n},\nu_{n}\big)\big\|_{\max}>\epsilon\Big)\leq\sum_{i,j=1}^{K}P\Big(\Big|(\Sigma^{\prime})_{ij}-\mathbb{E}\big((\Sigma^{\prime})_{ij}\mid\Psi_{n},\nu_{n}\big)\Big|>\epsilon\Big)\xrightarrow[n\to\infty]{}0. (62) Thus, \big\|\Sigma^{\prime}-\mathbb{E}\big(\Sigma^{\prime}\mid\Psi_{n},\nu_{n}\big)\big\|_{\max}\xrightarrow[n\to\infty]{\mathrm{p}}0. (63) Finally, by the triangle inequality, \big\|\Sigma^{\prime}-\Sigma^{0}\big\|_{\max}\leq\big\|\Sigma^{\prime}-\mathbb{E}\big(\Sigma^{\prime}\mid\Psi_{n},\nu_{n}\big)\big\|_{\max}+\big\|\mathbb{E}\big(\Sigma^{\prime}\mid\Psi_{n},\nu_{n}\big)-\Sigma^{0}\big\|_{\max}\xrightarrow[n\to\infty]{\mathrm{p}}0, (64) since the second term is o_{p}(1) by Equation 55 . Therefore, \Sigma^{\prime}\xrightarrow{\mathrm{p}}\Sigma^{0} , proving the result. ∎ Theorem S2.8 establishes the limiting behavior of \mu^{\prime} under the data-averaged LaD posterior, that is, integrating out the data. The factor of 2 in the covariance 2\Sigma^{0} reflects the combined contribution of (i) the sampling variability of the data and (ii) the posterior uncertainty given the data. Theorem S2.8 . Assume Conditions 2.1 and S2.1 hold, and (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 . Then \displaystyle\sqrt{n}(\mu^{\prime}-\mu^{0})\xrightarrow[n\to\infty]{\mathrm{d}}\mathcal{N}(0,2\Sigma^{0}). Proof of Theorem S2.8 . Write \sqrt{n}(\mu^{\prime}-\mu^{0})=\sqrt{n}(\mu^{\prime}-\mu_{n})+\sqrt{n}(\mu_{n}-\mu^{0}). (65) By Theorem S2.7 , we have \Sigma^{\prime}\xrightarrow{\mathrm{p}}\Sigma^{0} . Since the matrix square root and inverse are continuous on the space of positive definite matrices, the continuous mapping theorem yields (\Sigma^{\prime})^{-1/2}\xrightarrow{\mathrm{p}}(\Sigma^{0})^{-1/2} . Define the standardized quantities: V_{n}:=\sqrt{n}(\Sigma^{\prime})^{-1/2}\big(\mu^{\prime}-\mu_{n}\big),\qquad W_{n}:=\sqrt{n}(\Sigma^{\prime})^{-1/2}\big(\mu_{n}-\mu^{0}\big). We characterize the limiting distributions of V_{n} and W_{n} . For V_{n} , since \mu^{\prime}\mid\Sigma^{\prime},Z_{1:n}\sim\mathcal{N}\big(\mu_{n},\Sigma^{\prime}/\lambda_{n}\big) , we have V_{n}\mid\Sigma^{\prime},Z_{1:n}\sim\mathcal{N}(0,nI/\lambda_{n}), (66) Thus the conditional distribution V_{n} does not depend on \Sigma^{\prime} or Z_{1:n} , and hence V_{n}\sim\mathcal{N}(0,nI/\lambda_{n}) marginally for all n . Since n/\lambda_{n}\to 1 , it follows from Slutsky’s theorem that V_{n}\xrightarrow{\mathrm{d}}\mathcal{N}(0,I) . For W_{n} , from Theorem S2.5 and (\Sigma^{\prime})^{-1/2}\xrightarrow{\mathrm{p}}(\Sigma^{0})^{-1/2} , Slutsky’s theorem yields W_{n}\xrightarrow{\mathrm{d}}\mathcal{N}(0,I) . Moreover, since we can represent \mu^{\prime}=\mu_{n}+\Sigma^{\prime 1/2}\xi/\sqrt{\lambda_{n}} with \xi\sim\mathcal{N}(0,I) independent of (Z_{1:n},\Sigma^{\prime}) , it follows that V_{n}=\xi\sqrt{n/\lambda_{n}} depends only on \xi , whereas W_{n} depends only on (Z_{1:n},\Sigma^{\prime}) . Thus, V_{n} and W_{n} are independent. Therefore, V_{n}+W_{n}\xrightarrow{\mathrm{d}}\mathcal{N}(0,2I) . Finally, Slutsky’s theorem yields \sqrt{n}\big(\mu^{\prime}-\mu^{0}\big)=(\Sigma^{\prime})^{1/2}(V_{n}+W_{n})\xrightarrow[n\to\infty]{\mathrm{d}}\mathcal{N}(0,2\Sigma^{0}), (67) which proves the result. ∎ S2.4 Posterior consistency of hard min for the minimal KL Recall from Equation 12 that \mu^{0}=(\mu^{0}_{1},\ldots,\mu^{0}_{K})^{\texttt{T}} where \mu^{0}_{k}=\mathbb{E}\big(\ell_{k}(X;\theta^{*}_{k})\big) . Corollary S2.9 . Assume Conditions 2.1 and S2.1 hold, and (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 . Then \mu^{\prime}\xrightarrow[n\to\infty]{\mathrm{p}}\mu^{0}. Proof of Corollary S2.9 . By Lemma S2.8 , we have \sqrt{n}(\mu^{\prime}-\mu^{0})\xrightarrow{\mathrm{d}}\mathcal{N}(0,2\Sigma^{0}) as n\to\infty . Consequently, \mu^{\prime}-\mu^{0}=O_{p}(1/\sqrt{n}) , so \mu^{\prime}\xrightarrow{\mathrm{p}}\mu^{0} in \mathbb{R}^{K} . ∎ Recall from Equation 5 that we define M_{\delta}(\mu)=\big\{k:\mu_{k}\leq\min_{j}\mu_{j}+\delta\big\} for \delta\geq 0 . Theorem S2.10 . Assume Conditions 2.1 and S2.1 hold, and (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 . Then P\Big(\argmin_{1\leq k\leq K}\,\mu^{\prime}_{k}\in M_{0}(\mu^{0})\Big)\xrightarrow[n\to\infty]{}1. Proof of Theorem S2.10 . If M_{0}(\mu^{0})=\{1,\ldots,K\} , then the result is immediate. Suppose M_{0}(\mu^{0})\neq\{1,\ldots,K\} . Define m:=\min\big\{\mu^{0}_{k}-\mu^{0}_{\mathrm{min}}:k\notin M_{0}(\mu^{0})\big\} where \mu^{0}_{\mathrm{min}}=\min_{k}\mu^{0}_{k} , and note that m>0 . Choose any \eta\in(0,m/2) . By Corollary S2.9 , P\big(E_{n}(\eta)\big)\xrightarrow{}1 where E_{n}(\eta)=\big\{\max_{1\leq k\leq K}|\mu^{\prime}_{k}-\mu^{0}_{k}|<\eta\big\} . On the event E_{n}(\eta) , for all k\in M_{0}(\mu^{0}) , we have \mu^{\prime}_{k}<\mu_{k}^{0}+\eta=\mu^{0}_{\mathrm{min}}+\eta. (68) Meanwhile, for all k\notin M_{0}(\mu^{0}) , we have \mu_{k}^{0}-\mu^{0}_{\mathrm{min}}\geq m>0 , and so since \eta\mu^{0}_{k}-\eta\geq\mu^{0}_{\mathrm{min}}+m-\eta>\mu^{0}_{\mathrm{min}}+\eta. (69) Consequently, on event E_{n}(\eta) , we have \max_{k\in M_{0}(\mu^{0})}\,\mu^{\prime}_{k}<\min_{k\notin M_{0}(\mu^{0})}\,\mu^{\prime}_{k} , and hence, \argmin_{1\leq k\leq K}\mu^{\prime}_{k}\in M_{0}(\mu^{0}) . Therefore, P\Big(\argmin_{1\leq k\leq K}\mu^{\prime}_{k}\in M_{0}(\mu^{0})\Big)\geq P\big(E_{n}(\eta)\big)\longrightarrow 1. (70) ∎ S2.5 Posterior consistency for the complexity class Recall from Equation 6 that c_{\delta}^{*}(\mu)=\min\big\{c(k):k\in M_{\delta}(\mu)\big\} . Lemma S2.11 . The function \mu\mapsto c_{\delta}^{*}(\mu) is continuous at any point \mu\in\mathbb{R}^{K} such that \delta\neq\mu_{k}-\min_{j}\mu_{j} for all k . Proof of Lemma S2.11 . Since c_{\delta}^{*}(\mu) is integer valued, the claim is that \mu\mapsto c_{\delta}^{*}(\mu) is continuous at any point \mu\in\mathbb{R}^{K} such that \delta\neq\mu_{k}-\min_{j}\mu_{j} for all k . To show this, define g:\mathbb{R}^{K}\to\mathbb{R}^{K} by g(\mu)=\mu-\min_{k}\mu_{k} and define f:\mathbb{R}^{K}\to 2^{\{1,\ldots,K\}} by f(x)=\{k:x_{k}\leq\delta\} . We give \mathbb{R}^{K} the usual Euclidean topology and give 2^{\{1,\ldots,K\}} the discrete topology. Observe that f(g(\mu))=M_{\delta}(\mu) , so if f\circ g is continuous at some point \mu , then c_{\delta}^{*}(\mu) is also continuous at \mu , because it depends on \mu only through the discrete quantity M_{\delta}(\mu) . Note that g is continuous everywhere, since \mu\mapsto\min_{k}\mu_{k} is continuous. Next, note that f is continuous at any point x such that x_{k}\neq\delta for all k . Now, let \mu\in\mathbb{R}^{K} such that \delta\neq\mu_{k}-\min_{j}\mu_{j} for all k . Then, letting x=g(\mu) , we have x_{k}\neq\delta for all k , so f is continuous at x . Therefore, f\circ g is continuous at \mu . It follows that c_{\delta}^{*}(\mu) is continuous at \mu , proving the claim. ∎ Theorem S2.12 shows that the data-averaged posterior is consistent for the minimal \delta -optimal complexity class. More precisely, with probability tending to 1 , a sample of c_{\delta}^{*}(\mu^{\prime}) from the data-averaged posterior will be equal the true value c_{\delta}^{*}(\mu^{0}) . Theorem S2.12 . Assume Conditions 2.1 and S2.1 , and let (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 . Suppose \delta\neq\mu^{0}_{k}-\min_{j}\mu^{0}_{j} for all k=1,\ldots,K . Then P\big(c_{\delta}^{*}(\mu^{\prime})=c_{\delta}^{*}(\mu^{0})\big)\xrightarrow[n\to\infty]{}1. Proof of Theorem S2.12 . By Lemma S2.11 , \mu\mapsto c_{\delta}^{*}(\mu) is continuous at \mu^{0} . By Corollary S2.9 , \mu^{\prime}\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}\mu^{0} . Therefore, by the continuous mapping theorem, c_{\delta}^{*}(\mu^{\prime})\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}c_{\delta}^{*}(\mu^{0}) . Since c_{\delta}^{*}(\mu) is integer-valued, this implies P\big(c_{\delta}^{*}(\mu^{\prime})=c_{\delta}^{*}(\mu^{0})\big)\longrightarrow 1 . ∎ Theorem S2.13 shows that given Z_{1:n} , the posterior is consistent for the minimal \delta -optimal complexity class. This establishes consistency of the posterior, whereas Theorem S2.12 shows consistency of the data-averaged posterior. Theorem S2.13 . Under the same assumptions as Theorem S2.12 , P\big(c_{\delta}^{*}(\mu^{\prime})=c^{*}_{\delta}(\mu^{0})\,\big|\,Z_{1:n}\big)\xrightarrow[n\to\infty]{\mathrm{p}}1. Proof of Theorem S2.13 . Define the event A_{n}=\{c_{\delta}^{*}(\mu^{\prime})\neq c_{\delta}^{*}(\mu^{0})\} . By Theorem S2.12 , P(A_{n})\to 0 as n\to\infty . Consider the random variable P(A_{n}\mid Z_{1:n}) , which is the posterior probability of A_{n} given the data. By Markov’s inequality, for all \epsilon>0 , P\big(P(A_{n}\mid Z_{1:n})>\epsilon\big)\leq\frac{\mathbb{E}\big(P(A_{n}\mid Z_{1:n})\big)}{\epsilon}=\frac{P(A_{n})}{\epsilon}\longrightarrow 0. (71) Therefore, P(A_{n}\mid Z_{1:n})\xrightarrow{\mathrm{p}}0 , and hence, P ( c_ δ ^*( μ ’) = c^*_ δ ( μ ^0) — Z_1:n ) = 1 - P(A_n ∣Z_1:n) n→∞ p → 1. ∎ S2.6 Asymptotics of the soft selection score Lemma S2.15 establishes the asymptotics of the soft selection score in Equation 8 . First, we show a simple lemma that is used in the proof. Lemma S2.14 . If X_{n}=O_{p}(1) and Y_{n}\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}\infty then X_{n}+Y_{n}\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}\infty and \min\{X_{n},Y_{n}\}=O_{p}(1) . Proof. Let M\in\mathbb{R} and \epsilon>0 . Choose B>0 such that P(|X_{n}|>B)<\epsilon for all n sufficiently large. Since P(X_{n}+Y_{n}B)+P(Y_{n}0 are arbitrary, this shows that X_{n}+Y_{n}\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}\infty . Furthermore, since P(|\min\{X_{n},Y_{n}\}|>B)\leq P(|X_{n}|>B)+P(Y_{n}<-B) , lim sup_n P(—min{X_n,Y_n}— ¿ B) ≤lim sup_n P(—X_n— ¿ B) + lim sup_n P(Y_n ¡ -B) ≤ ϵ . This shows that \min\{X_{n},Y_{n}\}=O_{p}(1) . ∎ Lemma S2.15 . Suppose Y_{1},Y_{2},\ldots\in\mathbb{R}^{K} are random vectors such that \sqrt{n}(Y_{n}-\mu)\xrightarrow[n\to\infty]{\mathrm{d}}\mathcal{N}(0,\Sigma) for some \mu\in\mathbb{R}^{K} and a positive definite matrix \Sigma\in\mathbb{R}^{K\times K} . Suppose \alpha_{n}\to\infty such that \alpha_{n}/\sqrt{n}\to 0 as n\to\infty . Then \displaystyle\exp\big(-\alpha_{n}(Y_{nk}-Y_{n}^{*})\big)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}(\mu_{k}=\mu^{*}) (72) for all k=1,\ldots,K , where Y_{n}^{*}=\min_{k}Y_{nk} and \mu^{*}=\min_{k}\mu_{k} . Proof of Lemma S2.15 . For each k , decompose \displaystyle\alpha_{n}(Y_{nk}-Y_{n}^{*})=\alpha_{n}(Y_{nk}-\mu_{k})-\alpha_{n}(Y_{n}^{*}-\mu^{*})+\alpha_{n}(\mu_{k}-\mu^{*}). (73) Let S=\{k:\mu_{k}=\mu^{*}\} . For the third term in Equation 73 , since \alpha_{n}\to\infty , we have \displaystyle\alpha_{n}(\mu_{k}-\mu^{*})\longrightarrow\left\{\begin{array}[]{ll}0&\mbox{if }k\in S\\ \infty&\mbox{if }k\not\in S\end{array}\right. as n\to\infty . For the first term in Equation 73 , we have \displaystyle\alpha_{n}(Y_{nk}-\mu_{k})=\frac{\alpha_{n}}{\sqrt{n}}\sqrt{n}(Y_{nk}-\mu_{k})\xrightarrow[n\to\infty]{\mathrm{p}}0 (76) since \alpha_{n}/\sqrt{n}\to 0 and \sqrt{n}(Y_{nk}-\mu_{k})=O_{p}(1) , because \sqrt{n}(Y_{n}-\mu) converges in distribution by assumption. To study the second term, we write \displaystyle\sqrt{n}(Y_{n}^{*}-\mu^{*})=\min\Big\{\min_{k\in S}\sqrt{n}(Y_{nk}-\mu^{*}),\;\min_{k\in S^{c}}\sqrt{n}(Y_{nk}-\mu^{*})\Big\}. (77) Letting W\sim\mathcal{N}(0,\Sigma) , the first argument of Equation 77 is \displaystyle\min_{k\in S}\sqrt{n}(Y_{nk}-\mu^{*})\stackrel{{\scriptstyle\mathrm{(a)}}}{{=}}\min_{k\in S}\sqrt{n}(Y_{nk}-\mu_{k})\xrightarrow[n\to\infty]{\mathrm{d}}\min_{k\in S}W_{k}=O_{p}(1) (78) where (a) holds since \mu_{k}=\mu^{*} for k\in S , and the limit holds by the continuous mapping theorem since x\mapsto\min_{k\in S}x_{k} is a continuous function and \sqrt{n}(Y_{n}-\mu)\stackrel{{\scriptstyle\mathrm{d}}}{{\to}}W by assumption. Meanwhile, the second argument of Equation 77 is \displaystyle\min_{k\in S^{c}}\sqrt{n}(Y_{nk}-\mu^{*}) \displaystyle=\min_{k\in S^{c}}\Big(\sqrt{n}(Y_{nk}-\mu_{k})+\sqrt{n}(\mu_{k}-\mu^{*})\Big) \displaystyle\geq\min_{k\in S^{c}}\sqrt{n}(Y_{nk}-\mu_{k})+\min_{k\in S^{c}}\sqrt{n}(\mu_{k}-\mu^{*}). As in Equation 78 , \min_{k\in S^{c}}\sqrt{n}(Y_{nk}-\mu_{k})\stackrel{{\scriptstyle\mathrm{d}}}{{\to}}\min_{k\in S^{c}}W_{k} . Meanwhile, the second term satisfies \min_{k\in S^{c}}\sqrt{n}(\mu_{k}-\mu^{*})\to\infty since \min_{k\in S^{c}}\mu_{k}-\mu^{*}>0 by the definition of S and \mu^{*} . It follows that \displaystyle\min_{k\in S^{c}}\sqrt{n}(Y_{nk}-\mu^{*})\xrightarrow[n\to\infty]{\mathrm{p}}\infty, (79) by Lemma S2.14 . Combining Equations 78 and 79 with Equation 77 , we have that \sqrt{n}(Y_{n}^{*}-\mu^{*})=O_{p}(1) by Lemma S2.14 . Therefore, \displaystyle\alpha_{n}(Y_{n}^{*}-\mu^{*})=\frac{\alpha_{n}}{\sqrt{n}}\sqrt{n}(Y_{n}^{*}-\mu^{*})\xrightarrow[n\to\infty]{\mathrm{p}}0, (80) since \alpha_{n}/\sqrt{n}\to 0 . Plugging Sections S2.6 , 76 , and 80 into Equation 73 , we have \displaystyle\alpha_{n}(Y_{nk}-Y_{n}^{*})\xrightarrow[n\to\infty]{\mathrm{p}}\left\{\begin{array}[]{ll}0&\mbox{if }k\in S\\ \infty&\mbox{if }k\not\in S.\end{array}\right. For k\in S , we obtain Equation 72 by the continuous mapping theorem since x\mapsto\exp(-x) is continuous. For k\in S^{c} , Section S2.6 implies Equation 72 since for all \epsilon>0 , P ( exp ( - α _n(Y_n k - Y_n^*) ) ¿ ϵ ) = P ( α _n(Y_n k - Y_n^*) ¡ -log ϵ ) n→∞ → 0. This completes the proof. ∎ S2.7 Posterior consistency of the SLC score Before proving our main consistency result, Theorem 4.1 , we establish the following lemma. Lemma S2.16 shows that, under the data-averaged posterior, the within-class soft-selection score ( Equation 8 ) concentrates on models with the smallest KL among the set of minimal-complexity \delta -optimal models. Recall the definitions of c_{\delta}^{*}(\mu) , r_{nk}(\mu) , and \mu^{0} from Equations 6 , 8 , and 12 , respectively. Also recall that \mu_{\min,c(k)}=\min_{j\,:\,c(j)=c(k)}\mu_{j} . Lemma S2.16 . Assume Conditions 2.1 and S2.1 , and (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 . Suppose \alpha_{n}\to\infty with \alpha_{n}=o(\sqrt{n}) . Then, for all k such that c(k)=c_{\delta}^{*}(\mu^{0}) , \displaystyle r_{nk}(\mu^{\prime})=\exp\big(-\alpha_{n}(\mu^{\prime}_{k}-\mu^{\prime}_{\min,c(k)})\big)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\big). (83) This result pertains to the data-averaged posterior, that is, the marginal distribution of \mu^{\prime} integrating out data Z_{1:n} from the true distribution. Proof of Lemma S2.16 . Let C:=\big\{k:\,c(k)=c_{\delta}^{*}(\mu^{0})\big\} . By Theorem S2.8 , we have \sqrt{n}(\mu^{\prime}-\mu^{0})\xrightarrow{d}\mathcal{N}(0,2\Sigma^{0}) as n\to\infty . By the continuous mapping theorem, this convergence holds when restricted to the coordinates in C , that is, \sqrt{n}(\mu^{\prime}_{C}-\mu^{0}_{C})\xrightarrow{d}\mathcal{N}(0,2\Sigma^{0}_{C,C}) where \mu^{\prime}_{C}=(\mu^{\prime}_{j}:j\in C) , \mu^{0}_{C}=(\mu^{0}_{j}:j\in C) , and \Sigma^{0}_{C,C}=(\Sigma^{0}_{jk}:j,k\in C) . We apply Lemma S2.15 with \mu^{\prime}_{C} , \mu^{0}_{C} , and 2\Sigma^{0}_{C,C} playing the roles of Y_{n} , \mu , and \Sigma , respectively. This shows that for all k\in C , exp ( - α _n( μ ’_k - min_j∈C μ ’_j) ) n→∞ p → 1 ( μ ^0_k = min_j∈C μ ^0_j ) . The result in Equation 83 follows since k\in C implies c(k)=c_{\delta}^{*}(\mu^{0}) , and therefore, \mu^{\prime}_{\min,c(k)}=\min_{j\in C}\mu^{\prime}_{j} and \mu^{0}_{\min,c(k)}=\min_{j\in C}\mu^{0}_{j} . ∎ Lemma S2.17 . If X_{n} and Y_{n} are random variables such that X_{n} is bounded and X_{n}\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}c for some constant c\in\mathbb{R} , then \mathbb{E}(X_{n}\mid Y_{n})\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}c . Proof. These assumptions imply X_{n}\to c in L^{1} , that is, \mathbb{E}|X_{n}-c|\to 0 ( 21 , Theorem 17.4) . Thus, — E (X_n ∣Y_n) - c — = — E (X_n - c ∣Y_n) — ≤ E ( —X_n - c— — Y_n ) almost surely. Taking expectations, this yields E ( — E (X_n ∣Y_n) - c — ) ≤ E —X_n - c— →0 by the law of total expectation. Hence, \mathbb{E}(X_{n}\mid Y_{n})\to c in L^{1} , and thus \mathbb{E}(X_{n}\mid Y_{n})\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}c ( 21 , Theorem 17.2) . ∎ Proof of Theorem 4.1 . Recall from Equation 9 that we define the SLC score as \displaystyle w_{\delta}(k\mid Z_{1:n})=P\big(c_{\delta}^{*}(\mu^{\prime})=c(k)\mid Z_{1:n}\big)\,\mathbb{E}\big(r_{nk}(\mu^{\prime})\mid Z_{1:n}\big) (84) where r_{nk}(\mu)=\exp(-\alpha_{n}(\mu_{k}-\mu_{\min,c(k)})) and \mu_{\min,c(k)}=\min_{j:c(j)=c(k)}\mu_{j} for \mu\in\mathbb{R}^{K} . By assumption, \alpha_{n}\to\infty with \alpha_{n}=o(\sqrt{n}) , and (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 . Here, we write \mu^{\prime} and \Sigma^{\prime} to emphasize that these are random variables distributed according to the posterior, rather than arbitrary values of \mu and \Sigma . First, consider the first factor in Equation 84 . By Theorem S2.13 , for all k , P\big(c_{\delta}^{*}(\mu^{\prime})=c(k)\mid Z_{1:n}\big)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(c_{\delta}^{*}(\mu^{0})=c(k)\big). (85) Now, consider the second factor in Equation 84 . For all k such that c(k)=c_{\delta}^{*}(\mu^{0}) , we have r_{nk}(\mu^{\prime})\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}\mathds{1}\big(\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\big) by Lemma S2.16 , and therefore, \displaystyle\mathbb{E}\big(r_{nk}(\mu^{\prime})\,\big|\,Z_{1:n}\big)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\big) (86) by Lemma S2.17 , since r_{nk}(\mu^{\prime})\in[0,1] is bounded. We combine the two factors as follows. If c(k)\neq c_{\delta}^{*}(\mu^{0}) , then by Equations 84 and 85 , \displaystyle 0\leq w_{\delta}(k\mid Z_{1:n})\leq P\big(c_{\delta}^{*}(\mu^{\prime})=c(k)\mid Z_{1:n}\big)\xrightarrow[n\to\infty]{\mathrm{p}}0, (87) and hence, w_{\delta}(k\mid Z_{1:n})\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}0 . Meanwhile, if c(k)=c_{\delta}^{*}(\mu^{0}) , then applying Slutsky’s theorem to Equation 84 along with Equations 85 and 86 , we have \displaystyle w_{\delta}(k\mid Z_{1:n})\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\big). (88) Hence, for all k , \displaystyle w_{\delta}(k\mid Z_{1:n})\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(c_{\delta}^{*}(\mu^{0})=c(k)\big)\,\mathds{1}\big(\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\big), (89) by combining Equations 87 and 88 . Recalling the definition of M^{*}_{\delta}(\mu^{0}) , observe that \displaystyle M^{*}_{\delta}(\mu^{0})=\argmin_{k\,:\,c(k)=c_{\delta}^{*}(\mu^{0})}\mu^{0}_{k}=\Big\{k\,:\,c_{\delta}^{*}(\mu^{0})=c(k),\;\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\Big\}. (90) Therefore, by Equations 89 and 90 , w_{\delta}(k\mid Z_{1:n})\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}\mathds{1}(k\in M^{*}_{\delta}(\mu^{0})) as n\to\infty . ∎ S2.8 Posterior consistency of the approximate SLC score Proof of Theorem 4.2 . Recall Algorithm 2 defines \hat{w}_{\delta}(k) as the product of the between-class factor \hat{p}_{\delta}(k) and the within-class factor \hat{r}(k) , where \hat{p}_{\delta}(k):=\frac{1}{T}\sum_{t=1}^{T}\mathds{1}\big(c_{\delta}^{*}(\mu^{(t)})=c(k)\big)\quad\text{and}\quad\hat{r}(k):=\frac{1}{T}\sum_{t=1}^{T}r_{nk}(\mu^{(t)}). First, consider the between-class factor \hat{p}_{\delta}(k) . Since \mu^{\prime}\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 , by Theorem S2.12 , P\big(c_{\delta}^{*}(\mu^{\prime})=c(k)\big)\xrightarrow[n\to\infty]{}\mathds{1}\big(c_{\delta}^{*}(\mu^{0})=c(k)\big) (91) for all k . Each \mu^{(t)} has the same distribution as \mu^{\prime} , so for all t=1,\ldots,T , \mathds{1}\big(c_{\delta}^{*}(\mu^{(t)})=c(k)\big)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(c_{\delta}^{*}(\mu^{0})=c(k)\big). (92) Since T is fixed, averaging preserves the limit in probability, hence \hat{p}_{\delta}(k)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(c_{\delta}^{*}(\mu^{0})=c(k)\big). (93) Next, consider the within-class factor \hat{r}(k) . Fix any k such that c(k)=c_{\delta}^{*}(\mu^{0}) . For a single posterior draw \mu^{\prime} , we have r_{nk}(\mu^{\prime})\stackrel{{\scriptstyle\mathrm{p}}}{{\to}}\mathds{1}\big(\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\big) by Lemma S2.16 . Again, each \mu^{(t)} has the same distribution as \mu^{\prime} , so for all t=1,\ldots,T , r_{nk}(\mu^{(t)})\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\big). (94) Since T is fixed, averaging preserves the limit in probability, so \hat{r}(k)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(\mu^{0}_{k}=\mu^{0}_{\min,c(k)}\big). (95) By Slutsky’s theorem, combining Equations 93 and 95 yields ^ w _ δ (k) = ^ p _ δ (k) ^ r (k) n→∞ p → 1 ( c_ δ ^*( μ ^0) = c(k) ) 1 ( μ ^0_k = μ ^0_min,c(k) ) = 1 (k∈M_ δ ^*( μ ^0)) where the last equality holds by the same argument as in the proof of Theorem 4.1 . ∎ S2.9 Consistency of the estimated target set of models Proof of Theorem 4.3 . By Theorem 4.1 , for all k , w_{\delta}(k\mid Z_{1:n})\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(k\in M^{*}_{\delta}(\mu^{0})\big). (96) Fix \omega\in(0,1) . The function x\mapsto\mathds{1}(x>\omega) is continuous at 0 and 1 , so by the continuous mapping theorem, \mathds{1}\big(w_{\delta}(k\mid Z_{1:n})>\omega\big)\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(k\in M^{*}_{\delta}(\mu^{0})\big). (97) for all k . Equivalently, P(A_{nk})\to 0 as n\to\infty where A_{nk}:=\Big\{\mathds{1}\big(w_{\delta}(k\mid Z_{1:n})>\omega\big)\neq\mathds{1}\big(k\in M^{*}_{\delta}(\mu^{0})\big)\Big\}. (98) Since \hat{M}^{*}_{\delta}=\{k:w_{\delta}(k\mid Z_{1:n})>\omega\} , P\big(\hat{M}^{*}_{\delta}\neq M^{*}_{\delta}(\mu^{0})\big)=P\bigg(\bigcup_{k=1}^{K}A_{nk}\bigg)\leq\sum_{k=1}^{K}P(A_{nk})\xrightarrow[n\to\infty]{}0 (99) since P(A_{nk})\to 0 and K is finite. Therefore, P\big(\hat{M}^{*}_{\delta}=M^{*}_{\delta}(\mu^{0})\big)\to 1 as n\to\infty . ∎ S2.10 Instability of alternative approaches at ties Proof of Theorem 4.4 . Part (a). For brevity, let us denote M:=M^{*}_{\delta}(\mu^{0}) . Define C=\{k:c(k)=c_{\delta}^{*}(\mu^{0})\} , and note that M\subseteq C . Let (\mu^{\prime},\Sigma^{\prime})\sim p(\mu,\Sigma\mid Z_{1:n}) as in Equation 15 ; we use \mu^{\prime} and \Sigma^{\prime} to emphasize that these are random variables distributed according to the posterior. By the definition of M^{*}_{\delta}(\mu) , for all k , \mathds{1}\big(k\in M_{\delta}^{*}(\mu^{\prime})\big)=\mathds{1}\big(c_{\delta}^{*}(\mu^{\prime})=c(k),\;\mu^{\prime}_{k}=\mu^{\prime}_{\min,c(k)}\big). (100) Taking posterior expectations given Z_{1:n} and factoring, P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})=P(c_{\delta}^{*}(\mu^{\prime})=c(k)\mid Z_{1:n})\,P(\mu^{\prime}_{k}=\mu^{\prime}_{\min,c(k)}\mid Z_{1:n},\,c_{\delta}^{*}(\mu^{\prime})=c(k)). (101) We analyze four cases: (1) k\not\in C , (2) k\in C , (3) k\in C\setminus M , and (4) k\in M . (Case 1: k\not\in C .) By Theorem S2.13 , for all k , P(c_{\delta}^{*}(\mu^{\prime})=c(k)\mid Z_{1:n})\xrightarrow[n\to\infty]{\mathrm{p}}\mathds{1}\big(c_{\delta}^{*}(\mu^{0})=c(k)\big)=\mathds{1}(k\in C). (102) Hence, for all k\notin C , P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})\xrightarrow[n\to\infty]{\mathrm{p}}0 by Equations 101 and 102 . (Case 2: k\in C .) Fix k\in C . Define the events A:=\{\mu_{k}^{\prime}=\mu^{\prime}_{\min,c(k)}\} and B:=\{c^{*}_{\delta}(\mu^{\prime})=c(k)\} . Then Equation 101 can be written more succinctly as P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})=P(B\mid Z_{1:n})\,P(A\mid Z_{1:n},B). (103) By the law of total probability, P(A\mid Z_{1:n})=P(A\mid Z_{1:n},B)\,P(B\mid Z_{1:n})+P(A\mid Z_{1:n},B^{c})\,P(B^{c}\mid Z_{1:n}), (104) so P(A\mid Z_{1:n})-P(A\mid Z_{1:n},B)=P(B^{c}\mid Z_{1:n})\,\Big(P(A\mid Z_{1:n},B^{c})-P(A\mid Z_{1:n},B)\Big), (105) hence \Big|P(A\mid Z_{1:n})-P(A\mid Z_{1:n},B)\Big|\leq P(B^{c}\mid Z_{1:n}). (106) By Theorem S2.13 , P(B^{c}\mid Z_{1:n})\xrightarrow{\mathrm{p}}0 since k\in C , and thus, P(A\mid Z_{1:n},B)=P(A\mid Z_{1:n})+o_{p}(1). (107) Combining Equations 103 and 107 and the fact that P(B\mid Z_{1:n})\xrightarrow{\mathrm{p}}1 , we obtain P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})=P(B\mid Z_{1:n})P(A\mid Z_{1:n})+o_{p}(1)=P(A\mid Z_{1:n})+o_{p}(1), (108) that is, P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})=P\big(\mu_{k}^{\prime}=\mu^{\prime}_{\min,c(k)}\mid Z_{1:n}\big)+o_{p}(1). (109) Note that \mu^{\prime}_{\min,c(k)}=\min_{j\in C}\mu_{j}^{\prime} since k\in C . We reexpress Equation 109 in terms of an argmin over M , rather than the min over C . First, we show that \min_{j\in C}\mu_{j}^{\prime}=\min_{j\in M}\mu_{j}^{\prime} . If C=M , then this is trivial; otherwise, suppose C\setminus M\neq\varnothing . Define the within-class gap m:=\min_{j\in C\setminus M}\big(\mu_{j}^{0}-\min_{i\in M}\mu_{i}^{0}\big)>0, which is strictly positive since M\subseteq C and C\setminus M\neq\varnothing . For \eta>0 , define the event E_{n}(\eta):=\big\{\max_{1\leq k\leq K}|\mu^{\prime}_{k}-\mu^{0}_{k}|<\eta\big\} . By Corollary S2.9 , for any fixed \eta>0 , we have P\big(E_{n}(\eta)\big)\to 1 as n\to\infty . Choose \eta\in(0,m/2) . On E_{n}(\eta) , for all j\in M , \mu_{j}^{\prime}\leq\mu_{j}^{0}+\eta=\min_{i\in M}\mu_{i}^{0}+\eta, (110) while for all j\in C\setminus M , \mu_{j}^{\prime}\geq\mu_{j}^{0}-\eta\geq\min_{i\in M}\mu_{i}^{0}+m-\eta>\min_{i\in M}\mu_{i}^{0}+\eta. (111) Hence, \min_{j\in C\setminus M}\mu^{\prime}_{j}>\min_{j\in M}\mu^{\prime}_{j} , and thus, on event E_{n}(\eta) , \displaystyle\min_{j\in C}\mu_{j}^{\prime}=\min_{j\in M}\mu_{j}^{\prime}. (112) Under the NIW posterior ( Section 3.1 ), the conditional distribution \mu^{\prime}\mid Z_{1:n},\Sigma^{\prime} is multivariate normal with positive definite covariance matrix \Sigma^{\prime}/\lambda_{n} . Consequently, the posterior of \mu^{\prime}\mid Z_{1:n} has a density with respect to Lebesgue measure on \mathbb{R}^{K} , so for all i\neq j , we have P(\mu_{i}^{\prime}=\mu_{j}^{\prime}\mid Z_{1:n})=0 . Hence, the minimum \min_{j\in M}\mu^{\prime}_{j} is attained at only one index, almost surely, under the posterior. Along with Equation 112 , this implies P ( { μ _k’ = min_j ∈C μ _j’}∩E_n( η ) — Z_1:n) = P ( {{k} = arg min _j ∈M μ _j’}∩E_n( η ) — Z_1:n ) . For any events A,B,E , if P(A\cap E)=P(B\cap E) , then |P(A)-P(B)|\leq P(E^{c}) . Thus, \displaystyle\Big|P\big(\mu_{k}^{\prime}=\min_{j\in C}\mu_{j}^{\prime}\mid Z_{1:n}\big)-P\big(\{k\}=\argmin_{j\in M}\mu_{j}^{\prime}\mid Z_{1:n}\big)\Big|\leq P(E_{n}(\eta)^{c}\mid Z_{1:n}). (113) Since 0\leq P(E_{n}(\eta)^{c}\mid Z_{1:n})\leq 1 , Markov’s inequality yields, for all \epsilon>0 , P\Bigl(P(E_{n}(\eta)^{c}\mid Z_{1:n})>\epsilon\Bigr)\leq\frac{\mathbb{E}\bigl(P(E_{n}(\eta)^{c}\mid Z_{1:n})\bigr)}{\epsilon}\xrightarrow[n\to\infty]{}0, (114) so P(E_{n}(\eta)^{c}\mid Z_{1:n})\xrightarrow{\mathrm{p}}0 . Combining this with Equations 113 and 109 , along with the fact that \mu^{\prime}_{\min,c(k)}=\min_{j\in C}\mu^{\prime}_{j} since k\in C , P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})=P\Big(\{k\}=\argmin_{j\in M}\mu_{j}^{\prime}\;\Big|\;Z_{1:n}\Big)+o_{p}(1). (115) (Case 3: k\in C\setminus M .) If k\in C\setminus M , then \{\{k\}=\arg\min_{j\in M}\mu_{j}^{\prime}\}=\varnothing , so in this case, Equation 115 implies P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})\stackrel{{\scriptstyle\mathrm{p}}}{{\longrightarrow}}0 as n\to\infty . (Case 4: k\in M .) Finally, we analyze the asymptotics of the argmin over M in Equation 115 . Define V_{n}:=\sqrt{n}\big(\mu^{\prime}-\mu_{n}\big),\quad W_{n}:=\sqrt{n}\big(\mu_{n}-\mu^{0}\big), where \mu_{n} is the posterior mean defined in Section 3.1 , namely, \mu_{n}=\big(\lambda_{0}\mu_{0}+n\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{n}\big)/\lambda_{n} , where the Z values are evaluated at \hat{\theta}_{k} as in Section S2.3 . By the form of the NIW posterior in Section 3.1 , we can represent \mu^{\prime}\mid Z_{1:n},\Sigma^{\prime} as \mu^{\prime}=\mu_{n}+(\Sigma^{\prime})^{1/2}\xi/\sqrt{\lambda_{n}}, (116) with \xi\sim\mathcal{N}(0,I) independent of (Z_{1:n},\Sigma^{\prime}) . Thus, V_{n}\mid Z_{1:n},\Sigma^{\prime}\sim\mathcal{N}(0,n\Sigma^{\prime}/\lambda_{n}) , and W_{n}\mid Z_{1:n} is deterministic. In addition, we have n/\lambda_{n}\to 1 , \Sigma^{\prime}\xrightarrow{\mathrm{p}}\Sigma^{0} ( Theorem S2.7 ), and W_{n}\xrightarrow{d}\mathcal{N}(0,\Sigma^{0}) ( Theorem S2.5 ). Let k\in M and define \displaystyle Q_{nk}:=P\Big(\{k\}=\argmin_{j\in M}\mu_{j}^{\prime}\;\Big|\;Z_{1:n}\Big). (117) For all j\in M , since \mu_{j}^{0}=\mu_{k}^{0} , \sqrt{n}(\mu_{j}^{\prime}-\mu_{k}^{\prime})=(V_{nj}-V_{nk})+(W_{nj}-W_{nk}). (118) Note that \{k\}=\argmin_{j\in M}\mu^{\prime}_{j} if and only if \mu_{j}^{\prime}-\mu_{k}^{\prime}>0 for all j\in M\setminus\{k\} . Thus, Q_{nk}=P\big(L_{k}V_{n}+L_{k}W_{n}>0\mid Z_{1:n}\big), (119) where we define the linear difference map L_{k}:\mathbb{R}^{K}\to\mathbb{R}^{|M|-1} by L_{k}x:=(x_{j}-x_{k})_{j\in M\setminus\{k\}} , and > indicates componentwise inequality. We have L_{k}V_{n}\mid Z_{1:n},\Sigma^{\prime}\sim\mathcal{N}\big(0,\Gamma_{nk}(\Sigma^{\prime})\big),\qquad\Gamma_{nk}(\Sigma^{\prime}):=L_{k}\big(n\Sigma^{\prime}/\lambda_{n}\big)L_{k}^{\mathtt{T}}. Since n/\lambda_{n}\to 1 and \Sigma^{\prime}\xrightarrow{\mathrm{p}}\Sigma^{0} , \Gamma_{nk}(\Sigma^{\prime})\xrightarrow[n\to\infty]{\mathrm{p}}\Gamma_{k}:=L_{k}\Sigma^{0}L_{k}^{\mathtt{T}}. (120) Let \Phi_{\Gamma}(\cdot) denote the CDF of \mathcal{N}(0,\Gamma) . By the symmetry of Gaussian distributions with mean zero, along with the fact that W_{n}\mid Z_{1:n} is deterministic, \displaystyle Q_{nk} \displaystyle=P\big(-L_{k}V_{n}0 , P\Big(\mathbb{E}\big(|D_{n}|\big|Z_{1:n}\big)>\epsilon\Big)\leq\frac{\mathbb{E}|D_{n}|}{\epsilon}\xrightarrow[n\to\infty]{}0. (124) Therefore, \big|\mathbb{E}(D_{n}\big|Z_{1:n})\big|\leq\mathbb{E}\big(|D_{n}|\big|Z_{1:n}\big)=o_{p}(1) . Thus, for k\in M , \displaystyle Q_{nk} \displaystyle=\mathbb{E}\Big(\Phi_{\Gamma_{nk}(\Sigma^{\prime})}\big(L_{k}W_{n}\big)\,\Big|\,Z_{1:n}\Big) \displaystyle=\Phi_{\Gamma_{k}}\bigl(L_{k}W_{n}\bigr)+o_{p}(1) (125) \displaystyle=\pi_{k}(W_{n})+o_{p}(1). We now combine the cases to obtain the result. Since Equations 115 , 117 , and S2.10 hold coordinate-wise for each k\in M , they also hold jointly for k\in M , because coordinate-wise convergence in probability implies joint convergence in probability. Hence, \displaystyle\big(P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})\big)_{k\in M}=\big(\pi_{k}(W_{n})\big)_{k\in M}+o_{p}(1). (126) Meanwhile, by cases (1) and (3) above, for k\notin M we know P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})\xrightarrow{\mathrm{p}}0 , and by definition \pi_{k}(x)=0 for all x when k\not\in M ( Section 4.2 ). Thus, in fact, Equation 126 holds jointly over all k\in\{1,\ldots,K\} , not just k\in M . Since W_{n}\xrightarrow{\mathrm{d}}W\sim\mathcal{N}(0,\Sigma^{0}) and the map x\mapsto\big(\pi_{k}(x)\big)_{k=1}^{K} is continuous, the continuous mapping theorem implies \big(\pi_{k}(W_{n})\big)_{k=1}^{K}\xrightarrow{\mathrm{d}}\big(\pi_{k}(W)\big)_{k=1}^{K} . Therefore, by Slutsky’s theorem, \big(P(k\in M^{*}_{\delta}(\mu^{\prime})\mid Z_{1:n})\big)_{k=1}^{K}\xrightarrow[n\to\infty]{\mathrm{d}}\big(\pi_{k}(W)\big)_{k=1}^{K}. (127) Part (b). By the definition \tilde{r}_{nk}(\mu)=\mathds{1}(\mu_{k}=\mu_{\min,c(k)}) , we have \tilde{w}_{\delta}(k\mid Z_{1:n})=P\big(c_{\delta}^{*}(\mu^{\prime})=c(k)\mid Z_{1:n}\big)P\big(\mu_{k}^{\prime}=\mu^{\prime}_{\min,c(k)}\mid Z_{1:n}\big)=P(B\mid Z_{1:n})P(A\mid Z_{1:n}), (128) where A=\{\mu_{k}^{\prime}=\mu^{\prime}_{\min,c(k)}\} and B=\{c^{*}_{\delta}(\mu^{\prime})=c(k)\} as before. For k\in C , subtracting Equation 103 from Equation 128 and using Equation 107 , we obtain \tilde{w}_{\delta}(k\mid Z_{1:n})-P\big(k\in M_{\delta}^{*}(\mu^{\prime})\mid Z_{1:n}\big)=P(B\mid Z_{1:n})\big(P(A\mid Z_{1:n})-P(A\mid Z_{1:n},B)\big)=o_{p}(1), (129) since |P(B\mid Z_{1:n})|\leq 1 . Meanwhile, for k\notin C , Equation 102 implies that both \tilde{w}_{\delta}(k\mid Z_{1:n}) and P(k\in M_{\delta}^{*}(\mu^{\prime})\mid Z_{1:n}) converge to 0 in probability, as in case (1); thus, their difference also converges to 0 in probability. Therefore, for all k , regardless of whether k\in C , \tilde{w}_{\delta}(k\mid Z_{1:n})-P\big(k\in M_{\delta}^{*}(\mu^{\prime})\mid Z_{1:n}\big)\xrightarrow[n\to\infty]{\mathrm{p}}0. (130) Therefore, \big(\tilde{w}_{\delta}(k\mid Z_{1:n})\big)_{k=1}^{K}=\big(P\big(k\in M_{\delta}^{*}(\mu^{\prime})\mid Z_{1:n}\big)\big)_{k=1}^{K}+o_{p}(1). (131) Along with the result of part (a), this implies that \big(\tilde{w}_{\delta}(k\mid Z_{1:n})\big)_{k=1}^{K}\stackrel{{\scriptstyle\mathrm{d}}}{{\to}}\big(\pi_{k}(W)\big)_{k=1}^{K} by Slutsky’s theorem. ∎ Proof of Corollary 4.5 . By Theorem 4.4 (b), it suffices to show that \big(\pi_{k_{1}}(W),\pi_{k_{2}}(W)\big)\stackrel{{\scriptstyle d}}{{=}}(U,1-U) (132) where W\sim\mathcal{N}(0,\Sigma^{0}) and U\sim\mathrm{Uniform}(0,1) . Since M_{\delta}^{*}(\mu^{0})=\{k_{1},k_{2}\} , the definition of \pi_{k}(x) reduces to \pi_{k_{1}}(x)=P(W_{k_{2}}-W_{k_{1}}\leq x_{k_{2}}-x_{k_{1}}),\quad\pi_{k_{2}}(x)=P(W_{k_{1}}-W_{k_{2}}\leq x_{k_{1}}-x_{k_{2}}) (133) for all x\in\mathbb{R}^{K} . Let F be the cumulative distribution function of the Gaussian random variable W_{k_{2}}-W_{k_{1}} . Since \Sigma^{0} is positive definite, {\mathrm{Var}}(W_{k_{2}}-W_{k_{1}})>0 , so F is continuous and strictly increasing. Then, \pi_{k_{1}}(x)=F(x_{k_{2}}-x_{k_{1}}) and \pi_{k_{2}}(x)=P\big(-(W_{k_{2}}-W_{k_{1}})\leq-(x_{k_{2}}-x_{k_{1}})\big)=1-F(x_{k_{2}}-x_{k_{1}}). (134) Hence, for all x , \pi_{k_{1}}(x)+\pi_{k_{2}}(x)=1. (135) Now, we consider evaluating these functions at x=W . By the probability integral transform, \pi_{k_{1}}(W)=F(W_{k_{2}}-W_{k_{1}})\sim\mathrm{Uniform}(0,1). (136) By Equation 135 , \pi_{k_{2}}(W)=1-\pi_{k_{1}}(W) . Combined with Equation 136 , this implies \big(\pi_{k_{1}}(W),\pi_{k_{2}}(W)\big)\overset{\mathrm{d}}{=}(U,1-U),\quad U\sim\mathrm{Uniform}(0,1). (137) This completes the proof. ∎ S3 Additional details on the examples S3.1 Gaussian mixture example details This section provides additional details on the Gaussian mixture example in Section 5.1 . For a given number of mixture components k , we model the observations x_{1},\ldots,x_{n}\in\mathbb{R} by a k -component Gaussian mixture f_{k}(x)=\sum_{j=1}^{k}\phi_{j}\,\mathcal{N}\big(x\,\big|\,m_{j},s_{j}^{2}\big) with parameters \theta=\big\{(\phi_{j},m_{j},s_{j}^{2})\big\}_{j=1}^{k} , where \phi_{1},\ldots,\phi_{k}\geq 0 are mixture weights such that \sum_{j=1}^{k}\phi_{j}=1 , while m_{j}\in\mathbb{R} and s_{j}^{2}>0 are the mean and variance, respectively, of the j th component. Prior distributions. Following Fraley and Raftery 2007 , we use a normal-inverse-gamma prior, which is a conjugate prior for the means and variances: m_{j}\mid s_{j}^{2}\sim\mathcal{N}(m_{0},s_{j}^{2}/\kappa_{0}),\quad s_{j}^{2}\sim\mathrm{InverseGamma}(\nu_{0}/2,s_{0}^{2}/2), and we place a uniform prior on the vector of mixing proportions \phi=(\phi_{1},\ldots,\phi_{k}) , that is, \phi\sim\mathrm{Dirichlet(1,\ldots,1)} , which is also conjugate. In each run (for a given n and k ), we set m_{0}=\bar{x} , \kappa_{0}=0.01 , and s_{0}^{2}=\widehat{\mathrm{Var}}(x)/k^{2} , following the default prior choices in Fraley and Raftery 2007 , and we set the degrees of freedom to \nu_{0}=10 . Expectation-maximization (EM) algorithm. We estimate \theta=\big\{(\phi_{j},m_{j},s_{j}^{2})\big\}_{j=1}^{k} via the EM algorithm, using maximum a posteriori (MAP) updates to prevent likelihood singularities and improve stability ( 3 ) . The E-step consists of computing the responsibilities r_{ij}=\frac{\phi_{j}\,\mathcal{N}(x_{i}\mid m_{j},s_{j}^{2})}{\sum_{l=1}^{k}\phi_{l}\,\mathcal{N}(x_{i}\mid m_{l},s_{l}^{2})}. For the M-step, define n_{j}:=\sum_{i=1}^{n}r_{ij} and define the weighted mean \bar{x}_{j}:=\frac{1}{n_{j}}\sum_{i=1}^{n}r_{ij}x_{i} . The MAP M-step updates are then \phi_{j}^{\mathrm{new}}=\frac{n_{j}}{n},\qquad m_{j}^{\mathrm{new}}=\frac{\kappa_{0}m_{0}+n_{j}\bar{x}_{j}}{\kappa_{0}+n_{j}}, s_{j}^{2,\mathrm{new}}=\frac{\displaystyle s_{0}^{2}+\frac{\kappa_{0}n_{j}}{\kappa_{0}+n_{j}}(\bar{x}_{j}-m_{0})^{2}+\sum_{i=1}^{n}r_{ij}(x_{i}-\bar{x}_{j})^{2}}{\nu_{0}+n_{j}+3}. We run this EM algorithm using the mclust package in R ( 37 ; 32 ) with random initialization. For each k , we run the algorithm on 50 independent restarts and keep the run with the highest observed log-likelihood. Bayesian posterior on the number of components. In this example, the model index k represents the number of mixture components. As a comparison method, we obtain a posterior over the (unknown) number of components k by placing a prior on k and computing its posterior \pi(k\mid x_{1:n}) under a mixture of finite mixtures model (MFM; 29 ). Specifically, we take K\sim\mathrm{Uniform}\{1,2,\ldots,30\} ; so in particular, P(K>30)=0 . For the mixture weights, we take (\phi_{1},\ldots,\phi_{k})\mid K=k\sim\mathrm{Dirichlet(1,\ldots,1)} . For the component parameters, we use independent conditionally conjugate priors on the mean and variance; specifically, given K=k , we take m_j ∼ N (m_0, σ _0^2),  s_j^2 ∼InverseGamma(2,b), independently for j=1,\ldots,k , where m_{0}=(\min_{i}x_{i}+\max_{i}x_{i})/2 and \sigma_{0}^{2}=(\max_{i}x_{i}-\min_{i}x_{i})^{2} , and we further place a b\sim\mathrm{Gamma}\big(0.2,\,10/\sigma_{0}^{2}\big) hyperprior on b . We run an MCMC algorithm implemented in the Julia package BayesianMixtures.jl ( https://github.com/jwmi/BayesianMixtures.jl ) targeting the MFM posterior for 200{,}000 iterations, discard the first 100{,}000 as burn-in, and use the remaining 100{,}000 draws for posterior inference. The MCMC algorithm samples from the joint posterior over assignments of observations to components, component parameters, and the hyperparameter b . We obtain posterior draws of k using the technique of Miller and Harrison 2018 , specifically, we sample from k|t for each MCMC sample of t , where t denotes the number of occupied clusters in a posterior draw. S3.2 Sparse MVN example details In this section, we provide additional details on the sparse multivariate normal (MVN) model example in Section 5.2 . Kullback–Leibler (KL) divergence calculation. The KL divergence between two multivariate normal distributions \mathcal{N}(\mu_{1},\Sigma_{1}) and \mathcal{N}(\mu_{2},\Sigma_{2}) on \mathbb{R}^{k} is D\big(\mathcal{N}(\mu_{1},\Sigma_{1})\,\|\,\mathcal{N}(\mu_{2},\Sigma_{2})\big)=\frac{1}{2}\bigg(\mathrm{tr}(\Sigma_{2}^{-1}\Sigma_{1})+(\mu_{1}-\mu_{2})^{\texttt{T}}\Sigma_{2}^{-1}(\mu_{1}-\mu_{2})-k+\log\frac{|\det\Sigma_{2}|}{|\det\Sigma_{1}|}\bigg). In our example, \Sigma_{1}=\Sigma_{2}=I , so the KL divergence simplifies to D(\mathcal{N}(\mu_{1},\Sigma_{1})\,\|\,\mathcal{N}(\mu_{2},\Sigma_{2}))=\frac{1}{2}\|\mu_{1}-\mu_{2}\|^{2} . Thus, for a candidate MVN model k whose mean is constrained to a linear subspace \Theta_{k} , the minimal KL divergence from the true model \mathcal{N}(\theta_{0},I) is \min_{\theta_{k}\in\Theta_{k}}D\big(\mathcal{N}(\theta_{0},I)\,\|\,\mathcal{N}(\theta_{k},I)\big)=\min_{\theta_{k}\in\Theta_{k}}\frac{1}{2}\|\theta_{0}-\theta_{k}\|^{2}=\frac{1}{2}\|\theta_{0}-\theta^{*}_{k}\|^{2}, where \theta^{*}_{k}:=\argmin_{\theta\in\Theta_{k}}\|\theta_{0}-\theta\|^{2} , that is, \theta^{*}_{k} is the Euclidean orthogonal projection of \theta_{0} onto the parameter subspace for model k . Coarsened posterior (c-posterior). In Section 5.2.1 , we compare with the c-posterior of Miller and Dunson 2019 ; here we provide details on this method. For robustness to misspecification, the c-posterior is obtained by conditioning on the event that the empirical distribution generated by the model is “close” to that of the observed data. When closeness is measured by KL divergence (also known as relative entropy), the c-posterior can be approximated by a power posterior, obtained by raising the likelihood to a fractional power \zeta_{n} ( 28 ) . Concretely, if \pi_{k}(\theta) is the prior under model k , then the power posterior for model k is \pi_{\alpha,k}(\theta;x_{1:n})\propto\pi_{k}(\theta)\prod_{i=1}^{n}f_{k}(x_{i};\theta)^{\zeta_{n}}, where \zeta_{n}:=\alpha/(\alpha+n)\in(0,1] and \alpha>0 is a setting that control the precision of the power posterior. In particular, the power posterior recovers the standard posterior as a special case in the limit as \alpha\to\infty ( \zeta_{n}\to 1 ). The marginal power likelihood is defined as m_{\alpha,k}(x_{1:n}):=\int\pi_{k}(\theta)\prod_{i=1}^{n}f_{k}(x_{i};\theta)^{\zeta_{n}}d\theta, and can be used to construct a posterior on models k that provides robust model inference. For exponential families, the power likelihood has the attractive property that it preserves conjugacy. Thus, when using conjugate priors with exponential families, the power posterior and power marginal likelihood take closed form expressions. We specialize the c-posterior framework to the sparse MVN mean model in Section 5.2 . Recall that we observe x_{1},\ldots x_{n}\overset{\mathrm{iid}}{\sim}\mathcal{N}(\theta_{0},I) where \theta_{0}\in\mathbb{R}^{6} is the true mean, which is treated as unknown. Consider the K=7 models in Table 1 , with a uniform prior over models, \pi(k)=1/K . For each model k=1,\ldots,K , let J_{k}\subseteq\{1,\ldots,6\} be the set of free coordinates and define d_{k}=|J_{k}| ; coordinates in J_{k}^{c} are fixed at 0 . Denote the sample mean by \bar{x}=(\bar{x}_{1},\ldots,\bar{x}_{6})^{\texttt{T}} . Since the covariance matrix is I , the power likelihood for each model is \displaystyle\prod_{i=1}^{n}f_{k}(x_{i};\theta)^{\zeta_{n}}\propto\prod_{i=1}^{n}\exp\Big(-\frac{\zeta_{n}}{2}\|x_{i}-\theta\|^{2}\Big)\propto\exp\Big(-\frac{\zeta_{n}n}{2}\|\bar{x}-\theta\|^{2}\Big). (138) Denote \theta_{J_{k}}=(\theta_{j}:j\in J_{k}) and likewise for \theta_{J_{k}^{c}} , as well as \bar{x}_{J_{k}}=(\bar{x}_{j}:j\in J_{k}) . Under model k , we have \theta_{J_{k}^{c}}=0 , and hence, \|\bar{x}-\theta\|^{2}=\|\bar{x}_{J_{k}}-\theta_{J_{k}}\|^{2}+\|\bar{x}_{J_{k}^{c}}\|^{2} . Therefore, \displaystyle\prod_{i=1}^{n}f_{k}(x_{i};\theta)^{\zeta_{n}}\propto\exp\Big(-\frac{\zeta_{n}n}{2}\|\bar{x}_{J_{k}}-\theta_{J_{k}}\|^{2}\Big)\exp\Big(-\frac{\zeta_{n}n}{2}\|\bar{x}_{J_{k}^{c}}\|^{2}\Big). (139) For each model k , we place a Gaussian prior on the free coordinates and fix the rest to zero: \theta_{J_{k}}\sim\mathcal{N}\big(\theta_{0J_{k}},\kappa_{0}^{-1}I_{d_{k}}\big),\qquad\theta_{J_{k}^{c}}=0, where \theta_{0J_{k}}\in\mathbb{R}^{d_{k}} is a fixed prior mean and \kappa_{0}>0 is a fixed prior precision. Then, by conjugacy, the power posterior on \theta_{J_{k}} under model k is \theta_{J_{k}}\;\big|\;x_{1:n},\alpha\sim\mathcal{N}\bigg(\frac{\kappa_{0}\theta_{0J_{k}}+\zeta_{n}n\bar{x}_{J_{k}}}{\kappa_{0}+\zeta_{n}n},\,(\kappa_{0}+\zeta_{n}n)^{-1}I_{d_{k}}\bigg). Observe that the constant of proportionality does not depend on k in Equation 138 , and hence also in Equation 139 . Thus, writing the first factor of Equation 139 in the form of a Gaussian density, we have \displaystyle\prod_{i=1}^{n}f_{k}(x_{i};\theta)^{\zeta_{n}}=\frac{C}{(\zeta_{n}n)^{d_{k}/2}}\mathcal{N}(\bar{x}_{Jk}\mid\theta_{Jk},(\zeta_{n}n)^{-1}I_{d_{k}})\exp\Big(-\frac{\zeta_{n}n}{2}\|\bar{x}_{J_{k}^{c}}\|^{2}\Big) (140) where C does not depend on \theta or k . Then, the marginal power likelihood is \displaystyle m_{\alpha,k}(x_{1:n}) \displaystyle=\int\pi(\theta_{J_{k}})\prod_{i=1}^{n}f_{k}(x_{i};\theta)^{\zeta_{n}}d\theta_{J_{k}} \displaystyle=\frac{C\exp(-\frac{\zeta_{n}n}{2}\|\bar{x}_{J_{k}^{c}}\|^{2})}{(\zeta_{n}n)^{d_{k}/2}}\int\mathcal{N}(\bar{x}_{J_{k}}\mid\theta_{Jk},(\zeta_{n}n)^{-1}I_{d_{k}})\,\mathcal{N}(\theta_{J_{k}}\mid\theta_{0J_{k}},\kappa_{0}^{-1}I_{d_{k}})d\theta_{J_{k}} \displaystyle=\frac{C\exp(-\frac{\zeta_{n}n}{2}\|\bar{x}_{J_{k}^{c}}\|^{2})}{(\zeta_{n}n)^{d_{k}/2}}\mathcal{N}(\bar{x}_{J_{k}}\mid\theta_{0J_{k}},((\zeta_{n}n)^{-1}+\kappa_{0}^{-1})I_{d_{k}}). Therefore, the log marginal power likelihood is \displaystyle\log m_{\alpha,k}(x_{1:n})=c+\frac{d_{k}}{2}\log\Big(\frac{\kappa_{0}}{\kappa_{0}+\zeta_{n}n}\Big)-\frac{\kappa_{0}\zeta_{n}n}{2(\kappa_{0}+\zeta_{n}n)}\|\bar{x}_{J_{k}}-\theta_{0J_{k}}\|^{2}-\frac{\zeta_{n}n}{2}\|\bar{x}_{J_{k}^{c}}\|^{2} (141) where c is a constant that does not depend on k . Since we assume a uniform prior over models, \pi(k)=1/K , the coarsened posterior on models is \displaystyle\pi_{\alpha}(k;x_{1:n})=\frac{m_{\alpha,k}(x_{1:n})}{\sum_{j=1}^{K}m_{\alpha,j}(x_{1:n})}. (142) Equations 141 and 142 yield a closed-form expression that enables exact computation of the c-posterior probabilities for the K models, which we use in Section 5.2.1 . S3.3 Thermal performance curve example details This section provides additional details on the example from Section 5.3 . Data source. The dataset used for fitting was obtained from Kontopoulos et al. 2024a , available at https://doi.org/10.6084/m9.figshare.24106161.v2 . MLEs for noise model and flexible model. The MLE for the noise model is \hat{\varphi}=\frac{1}{n}\sum_{i=1}^{n}y_{i} and \hat{\sigma}_{\mathrm{noise}}^{2}=\frac{1}{n}\sum_{i=1}^{n}(y_{i}-\hat{\varphi})^{2} . The MLE for the flexible model is \hat{\beta}_{t}=\frac{1}{n_{t}}\sum_{i=1}^{n}y_{i}\mathds{1}(t_{i}=t) where n_{t}=\sum_{i=1}^{n}\mathds{1}(t_{i}=t) , and \hat{\sigma}_{\mathrm{flex}}^{2}=\frac{1}{n}\sum_{i=1}^{n}(y_{i}-\hat{\beta}_{t_{i}})^{2} . Functional forms of the TPC candidates. For completeness, we state the functional forms of the TPC candidate models here. All formulas are taken from the supplementary material of Kontopoulos et al. 2024b . Temperature parameters are real-valued, and in nonlinear least squares fitting we impose the constraints t_{pk}\in[0,150] and t_{\mathrm{min}}\leq t_{pk}\leq t_{\mathrm{max}} . 1. Eubank (3 parameters) g_{1}(t;\,a,b,t_{pk})=\frac{a}{(t-t_{pk})^{2}+b},\qquad a>0,\;b>0. 2. Gaussian (3 parameters) g_{2}(t;\,a,b_{pk},t_{pk})=b_{pk}\exp\Big(-\frac{1}{2a^{2}}(t-t_{pk})^{2}\Big),\qquad a>0,\;b_{pk}>0. 3. Mitchell–Angilletta (3 parameters) g_{3}(t;\,a,b,t_{pk})=\frac{a}{2b}\bigg(1+\cos\Big(\frac{t-t_{pk}}{b}\cdot\pi\Big)\bigg),\qquad a>0,\;b>0. 4. Brière I (3 parameters) g_{4}(t;\,a,t_{\mathrm{min}},t_{\mathrm{max}})=a\,t\,(t-t_{\mathrm{min}})\sqrt{t_{\mathrm{max}}-t},\qquad a>0,\;t_{\mathrm{min}}0,\;c>0,\;b_{pk}>0 . 6. Modified Gaussian (4 parameters) g_{6}(t;\,a,b,b_{pk},t_{pk})=b_{pk}\exp\Big(-\frac{1}{2}\Big(\frac{|t-t_{pk}|}{a}\Big)^{b}\Big),\qquad a>0,\;b>0,\;b_{pk}>0. 7. Extended Brière (5 parameters) g_{7}(t;\,a,b,c,t_{\mathrm{min}},t_{\mathrm{max}})=a\,t\,(t-t_{\mathrm{min}})^{b}\,(t_{\mathrm{max}}-t)^{c}, a>0,\;b>0,\;c>0,\;t_{\mathrm{min}}0,\;b_{0}>0,\;\Delta H_{L}>0,\;\Delta H_{A}^{\ddagger}>0,\;\Delta H_{H}>0 . Here, T_{\mathrm{ref}}=273.5 (temperatures in Kelvin) and R=1.987 . S3.4 Population structure admixture example details Here, we provide additional details on the admixture model example in Section 5.4 . STRUCTURE output: Estimated Ln Prob of Data STRUCTURE performs posterior inference for the admixture model using MCMC, and reports an “Estimated Ln Prob of Data” (ELP), which is intended as a rough approximation to the log marginal likelihood \log p(x\mid k) . For LaD, we use this quantity as a fit diagnostic to filter out failed MCMC runs; specifically, we run the STRUCTURE algorithm 20 times with random restarts, and keep the run with the highest ELP. This quantity is also used for implementing Evanno’s method, in which the ELP is used to compute L(k) . As described in Pritchard et al. 2000 , the ELP is constructed from the Bayesian deviance \mathrm{dev}(\phi,q,s):=-2\log p(x\mid\phi,q,s) evaluated on the MCMC output. Specifically, Pritchard et al. 2000 use a normal approximation to show that -2\log p(x\mid k)\approx\mathbb{E}\big(\mathrm{dev}(\phi,q,s)\mid x,k\big)+\frac{1}{4}{\mathrm{Var}}\big(\mathrm{dev}(\phi,q,s)\mid x,k\big). In practice, \mathbb{E}\big(\mathrm{dev}(\phi,q,s)\mid x,k\big) and {\mathrm{Var}}\big(\mathrm{dev}(\phi,q,s)\mid x,k\big) are approximated by Monte Carlo averages over the post–burn-in MCMC draws. The resulting approximation of \log p(x\mid k) is what STRUCTURE returns as “Estimated Ln Prob of Data” (ELP). Per-individual negative log-likelihoods. For the LaD technique, we need to compute the per-individual negative log-likelihood values \ell_{k}(x_{i};\hat{\phi},\hat{\psi})=-\log p(x_{i}\mid\hat{\phi},\hat{\psi}) , integrating out q_{i} and s_{ila} for all l,a , given the point estimate (\hat{\phi} , \hat{\psi}) for model k . We compute these using Monte Carlo approximations, as follows. Fix a given model k , and let \hat{\phi} and \hat{\psi} denote the posterior means obtained from MCMC. First, observe that p(x_{ila}=v\mid\phi,q_{i})=\sum_{j=1}^{k}p(s_{ila}=j\mid q_{i})\,p(x_{ila}=v\mid s_{ila}=j,\,\phi)=\sum_{j=1}^{k}q_{ij}\phi_{jl}(v). We write \phi_{jl}(v) instead of \phi_{jlv} for visual clarity of the notation in what follows. Thus, for the whole genotype of individual i , p(x_{i}\mid\phi,\psi)=\int_{\Delta^{k-1}}\bigg(\prod_{l=1}^{L}\prod_{a=1}^{2}\sum_{j=1}^{k}q_{ij}\phi_{jl}(x_{ila})\bigg)\mathrm{Dirichlet}(q_{i}\mid\psi)\,dq_{i}, where the integral is over the simplex \Delta^{k-1}=\{q\in\mathbb{R}_{+}^{k}:\sum_{j=1}^{k}q_{j}=1\} and \mathrm{Dirichlet}(q\mid\psi)\propto\prod_{j=1}^{k}q_{j}^{\psi_{j}-1} . For each individual i , draw q_{i}^{(b)}\sim\mathrm{Dirichlet}(\hat{\psi}) for b=1,\ldots,B . Then, for each sample b , compute \log p(x_{i}\mid\hat{\phi},q_{i}^{(b)})=\sum_{l=1}^{L}\sum_{a=1}^{2}\log\bigg(\sum_{j=1}^{k}q_{ij}^{(b)}\hat{\phi}_{jl}(x_{ila})\bigg). We would like to form a Monte Carlo approximation \tilde{p}(x_{i}\mid\hat{\phi},\hat{\psi})=\frac{1}{B}\sum_{b=1}^{B}p(x_{i}\mid\hat{\phi},q_{i}^{(b)}) , but we need to do so in a numerically stable way. To this end, we use the log-sum-exp trick; that is, we compute C:=\max_{b}\log p(x_{i}\mid\hat{\phi},q_{i}^{(b)}) and then \log\tilde{p}(x_{i}\mid\hat{\phi},\hat{\psi})=\log\bigg(\frac{1}{B}\sum_{b=1}^{B}\exp\Big(\log p(x_{i}\mid\hat{\phi},q_{i}^{(b)})-C\Big)\bigg)+C. This yields Monte Carlo approximations of the per-individual negative log-likelihood values via \tilde{\ell}_{k}(x_{i};\hat{\phi},\hat{\psi})=-\log\tilde{p}(x_{i}\mid\hat{\phi},\hat{\psi}) . The additional computational burden of this procedure is not significant relative to the time required for posterior sampling with the MCMC algorithm. S4 Alternative LaD model with diagonal covariance Our proposed LaD model ( Section 3.1 ) places a NIW prior on (\mu,\Sigma) , which uses a full covariance matrix for \Sigma . In Section 5.2.1 , we compare with an alternative LaD model that employs a diagonal covariance matrix. We provide the details of this diagonal covariance approach here. Assume \Sigma=\mbox{diag}(\sigma^{2}_{1},\ldots,\sigma^{2}_{K}) , and model each coordinate k independently with a normal-inverse-gamma prior. Specifically, for each k , consider modeling Z_{ik}\mid\mu_{k},\sigma^{2}_{k}\overset{\mathrm{iid}}{\sim}\mathcal{N}(\mu_{k},\sigma^{2}_{k}),\quad\sigma_{k}^{2}\sim\mathrm{InverseGamma}(a_{0k},b_{0k}),\quad\mu_{k}\mid\sigma_{k}^{2}\sim\mathcal{N}\big(\mu_{0k},\sigma_{k}^{2}/\lambda_{0k}\big), where \mathrm{InverseGamma}(a,b) denotes an inverse-gamma distribution with shape a and scale b . Let \,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{.k}=\frac{1}{n}\sum_{i=1}^{n}Z_{ik} . Then the posterior is, independently for k=1,\ldots,K , \sigma^{2}_{k}\mid Z_{1:n}\sim\mathrm{InverseGamma}(a_{nk},b_{nk}),\quad\mu_{k}\mid\sigma^{2}_{k},Z_{1:n}\sim\mathcal{N}\big(\mu_{nk},\sigma^{2}_{k}/\lambda_{nk}\big), where \lambda_{nk}=\lambda_{0k}+n,\quad\mu_{nk}=\frac{\lambda_{0k}\mu_{0k}+n\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{.k}}{\lambda_{nk}},\quad a_{nk}=a_{0k}+\frac{n}{2}, b_{nk}=b_{0k}+\frac{1}{2}\sum_{i=1}^{n}(Z_{ik}-\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{.k})^{2}+\frac{1}{2}\frac{\lambda_{0k}n}{\lambda_{nk}}(\,\vbox{\hrule height=0.5pt\kern 1.07639pt\hbox{\kern-1.00006pt$Z$\kern 0.0pt}}_{.k}-\mu_{0k})^{2}. We set the hyperparameters to match the NIW prior \Sigma\sim\mathrm{InverseWishart}(\Psi_{0},\nu_{0}) and \mu\mid\Sigma\sim\mathcal{N}(\mu_{0},\Sigma/\lambda_{0}) , by choosing a_{0k}=\frac{\nu_{0}-K+1}{2},\quad b_{0k}=\frac{\Psi_{0,kk}}{2},\quad\mu_{0k}=(\mu_{0})_{k},\quad\lambda_{0k}=\lambda_{0}, for k=1,\ldots,K . With this choice, the NIW marginal for \Sigma_{kk} match \mathrm{IG}(a_{nk},b_{nk}) and yields \mu_{k}\mid\sigma^{2}_{k}\sim\mathcal{N}(\mu_{0k},\sigma^{2}_{k}/\lambda_{0k}) under independence.