Positive-definiteness in separable priors: effects on prior interpretability and inference
Jack Storror Carter, David Rossell · 2026
Read Positive-definiteness in separable priors: effects on prior interpretability and inference by Jack Storror Carter, David Rossell (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22640.txt.
Introduction
Storror Carter and Rossell examine a popular prior for symmetric positive definite matrices. The prior assumes independent entries and truncates to force positive definiteness. They show careless truncation can warp interpretation, shrinkage and posterior inference when the cut prior drifts far from its uncut twin. The paper studies dense and sparse cases and shows how to set prior settings, such as off diagonal variance, so the truncation effect fades as matrix size grows. Sparse cases need special care, since poorly set priors push estimates toward spuriously sparse structures.
Catalog record
| CREATOR | Jack Storror Carter, David Rossell |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78669 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22640
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Positive-definiteness in separable priors: effects on prior interpretability and inference Jack Storror Carter, David Rossell Abstract A popular class of priors for symmetric positive-definite matrices assumes independent entries and adds a truncation to ensure positive-definiteness. While conceptually simple and often computationally convenient, unless done carefully this truncation can have unintended effects. If the truncated prior or its margins are significantly different from their untruncated counterpart, then its interpretability may suffer, its shrinkage properties become harder to characterise, and posterior inference may be affected in unanticipated ways. We investigate the effect of the truncation both for dense and sparse matrices, and show how to set prior parameters such as the variance of off-diagonal entries such that said effect is mitigated as the matrix dimension grows. We pay particular attention to sparse inference where, unless prior parameters are set carefully, the truncated prior and hence its corresponding posterior assign systematically higher mass to sparser structures than the untruncated prior. Keywords— Positive definite matrix, separable prior distribution, Wigner matrix Distributions over the space of symmetric positive-definite matrices play an important role in Bayesian statistics. This is mostly for their use as priors for covariance matrices and their transformations, such as correlation matrices, precision matrices or partial correlation matrices, but positive-definite matrices also appear as parameters in many multivariate and matrix-variate distributions such as the Wishart, matrix t and matrix F distributions. Let \Theta=(\theta_{ij}) be a random k\times k symmetric and positive-definite matrix. A conceptually simple class of prior distributions assumes that the density is \displaystyle p^{+}(\Theta)=\frac{p(\Theta)\mathbb{I}(\Theta\succ 0)}{c} (1) where \mathbb{I}(\Theta\succ 0) is an indicator for \Theta being positive-definite, p(\Theta)=\prod_{i\leq j}\pi_{ij}(\theta_{ij}), and \pi_{ii}:\mathbb{R}^{+}\to\mathbb{R}^{+} and \pi_{ij}:\mathbb{R}\to\mathbb{R}^{+} for i<j are density functions. In ( 1 ), the normalising constant c=\mathbb{P}_{p}(\Theta\succ 0)=\int\mathbb{I}(\Theta\succ 0)dP(\Theta) is the probability that \Theta\succ 0 under P , the distribution whose density is p . That is, p assumes that the entries of \Theta are independent and p^{+} truncates p onto the space of positive-definite matrices. We refer to p and p^{+} as separable and positive-definite separable (PD-separable) distributions respectively. We denote the marginal distribution of \theta_{ij} under p^{+}(\Theta) as \pi^{+}_{ij}(\theta_{ij}) . In the identical distributed case \pi_{ii}=\pi_{D} for all i and some \pi_{D} , and \pi_{ij}=\pi for all i<j and some common off-diagonal \pi , then also \pi^{+}_{ii}=\pi^{+}_{D} for all i and \pi^{+}_{ij}=\pi^{+} for all i<j , for some common (\pi_{D}^{+},\pi^{+}) . A setting in which PD-separable priors generated particular interest are graphical models. For example, in Gaussian graphical models with precision (inverse covariance) matrix \Theta , additionally to estimating \Theta , one also seeks to detect the zero off-diagonal entries in \Theta (its sparsity structure). The interest in PD-separable priors stems partly from their perceived interpretability, and perharps more importantly for their leading to computationally-convenient posterior sampling algorithms and posterior mode estimators that are directly connected to penalised likelihood methods. For example, one may obtain the posterior mode for the Bayesian graphical LASSO of 19 using standard graphical LASSO ( 8 ) algorithms, or alternatively use algorithms that encourage the presence of positive partial correlations ( 12 ) . 11 extended the work of 19 , and 3 ; 20 ; 9 ; 14 considered various PD-separable spike-and-slab priors, along with computational proposals. Beyond canonical Gaussian graphical models, PD-separable priors also found use for correlation matrices ( 1 ) , for regressing graphical models on network data ( 10 ) and for graphical models for mixed data types such as combinations of continuous, categorical and count variables ( 7 ) . In this paper we argue that, while the construction of p^{+} is appealingly simple, its interpretation is not. Beyond the fact that the entries of \Theta are no longer independent, as intended to ensure that \Theta is positive-definite, there may be unintended consequences. Specifically, the marginal distribution \pi_{ij}^{+} can be dramatically different from \pi_{ij} , and the prior beliefs about sparsity structures in \Theta may be significantly affected, with the consequent effects on posterior inference. For example, 19 showed that when \pi_{ii} are exponential and \pi_{ij} Laplace distributions, the truncation can significantly inflate the diagonal of \Theta (relative to \pi_{ii} ) and shrink its off-diagonal entries towards 0 (relative to \pi_{ij} ). This has practical implications. First, an important appeal of the Bayesian paradigm is that prior hyper-parameters often describe interpretable characteristics of the data-generating process, such as the proportion of non-zero entries in \Theta , or their variance. In contrast to approaches where hyper-parameters are treated in a black-box manner, one can use the hyper-parameters to incorporate prior knowledge. Further, one can describe quantities of interest (e.g., the proportion of non-zeroes in \Theta ) by obtaining point estimates for the hyper-parameters (or their full posterior distribution). Second, the fact that the truncation can significantly alter the prior implies that the prior shrinkage may have unanticipated effects, or simply be much stronger or milder than one may naively expect, which has an effect on posterior inference. As we show, this is particularly obvious for sparse structural learning tasks. Our first main contribution is describing precisely how p^{+} and its margins \pi_{ij}^{+} differ from the untruncated p and \pi_{ij} , and providing a series of results that show how to set prior parameters such that these differences are small. From ( 1 ), if c=1 then p=p^{+} . We show that the key aspects for ensuring that c is close to 1 are having sufficiently small off-diagonal variances or a sufficiently large minimum diagonal entry, and give conditions ensuring that c\to 1 as the matrix dimension k\to\infty . Our second main contribution is describing the impact of the positive-definite truncation in sparse settings. More specifically, on the prior distribution of the sparsity pattern (zero entries) of \Theta , a key feature for structural learning in graphical models. We show that sparsity in \Theta generally increases the probability of positive-definiteness, and the immediate implications on the prior probability of sparsity patterns under p^{+} relative to under p . We also show that setting a large off-diagonal variance, in the limit, results in assigning posterior probability 1 to \Theta being a diagonal matrix. We obtain bounds for the probability of \Theta\succ 0 that hold for any fixed k and sparsity structure, which indicate a possible way to set the prior off-diagonal variance to avoid such pathologies. Let \mu be the mean of the diagonal \pi_{D} , \sigma the variance of off-diagonal \pi and d_{z} the maximum row degree of \Theta (defined below). Roughly speaking, whereas the dense case requires \sigma/\mu<1/(2\sqrt{k}) for c to converge to 1, in the sparse case it suffices that \sigma/\mu=o(1/\sqrt{d_{z}}) if \Theta has fixed diagonal and \sigma/\mu=o(1/(k\sqrt{d_{z}}) if \theta_{ii}\sim\mbox{Exp}(1/\mu) (neglecting log terms). The paper proceeds as follows. In Section 1 we describe how p^{+} relates to p . We explain that the total variation distance (TVD) between p^{+} and p and their Kullback-Leibler (KL) divergence are simple functions of c . We then give a simple expression relating the marginal \pi_{ij}^{+} to the untruncated \pi_{ij} , and show that their TV distance and KL divergence are upper-bounded by functions of c . These results make explicit precisely how, as c approaches 1, p^{+} converges to p , but also how in some settings the marginals \pi_{ij} and \pi_{ij}^{+} can