A Standardization Procedure to Incorporate Variance Partitioning Based Priors in Latent Gaussian Models

Luisa Ferrari, Massimo Ventrucci · 2025

Read A Standardization Procedure to Incorporate Variance Partitioning Based Priors in Latent Gaussian Models by Luisa Ferrari, Massimo Ventrucci (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16057.txt.

Introduction

Luisa Ferrari and Massimo Ventrucci made prior setting easier in a family of Bayesian models in a 2025 paper. Latent Gaussian models are flexible and fast, but their variance knobs lack intuitive meaning, so choosing sensible prior beliefs for them frustrates practitioners. A recent idea reparametrizes the model by how total variance splits among effects, letting users state beliefs about shares instead of raw variances.

Existing share-based priors handled fixed and random effects separately. The authors give a standardization procedure that covers both together, widening the approach to a broader model class that includes spatial random fields in common use. Simulations and a survival analysis example show the practical payoff of priors users can actually reason about.

Catalog record

CREATORLuisa Ferrari, Massimo Ventrucci
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE97 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-16057

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Standardization Procedure to Incorporate Variance Partitioning Based Priors in Latent Gaussian Models Luisa Ferrari, Massimo Ventrucci Abstract Latent Gaussian Models (LGMs) are a subset of Bayesian Hierarchical models where Gaussian priors, conditional on variance parameters, are assigned to all effects in the model. LGMs are employed in many fields for their flexibility and computational efficiency. However, practitioners find prior elicitation on the variance parameters challenging because of a lack of intuitive interpretation for them. Recently, several papers have tackled this issue by rethinking the model in terms of variance partitioning (VP) and assigning priors to parameters reflecting the relative contribution of each effect to the total variance. So far, the class of priors based on VP has been mainly deployed for random effects and fixed effects separately. This work presents a novel standardization procedure that expands the applicability of VP priors to a broader class of LGMs, including both fixed and random effects. We describe the steps required for standardization through various examples, with a particular focus on the popular class of intrinsic Gaussian Markov random fields (IGMRFs). The practical advantages of standardization are demonstrated with simulated data and a real dataset on survival analysis. Keywords : Gaussian Markov Random Fields; Hierarchical Decomposition priors; P-splines; R2D2; PC priors Email for Correspondence : [email protected] 1 Introduction Latent Gaussian Models (LGMs, Rue et al., (2009) ) are a popular class of Bayesian Hierarchical models used by applied scientists in many fields for their flexibility and ability to include linear and smooth effects of continuous covariates, spatial and temporal random effects and more complex structures such as interactions. The linear predictor for an observation i is structured as: \eta_{i}=\mu+\sum_{p=1}^{P}x_{ip}\beta_{p}+\sum_{r=1}^{R}f_{r}(z_{ir}) where \beta_{p}\sim N(0,\sigma_{p}^{2}),p=1,\ldots,P are referred to as fixed effects, associated to P covariates, and \bm{f_{r}}\sim N(\bm{0},\bm{\Sigma}_{r}),r=1,\ldots\,R , are called random effects. Both fixed and random effects in an LGMs are Gaussian conditional on the parameters in their covariance. Variances for fixed effects, \sigma_{p}^{2} , are typically fixed at large values. For random effects, \bm{\Sigma}_{r} is often expressed as a correlation matrix scaled by a random variance \sigma_{r}^{2} who is assigned a prior (hence \sigma_{r}^{2} is an hyper-parameter). Along with proper Gaussian processes, LGMs frequently make use of Intrinsic Gaussian Markov Random Fields (IGMRFs, Rue and Held, (2005) and Fahrmeir et al., (2004) ). IGMRFs, which include as special cases the random walk of order 1 (RW1) and 2 (RW2) and the intrinsic conditional autoregressive process (ICAR, Besag and Kooperberg, (1995) ), are especially employed as priors for spatial and temporal random effects. Prior specification on the variances of the random effects, \sigma_{r}^{2} , is a long discussed problem due to the poor interpretability of these parameters, especially for IGMRFs ( Sørbye and Rue, (2014) ). Independent Inverse Gamma priors have been found to lead to overfitting ( Gelman, (2006) , Lunn et al., (2009) , Simpson et al., (2017) ) and are now being replaced by Penalized Complexity (PC) priors ( Simpson et al., (2017) ) in the R package INLA ( Rue et al., (2009) ), which provides fast approximate inference for LGMs. While variances are hard to elicit, experts often have some intuition about the relative importance of the different effects in terms of variance contribution. Recently, multiple works have tried to leverage this underlying information by designing joint priors on the variance parameters, either the \sigma_{p}^{2} ’s or the \sigma_{r}^{2} ’s above mentioned. Specifically, a joint prior can be specified using a reparameterization of the original parameters into a total variance and a set of proportions of variance. Proportions are more intuitive parameters for users, as they directly indicate the relative importance of effects on a (0-1) scale, thereby facilitating prior specification. This novel approach can be referred to generally as Variance Partitioning (VP) , and in this paper the term VP priors is used to define the class of priors derived using this approach. Two main lines of research can be recognized in the literature on VP priors. The first one follows the work by Fuglstad et al., (2020) who introduce the class of Hierarchical Decomposition (HD) priors, which can be seen as a subclass of VP priors focusing exclusively on random effects (i.e. variance partitioning is applied to \sigma_{r}^{2} ’s only). HD priors can be applied to all elements in the linear predictor or in a hierarchical manner through consecutive splits of the linear predictor; for applications see Hem et al., (2021) and Franco-Villoria et al., (2022) . Similarly, in disease mapping Wakefield, (2007) and Riebler et al., (2016) have proposed priors on the variance contribution of spatially structured and unstructured components ( Besag et al., (1991) ).The second line of research based on Zhang et al., (2022) considers priors on a measure of goodness-of-fit R^{2} , denoted as R2D2 priors, which initially focused exclusively on linear effects (i.e. variance partitioning is applied to \sigma_{p}^{2} ’s only) but have recently been extended to mixed model cases ( Aguilar and Bürkner, (2023) , Yanchenko et al., 2024a () , Yanchenko et al., 2024b () ). Similarly, Bhattacharya et al., (2015) define a global-local shrinkage prior for sparse regression. It is worth noting that the applicability of VP priors in LGMs relies on the user being able to interpret the variance parameters as variance contributions of the corresponding effects. Thus, all effects in the model should somehow live in the same scale so that, for instance, \sigma_{r}=1,\forall r , means equal a priori contribution for all random effects. In other words, an intuitive interpretation of the variance of each effect is required to safely use VP priors. Many popular effects (called heterogeneous by Fuglstad et al., (2020) ) do not satisfy such interpretation requirement. Notable examples are some popular IGMRFs (e.g., ICAR, RW1, RW2); as noted by Sørbye and Rue, (2014) , the variance of an IGMRF strongly depends on the domain in which the process is observed. Sørbye and Rue, (2014) proposed a solution based on rescaling the strucure matrix, which is recommended by Fuglstad et al., (2020) for inclusion of IGMRFs in the HD framework. R2D2 recommends standardization of all covariates, which solves the issue in an LGM with only linear effects. It is not clear how to guarantee the interpretation requirement in a broader LGM framework including both fixed and random effects. In this paper, we aim to extend the applicability of VP priors to a class of Latent Gaussian Models including both fixed and random effects. We start by defining variance contribution for fixed and random effects separately and then derive a novel standardization procedure that guarantees an exact match between variance parameters and variance contributions. We prove analytically that the proportions of variance parameters are correctly interpreted after standardization (Remark 1 ). The benefits of the proposed standardization are confirmed by simulations. On the practical side, our primary interest is to enable the use of VP priors in realistic settings. For this reason, we illustrate the proposed standardization through various examples frequently encountered in LGMs, such as linear and smooth effects of continuous covariates, spatial and temporal random effects, etc. We put emphasis on IGMRFs as these are popularly used not only to model spatial and temporal dependence but also to construct smooth effects via

Read the complete text