Inference on covariance structure in high-dimensional multi-view data

Lorenzo Mauri · 2025

Read Inference on covariance structure in high-dimensional multi-view data by Lorenzo Mauri (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-02772.txt.

Introduction

Lorenzo Mauri and David B. Dunson wrote a 2025 statistics paper on estimating covariance structure in multi-view data, where each subject is measured through several high-dimensional lenses, such as different omics assays. Factor models for such data usually need slow, fragile sampling algorithms.

Their method uses spectral decompositions to find latent factors that show up in one view or more, and to align them across views. Given those factors, conjugate priors make the posterior a simple product of standard distributions for each variable, skipping Markov chain sampling entirely. Theory gives reassuring large-dimension behaviour, including contraction and central limit results for point estimates. Simulations show accurate uncertainty quantification. An application integrates four views from a cancer cell multi-omics dataset, demonstrating speed and reliability together.

Catalog record

CREATORLorenzo Mauri
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-02772

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Inference on covariance structure in high-dimensional multi-view data Lorenzo Mauri Abstract This article focuses on covariance estimation for multi-view data. Popular approaches rely on factor-analytic decompositions that have shared and view-specific latent factors. Posterior computation is conducted via expensive and brittle Markov chain Monte Carlo (MCMC) sampling or variational approximations that underestimate uncertainty and lack theoretical guarantees. Our proposed methodology employs spectral decompositions to estimate and align latent factors that are active in at least one view. Conditionally on these factors, we choose jointly conjugate prior distributions for factor loadings and residual variances. The resulting posterior is a simple product of normal-inverse gamma distributions for each variable, bypassing MCMC and facilitating posterior computation. We prove favorable increasing-dimension asymptotic properties, including posterior contraction and central limit theorems for point estimators. We show excellent performance in simulations, including accurate uncertainty quantification, and apply the methodology to integrate four high-dimensional views from a multi-omics dataset of cancer cell samples. Keywords: Factor analysis; High-dimensional; Latent variable model; Multi-omics; Multi-view; Scalable Bayesian computation; Singular value decomposition 1 Introduction Multi-view data are routinely collected in many applied settings, ranging from biology to ecology. In biomedical and precision medicine applications, researchers commonly collect multi-omics data, ranging from gene expression to metabolomics. In such settings, there is often interest in inferring the within- and between-view covariance structure in the data. However, it is common for different data views to have very different dimensionalities, distributions, and signal-to-noise ratios, so that methods that ignore the multi-view structure and concatenate the data prior to analysis can produce sub-optimal results. This has motivated a rich and rapidly expanding literature on methods for integrating data from multiple sources, including Lock et al. 2013 ; Feng et al. 2018 ; Argelaguet et al. 2018 ; Argelaguet et al. 2020 ; Palzer et al. 2022 ; Velten et al. 2022 ; Ding et al. 2022 ; Yi et al. 2023 ; Dombowsky & Dunson 2025 ; Anceschi et al. 2024 ; Ma & Ma 2024 . Factor models ( Harman 1976 ; West 2003 ) provide a powerful way to reduce dimensionality and accurately estimate covariance in high dimensions. For multi-view data, a popular approach models the i -th observation from the m -th view as \begin{aligned} y_{mi}&=\Lambda_{m}\eta_{mi}+\epsilon_{mi},\quad\epsilon_{mi}\sim N_{p}(0,\Psi_{m}),\\ \eta_{mi}&=A_{m}^{\top}\eta_{i},\quad\eta_{i}\sim N_{k_{0}}(0,I_{k_{0}}),\end{aligned}\quad(i=1,\dots,n;m=1,\dots,M), (1) where \Psi_{m}=\text{diag}(\sigma_{1m}^{2},\dots,\sigma_{pm}^{2}) are residual variances and A_{m}\in\mathbb{R}^{k_{0}\times k_{s}} is a Boolean matrix that selects the k_{m} factors \eta_{mi} that are active in the m -th view from the vector of all factors \eta_{i} . Model ( 1 ) induces the following model for intra- and inter-view covariances cov(y_{mi})=\Lambda_{m}\Lambda_{m}^{\top}+\Psi_{m},\quad cov(y_{mi},y_{m^{\prime}i})=\Lambda_{m}A_{m}^{\top}A_{m^{\prime}}\Lambda_{m^{\prime}}^{\top} (2) and, using results for multivariate Gaussian distributions, for a pair of two views m\neq m^{\prime} , we have \displaystyle y_{mi} \displaystyle\mid y_{m^{\prime}i}\sim N_{p}(\mu_{i,m\mid m^{\prime}},\Sigma_{m\mid m}), (3) \displaystyle\mu_{i,m\mid m^{\prime}} \displaystyle=\Lambda_{m}A_{m}^{\top}A_{m^{\prime}}\Lambda_{m^{\prime}}^{\top}\begin{bmatrix}\Lambda_{m^{\prime}}\Lambda_{m^{\prime}}^{\top}+\Psi_{m^{\prime}}\end{bmatrix}^{-1}y_{m^{\prime}i}, \displaystyle\Sigma_{m\mid m} \displaystyle=\Lambda_{m}\Lambda_{m}^{\top}+\Psi_{m}-\Lambda_{m}A_{m}^{\top}A_{m^{\prime}}\Lambda_{m^{\prime}}^{\top}\begin{bmatrix}\Lambda_{m^{\prime}}\Lambda_{m^{\prime}}^{\top}+\Psi_{m^{\prime}}\end{bmatrix}^{-1}\Lambda_{m^{\prime}}A_{m^{\prime}}^{\top}A_{m}\Lambda_{m}^{\top}. The results in ( 3 ) can easily be generalized to more than two views. Equations ( 2 ) and ( 3 ) illustrate that the joint modeling of multiple views facilitates straightforward inferences on the covariance structure as well as the prediction of variables in certain views given the others. Argelaguet et al. 2018 adopted a Bayesian approach to fitting model ( 1 ) using spike-and-slab priors ( Ishwaran et al. 2005 ) to infer which factors are active in each view. Argelaguet et al. 2020 ; Velten et al. 2022 expanded this approach, allowing for group structure and spatial correlation across observations, respectively. The posterior is approximated using mean-field variational inference, which is likely to substantially underestimate uncertainty and lacks guarantees on accuracy of the posterior approximation. Alternatively, Gibbs samplers or other Markov chain Monte Carlo algorithms can be developed for posterior computation under model ( 1 ). However, these sampling algorithms can suffer from slow convergence and mixing even for single-view factor models when data are high dimensional. Additionally, shrinkage priors that are used to identify the number of active factors and in which view each factor is active can be highly sensitive to hyperparameter choice, calling for non-trivial and computationally intensive tuning routines. We also highlight another issue with full likelihood inference for model ( 1 ). In the presence of heterogeneity of dimensions and signal-to-noise ratios across data modalities, views with smaller dimension or lower signal-to-noise ratio have a relatively small contribution to the overall data likelihood. The latent factors that act only on these views might be poorly estimated. This issue is exacerbated by methods that employ information criteria or shrinkage priors to estimate the number of latent factors, which might completely drop these latent factors. The same problem affects most state-of-the-art factor analysis methods, such as Rockova & George 2016 ’s method, applied to the dataset obtained by concatenating the Y_{m} ’s. In a parallel line of research, Lock et al. 2013 ; Feng et al. 2018 proposed a matrix factorization approach ( JIVE , henceforth) that decomposes each data set into a low-rank joint structure, a low-rank view-specific component, and additive noise. Properties of JIVE have been studied in Yang & Ma 2025 . The key limitations of JIVE are that it assumes a very rigid structure by only letting relevant features to be shared by all views or view-specific and it does not provide any quantification of uncertainty. Chattopadhyay et al. 2024 recently proposed an alternative to Gibbs sampling for Bayesian analysis of high-dimensional single-view factor models. Their approach consists of estimating the latent factors via a matrix factorization and inferring loadings and residual error variances via surrogate regression problems, where the estimates for latent factors are treated as observed covariates. They showed posterior concentration and valid coverage for credible intervals of entries of the covariance. Their approach is related to joint estimation for factor models ( Chen et al. 2019 ; Chen et al. 2020 ; Lee et al. 2024 ; Mauri & Dunson 2025 ; Mauri et al. 2025 ) , which has been shown to be consistent in double asymptotic scenarios in which both sample size and data dimension diverge. Chattopadhyay et al. 2024 ’s method can have excellent empirical performance when applied to the concatenated Y_{m} ’s in recovering the overall covariance matrix, but suffers from the aforementioned problems in estimating certain intra- or inter-view covariances in presence of view heterogeneity, as we show in our numerical experiments. Motivated by these considerations, we propose Factor Analysis for Multi-view data via spectral Alignment ( FAMA ). First, FAMA estimates the active latent factors in each view (the matrices of \{\eta_{mi}\}_{i=1}^{n} for m=1,\dots,M ) by spectral decompositions treating each view separately. This enables accurate estimation of the latent factors for each view, regardless of signal-to-noise ratio and dimension heterogeneity across views. Next, we use another spectral decomposition to align factor estimates from each view. This step provides an estimate of the matrix of \eta_{i} ’s, the latent factors that are active in at least one view, up to an orthogonal transformation. Finally, we infer posterior distributions for the loadings and residual variances via surrogate Bayesian regression tasks using the estimate for the latent factors as observed covariates and employing conjugate normal-inverse gamma priors. These posteriors can be computed analytically in parallel for the variables in each view, and induce posteriors on the inter- and intra-view covariances. With a mild analytic

Read the complete text