Enhancing the Tensor Normal via Geometrically Parameterized Cholesky Factors
Quinn Simonis, Martin T. Wells · 2025
Read Enhancing the Tensor Normal via Geometrically Parameterized Cholesky Factors by Quinn Simonis, Martin T. Wells (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10645.txt.
Introduction
Quinn Simonis and Martin Wells explore in 2025 a Bayesian extension of the tensor normal model, a way to model data that vary along several axes at once. The key object is a multi way covariance, whose square root like factor carries the structure. Their expansion builds that factor from a geometric average of simpler multi way factors, under a log based metric.
That choice removes one of three structural parts of the covariance without losing a tractable likelihood. A known splitting of matrices turns out to be a special case. The model is applied with a sampling method to tests of which covariance parts matter, in both fixed and season changing settings, with fast gradient updates available.
Catalog record
| CREATOR | Quinn Simonis, Martin T. Wells |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-10645
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing the Tensor Normal via Geometrically Parameterized Cholesky Factors Quinn Simonis, Martin T. Wells Abstract In this article, we explore Bayesian extensions of the tensor normal model through a geometric expansion of the multi-way covariance’s Cholesky factor inspired by the Fréchet mean under the log-Cholesky metric. Specifically, within a tensor normal framework, we identify three structural components in the covariance of the vectorized data. By parameterizing vector normal covariances through such a Cholesky factor representation, analogous to a finite average of multiway Cholesky factors, we eliminate one of these structural components without compromising the analytical tractability of the likelihood, in which the multiway covariance is a special case. Furthermore, we demonstrate that a specific class of structured Cholesky factors can be precisely represented under this parameterization, serving as an analogue to the Pitsianis-Van Loan decomposition. We apply this model using Hamiltonian Monte Carlo in a fixed-mean setting for two-way covariance relevancy detection of components, where efficient analytical gradient updates are available, as well as in a seasonally-varying covariance process regime. Keywords: Bayesian models, covariances estimation, Fréchet mean, Hamiltonian Monte Carlo, Log-Cholesky metric, Geodesic Monte Carlo, multiway models, Kronecker product, Introduction This article explores the generalization of a classical likelihood function for multimodal data, where observations are represented as tensors \mathcal{Y}\in\mathbb{R}^{\times_{i=1}^{D}d_{i}} . The normal tensor model assumes Gaussianity in each mode, which means that individual elements follow y_{i_{1},i_{2},\cdots,i_{D}}\sim\mathcal{N}(0,1) . Mode-wise products enable the construction of a broad class of distributions that capture correlations along specific modes. Alternatively, tensor normal models can be viewed through the lens of vectorization. Assuming lexicographic order, if \mathcal{Y}\sim\mathcal{T}\mathcal{N}(M,\{\Sigma_{i}\}_{i=1}^{D}) , then its vectorized form follows vec(Y)\sim\mathcal{N}(vec(M),\otimes_{i=D}^{1}\Sigma_{i}) . This likelihood is particularly valuable in Bayesian analysis for examining correlations in high-dimensional data, as it naturally decomposes complex dependencies into lower-dimensional components. Applications of the tensor normal model include analysis of mortality data [ 8 ] , assessment of healthcare [ 15 ] , and modeling in space time [ 5 , 11 ] . Beyond the two-way case, higher-order examples arise in relational data analysis [ 18 ] , where a Bayesian approach extends the tensor normal model through the Tucker product. Although the tensor normal model offers computational advantages, it imposes structural assumptions on global covariance that may not accurately capture the underlying data-generating process [ 3 , 29 ] . Our work is closely aligned with the literature on Kronecker product expansions of covariances. This area of work was inspired by the work of [ 26 ] , where the authors show that in fact any arbitrary matrix Q\in\mathbb{R}^{d_{1}d_{2}\times d_{1}d_{2}} can be represented as: Q=\sum_{i=1}^{r^{2}}A_{i}\otimes B_{i},\quad A_{i}\in\mathbb{R}^{d_{1}\times d_{1}},\text{ }B_{i}\in\mathbb{R}^{d_{2}\times d_{2}},\text{ }r=\min\{d_{1},d_{2}\}. Literature in Kronecker product expansions of covariances has attracted attention within a statistical context, wherein the goal is often to find a minimal d to adequately represent a covariance matrix as a sum of Kronecker products: \Sigma\approx\sum_{i=1}^{d}A_{i}\otimes B_{i},\quad A_{i}\in\mathcal{P}^{+}(d_{1}),B\in\mathcal{P}^{+}(d_{2}), (1) where \mathcal{P}^{+}(d) denotes the manifold of d\times d symmetric positive definite (SPD) matrices. Examples include [ 30 ] , who investigated using Kronecker product expansions to model matrix-valued data using a penalized least squares approach. In this article, we develop a Bayesian model selection procedure to identify the level of structural sparsity in the strictly lower triangular entries of the Cholesky factor for a multiway covariance. This approach leverages a geometrically inspired parametrization of Cholesky factors for multiway covariances, addressing one of the key structural assumptions imposed by the tensor normal model. Our work is motivated by the log-Cholesky metric [ 23 ] , which observes that Cholesky factors of symmetric positive definite (SPD) matrices form a topological manifold, wherein the author endows this manifold with a Riemannian metric that has a closed form Fréchet mean. Building on this, we express the Cholesky factor of an arbitrary SPD matrix of Cholesky factors of multiway SPD matrices, introducing a simplex-distributed random variable to regulate the structural complexity of lower triangular entries. This alternative parametrization maintains full analytic expressivity, and we employ Hamiltonian Monte Carlo [ 25 ] to avoid the constraints of conditional conjugacy required by Gibbs sampling. The remainder of the paper is structured as follows: • Section 1 provides the mathematical background of the paper, with an introduction to Hamiltonian Monte Carlo in Section 1.1 , and the Tensor Normal distribution, and its structural limitations in Section 1.2 . • Section 2 introduces the differential geometry of Cholesky factors in Section 2.1 used to extend the multiway covariance parameterization of the tensor normal in Section 2.2 . The latter subsection also provides bounds on approximation quality of arbitrary Cholesky factors of symmetric positive definite matrices under our parameterization. • Section 3 provides auxiliary results for centering matrix priors in section 3.1 , and two Bayesian models are proposed in Sections 3.2 and 3.3 for static and seasonally dynamic covariance implementations, respectively. • Section 4 provide results detailing our choice in the use of stan as our sampling algorithm over a Geodesic Monte Carlo implementation. • Section 5 provides simulated data examples for assessing the quality of the model’s ability to adequately recognize degree of separability for a data generating processes’ covariance in static and seasonally dynamic covariances in Sections 5.1 and 5.2 , respectively. • Section 6 provides real data applications of our model in a static case to the Wisconsin Breast Cancer dataset in Section 6.1 and a dynamic implementation for the analysis of coefficients in a Bayesian regression of continental US climate data corresponding to annual cyclic patterns in Section 6.2 . • Section 7 provides potential future directions of work, and Section 8 provide supplementary details for efficient Hamiltonian Monte Carlo implementation of the static and seasonally dynamic parameterizations in Sections 8.1 and 8.2 , respectively. 1 Mathematical Background 1.1 Bayesian Inference Bayesian inference aims to characterize the latent structure of a parameter space \Gamma given a collection of observations Y=\{y_{1},\ldots,y_{n}\} . We do so assuming a prior distribution in our parameter space p(\Gamma) and the likelihood of our data parameterized by L(y;\Gamma) . The goal of Bayesian inference is to characterize the posterior distribution of \Gamma given Y according to Bayes theorem P(\Gamma|Y)=\frac{L(Y;\Gamma)P(\Gamma)}{\int_{\gamma}L(Y;\gamma)P(\gamma)d\gamma}, however, the calculation of the integral in the denominator is often intractable. As such, posterior inference often resolves to using computational techniques such as MCMC sampling or making approximations with known distributions such as variational inference. MCMC sampling is asymptotically exact and is generally referred to as the gold standard for computational inference. Historically, MCMC inference is characterized by Gibbs [ 9 ] and random walk samplers. Gibbs sampling makes use of sampling from the full conditional posteriors. That is, by sampling from P(\beta|Y,\Gamma_{-\beta}) . This, however, requires either prior distributions that are conditionally conjugate with the likelihood or analytically tractable expansions that are conditionally conjugate. Often these impose restrictions which limit expressivity of the prior and are difficult to derive. Random walk samplers, which are not constrained by the necessity of conditional conjugacy, are often slow to draw independent samples from the posterior distribution. Hamiltonian Monte Carlo [ 25 ] is an advanced MCMC technique which extends random walk samplers by leveraging gradient information combined with updates defined by energy-preserving symplectic dynamics [ 17 ] to build an ergodic Markov chain, which combined with a Metropolis