Information Geometry for Wasserstein KL Divergence of Gaussian Measures on ℝ n \mathbb{R}^{n}

Adwait Datar, Nihat Ay · 2025

Read Information Geometry for Wasserstein KL Divergence of Gaussian Measures on ℝ n \mathbb{R}^{n} by Adwait Datar, Nihat Ay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-24022.txt.

Introduction

Adwait Datar and Nihat Ay build in 2025 a geometry for a new distance between bell curve distributions, blending a transport based metric with a classic divergence. On the space of such curves, swapping the classic metric for the transport one yields a new connection and the divergence in question. The authors prove the paths of this geometry run complete, sort their end behaviors, and show each ordered pair of curves is joined by a single path born of a simple potential.

They give an explicit formula for the divergence, split into parts from the center and the spread of the curves. It is never negative and is zero only for equal curves. Edge cases with flat targets are also settled, with care at the sharpest limits.

Catalog record

CREATORAdwait Datar, Nihat Ay
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE118 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-24022

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Information Geometry for Wasserstein KL Divergence of Gaussian Measures on ℝ n \mathbb{R}^{n} Adwait Datar, Nihat Ay Abstract We study the Wasserstein Kullback–Leibler divergence (WKL divergence) on the manifold of nondegenerate Gaussian measures over ℝ n \mathbb{R}^{n} . In the canonical-divergence construction, the Fisher–Rao metric recovers forward KL along intrinsic mixture geodesics and reverse KL along geodesics of the conjugate exponential connection. Replacing the Fisher–Rao metric by the Otto metric and following the latter route produces the e 1 e_{1} -connection underlying WKL divergence. We establish its geodesic completeness, classify its forward limits, prove that every ordered pair is joined by a unique e 1 e_{1} -connector generated by a quadratic potential, and derive an explicit WKL divergence formula with separate mean and covariance contributions. WKL divergence is nonnegative and separating. At equal covariances WKL divergence equals one half of the squared Euclidean mean distance. Finally, WKL divergence extends finitely and continuously to singular targets from a nondegenerate source, but diverges when the target remains nondegenerate and the source covariance becomes singular. Path-dependent joint limits at Dirac pairs preclude a continuous extension to the full positive-semidefinite covariance product, although a lower-semicontinuous extended-real extension exists. keywords Wasserstein geometry, Kullback–Leibler divergence, Gaussian measures, Otto metric 1 Introduction Classical information geometry equips statistical models with intrinsic objects such as the Fisher–Rao metric and the Kullback–Leibler divergence (KL divergence), without requiring a metric on the sample space Amari and Nagaoka (2000) ; Ay et al. (2017) . In this article the sample space is \mathbb{R}^{n} , endowed with its Euclidean geometry, and we ask for an information divergence that reflects this ground geometry. The classical KL divergence does not do so: if x\neq y , then the KL divergence between \delta_{x} and \delta_{y} is infinite, independently of the Euclidean distance \lVert x-y\rVert . Our starting point is Ay’s canonical-divergence construction Ay and Amari (2015) ; see also Felice and Ay (2021) . For a Riemannian metric g , an affine connection \nabla , and its \nabla -geodesic \gamma from \mu to \nu , the canonical divergence is the weighted energy \displaystyle D_{g}^{(\nabla)}(\mu\|\nu)=\int_{0}^{1}t\,\lVert\dot{\gamma}(t)\rVert_{\gamma(t),g}^{2}\,dt. With the Fisher--Rao metric, the intrinsic Gaussian mixture 1 1 1 Here “mixture” is intrinsic to the Gaussian family, not a literal density mixture. geodesic yields the forward divergence D_{\rm KL}(\mu\|\nu) . Its Fisher–Rao conjugate is the usual exponential connection, denoted e_{0} here to distinguish it from e_{1} ; the e_{0} -geodesic yields the reverse divergence D_{\rm KL}(\nu\|\mu) . Both calculations are included for completeness in Appendices A.3 and B.1 . We follow the same conjugate-connection route after replacing the Fisher–Rao metric by the Otto metric. Computing the canonical divergence for the intrinsic mixture geodesic with the norm induced by the Otto metric gives an exact integral representation, but not an explicit closed form in general; see Appendix A.3 . Instead, conjugating the mixture connection with respect to the Otto metric gives the e_{1} -connection Ay (2025b) ; Ay (2025a) . Its Gaussian geodesics can be computed explicitly, and the corresponding canonical divergence yields a closed endpoint formula; see Appendix B.2 . This is the Otto analogue of the reverse-KL route above. Since the Otto metric induces the 2 -Wasserstein distance as its Riemannian distance Otto (2001) ; Khan and Zhang (2022) , we call the resulting functional the Wasserstein KL divergence , abbreviated WKL divergence, following our preliminary work Datar and Ay (2026) . The Gaussian WKL divergence construction and its central closed formula first appeared in our Geometric Science of Information conference paper Datar and Ay (2026) . That paper derived the closed formula for WKL divergence, including commuting and univariate specializations; compared the equal-covariance case with reverse KL; and presented preliminary equal-covariance Dirac-limit observations and numerical illustrations. The conference paper applied the analytic formulation in Ay (2025b) ; Ay (2025a) , which assumes a compact sample-space manifold, to Gaussian manifold over the non-compact sample space \mathbb{R}^{n} without supplying the corresponding noncompact justification. Relative to that conference paper, the present journal article supplies an information geometric foundation for the WKL divergence on the Gaussian manifold over the noncompact sample space \mathbb{R}^{n} . The main geometric advances here place the previously derived e_{1} endpoint connector within a rigorous framework, prove geodesic completeness of the Gaussian e_{1} -connection, give the forward limits of its geodesics. For completeness, the appendices describe the intrinsic Gaussian mixture connection, verify its Fisher–Rao/ e_{0} and Otto/ e_{1} metric-conjugacy relations, and recover the classical forward-KL identity; they also derive the Otto/mixture integral representation and its closed univariate specialization. The present article also strengthens the analytic study of WKL divergence. A general boundary theorem replaces the preliminary Dirac observation: it characterizes both one-sided singular-covariance limits, analyzes joint Dirac limits, constructs a lower-semicontinuous extension, and proves that no continuous extension exists on the full product of positive-semidefinite covariance cones. Subsequent work, not part of the present article, places WKL divergence in a broader family of state-space-aware KL divergences, extends its Gaussian formula to elliptic families, introduces Kalman–Wasserstein and other metric-dependent variants, and applies these divergences to low-noise stochastic control Stein et al. (2026) . That work takes the Gaussian WKL divergence formula as its starting point; the present article is concerned with the intrinsic Gaussian geometry, global connector, local expansion, and singular boundary behavior of WKL divergence itself. The remainder of the paper is organized as follows. Section 2 develops the Gaussian manifold, the Fisher–Rao and Otto metrics, the e_{0} - and e_{1} -connections, and their global properties. Section 3 derives the explicit WKL divergence formula and studies its transport interpretation, local behavior, comparisons, univariate specialization, and singular-covariance limits. Appendix A develops the intrinsic Gaussian mixture connection, its metric conjugates, and the corresponding canonical divergences; Appendix B derives the reverse-KL divergence and WKL divergence identities along the e_{0} - and e_{1} -geodesics. 2 Gaussian Information Geometry: Metrics, Connections, and Canonical Divergences In this section, we introduce the information geometric constructions leading to the Wasserstein KL divergence. We follow the development from Ay (2025b) ; Ay (2025a) and verify the required constructions directly in the Gaussian setting; see those references for further details. On one hand, the results in the present paper can be seen as a special case for instantiation of the results from Ay (2025b) ; Ay (2025a) to the case of finite dimensional Gaussian measures. On the other hand, these earlier results assume a compact sample manifold structure which excludes Gaussian measures defined on \mathbb{R}^{n} . The following calculations can thus be also seen as an extension of earlier results in the Gaussian setting, without assuming compactness of the underlying manifold. Notation and conventions. Throughout, I denotes the identity matrix of the size determined by context, A^{\top} the transpose of A , and \textnormal{Tr}\left({A}\right) its trace. We write \lVert\cdot\rVert for the Euclidean norm and \lVert\cdot\rVert_{F} for the Frobenius norm. For a symmetric matrix A , the relations A\succ 0 and A\succeq 0 mean positive definiteness and positive semidefiniteness, respectively. The matrix exponential is denoted by e^{A} , and \nabla{f} denotes the Euclidean gradient of f . 2.1 The Gaussian Manifold and Its Tangent and Cotangent Spaces Let \lambda denote the Lebesgue measure on \mathbb{R}^{n} , and let C^{\infty}(\mathbb{R}^{n}) denote the space of smooth real-valued functions on \mathbb{R}^{n} .

Read the complete text