Gaussian Mixture Models in Hilbert Spaces via Kernel Methods

Daniel López-Montero, Antonio Álvarez-López, Marcos Matabuena · 2026

Read Gaussian Mixture Models in Hilbert Spaces via Kernel Methods by Daniel López-Montero, Antonio Álvarez-López, Marcos Matabuena (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05996.txt.

Introduction

Daniel López-Montero, Antonio Álvarez-López, and Marcos Matabuena build Gaussian mixture models for data that live in Hilbert spaces. Modern data often consist of evolving, possibly infinite dimensional objects, such as functional measurements over time. In such spaces, defining probability through densities can be ill posed or technically hard, which blocks standard mixture modeling.

Their framework rests on kernel mean embeddings, a way to represent distributions through their averages in a feature space. They develop fast tuning methods for fitting the mixtures. Theory shows the procedure is well defined and that the model class can approximate a dense range of distributions. Experiments cover functional data and random graphs in Laplacian spaces drawn from medical settings.

Catalog record

CREATORDaniel López-Montero, Antonio Álvarez-López, Marcos Matabuena
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE119 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.05996

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Gaussian Mixture Models in Hilbert Spaces via Kernel Methods Daniel López-Montero, Antonio Álvarez-López, Marcos Matabuena Abstract Modern datasets across many disciplines increasingly consist of time-evolving, potentially infinite-dimensional random objects, such as dynamic functional data, which are naturally modeled in Hilbert spaces. In these settings, characterizing probability measures, for example, through densities, can be ill-defined or technically challenging. Motivated by clustering applications, we propose a Gaussian mixture framework for Hilbert-space-valued data based on kernel mean embeddings and develop efficient optimization algorithms for estimation. We establish theoretical guarantees showing that the proposed algorithm is well defined and that the model yields a dense class of approximations in infinite-dimensional spaces. We evaluate the framework through extensive experiments on diverse structures and data geometries, including L 2 L^{2} -functional data and random graphs in Laplacian spaces arising in modern medical applications. 0 0 footnotetext: Code, data and experiments are available at https://github.com/dani2442/rkhs-mixture-models/ . 1 Introduction Technological progress across multiple domains is generating high-dimensional datasets that demand new analytical tools. Modern data structures, such as random functions, spatial fields, images, signals, graphs, or trajectories are often more naturally represented as random objects in Hilbert spaces than as finite-dimensional Euclidean vectors in \mathbb{R}^{d} . Analyzing the data directly using Hilbert space methods preserves their intrinsic geometric structure and allows their full potential to be exploited [ 53 , 26 ] . In many applications, observed data are heterogeneous and may arise from K latent populations, each characterized by a probability law P_{i} , with i=1,\dots,K . Mixture models [ 28 ] provide a flexible framework for representing such data-generating mechanisms and, under suitable regularity conditions, can approximate broad classes of probability measures. This paper proposes a mixture-based framework for probability measures on a separable Hilbert space \mathcal{X} , extending the classical idea of Gaussian mixture models (GMM) for finite-dimensional data [ 22 ] . Extending GMMs beyond finite-dimensional spaces is challenging. Infinite-dimensional Hilbert spaces do not admit a Lebesgue measure, so densities cannot be defined in the standard way, and likelihood-based methods typically require the choice of a reference measure [ 19 , 41 ] . We therefore adopt a density-free perspective: we fit Gaussian mixture models by minimizing the Maximum Mean Discrepancy (MMD) [ 33 ] between the kernel mean embedding of the empirical measure and that of the target mixture model. We now formulate the problem. Model and objective. Let P be an unknown probability measure on a separable Hilbert space \mathcal{X} . A Gaussian measure on \mathcal{X} , denoted by \mathcal{N}(m,\mathcal{K}) , is uniquely determined by a mean element m\in\mathcal{X} and a covariance operator \mathcal{K}\colon\mathcal{X}\to\mathcal{X} , which generalizes the finite-dimensional covariance matrix. To ensure non-Dirac Gaussian components with finite total variance, \mathcal{K} must be non-zero, self-adjoint, positive semi-definite, and trace-class. For a fixed number of components K\in\mathbb{N} , we approximate P with a finite Gaussian mixture Q_{\theta}\coloneqq\sum_{k=1}^{K}\pi_{k}\,\mathcal{N}(m_{k},\mathcal{K}_{k}),\qquad\pi_{k}\geq 0\quad\forall k,\qquad\sum_{k=1}^{K}\pi_{k}=1, (1) parameterized by \theta\coloneqq(\pi_{k},m_{k},\mathcal{K}_{k})_{k=1}^{K}\in\Theta_{K} , where \Theta_{K} denotes the admissible parameter space. We fit Q_{\theta} to P by minimizing the Maximum Mean Discrepancy induced by a measurable positive-definite kernel \kappa\colon\mathcal{X}\times\mathcal{X}\to\mathbb{R} : \operatorname{MMD}_{\kappa}^{2}(P,Q_{\theta})=\mathbb{E}_{X,X^{\prime}\sim P}[\kappa(X,X^{\prime})]-2\,\mathbb{E}_{X\sim P,\;Y\sim Q_{\theta}}[\kappa(X,Y)]+\mathbb{E}_{Y,Y^{\prime}\sim Q_{\theta}}[\kappa(Y,Y^{\prime})], (2) where X and X^{\prime} are independent copies drawn from P , Y and Y^{\prime} are independent copies drawn from Q_{\theta} , and all four random variables are mutually independent. This leads to a density-free fitting criterion that depends only on kernel expectations. When the kernel \kappa is characteristic , the MMD is a metric on probability measures. Important examples include Gaussian radial kernels in Euclidean spaces and, more generally, in suitable metric spaces of strong negative type [ 60 ] . The formal definitions and statistical background are deferred to Supplemental Material Appendix A . In practice, given X_{1},\dots,X_{n}\overset{\text{i.i.d.}}{\sim}P , we replace P by the empirical measure P_{n}\coloneqq\frac{1}{n}\sum_{i=1}^{n}\delta_{X_{i}} and solve the problem \min_{\theta\in\Theta_{K}}\operatorname{MMD}_{\kappa}^{2}(P_{n},Q_{\theta}). (3) Motivation. The motivation for this work is to perform clustering of dynamic, time-varying functional objects [ 23 , 24 ] that take values in a possibly infinite-dimensional space \mathcal{X} . Figure 1 shows examples of contemporary data structures that arise in this setting. Let \mathcal{X} be a Hilbert space, and let (X(t))_{t\in[0,T]} be a stochastic process valued in \mathcal{X} . We observe independent sample trajectories X_{1}(t),\dots,X_{n}(t) and denote by P_{t} the law of X(t) at time t . We estimate a time-varying mixture model by minimizing a temporal finite-sample approximation of the integrated discrepancy \int_{0}^{T}\operatorname{MMD}_{\kappa}^{2}\bigl(P_{n,t},Q_{\theta,t}\bigr)\,\mathrm{d}t, where P_{n,t} is the empirical law of X_{1}(t),\dots,X_{n}(t) at time t . In Section 4.2 , this methodology is applied to continuous glucose monitoring data from a diabetes clinical trial evaluating a new therapy for juvenile diabetes [ 67 ] . Notation. Throughout, \mathcal{X} denotes a separable real Hilbert space with inner product \langle\cdot,\cdot\rangle_{\mathcal{X}} , norm \|\cdot\|_{\mathcal{X}} , and identity operator I_{\mathcal{X}} . We let \mathcal{L}(\mathcal{X}) denote the space of bounded linear operators on \mathcal{X} . For a trace-class operator T\in\mathcal{L}(\mathcal{X}) with eigenvalues (\lambda_{j})_{j\geq 1} , its Fredholm determinant is \det_{F}(I_{\mathcal{X}}+T)\coloneqq\prod_{j=1}^{\infty}(1+\lambda_{j}) . Let \mathcal{P}(\mathcal{X}) denote the set of Borel probability measures on \mathcal{X} , and for p\geq 1 , let \mathcal{P}_{p}(\mathcal{X}) be the subset of measures with finite p -th moments. For finite-dimensional spaces, boldface lowercase letters ( \mathbf{x},\mathbf{m} ) denote vectors and boldface uppercase letters ( \mathbf{K},\mathbf{I} ) denote matrices. We write \mathbf{1} for the all-ones vector and \Delta^{K-1} for the probability simplex. 1.1 Contributions and paper organization Figure 1 : Representative structured data samples. Left: Hourly continuous glucose monitoring paths. Center: Signals on a fixed graph with varying node intensities. Right: Correlation matrices. The main contributions of this paper are threefold: • Closed-form MMD fitting for Hilbert-space mixtures. We derive exact closed-form expressions for \operatorname{MMD}_{\kappa}^{2}(P_{n},Q_{\theta}) for Gaussian radial basis and polynomial kernels, extending known \mathbb{R}^{d} formulas to separable Hilbert spaces [ 1 , 13 ] . Moreover, we prove that, for every p\geq 1 , finite Gaussian mixtures of the form ( 1 ) are dense in \mathcal{P}_{p}(\mathcal{X}) under the Wasserstein distance W_{p} . This result provides theoretical support for the approximation capacity of the proposed model in infinite-dimensional applications. • Consistent finite-dimensional computation. We introduce a spectral projection framework that reduces infinite-dimensional computation to standard linear algebra in \mathbb{R}^{M} . We prove that both the projected MMD objective and the posterior probabilities converge to their exact infinite-dimensional counterparts as the number of terms in the finite-dimensional approximation satisfies M\to\infty ; see Propositions 2.1 and 2.2 . • Real-world performance. We evaluate the empirical performance of the proposed method across several data structures, including random functions in L^{2} , spatial fields, rotations in \mathrm{SO}(3) , and graph Laplacians. We compare our method against state-of-the-art approaches on established clustering benchmarks, demonstrating competitive performance; see Sections 4 and F . To further assess its performance on dynamic functional data, we analyze a clinical trial dataset with continuous glucose monitoring data, aiming to identify subgroups of individuals with heterogeneous responses to therapy. The supplementary material contains all proofs, together with additional analytical results. Related work Functional data analysis and clustering. Functional data analysis (FDA) studies samples naturally viewed as functions, curves, or surfaces, typically using basis expansions, smoothing, and functional principal component analysis [ 53 , 26 ] . Our spectral projection framework is related to Karhunen–Loève expansions for Gaussian measures [ 53 , 26 , 4 ] ,

Read the complete text