Tracking Temporal Dynamics of Vector Sets with Gaussian Process
Taichi Aida, Mamoru Komachi, Toshinobu Ogiso, Hiroya Takamura, Daichi Mochihashi · 2025
Read Tracking Temporal Dynamics of Vector Sets with Gaussian Process by Taichi Aida, Mamoru Komachi, Toshinobu Ogiso, Hiroya Takamura, Daichi Mochihashi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-15538.txt.
Introduction
Taichi Aida and co authors present a method for tracking how sets of vectors change over time in a 2025 paper. Such sets appear in ecology, crime mapping, and word embeddings, where each set has its own structure that also shifts. Comparing sets across time is hard because both members and structure move.
The team models the distribution behind each set with infinite dimensional Gaussian processes. Approximating the latent function with random Fourier features gives compact representations that can be compared across time. Plotting those representations in low dimensions shows transitions clearly. Tests on crime distributions and word embeddings show the method captures change in an interpretable and robust way.
Catalog record
| CREATOR | Taichi Aida, Mamoru Komachi, Toshinobu Ogiso, Hiroya Takamura, Daichi Mochihashi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-15538
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Tracking Temporal Dynamics of Vector Sets with Gaussian Process Taichi Aida, Mamoru Komachi, Toshinobu Ogiso, Hiroya Takamura, Daichi Mochihashi Abstract Understanding the temporal evolution of sets of vectors is a fundamental challenge across various domains, including ecology, crime analysis, and linguistics. For instance, ecosystem structures evolve due to interactions among plants, herbivores, and carnivores; the spatial distribution of crimes shifts in response to societal changes; and word embedding vectors reflect cultural and semantic trends over time. However, analyzing such time-varying sets of vectors is challenging due to their complicated structures, which also evolve over time. In this work, we propose a novel method for modeling the distribution underlying each set of vectors using infinite-dimensional Gaussian processes. By approximating the latent function in the Gaussian process with Random Fourier Features, we obtain compact and comparable vector representations over time. This enables us to track and visualize temporal transitions of vector sets in a low-dimensional space. We apply our method to both sociological data (crime distributions) and linguistic data (word embeddings), demonstrating its effectiveness in capturing temporal dynamics. Our results show that the proposed approach provides interpretable and robust representations, offering a powerful framework for analyzing structural changes in temporally indexed vector sets across diverse domains. (a) Estimation. (b) Tracking. Figure 1: Overview. (a) We randomly sample K cosine basis functions and estimate their corresponding weights from the observed vector set \mathbb{R}^{N\times 2} at each time step t\in T . This allows us to represent the temporal dynamics of the vector set as a K -dimensional vector \mathbf{w}_{t}\in\mathbb{R}^{K} , for each time step. (b) After that, we apply Principal Component Analysis (PCA) to the set of weight vectors \{\mathbf{w}_{t}\}_{t=1}^{|T|} to obtain two-dimensional projections \mathbf{v}_{t}\in\mathbb{R}^{2} . This enables compact visualization and analysis of temporal transitions of vector sets over time. 1 Introduction Analyzing dynamically evolving vector sets is a critical challenge across various domains. Here, each vector typically represents a data point characterized by multiple attributes (e.g., spatial coordinates, statistical properties, or semantic features), and the entire set encodes the distribution of such points at a given time. For example, in ecology, the interaction among plants, herbivores, and carnivores leads to temporal changes in habitats ( Odum 1971 ) , and analyzing these temporal transitions ( Kass et al. 2018 ) can help reveal ecosystem dynamics. In sociology, social phenomena such as population movements and crime incidents can be studied over time across specific regions ( Murakami & Yamagata 2019 ; Murakami et al. 2020 ) , enabling an understanding of changing patterns. In linguistics, understanding how the meaning of a particular word changes over time ( Kulkarni et al. 2015 ; Hamilton et al. 2016 ; Aida et al. 2021 ) can be aided by acquiring embeddings from usage examples and analyzing the resulting collections ( Aida & Bollegala 2023 ; Nagata et al. 2023 ) . Thus, understanding the temporal transitions of spatially distributed sets of vectors is foundational for revealing the underlying structures and interactions of such phenomena. Previous studies have proposed various methods to estimate vector sets at particular time periods (e.g., species distributions, crime locations, word usage embeddings) ( Kass et al. 2018 ; Murakami & Yamagata 2019 ; Murakami et al. 2020 ; Weisburd et al. 2006 ; Hachadoorian et al. 2011 ; Aida & Bollegala 2023 ) . However, while these approaches are effective at modeling states at specific times, they are limited in their ability to reveal the mechanisms of temporal variation across the entire vector set and the relationships between different sets of vectors . To address these limitations, we propose a novel method that represents each vector set at a given time as a single distribution using a compact approximation of Gaussian processes (GPs) via Random Fourier Features (RFF) ( Rahimi & Recht 2007 ) . RFF approximates a distribution using a combination of K cosine functions, allowing complicated distributions to be represented as K -dimensional real vectors in frequency space. This compact representation is derived from GPs, which can be viewed as an infinite-dimensional generalization of Gaussian distributions over function spaces. GPs are known for their flexibility in capturing complicated, nonparametric structures, making them well-suited for modeling the temporal changes of vector sets. However, traditional GPs retain information about all data points to model complicated distributions, making it challenging to extract essential distributional characteristics. By leveraging RFF, we retain the expressive power of GPs while enabling a tractable and interpretable analysis of transitions of vector sets over time. We evaluate the effectiveness of the proposed method by applying it to spatial crime data and analyzing semantic changes in words via sets of word usage vectors. 2 Tracking Vector Set Transitions via RFF To analyze how sets of vectors evolve over time, we need a way to compactly and meaningfully represent each set at a given time. Rather than treating vectors individually, we represent each set as a probability density function over the vector space, capturing global structures. This enables us to move beyond pointwise statistics or aggregate moments and to treat the entire set as a unified object with rich geometric and statistical properties. To achieve this, we model the density of vectors on a target space \mathcal{X} using the Gaussian Process Density Sampler (GPDS) ( Murray et al. 2008 ) : \displaystyle p(\mathbf{x}) \displaystyle=\frac{\sigma(f(\mathbf{x}))}{Z},~~Z=\int_{\cal X}\!\sigma(f(\mathbf{x}))\mathrm{d}\mathbf{x}. (1) Here, \sigma(x)=1/(1+e^{-x}) is the sigmoid function, f is a sample from a Gaussian process \mathrm{GP}(0,k(\mathbf{x},\mathbf{x}^{\prime})) , where k is a kernel function that defines similarity between vectors in space \mathcal{X} . The function f serves as the main learnable component, defining the unnormalized log-density over the space. However, the density equation 1 is intractable due to the normalization constant Z , which involves integration over the entire space \mathcal{X} . To circumvent this issue, we apply contrastive learning ( Gutmann & Hyvärinen 2012 ) , treating Z as a latent variable to be learned by maximizing the following discriminative objective: \displaystyle\sum_{\mathbf{x}\in{\cal D}}\log p(z\!=\!1\,|\,\mathbf{x})+\sum_{\mathbf{x}^{\prime}\in\overline{\cal D}}\log p(z\!=\!0\,|\,\mathbf{x}^{\prime}), (2) where \mathcal{D} is the set of positive, observed samples and \overline{\mathcal{D}} is a set of randomly sampled negative examples and binary variable z indicates whether data is generated from the target distribution or random distribution. Conventionally, the function f is represented non-parametrically as a vector of function values at N data points \mathbf{f}\!=\!(f(\mathbf{x}_{1}),\cdots,f(\mathbf{x}_{N}))^{\mathsf{T}} . To model the density function f non-parametrically from the data, GP-based approach usually requires computing the kernel matrix between N points, which incurs O(N^{3}) complexity and lacks explicit global information. Instead of explicitly computing the kernel function k(\mathbf{x},\mathbf{x}^{\prime}) , which corresponds to an inner product in an infinite-dimensional feature space, we approximate it as a dot product in a randomized K -dimensional space k(\mathbf{x},\mathbf{x}^{\prime})\approx\phi(\mathbf{x})^{\mathsf{T}}\phi(\mathbf{x}^{\prime}) . Here, \phi(\mathbf{x}) is a finite-dimensional random feature map constructed via Fourier transform and Monte Carlo integration, defined as: \phi(\mathbf{x})=(\phi_{1}(\mathbf{x}),\phi_{2}(\mathbf{x}),\ldots,\phi_{K}(\mathbf{x}))^{\mathsf{T}} . For the Gaussian kernel, the feature function can be expressed as \left\{\begin{array}[]{rl}\phi_{k}(\mathbf{x})&=\sqrt{\dfrac{2}{K}}\cos(\bm{\omega}_{k}^{\mathsf{T}}\mathbf{x}+b_{k}),\\[10.00002pt] \bm{\omega}_{k}&\sim\mathcal{N}(\mathbf{0},\sigma^{2}\mathbf{I}),~b_{k}\sim\mathrm{Unif}[0,1].\end{array}\right. This is known as Random Fourier Features (RFF) ( Rahimi & Recht 2007 ) . Each \bm{\omega}_{k} is a frequency vector and b_{k} is a phase offset. Under this representation, a sample f from the GP can then be expressed as a linear model \mathbf{f}=\mathbf{\Phi}\mathbf{w} : \displaystyle\underbrace{\begin{pmatrix}f(\mathbf{x}_{1})\\ f(\mathbf{x}_{2})\\ \vdots\\ f(\mathbf{x}_{N})\end{pmatrix}\vphantom{\begin{pmatrix}\phi(\mathbf{x}_{1})^{\mathsf{T}}\\ \phi(\mathbf{x}_{2})^{\mathsf{T}}\\ \vdots\\ \phi(\mathbf{x}_{N})^{\mathsf{T}}\end{pmatrix}}}_{\textstyle\mathbf{f}}\;=\;\underbrace{\begin{pmatrix}\phi(\mathbf{x}_{1})^{\mathsf{T}}\\ \phi(\mathbf{x}_{2})^{\mathsf{T}}\\ \vdots\\ \phi(\mathbf{x}_{N})^{\mathsf{T}}\end{pmatrix}}_{\textstyle\bm{\Phi}}\;\underbrace{\begin{pmatrix}w_{1}\\ \vdots\\