Attention-Guided Deep Generative Temporal Subspace Clustering for Multivariate Spatiotemporal Data Conference: ACM SIGSPATIAL Conference; November 3 - 6; Minneapolis, MN, USA

Francis Ndikum Nji, Vandana Janeja, Jianwu Wang · 2025

Read Attention-Guided Deep Generative Temporal Subspace Clustering for Multivariate Spatiotemporal Data Conference: ACM SIGSPATIAL Conference; November 3 - 6; Minneapolis, MN, USA by Francis Ndikum Nji, Vandana Janeja, Jianwu Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18004.txt.

Introduction

Francis Ndikum Nji, Vandana Janeja, and Jianwu Wang group complex data that vary across both space and time. Uses range from snowmelt and sea ice tracking to crop health and disease spread. Older grouping tools miss long-range links, local shape, and the errors made during grouping itself, the authors argue.

Their A-DATSC model adds attention, an adversarial setup, and a deeper encoder to the standard subspace approach. Samples are projected into a latent space where hidden groups form simpler patterns. For analysts facing four-dimensional records, the paper offers a design built for the tangled time behavior that simpler methods flatten away.

Catalog record

CREATORFrancis Ndikum Nji, Vandana Janeja, Jianwu Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-18004

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Attention-Guided Deep Generative Temporal Subspace Clustering for Multivariate Spatiotemporal Data Conference: ACM SIGSPATIAL Conference; November 3 - 6; Minneapolis, MN, USA Francis Ndikum Nji, Vandana Janeja, Jianwu Wang Abstract . Deep subspace clustering models provide solutions often needed in applications such as snow melt detection, sea ice tracking, crop health monitoring, tracking infectious disease spread, network load prediction, location-based advertising and land-use planning, where multivariate spatiotemporal data exhibit complex temporal dependencies and lie on multiple non-linear manifolds whose internal structure cannot be effectively captured by traditional clustering methods. These models learn non-linear mappings by projecting data unto a latent space in which data lie in linear subspaces and exploit the self expressiveness property. While these approaches have shown impressive performance, they have shortcomings. First, they employ "shallow" autoencoders that completely rely on the self expressiveness of latent features and disregard potential clustering errors. Second, they focus solely on global features while overlooking local features in subspace self-expressiveness learning. Third, they do not capture long-range dependencies or positional information, both of which are crucial for effective spatial and temporal feature extraction and often lead to sub-optimal clustering outcomes. Fourth, their application to 4D multivariate spatiotemporal data remains underexplored. To address these limitations, we propose a novel Attention-Guided Deep Adversarial Temporal Subspace Clustering (A-DATSC) model for multivariate spatiotemporal data. A-DATSC incorporates a deep subspace clustering generator and a quality-verifying discriminator that work in tandem. Inspired by the U-Net architecture, the generator preserves the spatial and time-wise structural integrity, reduces the number of trainable parameters and improves generalization through the use of stacked TimeDistributed convLSTM2D layers. It further introduces a graph attention transformer-based self expressive network which captures local spatial relationships, global dependencies and both short and long range correlations crucial for understanding how distant regions and time periods influence each other. When evaluated on three real-world multivariate spatiotemporal datasets, A-DATSC outperforms deep subspace clustering models with significant margins. 1. Introduction Recent years have seen increased availability in spatiotemporal data from common sources such as government surveys, mobile and wearable devices, launched satellite and weather sensors. These data sources acquire, compress, store, transmit, and process massive amounts of complex high-dimensional multivariate spatiotemporal data. Although this data is high dimensional ( yang2021investigating ) , their intrinsic dimension (i.e number of variables needed to describe a data distribution) is often much smaller than the dimension of the ambient space ( pope2021intrinsic ) . For example; in image data processing, the number of pixels in an image can be rather large, yet most image processing models use only a few parameters to describe, for instance the appearance, geometry, and dynamics of a scene. This has motivated the development of techniques like autoencoders and regularization methods ( gonzalez2018deep ; zhu2018ldmnet ) for representing high-dimensional data in a lower dimension. Another technique for representing high-dimensional dataset in a lower dimension is the Principal Component Analysis (PCA) ( kurita2019principal ) . It assumes that the data is drawn from a single low-dimensional subspace within a high-dimensional space. However, in practice, data points may come from multiple subspaces, and the membership of these points to their respective subspaces is often unknown. This creates a complex sample distribution problem, particularly in multidimensional spatiotemporal data. Therefore, it is necessary to group data points into clusters, where each cluster contains points from the same subspace. This approach assumes that data lies in different subspaces ( chen2020stochastic ) . A category of classical subspace clustering methods have been proposed ( chen2020stochastic ; liu2012robust ; xu2021selective ; ding2024survey ) . A few researchers ( yang2019deep ; dang2020multi ; li2021lrsc ; ji2017deep ) showed that joint subspace clustering and deep learning have promising performance on benchmark datasets. However, these approaches can hardly be extended to large-scale datasets because they need to learn a self-expressive matrix leading to quadratic time and space complexities. Consequently, some latest works ( zhang2018scalable ; fan2021large ; zhang2021learning ) dedicate to improving the efficiency of subspace clustering but to the best of our knowledge, there is currently no literature on applying joint subspace clustering and deep learning on multidimensional multivariate spatiotemporal data. In this paper, we advance research in this area by designing an end-to-end deep temporal subspace clustering model tailored for complex multidimensional multivariate spatiotemporal data. It consists of a deep subspace clustering generator and a quality-verifying discriminator that learns to supervise the generator by evaluating clustering quality in an unsupervised manner. Drawing inspiration from the recent success of the U-net architecture ( ronneberger2015u ) in representation learning, our generator incorporates a deep autoencoder composed of stacked convLSTM2D layers and graph attention transformer-based self expressive network and a clustering layer organized in series to capture compact and informative representations of spatial, temporal and salient features of the data. These components capture both local and global patterns, long-range dependencies and positional awareness essential for learning meaningful spatial and temporal patterns and relationships. The clustering layer uses the inherent logic of the Student’s t-distribution and iteratively improves clustering result. At the same time, the decoder module adjusts its weights to reduce the disparity between the input and reconstructed data while learning to reconstruct the multidimensional spatiotemporal input data from lower-dimensional latent features. To sum up, this paper makes the following contributions: 1) We propose a novel Attention-guided deep adversarial temporal subspace clustering (A-DATSC) model for 4D multivariate spatiotemporal data. The generator preserves the spatial and time-wise structural integrity, reduces the number of trainable parameters and improves generalization. 2) We design a unified graph attention transformer-based self expressive network which captures local spatial relationships, global dependencies and both short and long range correlations. 3) We design an energy-based, time-varying mini-batch discriminator that leverages temporal subspace modeling to better distinguish between real and fake feature sequences. The remainder of the paper is structured as follows. Section 2 summarizes the background while 3 discusses related works. Section 4 describes the problem in detail while Section 5 presents our proposed solution. Section 6 details the experiment and presents our results while Section 7 concludes our research. 2. Background and Motivation The exponential growth of multivariate spatiotemporal data across disciplines has created both unprecedented opportunities and formidable analytical challenges. These data are high-dimensional, noisy, heterogeneous, and often exhibit strong nonlinear dependencies across space, time, and variables. Conventional clustering methods, which treat samples as independent and identically distributed (iid), fail to capture these intricate dependencies and often miss the latent low-dimensional subspace structure that governs real-world dynamics. This motivates the pursuit of deep subspace clustering (DSC) methods that can uncover meaningful representations of complex spatiotemporal systems, disentangle overlapping patterns, and group data into coherent clusters that are physically interpretable and temporally consistent. For multivariate spatiotemporal data, these subspaces may represent distinct climate regimes, transportation flow patterns, disease outbreak waves, or other structured phenomena. The development of deep neural architectures particularly those leveraging convolutional, recurrent, and attention-based modules enable learning hierarchical feature representations that preserve spatial locality, model temporal continuity, and capture complex cross-variable correlations. Integrating subspace clustering with representation learning is therefore a powerful paradigm: it simultaneously discovers a latent feature space and a segmentation of the data into meaningful subspaces, improving robustness to noise and scalability to large datasets. The motivation for this research is also deeply societal.

Read the complete text