Temporal Graph MLP Mixer for Spatio-Temporal Forecasting

Muhammad Bilal, Luis Carretero López · 2025

Read Temporal Graph MLP Mixer for Spatio-Temporal Forecasting by Muhammad Bilal, Luis Carretero López (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-10214.txt.

Introduction

Muhammad Bilal and Luis Carretero López design a forecasting model for data that vary across both places and time, in a 2025 paper. Traffic flow, air quality, and solar output are recorded by sensor networks whose readings are often missing. Their model, the Temporal Graph MLP-Mixer, treats sensors as graph nodes and processes local patches of the network before mixing information across time, space, and features.

Tests on air quality, engineering, solar, and Los Angeles traffic datasets showed the model forecasting well even with large gaps in the data. It did not beat the best specialized models everywhere, but it learned long-range links between distant sensors especially well. The authors present it as a robust, scalable option for messy real-world sensor networks.

Catalog record

CREATORMuhammad Bilal, Luis Carretero López
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-10214

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Temporal Graph MLP Mixer for Spatio-Temporal Forecasting Muhammad Bilal, Luis Carretero López Abstract Spatiotemporal forecasting is critical in applications such as traffic prediction, climate modeling, and environmental monitoring. However, the prevalence of missing data in real-world sensor networks significantly complicates this task. In this paper, we introduce the Temporal Graph MLP-Mixer (T-GMM), a novel architecture designed to address these challenges. The model combines node-level processing with patch-level subgraph encoding to capture localized spatial dependencies while leveraging a three-dimensional MLP-Mixer to handle temporal, spatial, and feature-based dependencies. Experiments on the AQI, ENGRAD, PV-US and METR-LA datasets demonstrate the model’s ability to effectively forecast even in the presence of significant missing data. While not surpassing state-of-the-art models in all scenarios, the T-GMM exhibits strong learning capabilities, particularly in capturing long-range dependencies. These results highlight its potential for robust, scalable spatiotemporal forecasting. 1 1 footnotetext: Department of Computer Science, ETH Zurich, Zurich, Switzerland 1 Introduction Most timeseries data is collected through sensor networks (SN) over time across different locations. Analyzing and forecasting this data is crucial in a variety of applications such as traffic flow prediction, climate modeling, and environmental monitoring as it helps with decision-making and resource allocation ( Ghaderi et al. 2017 ) . This is a prominent problem in deep learning called spatio-temporal forecasting ( Ghaderi et al. 2017 ) . A key challenge in spatiotemporal forecasting, particularly in real-world SNs, is the prevalence of missing data ( Marisca et al. 2024 ; Cini et al. 2021 ) . Sensor failures, communication breakdowns, or other operational issues frequently disrupt data collection, resulting in incomplete or irregular sequences ( Marisca et al. 2024 ; Cini et al. 2021 ) . Most existing spatiotemporal models assume the availability of complete and regularly sampled input data, limiting their applicability to these real-world scenarios ( Marisca et al. 2024 ) . Missing data disrupts the spatial and temporal dependencies that these models rely on, leading to a degradation in predictive performance ( Cini et al. 2023a ; Cini et al. 2023b ) . When missing data occurs randomly and sporadically, the localized processing capabilities of the models can act as an effective regularization mechanism. Observations close in time and space can provide sufficient information to impute missing values and maintain forecasting accuracy ( Cini et al. 2021 ) . However, significant challenges arise when missing data occurs in large, contiguous blocks, such as when sensor failures persist over extended periods or affect entire portions of a network. In these cases, the underlying spatiotemporal dynamics can only be captured by reaching valid observations that are spatially or temporally distant ( Marisca et al. 2022 ) . This requires models to expand their receptive fields significantly, incorporating distant yet relevant information. Expanding a model’s receptive field introduces its own set of trade-offs. Deeper layers and more extensive processing can attenuate faster temporal dynamics, making it difficult for models to capture localized patterns in data. ( Rusch et al. 2023 ) . 1.1 The State of the Field Existing approaches to spatiotemporal forecasting can be broadly categorized into multivariate forecasting methods and graph-based models ( Longa et al. 2023 ) . Multivariate forecasting approaches primarily focus on time series patterns, while Spatio-Temporal Graph Neural Networks (STGNN) model spatial and temporal dependencies explicitly. While STGNNs have gained significant attention, their robustness and effectiveness in handling scenarios with missing data—especially in complex cases involving substantial missing data—remains relatively underexplored ( Marisca et al. 2024 ) . Empirically, GNN-based models have consistently demonstrated superior performance in spatiotemporal forecasting tasks, earning them widespread popularity in the research community ( Longa et al. 2023 ) . However, this trend raises a critical question: why do GNNs perform better, especially when multivariate models sometimes show higher accuracy on similar datasets in specific studies ( He et al. 2024 ; Luo et al. 2024 ; Wang et al. 2024 ; Cai et al. 2020 ; Liu et al. 2023 ) ? Notably, existing studies often fail to investigate or explain the reasons behind this phenomenon, leaving a significant gap in understanding the relative strengths and limitations of these techniques. 1.2 Contributions In this paper, we address these open questions and make the following key contributions: 1. Understanding Performance Disparities: We provide a comprehensive analysis of the scenarios under which GNN-based models outperform multivariate forecasting models. By examining the interplay between spatial and temporal inductive bias, we identify the conditions where each technique excels, offering a clearer picture of their comparative advantages. 2. Proposing a Novel Temporal Graph-Mixer Model: We introduce Temporal Graph-Mixer (TGM), a novel architecture specifically designed for temporal graph data. This model builds on the Graph ViT/MLP-Mixer ( He et al. 2023 ) by integrating a temporal channel into the model, allowing it to learn temporal dynamics such as trends, periodicities, and localized patterns. 3. Improved Robustness in Missing Data Scenarios: One of the key strengths of the Graph-Mixer model is its ability to model long range dependencies. By being able to model long range dependencies, TGM is able to utilize a larger receptive field to accurately forecast even with large contiguous missing data. 2 Models and Methods We adopted a first principles approach to formulate a hypothesis to address the problem of missing large contiguous data in the input for spatiotemporal forecasting. This process began with an extensive review of existing literature to understand the efficacy and limitations of various methods. The literature review revealed two predominant approaches: low-inductive-bias models such as transformers and recurrent neural networks (RNNs) and spatiotemporal graph networks ( Longa et al. 2023 ) . The latter are popular because they explicitly encode the spatial and temporal dependencies. Given the nature of our problem, we sought to determine the conditions under which each approach performs optimally and why. Our hypothesis was that for static graphs, where nodes and edges remain unchanged over time, models with low inductive bias could perform well. This was based on recent findings emphasizing the role of inductive bias in model generalization, particularly when sufficient data and computational resources are available ( Bachmann et al. 2024 ) . We posited that low-spatial-bias models could generalize effectively in the presence of large missing data and, if successful, could guide the design of a low-inductive-bias architecture tailored to spatiotemporal forecasting under such conditions. 2.1 Comperative Study To test our hypothesis, we conducted experiments using the METR-LA dataset, the largest benchmark dataset for static temporal graphs ( Li et al. 2017 ) . The dataset consists of traffic flow data collected by 207 sensors in the Los Angeles County highway system, aggregated in five-minute intervals, spanning March to June 2012. We split the data into training (70%), validation (10%), and test (20%) sets, evaluating model performance using the Mean Absolute Error (MAE) metric on 12-step-ahead predictions. Our proposed low-spatial-inductive-bias model consisted of a five-layered Long Short-Term Memory (LSTM) network, followed by two fully connected layers. The model was trained for 500 epochs with batch normalization, dropout, and regularization strategies to prevent over fitting. Dropout rates were incrementally reduced from 0.8 to 0.4 across successive layers. For baseline comparisons, we implemented three top performing models from the literature: the Diffusion Convolutional Recurrent Neural Network (DCRNN), which integrates

Read the complete text