Reversible Residual Normalization Alleviates Spatio-Temporal Distribution Shift
Zhaobo HU, Vincent Gauthier, Mehdi Naima · 2026
Read Reversible Residual Normalization Alleviates Spatio-Temporal Distribution Shift by Zhaobo HU, Vincent Gauthier, Mehdi Naima (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15838.txt.
Introduction
Hu, Gauthier, and Naima tackle drift in spatiotemporal forecasting. Both single series and links across a network change over time, so models trained on old patterns fail. Their framework, Reversible Residual Normalization, uses place-aware transforms that can run backward inside its blocks. It learns in a tidy space, then undoes the change at the end. Center scaling plus graph nets with capped spectra make the method work with any model, aiming for steady results on changing networks.
Catalog record
| CREATOR | Zhaobo HU, Vincent Gauthier, Mehdi Naima |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34445 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15838
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reversible Residual Normalization Alleviates Spatio-Temporal Distribution Shift Zhaobo HU, Vincent Gauthier, Mehdi Naima Abstract Distribution shift severely degrades the performance of deep forecasting models. While this issue is well-studied for individual time series, it remains a significant challenge in the spatio-temporal domain. Effective solutions like instance normalization and its variants can mitigate temporal shifts by standardizing statistics. However, distribution shift on a graph is far more complex, involving not only the drift of individual node series but also heterogeneity across the spatial network where different nodes exhibit distinct statistical properties. To tackle this problem, we propose Reversible Residual Normalization (RRN), a novel framework that performs spatially-aware invertible transformations to address distribution shift in both spatial and temporal dimensions. Our approach integrates graph convolutional operations within invertible residual blocks, enabling adaptive normalization that respects the underlying graph structure while maintaining reversibility. By combining Center Normalization with spectral-constrained graph neural networks, our method captures and normalizes complex Spatio-Temporal relationships in a data-driven manner. The bidirectional nature of our framework allows models to learn in a normalized latent space and recover original distributional properties through inverse transformation, offering a robust and model-agnostic solution for forecasting on dynamic spatio-temporal systems. Keywords: Spatio-Temporal Forecasting, Distribution Shift, Invertible Neural Networks, Graph Neural Networks. 1 Introduction Spatio-temporal forecasting is fundamental to managing dynamic systems, ranging from urban traffic [ 9 , 27 , 7 ] and environmental monitoring [ 21 , 11 , 14 ] to public health surveillance. Effective models must capture both the spatial dependencies among interconnected locations and the temporal dynamics governing system evolution. Despite deep learning advancements, a critical challenge persists: distribution shift. This phenomenon occurs when statistical properties change, violating the assumption that training and testing data follow identical distributions. In spatio-temporal systems, this shift is dual-natured. Temporally, observations drift due to evolving dynamics; spatially, locations exhibit statistical heterogeneity despite sharing underlying correlations. This Spatio-Temporal distribution shift [ 12 ] severely degrades generalization when networks trained on historical data confront evolved patterns during inference. Current mitigation strategies primarily focus on time series, using methods like instance normalization [ 28 ] to standardize sequences. However, these techniques do not naturally extend to graph-structured data. Applying normalization independently to each node neglects spatial heterogeneity and may disrupt topological relationships. Furthermore, most spatio-temporal architectures focus on correlation modeling while implicitly assuming stationarity, creating a gap between model assumptions and the non-stationary nature of real-world data. To bridge this gap, we propose Reversible Residual Normalization (RRN), a novel framework that extends normalization to the spatio-temporal domain via invertible transformations. RRN integrates three key components: Center Normalization [ 26 ] , a Lipschitz-continuous alternative [ 30 , 1 , 25 ] that removes shifts while maintaining invertibility; graph convolutional operations within invertible residual blocks [ 25 ] for spatially-aware normalization; and spectral normalization [ 24 ] to constrain the transformation. This bidirectional framework maps data into a stationary latent space for easier learning and uses the inverse transformation to restore location-specific characteristics. Extensive experiments on real-world benchmarks demonstrate that our model-agnostic approach consistently improves performance, validating the necessity of addressing distribution shift in spatio-temporal systems. 2 Related Works 2.1 Distribution Shift in Time Series and Spatio-Temporal Data Distribution shift arises from non-stationarity, causing significant discrepancies between training and testing distributions [ 15 , 23 , 5 , 6 ] . To mitigate this, instance normalization and its variants, such as reversible and adaptive normalization effectively standardize temporal sequences by removing non-stationary components, yet they typically process nodes independently. Other advanced approaches explicitly model distribution shifts by learning intra-space and inter-space variations, but these remain confined to the temporal dimension. Consequently, the spatial dimension of distribution shift is largely neglected, applying temporal normalization strategies directly to spatio-temporal data ignores the underlying graph structure, failing to account for spatial heterogeneity where distinct nodes exhibit unique statistical properties. 2.2 Invertible Networks Invertible neural networks learn bijective transformations that allow for exact computation of both forward and inverse mappings, facilitating tasks such as generative modeling and density estimation [ 4 , 31 , 22 , 1 , 16 ] . While normalizing flows typically rely on coupling layers to construct invertible mappings [ 4 , 16 , 3 , 17 ] , Invertible Residual Networks [ 1 , 30 , 25 ] extend this capability to standard residual architectures. Rather than relying on partitioning dimensions, these models utilize residual blocks [ 10 ] that guarantee invertibility provided the residual function satisfies specific Lipschitz continuity conditions (requiring a constant strictly less than one). This constraint is often enforced via spectral normalization to enable exact inverse computation through fixed-point iteration. Despite their success in time series, the potential of invertible architectures to simultaneously address spatial and temporal distribution shift while preserving graph topology remains underexplored. 3 Problem Formulations 3.1 Spatio-Temporal Forecasting Let a spatio-temporal system be represented by a graph \mathcal{G}=(V,E) , where V is a set of N spatial locations (e.g., sensors, regions) with |V|=N , and E is a set of edges representing the spatial relationships between these locations. At each discrete time step t , the system exhibits a feature matrix X^{(t)}\in\mathbb{R}^{N\times D} , where D is the number of features recorded at each location. This feature matrix is also referred to as a graph signal. The task of spatio-temporal forecasting aims to predict future system states based on historical observations. Given a lookback window of length L , the historical data can be denoted as a tensor \mathcal{X}_{t-L+1:t}=(X^{(t-L+1)},\dots,X^{(t)})\in\mathbb{R}^{L\times N\times D} . The objective is to forecast the subsequent data over a horizon window of length H , denoted as \mathcal{Y}_{t+1:t+H}=(X^{(t+1)},\dots,X^{(t+H)})\in\mathbb{R}^{H\times N\times D} . Formally, the goal is to learn a function f_{\theta} parameterized by \theta that maps the historical observations and the graph structure to the future sequence: \hat{\mathcal{Y}}_{t+1:t+H}=f_{\theta}(\mathcal{X}_{t-L+1:t};\mathcal{G}) where \hat{\mathcal{Y}}_{t+1:t+H} represents the predicted future sequence. 3.2 Distribution Shift in Spatio-Temporal Series A primary challenge in real-world spatio-temporal forecasting is the non-stationarity of the data, which manifests as a distribution shift. Many deep learning models implicitly assume that the data-generating process is consistent between the training and test periods, an assumption that is frequently violated in practice. This discrepancy occurs as the joint distribution of the observed spatio-temporal data changes over time, hindering a model’s ability to generalize from past observations to future predictions. Formally, for any two distinct time steps t_{u} and t_{v} , the underlying conditional probability that governs the system’s evolution is not constant. This distribution shift can be expressed as: P({\mathcal{Y}}_{t_{u}+1:t_{u}+H}|{\mathcal{X}}_{t_{u}-L+1:t_{u}};\mathcal{G})\neq P({\mathcal{Y}}_{t_{v}+1:t_{v}+H}|{\mathcal{X}}_{t_{v}-L+1:t_{v}};\mathcal{G}) Therefore, our objective is not only to predict future spatio-temporal states but also to explicitly model and mitigate these distribution shifts to enhance forecasting accuracy and generalization. To address this, we propose extending the principles of normalizing flows, which have shown promise in handling temporal distribution shifts, to the more complex spatio-temporal domain. 4 Methodology 4.1 Invertible Residual Structure Figure 1 : Overview of the Reversible Residual Normalization framework for spatio-temporal forecasting. The framework operates through four stages: (1) Input Space contains raw spatio-temporal data with distribution shifts across nodes and time; (2) Transformed Space where the invertible transformation T_{\phi} normalizes the data by removing Spatio-Temporal distribution shifts while preserving correlational structures; (3) Forecasting Space where any forecasting model f_{\theta} makes predictions in