On the Role of Reversible Instance Normalization

Gaspard Berthelier, Tahar Nabil, Etienne Le Naour, Richard Niamke, Samir Perlaza, Giovanni Neglia · 2026

Read On the Role of Reversible Instance Normalization by Gaspard Berthelier, Tahar Nabil, Etienne Le Naour, Richard Niamke, Samir Perlaza, Giovanni Neglia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11869.txt.

Introduction

Berthelier and co-authors revisit Reversible Instance Normalization, a widely used step in time series forecasting. Normalization matters in deep learning, yet its role in forecasting stays poorly understood. The authors name three hard cases: shifts in input over time, shifts across space, and shifts in the output given the input. Through ablation studies they show several RevIN parts are redundant, and some even hurt. From those findings they draw new ideas for making RevIN sturdier and better at generalizing.

Catalog record

CREATORGaspard Berthelier, Tahar Nabil, Etienne Le Naour, Richard Niamke, Samir Perlaza, Giovanni Neglia
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54555
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.11869

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On the Role of Reversible Instance Normalization Gaspard Berthelier, Tahar Nabil, Etienne Le Naour, Richard Niamke, Samir Perlaza, Giovanni Neglia Abstract Data normalization is a crucial component of deep learning models, yet its role in time series forecasting remains insufficiently understood. In this paper, we identify three central challenges for normalization in time series forecasting: temporal input distribution shift, spatial input distribution shift, and conditional output distribution shift. In this context, we revisit the widely used Reversible Instance Normalization (RevIN), by showing through ablation studies that several of its components are redundant or even detrimental. Based on these observations, we draw new perspectives to improve RevIN’s robustness and generalization. \keepXColumns 1 Introduction Data normalization is a fundamental preprocessing step, long recognized as essential for the stable and efficient training of neural networks. Its significance was established in early studies on network optimization ( LeCun et al. 2012 ) . Normalization is particularly critical in time series forecasting, a domain in which deep learning models have achieved state-of-the-art performance ( Nie et al. 2023 ) . Intrinsic properties of time series, such as non-stationarity, trends, and seasonality, introduce challenges that standard normalization methods fail to address ( Ogasawara et al. 2010 ) . Three major aspects must be considered when designing a normalization strategy for time series forecasting (formalized in Appendix A ): 1. Temporal distribution shift. When training a neural forecaster over a training period, the input data distribution in a future test period may differ substantially. For instance, electricity consumption or road traffic often increase over time (see Fig. 1(a) ). 2. Spatial distribution shift. Neural forecasters are typically trained on multiple time series and must generalize to unseen time series at inference. Even when series describe similar phenomena (e.g., solar power generation at different locations), their overall distributions may differ, for example, due to scale or level shifts (see Fig. 1(b) ). 3. Conditional distribution shift. A neural forecaster maps past look-back windows to future horizon windows. The conditional distribution of the horizon given the look-back may vary both in space and time. This shift remains particularly challenging to handle (see Fig. 1(c) ). (a) Temporal shift. (b) Spatial shift. (c) Conditional shift. Figure 1: Illustration of the three distribution shifts: (a) temporal shift between training and test periods (rolling average of a Traffic sensor), (b) spatial shift between users (two Solar sensors), (c) conditional shift (different horizons for similar look-back windows, from one Electricity user). In deep learning, a common approach is to apply standard normalization to all data points, using a single global mean \mu and standard deviation \sigma computed empirically over the training data. Each data point is processed by subtracting the mean and dividing by the standard deviation. However, this global normalization fails to address the challenges of the three distribution shifts described above ( Ogasawara et al. 2010 ) . In time series forecasting, a more sophisticated and widely adopted normalization strategy is Reversible Instance Normalization (RevIN) ( Kim et al. 2022 ) . The RevIN method has been adopted in numerous recent forecasting studies ( Nie et al. 2023 ; Xu et al. ; Auer et al. 2024 ; Huang et al. 2025 ; Kim et al. 2022 ) . Its authors claim that RevIN mitigates the distribution shift problem in time series. In this work, we challenge this claim. Our contributions are summarized as follows: • We review normalization techniques for neural forecasters and identify the key challenges they face in time series applications. • We conduct extensive ablation studies on RevIN using standard forecasting benchmarks and show which components are truly necessary. • We discuss the effects of instance normalization on data distributions and highlight the limitations of RevIN. 2 Setting Time series forecasting consists in finding an optimal parameterized model f_{\theta} of the form: f_{\theta}:\;\begin{array}[]{@{}c@{~}c@{~}l@{}}\mathbb{R}^{L}&\longrightarrow&\mathbb{R}^{H},\\ x&\longmapsto&\hat{y}\approx y,\end{array}\qquad\begin{aligned} L&:\ \text{look-back window size},\\ H&:\ \text{horizon size}.\end{aligned} (1) where \theta corresponds to the parameters of the model (e.g. a neural network), fitted on a training dataset \mathcal{D} of N pairs of windows (x,y) , with x a past look-back window and y a horizon to forecast. Naturally, the goal is to have predictions \hat{y} as close as possible to the ground-truth horizons y . We say a model generalizes well when this is the case for unseen windows. Eq. 1 corresponds more precisely to the auto-regressive and univariate setting. A more general setting would include multivariate inputs and outputs, exogenous covariates, and probabilistic predictions. Although we anticipate similar conclusions, we do not consider it here. 3 Related Work In this section, we describe the common normalization techniques for neural networks and review their limitations. More detailed mathematical formulations are provided in Appendix B . Normalization techniques in deep learning. Early studies on neural networks optimization showed that having similarly distributed input features centered around the origin was beneficial for convergence and performance ( LeCun et al. 2012 ) . A common normalization technique is standard normalization , which centers and reduces data distributions using the training data’s statistics: \tilde{{x}}=\frac{{x}-{\mu}}{{\sigma}},\quad\tilde{{y}}=\frac{{y}-{\mu}}{{\sigma}} (2) where the statistics are computed empirically as: \mu=\frac{1}{N}\sum_{x\in\mathcal{D}}\frac{1}{L}\sum_{i=1}^{L}x_{i}\;\in\mathbb{R}\quad\textrm{and}\quad{{\sigma}}^{2}=\frac{1}{NL-1}\sum_{x\in\mathcal{D}}\sum_{i=1}^{L}(x_{i}-\mu)^{2}\;>0. (3) Later on, various normalization schemes were introduced between layers of deep neural networks to accelerate convergence, e.g. batch normalization ( Ioffe & Szegedy 2015 ) and layer normalization ( Ba et al. 2016 ) . Using the inputs’ own statistics was introduced with instance normalization ( Ulyanov et al. 2016 ) . A thorough analysis of these methods can be found in ( Lubana et al. 2021 ) . Normalization for time series forecasting. Normalization has long been used not only to facilitate optimization but also to mitigate time-series non-stationarity, as recognized early on, e.g. by Ogasawara et al. 2010 . Indeed, standard normalization cannot handle distribution shifts, since (\mu,\sigma) no longer correspond to the correct statistics at inference time. Nevertheless, until recently, standard normalization—sometimes called standardization or z-normalization —has remained the default strategy ( Zhou et al. 2021 ; Passalis et al. 2019 ) . Inspired by the normalization techniques from other domains, Kim et al. 2022 proposed Reversible Instance Normalization for time series forecasting. With RevIN, the prediction pipeline associated with a model f_{\theta}:x\mapsto\hat{y} becomes: \tilde{{x}}=\alpha\frac{{x}-{\mu}_{x}}{{\sigma_{x}}}+\beta,\quad\hat{y}=\sigma_{x}\left(\frac{f_{\theta}(\tilde{x})-\beta}{\alpha}\right)+\mu_{x}\quad(\alpha,\beta\in\mathbb{R}), (4) where the instance x ’s statistics are computed as: \mu_{x}=\frac{1}{L}\sum_{i=1}^{L}x_{i}\;\in\mathbb{R},\quad\sigma_{x}^{2}=\frac{1}{L-1}\sum_{i=1}^{L}(x_{i}-\mu_{x})^{2}\;>0. (5) The pipeline can be decomposed as applying the following steps: 1. apply instance normalization, i.e. standardize using the input statistics \mu_{x} and \sigma_{x} ; 2. apply a learnable affine transformation with global parameters \alpha and \beta ; 3. pass the transformed input through the forecaster f_{\theta} ; 4. apply the inverse affine transformation; and 5. denormalize using the inverse of the normalization. A diagram of the whole process is presented in Fig. 2 . Figure 2: Illustration of the RevIN process on three synthetic examples. By normalizing input windows using their own statistics, RevIN addresses temporal distribution shifts and, consequently, substantially improves performance ( Kim et al. 2022 ) . One key ingredient is the denormalization step that enables the model to make predictions at the correct offset and scale. Extensions to RevIN. A few works have identified limitations of RevIN and proposed extensions or alternatives. In DAIN ( Passalis et al. 2019 ) , per instance normalization is applied using learned scale

Read the complete text