Forecasting extreme temperature events by Hankel-augmented contrastive learning Authors: Chengyang Qin, Yueyang Ding, Peng Tao, Luonan Chen Published: 2026 ABSTRACT ABSTRACTAccurate forecasting of extreme temperature events is critically important for climate adaptation and disaster preparedness, yet it remains a fundamental challenge due to the abrupt and non-stationary nature of such phenomena. Current time-series models often fail to capture rapid transitions and anomalous patterns that deviate significantly from historical behaviors. Here, we propose Hankelformer, a novel deep learning architecture that integrates structured Hankel-based augmentation with contrastive learning to significantly improve the forecasting of extreme weather events. The model employs a structured augmentation module that constructs Hankel matrices to capture local spatiotemporal dynamics without disrupting temporal coherence, generating delay-embedding-inspired views of the original input data in terms of states. These views are processed together with the original input in a dual-stream Transformer encoder, followed by a contrastive learning between them that forms spatiotemporal representations, thus significantly enhancing feature invariance and robustness against distribution shifts. We evaluated Hankelformer on the curated extreme weather datasets: the TexasFreeze, the Pacific Northwest Heatwave and the Antarctic Heat, in addition to six standard benchmarks in energy and transportation. Hankelformer consistently achieves state-of-the-art performance, demonstrating remarkable accuracy in predicting extreme temperature collapses and spikes, with up to 34% improvement in mean squared error over leading baselines. The framework offers a promising tool for reliable extreme temperature forecasting and underscores the value of constructing topologically equivalent sequences to the original sequence for spatiotemporal representation learning in handling real-world non-stationary time series. INTRODUCTION Multivariate time-series (MTS) forecasting underpins critical decisions across domains from energy infrastructure management to climate adaptation, yet failures in prediction can have severe consequences. Accurate forecasting is essential for energy systems requiring precise demand prediction for grid stability and resource allocation [1–3], transportation networks depending on traffic flow forecasting for efficient routing [4,5] and climate monitoring systems relying on robust predictions for disaster preparedness [6–8]. Applications extend to financial risk management, healthcare monitoring and industrial process control [9,10]. Recent extreme events underscore the critical importance of robust forecasting capabilities. The 2021 Texas winter storm exemplifies how inadequate prediction can lead to cascading system failures [11–13]. Economic analyses reveal that electricity demand mispredictions cost the global energy sector billions annually [14,15], while traffic flow prediction errors contribute significantly to congestion-related losses [16,17]. As climate change intensifies extreme event frequency [18,19], developing robust MTS forecasting models capable of handling competitive conditions has become imperative. The evolution of MTS forecasting has progressed through several paradigmatic shifts. Classical statistical methods such as autoregressive integrated moving average (ARIMA) and vector autoregression (VAR) models [20–22] provided interpretable results but struggled with complex non-linear patterns. Machine learning approaches including Support Vector Machines and Random Forests [23–25] improved non-linear modeling but remained limited in capturing long-range dependencies. The advancement of deep learning introduced recurrent neural networks (RNNs), and long short-term memory (LSTM) networks [26–28], convolutional neural networks (CNNs) for local pattern capture [29,30], and attention mechanisms enabling direct long-range dependency modeling [31,32]. Recent Transformer architectures [32] have shown exceptional promise, spawning innovations including Informer [33], Autoformer [34], TimesNet [35], PatchTST [36], iTransformer [37], FEDformer [38], ETSformer [39], Pyraformer [40], LogTrans [41], Reformer [42] and FiLM [43]. Despite these advances, current approaches face critical limitations: insufficient local pattern extraction during extreme events, limited robustness to distribution shifts [44–46] and suboptimal augmentation strategies that destroy temporal coherence [47,48]. Self-supervised learning has emerged as an important approach with theoretical foundations in information theory [49,50]. Contrastive learning maximizes agreement between augmented views while minimizing agreement with other samples [51–53], significantly advancing computer vision (SimCLR, MoCo) and NLP (BERT variants) [53–55]. Applications to time-series analysis show promise [56,57], with works like TS2Vec [58], CoST [59], TF-C [60], SimMTM [61] and TNC [62] demonstrating improved representations. However, these typically employ generic augmentation strategies potentially destroying temporal coherence and treat self-supervision as separate pretraining, creating representation gaps. Here, we introduce Hankelformer, addressing these challenges through mathematically principled structured augmentation and end-to-end contrastive learning. Drawing inspiration from delay embedding theorem and Hankel matrix decomposition for spatiotemporal patterns [63–67], we construct topologically equivalent sequences/views to the original sequence/input (see Note S1 for the proof), thus contrastive learning can be efficiently applied between them to form spatiotemporal representations for accurate prediction. Although delay embedding/Hankel matrices and contrastive learning are both well-established techniques, to our knowledge, utilizing Hankel matrices for topologically equivalent augmented representation of time series in conjunction with contrastive learning remains unexplored, constituting one of the core innovations of Hankelformer (see Note S2 for a detailed comparison of Hankelformer with existing contrastive learning methods and delayed embedding methods). Specifically, our approach makes two key innovations: (i) a Hankel-Augmentation (Hankel-AUG) module constructing spatiotemporal patterns/sequences associated with the original sequence through structured sliding windows while maintaining temporal coherence; (ii) a dual-stream contrastive learning framework processing original and augmented sequences with shared-weight Transformer encoders, mitigating potential pretraining-finetuning mismatch. In our context, this mismatch refers to the suboptimality that can arise when representations are first optimized for a generic self-supervised objective and may require substantial adaptation during fine-tuning for the specific downstream forecasting task. The constructed dynamics/sequences for contrastive learning may be topologically equivalent to the original input by the delay embedding theorem or spatiotemporal information transformation [65–69]. The overall Hankelformer architecture is illustrated in Fig. 1. We demonstrate Hankelformer’s effectiveness across nine benchmark datasets, including three custom datasets capturing extreme temperature events: the 2021 North American cold wave (TexasFreeze), the Pacific Northwest heat dome (Heatwave) and the Antarctic Heat. Hankelformer achieves state-of-the-art performance in all scenarios, with strong robustness in extreme event forecasting. The Hankelformer architecture. (a) Predicting extreme temperature events remains challenging due to abnormal dynamic changes. (b) The overall dual-stream architecture of Hankelformer. The original input sequence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}$\end{document} is processed in two parallel streams. The top stream processes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}$\end{document} directly. The bottom stream first passes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}$\end{document} through the Hankel-AUG module to generate a structurally augmented sequence or view \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{X}}}^{{\mathrm{Aug}}}$\end{document}. Both sequences are embedded and fed into a shared-weight Transformer encoder (N layers). The output representations \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{Z}}}_{{\mathrm{emb}}}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{Z}}_{{\mathrm{emb}}}^{{\mathrm{Aug}}}$\end{document} are used to compute a contrastive loss, promoting robust feature learning. The representations are then fused and passed to an output projection layer to generate the final forecast \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{\hat{Y}}}$\end{document}. (c) Detailed workflow of the Hankel-AUG module. A block-Hankel matrix is constructed from its original time series/sequence of a specified window size W, and its dynamics is topologically equivalent to the original input. The matrix is transformed by a multilayer perceptron (MLP), padded and transposed again to form the Hankel-augmented sequence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{X}}}^{{\mathrm{Aug}}}$\end{document}. (d) The intra-batch contrastive learning mechanism between the original and augmented sequences. After pooling, the representation of an augmented sample (e.g. green square) is considered a positive sample to its original counterpart (red square) and a negative sample to all other original samples in the batch. RESULTS The Hankelformer architecture Given an MTS \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}\in{\mathbb{R}}^{B \times M \times L}$\end{document} with batch size B, M variables and L historical time steps, the objective is to predict future H time steps: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{\hat{Y}}}\in{\mathbb{R}}^{B \times M \times H}$\end{document}. Each time step \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{X}}}^t = [ {x_1^t,x_{2,}^t\ldots,x_M^t} ]{\mathrm{\ }}( {t = 1,\ldots,L} )$\end{document} represents observations of M variables at time t. As illustrated in Fig. 1b, Hankelformer employs a dual-stream architecture that processes original and structurally augmented sequences through shared-weight Transformer encoders. The architecture consists of four core components: Hankel-AUG, dual-stream encoding, contrastive learning and feature fusion. Hankel-AUG module. The Hankel-AUG module (Fig. 1c) generates structurally augmented views/sequences by extracting local spatiotemporal patterns through Hankel matrix decomposition. For each sample \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{X}}}_b\in{\mathbb{R}}^{M \times L}$\end{document} within the batch B, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $b = 1,\ 2,\ldots,B$\end{document}, it is transformed into a block-Hankel matrix by multivariable delayed embedding, where W is the window size. This yields a tensor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{H}}}_b\in{\mathbb{R}}^{MW \times ( {L - W + 1} )}$\end{document} and it is transposed to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\mathbb{R}}^{( {L - W + 1} ) \times MW}$\end{document}, processed through an MLP, padded and transposed to generate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}_{\boldsymbol{b}}^{{{\mathbf{Aug}}}}\in{\mathbb{R}}^{M \times L}$\end{document}, maintaining dimensional compatibility while capturing complementary local patterns. Note that the Hankel matrix is one delay embedding form of the original time series based on delay embedding theorem (see Note S1 for the proof), and thus its dynamics is topologically equivalent to the original one and also considered as a temporal/dynamical representation of the original time series [63–69]. Dual-stream encoding. The original input \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}$\end{document} undergoes two parallel pathways. The upper stream processes multivariate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}$\end{document} directly through input projection to obtain \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{X}}}_{{\mathrm{emb}}}\in{\mathbb{R}}^{B \times M \times D}$\end{document}. The lower stream first applies the Hankel-AUG module in the delay embedding form to generate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{X}}}^{{\mathrm{Aug}}}$\end{document}, followed by input projection to obtain \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}_{{\mathrm{emb}}}^{{\mathrm{Aug}}}\in{\mathbb{R}}^{B \times M \times D}$\end{document}, where D is the model dimension. Both embedded sequences are processed through a shared-weight multilayer Transformer encoder, yielding representations \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{Z}}}_{{\mathrm{emb}}}\in{\mathbb{R}}^{B \times M \times D}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{Z}}_{{\mathrm{emb}}}^{{\mathrm{Aug}}}\in{\mathbb{R}}^{B \times M \times D}$\end{document}. Note that the two representations originate from two topologically equivalent inputs (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\boldsymbol{X}}$\end{document} and its corresponding Hankel matrix), and thus contrastive learning can be efficiently applied to construct the spatiotemporal representation of the original MTS. Contrastive learning. After average pooling of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{Z}}}_{{\mathrm{emb}}}\ and\ {\boldsymbol{Z}}_{{\mathrm{emb}}}^{{\mathrm{Aug}}}$\end{document}, we obtain \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${{\boldsymbol{Z}}}_{{\mathrm{pool}}}{\mathrm{\ }}\in{\mathbb{R}}^{B \times D}\ {\mathrm{and}}\ {\boldsymbol{Z}}_{{\mathrm{pool\ }}}^{{\mathrm{Aug\ }}}\in{\mathbb{R}}^{B \times D}$\end{document}. The contrastive learning objective maximizes agreement between original and augmented representations while minimizing agreement with other samples in the batch. Given the batch size B with samples indexed by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i,j\in\{ {1,\ 2,\ldots,B} \}$\end{document}, a sample’s representation with its augmentation (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${z}_i\in{{\boldsymbol{Z}}}_{{\mathrm{pool\ }}},z_i^{{\mathrm{Aug\ }}}\in{\boldsymbol{Z}}_{{\mathrm{pool\ }}}^{{\mathrm{Aug\ }}}$\end{document}) are defined as a positive pair. Conversely, negative pairs consist of a given \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${z}_i$\end{document} and all other augmented representations \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\{ {z_j^{{\mathrm{Aug}}}} \}}_{j\ne i}$\end{document} within the same batch. Both original and augmented representations are first L2-normalized as where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\| \cdot {\|}_2$\end{document} denotes the Euclidean norm. Finally, inspired by the work of Gao et al. [55], the contrastive loss \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${L}_{{\mathrm{cont}}}$\end{document} is defined as where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${\mathrm{sim}}( { \cdot , \cdot } )$\end{document} denotes cosine similarity and τ is the temperature parameter. Feature fusion. The fused representation combines information from both streams as the spatiotemporal representation: Final predictions are obtained through output projection: The total loss function integrates forecasting and contrastive objectives: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${L}_{{\mathrm{pred}}}$\end{document} is the mean squared error (MSE) and λ is a weight parameter of the contrastive objective. Validation of Hankelformer on a chaotic 90D-Lorenz system To validate the effectiveness of Hankelformer, we first evaluated its predictive performance on a 90-dimensional (90D) chaotic Lorenz system, whose ground-truth solution is known. Using 96 observed time steps, we forecasted the next 96 steps and compared Hankelformer against several representative baselines: WPMixer, iTransformer, TimeMixer, TiDE, PatchTST, TimesNet, CoST, SimMTM and Naïve (consistently utilizes the last value as the prediction), covering Transformer-based architectures, decomposition-based models and MLP/Mixer-style methods. Figure 2a and b compares the predictions for two example Lorenz dimensions and the predictions of Hankelformer align closest to the true dynamics. Quantitatively, Hankelformer achieves an MSE of 0.348 and an mean absolute error (MAE) of 0.380, outperforming the second-best method, TimeMixer, which records 0.405 and 0.424, respectively (Table 1). Hankelformer demonstrates superior robustness to noise on the chaotic Lorenz system. (a–f) Qualitative prediction results of two example dimensions for noise levels σ = 0, σ = 0.5 and σ = 1, respectively. As noise increases, Hankelformer’s predictions remain stable and accurate, while competitors' predictions diverge significantly. (g) Quantitative plot showing MSE as a function of noise level. The error bar represents the range of prediction error across 90 variables. Hankelformer’s error is consistently the lowest and exhibits the slowest rate of increase, highlighting its superior noise robustness. Table 1.Evaluation of Hankelformer on the 90D-Lorenz dataset. Forecasting performance (MSE/MAE) under increasing levels of additive Gaussian noise (σ from 0.0 to 1.0). All models predict 96 future steps based on 96 past steps. Best results are in bold. ‘1st Cnt’ indicates the number of times a model achieved the best score.Hankelformer (Ours)WPMixer (2025)iTransformer (2024)TimeMixer (2024)TiDE (2023)PatchTST (2023)SimMTM (2023)CoST (2022)TimesNet (2022)NaïveσMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE0 0.348 0.380 0.5390.5150.5250.5060.4050.4240.9240.7200.5410.5200.4460.4490.9880.7991.1100.8551.7071.0110.1 0.353 0.384 0.5410.5160.5250.5070.4070.4250.9250.7210.5420.5200.4470.4500.9970.8011.1100.8541.7071.0110.2 0.348 0.381 0.5400.5160.5260.5070.4060.4240.9260.7220.5440.5220.4520.4550.9890.7991.1090.8541.7071.0110.3 0.356 0.386 0.5550.5230.5270.5090.4080.4260.9280.7240.5470.5250.4610.4620.9930.7991.1090.8541.7081.0110.4 0.367 0.396 0.5400.5160.5290.5100.4050.4260.9300.7260.5510.5270.4690.4690.9930.8021.1090.8541.7071.0110.5 0.374 0.401 0.5470.5240.5330.5120.4100.4290.9320.7280.5540.5300.4800.4780.9940.8011.1090.8541.7081.0120.6 0.369 0.398 0.5580.5330.5360.5150.4120.4320.9350.7300.5590.5340.4900.4860.9910.8001.1100.8541.7101.0130.7 0.373 0.402 0.5660.5380.5360.5160.4170.4370.9360.7310.5640.5370.5010.4940.9890.7991.1090.8541.7091.0130.8 0.393 0.417 0.5480.5210.5410.5200.4240.4420.9390.7330.5680.5400.5070.5000.9950.8001.1090.8541.7101.0130.9 0.393 0.418 0.5460.5220.5430.5210.4260.4440.9410.7340.5720.5430.5150.5070.9940.8021.1090.8541.7111.0141.0 0.407 0.428 0.5540.5290.5470.5250.4280.4470.9420.7360.5760.5470.5220.5120.9890.7991.1090.8541.7111.0151st Cnt 11 11 000000000000000000 Evaluation of Hankelformer on the 90D-Lorenz dataset. Forecasting performance (MSE/MAE) under increasing levels of additive Gaussian noise (σ from 0.0 to 1.0). All models predict 96 future steps based on 96 past steps. Best results are in bold. ‘1st Cnt’ indicates the number of times a model achieved the best score. To further assess the robustness of Hankelformer, we injected Gaussian noise of controlled intensity during Lorenz simulation (see ‘Methods’ section). Figure 2c–f compares the predictions under noise levels of 0.5 and 1.0, while Fig. 2g summarizes the MSE and corresponding standard deviations under different noise conditions. Regardless of noise intensity, Hankelformer consistently incurs significantly lower error than the those competing methods, corroborating its noise robustness (Table 1). In addition, to evaluate the out-of-distribution (OOD) generalization performance of Hankelformer, we added a cross-system experiment. Specifically, all models were trained exclusively on trajectories generated from a 90D coupled Lorenz system and then evaluated directly, without retraining, on trajectories generated from a 90D coupled Rössler system. No Rössler samples were used during training or hyperparameter optimization. The Lorenz and Rössler systems exhibit substantially different chaotic dynamics and attractor structures, making this zero-shot cross-system transfer setting a controlled OOD benchmark. The results are summarized in Table S1 and Fig. S1. Although all methods exhibit significant prediction errors when evaluated on the Rössler system without retraining, Hankelformer still demonstrates relatively superior performance. These results provide empirical evidence that the proposed Hankel-augmented contrastive framework learns representations that are more transferable under certain non-linear distribution shifts compared with standard forecasting architectures. Evaluation of Hankelformer on six public datasets We next conducted a systematic evaluation of Hankelformer on six public datasets from the energy and transportation domains: Electricity (ECL), Traffic and PEMS03/04/07/08. For ECL and Traffic, we employed an input window of 96 time steps to forecast horizons of 96, 192, 336 and 720 steps; for the PEMS datasets, the model kept an input window of 96 time steps and predicted 12, 24, 48 and 96 steps ahead. Table 2 summarizes the MSE and MAE values for Hankelformer and all baseline models. Hankelformer consistently achieves the lowest errors across all datasets and forecast horizons. Table 2.Main forecasting results on six public benchmark datasets. Forecasting performance (MSE/MAE) for multistep forecasting across various horizons. Predictions are based on 96 historical time steps. Best results for each metric and horizon are highlighted in bold. The ‘Avg’ row reports the average score across all horizons. ‘1st Cnt’ indicates the number of times a model achieved the best score.Hankelformer (Ours)WPMixer (2025)iTransformer (2024)TimeMixer (2024)TiDE (2023)PatchTST (2023)SimMTM (2023)CoST (2022)TimesNet (2022)NaïveDatasetTimestepMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEECL96 0.141 0.237 0.1500.2410.1480.2400.1530.2470.2370.3290.1810.2700.1640.2550.2750.3720.1680.2721.5880.946192 0.157 0.252 0.162 0.252 0.1620.2530.1660.2560.2360.3300.1880.2740.1780.2680.2840.3820.1840.2891.5950.950336 0.169 0.265 0.1790.2700.1780.2690.1850.2770.2490.3440.2040.2930.1900.2800.2970.3930.1980.3001.6170.961720 0.199 0.296 0.2170.3040.2250.3170.2250.3100.2840.3730.2460.3240.2350.3180.3030.3930.2200.3201.6470.975Avg 0.167 0.263 0.1770.2670.1780.2700.1820.2730.2520.3440.2050.2900.1920.2800.2900.3850.1930.2951.6120.958Traffic96 0.395 0.266 0.4650.286 0.395 0.2680.4620.2850.8050.4930.6490.3890.5500.3530.7350.4450.5930.3212.7231.079192 0.413 0.273 0.4750.2900.4170.2760.4730.2960.7560.4740.6010.3660.5630.3650.7190.4250.6170.3362.7561.087336 0.423 0.278 0.4890.2960.4330.2830.4980.2960.7620.4770.6090.3690.5870.3680.7350.4380.6290.3362.7891.094720 0.463 0.300 0.5270.3180.4670.3020.5060.3130.7190.4490.6470.3870.6020.3850.7560.4450.6400.3502.8101.097Avg 0.424 0.279 0.4890.2980.4280.2820.4850.2980.7610.4730.6270.3780.5760.3680.7360.4380.6200.3362.7701.089PEMS0396 0.063 0.169 0.0710.1780.0710.1740.0780.1880.1780.3050.0990.2160.0820.1890.1160.2290.5930.1920.1360.250192 0.086 0.196 0.1080.2190.0930.2010.1160.2300.2570.3710.1420.2590.1330.2390.1350.2500.6170.2230.2650.350336 0.121 0.233 0.2020.3000.1250.2360.2000.3010.3790.4630.2110.3190.2330.3260.2530.3650.6290.2600.5960.544720 0.138 0.252 0.2930.3620.1640.2750.3260.3900.4900.5390.2690.3700.2470.3320.2540.3650.6400.3171.2910.857Avg 0.102 0.213 0.1690.2650.1130.2220.1800.2770.3260.4200.1800.2910.1740.2720.1900.3020.6200.2480.5720.500PEMS0496 0.072 0.177 0.0830.1880.0780.1830.0930.2040.2190.3400.1050.2240.1000.2050.1110.2210.0850.1950.1570.269192 0.080 0.187 0.1180.2290.0950.2050.1330.2470.2920.3980.1530.2750.1250.2410.1190.2330.1180.2150.2890.372336 0.096 0.209 0.2050.3070.1200.2330.2410.3430.4090.4780.2290.3390.1900.2930.1490.2670.1550.2500.6420.572720 0.106 0.219 0.2870.3670.1500.2620.3360.4070.4920.5320.2910.3890.2010.2990.1720.2930.2280.3031.4100.904Avg 0.089 0.198 0.1730.2730.1110.2210.2010.3000.3530.4370.1950.3070.1540.2600.1380.2540.1470.2410.6250.529PEMS0796 0.058 0.152 0.0660.1650.0670.1650.0720.1780.1730.3040.0950.2070.0720.1800.1760.2530.0870.1810.1320.247192 0.072 0.172 0.1110.2190.0880.1900.1070.2150.2710.3830.1500.2620.1110.2200.1800.2610.1030.2040.2730.355336 0.093 0.197 0.1980.2920.1100.2150.1780.2730.4460.4950.2530.3400.1910.2910.2020.2880.1360.2380.6450.557720 0.112 0.214 0.3120.3680.1390.2450.2780.3420.6280.5770.3460.4040.1980.3020.2550.3390.1900.2791.3820.872Avg 0.084 0.184 0.1720.2610.1010.2040.1590.2520.3800.4400.2110.3030.1430.2480.2030.2850.1290.2260.6080.508PEMS0896 0.074 0.177 0.0860.1890.0790.1820.0930.2010.2270.3430.1680.2320.1100.2230.2550.2800.1120.2120.1480.263192 0.103 0.209 0.1360.2430.1150.2190.1430.2500.3180.4090.2240.2810.1820.2890.2710.2950.1410.2380.2830.369336 0.166 0.228 0.2450.3220.1860.2350.2630.3440.4970.5100.3210.3540.2870.3330.3400.3550.1980.2830.6550.577720 0.211 0.259 0.3900.3890.2210.2670.4350.4320.7210.5920.4080.4170.3020.3490.4090.4070.3200.3511.4610.909Avg 0.139 0.218 0.2140.2860.1500.2260.2340.3070.4410.4640.2800.3210.2200.2990.3190.3340.1930.2710.6370.5301st Cnt 30 30 011000000000000000 Main forecasting results on six public benchmark datasets. Forecasting performance (MSE/MAE) for multistep forecasting across various horizons. Predictions are based on 96 historical time steps. Best results for each metric and horizon are highlighted in bold. The ‘Avg’ row reports the average score across all horizons. ‘1st Cnt’ indicates the number of times a model achieved the best score. On the highly volatile electricity load series (Fig. 3a and b), Hankelformer not only captures the dominant trend but also accurately reproduces the true peaks and troughs, whereas the predictions from baseline models exhibit noticeable deviations. Similarly, on the normalized traffic flow series (Fig. 3c–l), Hankelformer maintains comparable superiority. These results confirm Hankelformer’s ability to model long-range dependencies and complex non-linear dynamics, demonstrating strong generalization and robustness across diverse domains and data characteristics. Forecasting results on standard benchmark datasets. The figure displays representative 96-step ahead forecasting results on various datasets. (a and b) Two distinct cases from the ECL dataset. (c and d) Two cases from the Traffic dataset. (e and f) Two cases from the PEMS03 dataset. (g and h) Two cases from the PEMS04 dataset. (i and j) Two cases from the PEMS07 dataset. (k and l) Two cases from the PEMS08 dataset. Across all scenarios, Hankelformer demonstrates a superior ability to align with the ground truth compared to other models. Forecasting of extreme temperature events Accurate forecasting of extreme temperature events remains one of the most critical yet challenging tasks in meteorological science. Conventional time-series models often rely on the assumption of weak stationarity, which limits their ability to capture abrupt onsets and extreme amplitudes characteristic of such phenomena. Consequently, developing forecasting tools that remain reliable under extreme conditions is of significant scientific and societal value. To evaluate Hankelformer’s performance in such scenarios, we constructed three tailored datasets based on recent real-world events: TexasFreeze (the February 2021 North American cold wave), Heatwave (the June 2021 Pacific Northwest heat dome) and Antarctic Heat (the February 2020 Antarctic Peninsula heat event). As summarized in Table 3, Hankelformer achieves the lowest MSE and MAE across all forecast horizons and outperforms most baseline models. Table 3.Forecasting results on extreme temperature datasets. Performance (MSE/MAE) for multistep temperature prediction on the TexasFreeze, Heatwave and Antarctic Heat datasets. Predictions are based on 96 past hours. Best results are in bold. The ‘Avg’ row reports the average score across all horizons. ‘1st Cnt’ indicates the number of times a model achieved the best score.Hankelformer (Ours)WPMixer (2025)iTransformer (2024)TimeMixer (2024)TiDE (2023)PatchTST (2023)SimMTM (2023)CoST (2022)TimesNet (2022)NaïveDatasetTimestepMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAEMSEMAETexasFreeze12 0.105 0.212 0.1600.2690.1400.2510.1530.2570.1650.2800.1590.2770.1090.2100.1170.2250.1630.2800.2870.35324 0.179 0.296 0.2810.3800.2500.3600.2650.3610.2780.3840.2710.3780.1850.2960.2210.2540.2210.3340.3790.42648 0.347 0.431 0.5350.5450.5510.5480.5180.5320.5210.5420.5100.5350.3580.4390.3610.4420.3300.4320.5410.52996 0.628 0.6191.0860.8051.1080.8070.9820.7721.0350.7981.0550.8040.629 0.617 0.6670.6530.6980.6440.9300.726Avg 0.315 0.390 0.5160.5000.5120.4920.4800.4810.5000.5010.4990.4990.3200.3910.3420.3940.3530.4230.5340.509Heatwave12 0.100 0.205 0.1500.2550.1280.2360.1410.2460.1580.2690.1580.2690.1090.2140.1030.2180.1210.2380.9270.74724 0.170 0.276 0.2210.3200.1890.2920.2130.3100.2240.3270.2290.3310.1810.2800.1920.2890.1720.2850.9440.75148 0.263 0.3580.3380.4130.3050.3880.3310.3980.3330.4130.3400.4160.271 0.355 0.3030.3580.3160.3861.0170.78496 0.451 0.466 0.4720.4880.4990.5060.4790.4850.4940.5060.4860.5000.4650.4690.4720.4930.6130.5551.1800.839Avg 0.246 0.326 0.2950.3690.2800.3560.2910.3600.3020.3790.3030.3790.2570.3300.2680.3400.3060.3661.0170.780Antarctic Heat12 0.147 0.263 0.1680.2770.1560.2680.1610.2710.1760.2810.1680.2770.1710.2840.1610.2910.1750.2990.5240.49124 0.203 0.320 0.2340.3420.2120.3260.2220.3320.2330.3380.2270.3370.2290.3410.2290.3280.2210.3430.5830.53748 0.272 0.382 0.2900.3940.2750.3840.2790.3870.2890.3910.2840.3900.2910.3970.2940.3990.2880.3990.6890.59796 0.316 0.419 0.3310.4280.3490.4420.3240.4230.3370.4310.3270.4250.3370.4320.3480.4260.3250.4270.7830.645Avg 0.235 0.346 0.2560.3600.2480.3550.2470.3530.2590.3600.2520.3570.2570.3640.2580.3610.2520.3670.6450.5681st Cnt 15 13 000000000002000000 Forecasting results on extreme temperature datasets. Performance (MSE/MAE) for multistep temperature prediction on the TexasFreeze, Heatwave and Antarctic Heat datasets. Predictions are based on 96 past hours. Best results are in bold. The ‘Avg’ row reports the average score across all horizons. ‘1st Cnt’ indicates the number of times a model achieved the best score. Specifically, during the TexasFreeze cold surge (Fig. 4a and d), Hankelformer accurately reproduces the precipitous temperature drop and subsequent recovery, whereas the baseline models systematically underestimate both the rate and magnitude of the decline. Similarly, during the Heatwave and Antarctic Heat event (Fig. 4b, c, e and f), Hankelformer closely tracks the anomalous temperature spike with high fidelity, while competing models exhibit noticeable deviations. These results demonstrate that Hankelformer generalizes effectively across radically different and highly non-linear extreme weather patterns. Notably, the performance advantage of Hankelformer becomes more pronounced at longer forecasting horizons (Fig. 4g–i). Comprehensive analysis of forecasting performance during extreme weather events. (a, d) Two qualitative case studies from the TexasFreeze dataset, showing a 48-hour temperature forecast. Hankelformer accurately captures the severe temperature drop, while baselines fail. (b, e) Two similar cases from the Heatwave dataset, where Hankelformer successfully predicts the sharp peak in temperature. (c, f) Two representative cases from the Antarctic Heat dataset. (g–i) Normalized root mean squared error as a function of prediction horizon for the TexasFreeze, Heatwave and Antarctic Heat datasets, respectively. Hankelformer maintains the lowest error across the entire forecast window. RMSE, root mean squared error. In addition to the MSE and MAE metrics, to directly evaluate performance on extreme events, we further incorporated the Symmetric Extremal Dependence Index (SEDI) and the Frequency Bias Index (FBI) as additional evaluation metrics. SEDI is a metric specifically designed to evaluate the predictive performance of extreme weather events, where positive values indicate that the prediction outperforms random guess, and the closer the value is to 1, the more accurate the prediction. FBI is used to measure the discrepancy between the forecast frequency of an event and the actual observed frequency of the event; the closer the value is to 1, the greater the agreement between the forecast frequency and the observed frequency. For details on how these metrics are calculated, see the ‘Methods’ section. The results are summarized in Table 4. Overall, Hankelformer exhibits more consistent alignment with observed extreme-event amplitudes and frequencies across multiple forecasting horizons compared with the baseline models. The results indicate that Hankelformer excels not only in reducing average forecast error but also in specifically capturing the dynamics of extreme weather events. Table 4.Extreme event forecasting performance measured by SEDI and FBI.​ Results are shown for multistep (12, 24, 48 and 96) hourly temperature prediction on the Antarctic Heat, Heatwave and TexasFreeze datasets. SEDI measures detection skill (higher is better). FBI measures frequency calibration (closer to 1 is better). The best SEDI and FBI scores for each horizon and dataset are highlighted in bold. The ‘Avg’ row reports the average score across all horizons. ‘1st Cnt’ indicates the number of times a model achieved the best score for a given metric.HankelformerWPMixeriTransformerTimeMixerTiDEPatchTSTSimMTMCoSTTimesNet(Ours)(2025)(2024)(2024)(2023)(2023)(2023)(2022)(2022)DatasetTimestepSEDIFBISEDIFBISEDIFBISEDIFBISEDIFBISEDIFBISEDIFBISEDIFBISEDIFBIAntarctic Heat120.7150.7930.7200.7770.7190.759 0.723 0.7460.7140.778 0.723 0.6580.6830.6100.6880.6300.705 0.857 24 0.699 0.6470.6840.6270.6900.6670.6940.6310.6880.7020.6820.5400.6970.6080.5530.5170.672 0.712 48 0.657 0.706 0.6370.6250.6340.0020.6380.5270.6350.6290.6210.5180.5770.6800.5620.5060.6160.58596 0.595 0.641 0.5790.5570.5570.3770.5620.4830.5690.6040.5560.4860.5540.5830.4650.5800.5520.651Avg 0.667 0.6970.6550.6470.6500.4510.6540.5970.6520.6780.6460.5510.6280.6200.5670.5580.636 0.701 Heatwave120.695 0.922 0.7290.6830.7160.8370.7780.6910.7920.674 0.798 0.5950.4130.6610.6930.8600.7270.855240.779 0.808 0.7050.5760.7650.7400.7810.546 0.782 0.5680.7790.4930.2450.5920.7700.8030.7570.73748 0.746 0.681 0.6170.4300.737-0.1050.7100.3740.7100.4260.7030.3600.3050.3500.5330.6530.6170.28296 0.542 0.131 0.4590.0780.4680.0550.4770.0560.4670.0660.4380.0490.2980.1270.0190.0960.4570.028Avg 0.691 0.636 0.6280.4420.6720.3820.6870.4170.6880.4340.6900.3740.3150.4330.5040.6030.6400.476TexasFreeze120.7060.8980.7010.9850.7480.7700.7030.9750.695 0.993 0.7170.9320.6270.8790.5150.661 0.756 0.651240.7250.8420.6971.0130.7410.7150.698 1.004 0.6901.0140.7150.9340.6910.8260.3090.636 0.755 0.638480.700 1.023 0.6891.099 0.721 0.8060.7031.0360.6891.0780.7051.0250.6300.9130.0200.5300.7090.63696 0.700 1.0710.6711.2390.673 1.023 0.6861.0860.6681.2120.6861.0810.6860.9080.0200.6430.6810.867Avg0.7080.9590.6901.0840.7210.8290.6981.0250.6861.0740.706 0.993 0.6590.8820.2070.618 0.725 0.6981st Cnt 8 8 0011111121000033 Extreme event forecasting performance measured by SEDI and FBI.​ Results are shown for multistep (12, 24, 48 and 96) hourly temperature prediction on the Antarctic Heat, Heatwave and TexasFreeze datasets. SEDI measures detection skill (higher is better). FBI measures frequency calibration (closer to 1 is better). The best SEDI and FBI scores for each horizon and dataset are highlighted in bold. The ‘Avg’ row reports the average score across all horizons. ‘1st Cnt’ indicates the number of times a model achieved the best score for a given metric. Ablation study To evaluate the individual contributions of Hankel-AUG and the contrastive learning module, we conducted ablation studies on the full Hankelformer model using four representative datasets: 90D-Lorenz, ECL, Traffic and TexasFreeze, which span the dynamics, energy, transportation and extreme weather domains, respectively. Figure 5a–d presents the corresponding MSE and MAE results. Notably, introducing Hankel-AUG without the contrastive loss often degrades performance, resulting in even higher errors than the baseline. Neither module alone is sufficient for achieving optimal performance. The reason can be explained as follows. While Hankel-based augmentation increases representation diversity and exposes informative dynamical patterns, it also introduces a nontrivial distributional discrepancy between the original sequence representation and the augmented representation. Without the contrastive objective, the encoder has no explicit mechanism to align these heterogeneous representations in the latent space. Consequently, directly fusing the original and augmented features may introduce optimization conflicts and unstable feature interactions, which can negatively affect forecasting performance. From a representation learning perspective, the Hankel-AUG module increases structural diversity, while the contrastive loss regularizes the latent geometry to ensure consistent and robust representations. The performance of the full model is compared against variants without key components. (a–d) The effectiveness of the Hankel-AUG and contrastive loss modules on the Lorenz, Electricity, PEMS07, and TexasFreeze datasets, respectively. The consistent lower error of the full model highlights the significant contribution of both the Hankel-AUG modules, the contrastive learning objective and the learnable MLP layer in the Hankel-AUG module. (e–h) The effectiveness of replacing the MLP in the Hankel-AUG module with a fixed linear projection and of removing the ReLU activation function on the same datasets. In addition, a learnable MLP has been incorporated into Hankel-AUG module. To empirically investigate the necessity and effect of this learnable component, we conducted an extended ablation study comparing three variants of the Hankel-AUG module: the full Hankelformer model with non-linear MLP projection, a learnable linear projection without non-linear activation and a fixed linear projection. As shown in the Fig. 5e–h (see also Tables S2 and S3), the full model consistently achieves the best performance. These results demonstrate that while the Hankel construction provides a critical structured starting point, the learnable non-linear transformation is essential for adapting this representation to achieve robustness and high accuracy in practical forecasting scenarios. Finally, compared to existing explicit spatial dynamics modeling methods, our approach employs a dual explicit modeling, which are handled by the Hankel-AUG and Transformer encoder, respectively. Notably, the accuracy of spatiotemporal dynamics modeling depends on the proportion of Hankel-augmented variables. To validate this point, we conducted an ablation study where Hankel augmentation was applied to only a subset of input variables (0%, 25%, 50%, 75% and 100%). The results, presented in Table S4, show that forecast performance improves as the proportion of augmented variables increases, with the full model (100%) achieving the best overall results on average across all datasets and horizons. DISCUSSION We introduced Hankelformer, a novel MTS forecasting architecture with the spatiotemporal representation, which achieves satisfactory performance through the integration of structured Hankel-AUG and end-to-end contrastive learning. Our approach addresses fundamental limitations in current deep learning methods, insufficient local spatiotemporal pattern extraction and limited robustness to distribution shifts, through mathematically principled innovations grounded in dynamical representation and information-theoretic principles, i.e. the delay embedding theorem and Hankel matrix decomposition. Comprehensive evaluation across nine benchmark datasets demonstrates state-of-the-art performance, with Hankelformer ranking first in almost all metric-horizon combinations. Notably, the architecture exhibits notable robustness in extreme temperature forecasting, achieving substantial improvements of up to 34% MSE reduction compared to leading baselines. The notable effectiveness during extreme weather events (Table 3 and Fig. 4) reveals fundamental insights into temporal pattern recognition under distribution shifts. The Hankel-based augmentation naturally captures local precursor patterns that often precede extreme transitions, while contrastive learning encourages the model to learn features invariant to such local perturbations. This combination enables prediction of unprecedented patterns by leveraging learned invariances rather than relying solely on historical precedent. Rigorous ablation studies confirm the necessity of each architectural component, while noise robustness analysis validates the intrinsic stability of learned representations. The success of structured augmentation principles suggests promising extensions to other time-series tasks including anomaly detection, classification and multimodal forecasting. The demonstrated robustness properties make Hankelformer particularly suitable for safety-critical applications in energy systems, transportation networks and climate monitoring where prediction failures can have severe consequences. The strong performance of Hankelformer stems from three complementary design principles that address fundamental limitations in current time-series forecasting. First, the Hankel-AUG module leverages structured matrix decomposition theory to extract local spatiotemporal patterns while preserving temporal coherence—a critical advantage over generic augmentation strategies that often destroy sequential dependencies [47,48]. Unlike traditional sliding window approaches, Hankel matrices systematically capture all possible subsequence interactions within the specified window, providing a mathematically principled foundation for augmentation that aligns with classical signal processing theory [63,64]. Second, the contrastive learning framework operates on information-theoretic principles [49,50], maximizing mutual information between structured views to fuse the spatiotemporal representations while promoting invariant feature learning. Our noise robustness results (Table 1 and Fig. 2g) provide empirical validation that this approach yields representations inherently robust to stochastic perturbations—a crucial property for handling extreme events where historical distributions become unreliable. The graceful degradation (only 17% MSE increase across σ = 0–1.0) confirms that contrastive learning creates representations that capture essential temporal dynamics rather than spurious correlations. Third, the dual-stream architecture with explicit feature fusion ensures that robust representations learned through contrastive objectives directly contribute to forecasting performance, mitigating potential pretraining–finetuning mismatch that plagues separate pretraining approaches [56–62]. This end-to-end integration enables simultaneous optimization for both representation robustness and predictive accuracy. Compared to state-of-the-art Transformer variants [33–43], Hankelformer’s unique advantage lies in explicitly addressing the local pattern extraction limitation through structured augmentation, which provides effective dynamic constraints for contrastive learning. While models like PatchTST [36] and iTransformer [37] excel at capturing global dependencies through refined attention mechanisms, they often miss critical local dynamics that serve as early warning signals for extreme events. Our ablation results (Fig. 5) demonstrate that replacing structured Hankel augmentation with simple noise degrades performance by 53.6%—worse than baseline—highlighting that naïve contrastive learning is insufficient. Relative to existing time-series contrastive learning methods (TS2Vec [58], CoST [59] and TF-C [60]), our approach distinguishes itself through: (i) mathematically motivated augmentation preserving temporal structure due to the delay embedding strategy, (ii) end-to-end integration mitigating potential pretraining–finetuning mismatch and (iii) explicit focus on local spatiotemporal pattern diversity. The consistent improvements across most metric-horizon combinations validate this principled approach. Several limitations of this study warrant acknowledgment. First, the computational overhead associated with the dual-stream architecture increases training time by ∼1.3× compared to single-stream baselines, highlighting the need for future work on more efficient variants or adaptive augmentation strategies. For detailed metrics related to computational efficiency and comparisons with other benchmark methods, see Table S5. Second, the geographic scope of the extreme weather datasets is currently limited to limited regimes, validation across diverse climate regimes and geographical regions is necessary to strengthen claims regarding generalizability. Third, while the model demonstrates superior performance, its interpretability remains limited, similar to other existing works. Identifying which specific local patterns contribute to predictions of extreme events could enhance trustworthiness in high-stakes operational settings. Future research could incorporate attention visualization techniques or develop explainable variants of the Hankel-AUG mechanism. Fourth, our method is currently primarily applicable to extreme temperature events. For scenarios involving more rapid and abrupt changes, such as heavy rainfall events (see Fig. S2 and Table S6 for test results), more advanced theories and methods need to be introduced. Another promising direction is integrating graph neural networks [70,71] into Hankelformer to more explicitly incorporate geographical topology (e.g. 2D or 3D meteorological fields) and spatial connectivity, employing localized or sparse attention masks to restrict information exchange to physically relevant neighborhoods, and exploring hierarchical multiscale architectures for jointly modeling local and large-scale dynamics. Finally, hyperparameter sensitivity—particularly in the choice of Hankel window size W and contrastive loss weight λ —currently requires empirical tuning (see Fig. S3 and Tables S7–S9 for sensitivity analyses on several representative datasets). The development of adaptive or theoretically guided parameter selection strategies would improve practical applicability across varying temporal scales and application domains. Future work will focus on extending these principles to broader temporal analysis tasks, developing more efficient architectural variants and enhancing model interpretability for high-stakes applications. The integration of self-supervised learning with structured domain knowledge represents an important step toward more principled approaches in time-series modeling, moving beyond generic deep learning adaptations toward methods that respect the unique characteristics of temporal data. This work establishes a foundation for future research in robust temporal representation learning under real-world, non-stationary conditions. METHODS Detailed methods are given in the online Supplementary data. Supplementary Material