CoRA: Boosting Time Series Foundation Models for Multivariate Forecasting through Correlation-aware Adapter
Hanyin Cheng, Xingjian Wu, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, Chenjuan Guo · 2026
Read CoRA: Boosting Time Series Foundation Models for Multivariate Forecasting through Correlation-aware Adapter by Hanyin Cheng, Xingjian Wu, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, Chenjuan Guo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21828.txt.
Introduction
Cheng and co-authors add a small adapter that teaches time series foundation models to use cross-channel links. Most such models treat each channel alone and miss how series move together. Their CoRA adapter splits the correlation matrix into parts that stay fixed and parts that drift over time. Learnable polynomials track the drifting part through trends and cycles. A dual contrastive scheme picks up positive and negative links that involve only some channels. Fine-tuning with CoRA lifted forecast scores across ten real-world datasets, they report.
Catalog record
| CREATOR | Hanyin Cheng, Xingjian Wu, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, Chenjuan Guo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65691 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21828
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CoRA: Boosting Time Series Foundation Models for Multivariate Forecasting through Correlation-aware Adapter Hanyin Cheng, Xingjian Wu, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, Chenjuan Guo Abstract Most existing Time Series Foundation Models (TSFMs) use channel independent modeling and focus on capturing and generalizing temporal dependencies, while neglecting the correlations among channels or overlooking the different aspects of correlations. However, these correlations play a vital role in Multivariate time series forecasting. To address this, we propose a Co R relation-aware A dapter ( CoRA ), a lightweight plug-and-play method that requires only fine-tuning with TSFMs and is able to capture different types of correlations, so as to improve forecast performance. Specifically, to reduce complexity, we innovatively decompose the correlation matrix into low-rank Time-Varying and Time-Invariant components. For the Time-Varying component, we further design learnable polynomials to learn dynamic correlations by capturing trends or periodic patterns. To learn positive and negative correlations that appear only among some channels, we introduce a novel dual contrastive learning method that identifies correlations through projection layers, regulated by a Heterogeneous-Partial contrastive loss during training, without introducing additional complexity in the inference stage. Extensive experiments on 10 real-world datasets demonstrate that CoRA can improve TSFMs in multivariate forecasting performance. 1 Introduction Time Series Foundation Models (TSFMs) that show strong generalization are proposed recently. Through pre-training on large-scale time series data ( 62 ; 14 ; 31 ; 64 ) or the use of large language models ( 82 ; 30 ; 25 ; 18 ) , these models maintain strong reasoning abilities when handling new or unseen data. At the same time, multivariate time series forecasting, as a pivotal domain in data analysis, is widely applied in various industries ( 44 ; 71 ; 26 ; 46 ; 27 ; 65 ; 43 ) . Properly modeling and utilizing correlations in multivariate time series can significantly improve the performance of forecasting models ( 78 ; 29 ; 72 ) . Based on different interaction characteristics among channels, as shown in Figure 1 a, correlation can be summarized into three aspects: dynamic correlation (DCorr) describes the variation of channel relationships over time ( 79 ; 7 ) ; heterogeneous correlation (HCorr) focuses on how channels interact with each other by considering positive and negative correlations ( 16 ) ; partial correlation (PCorr) emphasizes that correlation exists only among certain channels, and modeling interactions across all channels can easily introduce noise ( 3 ; 45 ; 24 ) . Considering more comprehensive correlations provides richer information for the models. However, most existing TSFMs focus on capturing and generalising temporal dependencies and neglect relationships among channels ( 14 ; 1 ; 31 ; 25 ; 18 ; 52 ) . Although some models like TTM ( 11 ) , UniTS ( 13 ) , and Moirai ( 66 ) employ different methods to model the correlations among channels, they do not comprehensively consider multiple aspects of the correlations. For example, TTM employs an MLP-based channel mixing approach, but the MLP weights remain unchanged across different time steps, thereby failing to model DCorr while indiscriminately modelling interactions among all channels, and thus failing to capture HCorr and PCorr explicitly. Figure 1: (a) Illustration of three different types of correlations, the formal definitions are provided in Appendix A . (b) Comparisons of different plugins for learning correlations While the attention mechanisms used in UniTS and Moirai assign different attention scores at different time points, they still interact all channels simultaneously without considering HCorr and PCorr, thus leading to suboptimal correlation modeling. Furthermore, due to the variations in correlations across different datasets, it is difficult to capture generalized correlations during the pre-training phase ( 11 ) . Thus, it motivates us to design a plugin that can be fine-tuned alongside TSFMs, which avoids issues caused by correlation differences across datasets during the pre-training phase. Meanwhile, it possesses the ability to depict various correlations while also incorporating a lightweight design. However, this faces a major challenge: balancing the complete modeling of various correlations with the lightweight design. It is intrinsically difficult to model all three correlations in a unified manner. Although some models could address DCorr ( 79 ; 7 ) , HCorr ( 16 ) and PCorr ( 45 ; 24 ) individually, they struggle to effectively encompass various correlations simultaneously. Moreover, existing channel interaction methods often rely on MLPs ( 12 ; 10 ) , Transformers ( 29 ; 17 ) and GNNs ( 72 ; 2 ) , etc., which have a time complexity of \mathcal{O}(N^{2}) , where N denotes the number of channels. Some methods ( 78 ; 3 ; 38 ) have made efforts in reducing the complexity. However, end-to-end models such as Crossformer ( 78 ) require modifying or redesigning the entire model structure, and thus cannot be directly used as plugins for TSFMs. Existing plugins are primarily designed for end-to-end forecasting models. CCM ( 3 ) requires additional pre-training together with the end-to-end models before it can be plugged in. C-LoRA ( 38 ) is designed to be trained with an end-to-end backbone from scratch. Overall, there is a lack of an efficient plugin specifically designed for downstream fine-tuning of TSFMs. More importantly, considering various correlations in these methods would lead to a higher complexity. To address this, we propose CoRA, a lightweight plug-and-play method that only requires training on a few samples with TSFMs during the fine-tuning phase. By considering various correlations, CoRA utilises internal representations and original predictions from TSFMs to generate an enhanced prediction, as shown in Figure 1 b. To complete modeling the mentioned three types of correlation, we first propose the Dynamic Correlation Estimation (DCE) module which can learn dynamic correlation matrices. Then we design the Heterogeneous-Partial Correlation Contrastive Learning (HPCL) that uses the correlation matrices from DCE to learn HCorr and PCorr adaptively. Specifically, to achieve lightweight, we innovatively decompose the correlation matrices into two low-rank components: Time-Varying and Time-Invariant in DCE module. To better understand how DCorr evolves, we propose a learnable polynomial to capture trend or periodic patterns within the DCorr effectively. Afterwards, to better distinguish of HCorr, we propose channel-aware projections to map the representations into positive and negative correlation spaces. The projections are guided by the novel Heterogeneous-Partial Contrastive Loss during the training process, which enables adaptive learning of PCorr in the two HCorr spaces. As a result, we can capture the mentioned three types of correlations with linear complexity w.r.t. the number of channels during inference. Our contributions are summarized as follows: • We design a universal, lightweight plugin that allows TSFMs to capture the mentioned three types of correlations without re-pre-training the TSFMs. • We propose a lightweight Dynamic Correlation Estimation module that explicitly models the dynamic patterns of correlations in a lightweight manner. • We propose a novel Heterogeneous-Partial Correlation Contrastive Learning, which learns HCorr and PCorr through projection layers regulated by dual contrastive loss. • We conducted extensive experiments on 10 real-world datasets. The results show that CoRA effectively improves the performance of TSFMs in multivariate forecasting. 2 Related Work 2.1 Foundation Models for Time Series Forecasting TSFMs for forecasting can be divided into two sections: 1) LLM-based Models: These methods leverage