Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecasting
Xingyu Zhang, Hanyun Du, Zeen Song, Siyu Zhao, Changwen Zheng, Wenwen Qiang · 2025
Read Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecasting by Xingyu Zhang, Hanyun Du, Zeen Song, Siyu Zhao, Changwen Zheng, Wenwen Qiang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16308.txt.
Introduction
Xingyu Zhang and co-authors rethink forecasting for many linked series at once in a 2025 paper. Standard tools feed all series into one model with no sense of which ones drive which. That blurs true cause ties with fake ones and can bend the calls.
Their plan calls each target on its own, from parts sorted by a cause map. History splits into self, direct-cause, clash-cause, and fake-tie parts for each target. A transformer with a live cause tuner then learns from those parts and can fix a flawed map as it trains. A guard rule curbs the bias that clash parts can bring. Tests on set data banks back the plan.
Catalog record
| CREATOR | Xingyu Zhang, Hanyun Du, Zeen Song, Siyu Zhao, Changwen Zheng, Wenwen Qiang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 110 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16308
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecasting Xingyu Zhang, Hanyun Du, Zeen Song, Siyu Zhao, Changwen Zheng, Wenwen Qiang Abstract Most existing multivariate time series forecasting methods adopt an all-to-all paradigm that feeds all variable histories into a unified model to predict their future values without distinguishing their individual roles. However, this undifferentiated paradigm makes it difficult to identify variable-specific causal influences and often entangles causally relevant information with spurious correlations. To address this limitation, we propose an all-to-one forecasting paradigm that predicts each target variable separately. Specifically, we first construct a Structural Causal Model from observational data and then, for each target variable, we partition the historical sequence into four subsegments according to the inferred causal structure: endogenous, direct causal, collider causal, and spurious correlation. Furthermore, we propose the Causal Decomposition Transformer (CDT), which integrates a dynamic causal adapter to learn causal structures initialized by the inferred graph, enabling correction of imperfect causal discovery during training. Furthermore, motivated by causal theory, we apply a projection-based output constraint to mitigate collider induced bias and improve robustness. Extensive experiments on multiple benchmark datasets demonstrate the effectiveness of the CDT. 1 Introduction Multivariate Time Series Forecasting (MTSF) is a fundamental problem in various fields, including energy consumption ( Bilal et al., 2022 ) , economic planning ( Hidalgo, 2009 ) , weather prediction ( Duchon & Hale, 2012 ) , and traffic forecasting ( Li et al., 2015 ) . The goal of MTSF is to predict the future values of multiple interrelated variables based on their historical observations ( Box et al., 2015 ) . Unlike univariate forecasting, MTSF must capture not only individual temporal patterns but also the interactions among multiple interdependent variables. This makes it crucial to identify which variables influence the target and how. Failing to distinguish relevant from irrelevant inter-variable dependencies properly could either result in information loss or introduce spurious correlations that degrade forecasting performance. Based on the above statement, an MTSF method should be capable of capturing the intrinsic temporal patterns of each variable, correctly identifying how other variables causally influence the target, and eliminating spurious correlations that obscure true dependence. However, most existing MTSF methods overlook this structural heterogeneity, including Transformer-based models ( Wen et al., 2023 ; Tang & Zhang, 2023 ) . These models typically take all variables’ histories without differentiation, and train a single shared model to jointly forecast all targets in one forward pass ( Liu et al., 2025b ; Liu et al., 2022b ) . This design, whether channel-independent ( Nie et al., 2023 ) or channel-mixed ( Liu et al., 2024b ) , makes no distinction in contribution among variables, ignoring the distinct causal roles they may play with respect to the target. Although this all-to-all design is easy to implement, it overlooks an important observation: when focusing on forecasting a specific target variable, the history segments of different variables often play very different roles. For instance, in weather forecasting, temperature and humidity are influenced by wind direction, yet they have no direct causal relationship. Meanwhile, temperature and atmospheric pressure jointly affect precipitation, forming a collider structure: \text{temperature}\to\text{precipitation}\leftarrow\text{pressure} ( Wilks, 2011 ) . When forecasting temperature, different variables influence in distinct ways: (i) temperature’s own past provides an autoregressive signal; (ii) wind direction exerts a direct causal influence by determining the inflow of warm or cold air; (iii) through the collider structure, precipitation activates conditional dependence between temperature and pressure; and (iv) humidity appears correlated with temperature only through their shared cause wind direction, but becomes independent once wind direction is conditioned on. Feeding all of these histories indiscriminately into an all-to-all model conflates true causal drivers with spurious signals, leading to noisy attention weights, entangled parameter learning, and ultimately degraded forecasting performance. To address these challenges, we adopt an all-to-one forecasting paradigm, where each target variable is predicted individually with an explicit structural decomposition of its historical window. Specifically, guided by the d-separation criterion in structural causal models (SCMs), we partition the multivariate history for each target variable into four sub-segments: (1) Endogenous Subsegment (ES), the target’s own history; (2) Direct Causal Subsegment (DCS), histories of variables that are directly connected to variable in the causal graph; (3) Collider Causal Subsegment (CCS), histories involved in collider patterns with the target variable; and (4) Spurious Correlation Subsegment (SCS), histories that become independent of the target variable after conditioning on the relevant segments. To operationalize this decomposition, we learn a target specific DAG from observational training data using the Peter-Clark (PC) algorithm ( Spirtes et al., 2001 ) , from which we derive the direct causal and collider causal variable sets for segment construction. To ensure robustness to inevitable estimation errors, we further introduce a learnable dynamic causal adapter that treats the learned DAG as a structural prior and refines during training. Building on the above, we propose the Causal Decomposition Transformer (CDT), a target specific forecasting framework that aligns model computation with the causal roles of historical segments of other variables. For each target, CDT constructs three complementary contexts: an endogenous context capturing the target’s own dynamics, a direct causal context summarizing the influence of directly related variables, and a collider causal context accounting for collider induced dependencies. These contexts are processed in a structured manner with attention constraints that respect their causal roles, ensuring that multivariate information is incorporated without entangling spurious signals. Finally, motivated by generalization analysis, CDT applies a causal output refinement to suppress collider induced spurious dependence and improve robustness. Our contributions: 1 ) We propose an all-to-one forecasting paradigm for MTSF, in which each target variable’s future is predicted individually; 2 ) We propose CDT, which refines the structural prior, incorporates causal contexts, and introduces an output spouse projection motivated by theory to mitigate collider induced spurious dependence; 3 ) Extensive experiments and ablation studies on multiple benchmark datasets demonstrate that CDT achieves superior predictive accuracy, robustness, and interpretability compared to existing methods. 2 Related Works Multivariate Time Series Forecasting With the development of deep learning ( LeCun et al., 2015 ) , numerous models have been proposed for MTSF ( Hu & Xiao, 2022 ; Wen et al., 2023 ) , including CNN-based ( Zhan et al., 2023 ) , RNN-based ( Hewamalage et al., 2021 ; Tang et al., 2021 ) , MLP-based ( Zhang et al., 2024c ) , and Transformer-based ( Zhang et al., 2024b ) architectures. These approaches are all based on all‑to‑all strategy and can be broadly categorized based on their modeling focus into three groups: temporal-domain, frequency-domain, and variable-domain methods. Temporal domain methods, such as PatchTST ( Nie et al., 2023 ) and TimesNet ( Wu et al., 2023 ) , focus on intra-variable dependencies by modeling patch-wise or point-wise relations. Frequency domain methods, such as FEDFormer ( Zhou et al., 2022c ) and FreDF ( Zhang et al., 2024b ) , transform sequences into the Fourier domain to capture frequency-specific dynamics. Variable domain methods, such as iTransformer ( Liu et al., 2024b ) , model inter-variable dependencies via attention mechanisms, and TimeXer ( Wang et al., 2024b ) proposes to treat variables’ endogenous and exogenous signals differently. Modeling Variable