XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis
Israel Zexer [email protected], Omri Azencot [email protected] · 2026
Read XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis by Israel Zexer [email protected], Omri Azencot [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18534.txt.
Introduction
Israel Zexer and Omri Azencot present XCTFormer, a 2026 transformer-based channel-dependent model for multivariate time-series analysis capturing cross-temporal and cross-channel dependencies directly. Tokens model pairwise dependencies across time and channels via a Cross-Relational Attention Block, with an optional Dependency Compression Plugin for scalability. On three benchmarks it performs strongly, taking state-of-the-art imputation results by beating the second-best method by 20.8 percent in MSE and 15.3 percent in MAE on average.
Catalog record
| CREATOR | Israel Zexer [email protected], Omri Azencot [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 117832 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.18534
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis Israel Zexer [email protected], Omri Azencot [email protected] Abstract Multivariate time-series analysis involves extracting informative representations from sequences of multiple interdependent variables, supporting tasks such as forecasting, imputation, and anomaly detection. In real-world scenarios, these variables are typically collected from a shared context or underlying phenomenon, suggesting the presence of latent dependencies across time and channels that can be leveraged to improve performance. However, recent findings show that channel-independent (CI) models, which assume no inter-variable dependencies, often outperform channel-dependent (CD) models that explicitly model such relationships. This surprising result indicates that current CD models may not fully exploit their potential due to limitations in how dependencies are captured. Recent studies have revisited channel dependence modeling with various approaches; however, these methods often employ indirect modeling strategies, which can lead to meaningful dependencies being overlooked. To address this issue, we introduce XCTFormer , a transformer-based channel-dependent (CD) model that explicitly captures cross-temporal and cross-channel dependencies via an enhanced attention mechanism. The model operates in a token-to-token fashion, modeling pairwise dependencies between every pair of tokens across time and channels. The architecture comprises (i) a data processing module, (ii) a novel Cross-Relational Attention Block (CRAB) that increases capacity and expressiveness, and (iii) an optional Dependency Compression Plugin (DeCoP) that improves scalability. Through extensive experiments on three time-series benchmarks, we show that XCTFormer achieves strong results compared to widely recognized baselines; in particular, it attains state-of-the-art performance on the imputation task, outperforming the second-best method by an average of 20.8% in MSE and 15.3% in MAE. Our code is publicly available at https://github.com/azencot-group/XCTFormer . 1 Introduction Forecasting, anomaly detection, and imputation are critical tasks across a wide range of real-world domains ( 22 ) . For instance, forecasting is utilized in energy management, weather prediction, healthcare, and more ( 34 ; 6 ; 12 ) . Time-series analysis plays a vital role in extracting key information from sequential data to facilitate these tasks. The effectiveness of this information extraction is crucial, as it directly impacts the performance of subsequent time-series tasks ( 49 ) . Accurate time-series analysis enables organizations to enhance decision-making and optimize resource allocation ( 7 ; 54 ) , highlighting the importance of the information extraction component as a key area of research. Time-series data can be modeled using two main approaches ( 17 ) . Univariate approaches treat each channel independently, disregarding any potential relationships between them. In contrast, multivariate approaches take into account not only the temporal behavior within each channel but also potential dependencies across channels. In real-world scenarios, multivariate datasets are often derived from a common underlying process or phenomenon, which typically leads to dependencies among the features ( 8 ) . Incorporating relevant signals enhances representation quality and improves accuracy in downstream tasks ( 21 ; 11 ) . As a result, multivariate models are generally expected to outperform univariate models by leveraging both cross-channel dependencies and temporal dynamics. Therefore, time-series analysis can benefit significantly from richer representations when cross-channel dependencies are utilized. However, recent work in time-series forecasting by 17 challenged this assumption by showing that channel-independent (CI) models, which treat multivariate time-series as separate univariate channels and ignore potential inter-channel correlations, outperform channel-dependent (CD) models that explicitly model such dependencies. They attribute this surprising outcome to a trade-off between capacity, defined as a model’s ability to fit complex patterns, and robustness, defined as its ability to remain accurate in the presence of noise, input variation, or distribution shifts. While CD models gain capacity by incorporating cross-channel information, this often comes at the expense of robustness, making them more sensitive to distribution shifts. In contrast, CI models sacrifice some capacity by ignoring cross-channel dependencies, thereby improving robustness and generalization accuracy. These findings challenge the common belief that adding relevant information typically improves representation quality and accuracy, revealing a gap between channel-dependent methods and their unrealized potential. Motivated by these findings, we seek in this paper to address the following question: How should we model sequential cross-channel information to realize its potential? Recent research has revisited channel dependence with cross-channel modeling approaches that often outperform channel-independent (CI) baselines. iTransformer ( 30 ) targets cross-channel dependencies by treating each channel as a token and applying a Transformer on the token sequence. CrossFormer ( 64 ) and CARD ( 55 ) address both cross-channel and cross-time relationships, by employing a two-stage pipeline for sequence modeling and channel processing. Despite recent advancements, most methods model dependencies across different channels and time indirectly , thereby potentially overlooking important interactions. Additionally, cross-channel dependencies are often unknown in advance, as the underlying generative process is typically unknown. These dependencies may also change over time ( 65 ) , raising the need for simultaneous cross-channel cross-time modeling. To address these challenges, we propose a direct modeling strategy with a token-to-token approach that explicitly captures each token’s pairwise dependencies across all channels and time-steps. This potentially minimizes essential information loss associated with existing indirect models. To accomplish this, we introduce XCTFormer , a Transformer-based framework that models all pairwise dependencies directly within a single attention block, token-to-token, effectively identifying the most relevant dependencies for downstream tasks. The backbone of the XCTFormer consists of three novel components: (i) a data processing unit, (ii) the Cross-Relational Attention Block (CRAB), and (iii) the Dependency Compression Plugin (DeCoP). First, we independently patch each channel and tokenize the data. Next, we flatten the channel and time dimensions, which allows CRAB and DeCoP to capture all pairwise dependencies in a token-to-token manner. CRAB extends the standard attention block ( 50 ) with two key modifications to improve expressivity and robustness. First, it introduces a learnable, non-boolean masking mechanism that supplements conventional binary masks by weighting dependencies according to their learned importance. This allows the model to focus on the most crucial dependencies for the downstream task. Second, CRAB replaces the standard softmax function with a new normalization technique that retains the properties needed for attention activation ( 46 ) while allowing negative weights. This extension increases the model’s expressiveness by enabling it to capture a wider range of relationships, as suggested by 32 . Lastly, DeCoP is an optional module that partly modifies CRAB with the addition of a learnable compression mechanism, designed to enhance scalability for datasets with numerous channels. It addresses the memory limitations imposed by the transformer’s quadratic attention mechanism. DeCoP compresses the quadratic attention into a linear form while minimizing information loss through a learnable compression transformation. We evaluated XCTFormer against various baseline models on multiple downstream tasks, including forecasting, anomaly detection, and imputation, demonstrating strong results. Our main contributions are: 1. We identify a key limitation in the current literature on time-series modeling: while analysis methods have advanced substantially, little emphasis has been placed on explicitly capturing both cross-channel and cross-time dependencies in a unified manner. Most existing approaches either model temporal patterns or inter-channel relations separately, which restricts their ability to exploit the full structure of multivariate time-series data. 2. To address this gap, we propose XCTFormer, a general-purpose framework that models all pairwise cross-channel and cross-time dependencies directly through token-to-token mappings. XCTFormer integrates two complementary components: (i) CRAB, which enhances expressiveness by learning