Mixing It Up: Exploring Mixer Networks for Irregular Multivariate Time Series Forecasting

Christian Klötergens ✉, Tim Dernedde, Lars Schmidt-Thieme, Vijaya Krishna Yalavarthi · 2025

Read Mixing It Up: Exploring Mixer Networks for Irregular Multivariate Time Series Forecasting by Christian Klötergens ✉, Tim Dernedde, Lars Schmidt-Thieme, Vijaya Krishna Yalavarthi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11816.txt.

Introduction

Christian Klötergens, Tim Dernedde, Lars Schmidt-Thieme, and Vijaya Krishna Yalavarthi present a 2025 paper on forecasting time series that arrive at uneven times with missing values. Such data are common in health care, climate science, and biology. Their model adapts a simple design, built from plain multi-layer networks, that has rivaled attention-based models in other fields.

The model, called IMTS-Mixer, has two key parts. One part turns scattered readings from each channel into fixed-size vectors. The other part decodes in continuous time, so forecasts can be made for any time point. On standard benchmark datasets, the model set new best results for accuracy and speed. It also used fewer parameters than the rival methods.

Catalog record

CREATORChristian Klötergens ✉, Tim Dernedde, Lars Schmidt-Thieme, Vijaya Krishna Yalavarthi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11816

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Mixing It Up: Exploring Mixer Networks for Irregular Multivariate Time Series Forecasting Christian Klötergens ✉, Tim Dernedde, Lars Schmidt-Thieme, Vijaya Krishna Yalavarthi Abstract Forecasting irregularly sampled multivariate time series with missing values (IMTS) is a fundamental challenge in domains such as healthcare, climate science, and biology. While recent advances in vision and time series forecasting have shown that lightweight MLP-based architectures (e.g., MLP-Mixer, TSMixer) can rival attention-based models in both accuracy and efficiency, their applicability to irregular and sparse time series remains unexplored. In this paper, we propose IMTS-Mixer , a novel architecture that adapts the principles of Mixer models to the IMTS setting. IMTS-Mixer introduces two key components: (1) ISCAM , a channel-wise encoder that transforms irregular observations into fixed-size vectors using simple MLPs, and (2) ConTP , a continuous time decoder that supports forecasting at arbitrary time points. In our experiments on established benchmark datasets we show that our model achieves state-of-the-art performance in both forecasting accuracy and inference time, while using fewer parameters compared to baselines. 1 Introduction Time \text{c}_{1} \text{c}_{2} \text{c}_{3} Observation Range Forecasting Horizon Figure 1: Example of an IMTS Forecasting task. The observations and forecasting targets are irregularly spaced. Time series forecasting is a critical task across various domains. While a significant body of research has focused on forecasting regularly sampled, fully observed multivariate time series, many real-world applications, such as healthcare, economics, astronomy, and climate, deal with time series data that are irregularly sampled, with observations missing across different channels and timestamps. When such data are aligned to a global timeline, the result is a highly sparse and uneven multivariate structure, as illustrated in Figure 1 . We refer to such data as irregularly sampled multivariate time series with missing values (IMTS) . Traditional approaches to modeling IMTS have relied on Neural-ODE frameworks, where the time series is modeled as the solution to an ordinary differential equation (ODE) with a neural network-defined vector field [ 18 , 5 , 19 ] . While powerful in principle, these models tend to be computationally expensive due to the sequential nature of ODE solvers, and they lack a systematic way to handle missing values. Recently, attention-based models have emerged as a more efficient and accurate alternative, achieving competitive performance on IMTS forecasting tasks [ 29 , 34 , 13 ] . These models typically transform irregular channel-wise observations into fixed-size representations and apply attention mechanisms to capture inter- and intra-channel dependencies. However, such models are often resource-intensive, requiring a large number of parameters and high memory requirements . A recent work in computer vision has demonstrated that carefully arranged MLP-based models can rival or even surpass attention-based models [ 23 ] . This insight has also been applied to forecasting tasks with regularly sampled time series [ 8 , 4 ] . However, this line of research has not yet been extended to irregularly sampled time series with missing values. In this paper, we introduce IMTS-Mixer , a novel architecture for forecasting with IMTS that combines the simplicity of MLP-based designs with specialized modules to address irregular sampling and forecasting at arbitrary time points. First, we propose the I rregularly S ampled C hannel A ggregation M odule (ISCAM), a lightweight encoder that maps each irregular channel into a fixed-size representation using simple MLPs. These embeddings are then concatenated and processed by an MLP-Mixer-style architecture to learn inter- and intra-channel interactions. Second, to support continuous-time forecasting, we introduce the Con tinuous T emporal P rojection ( ConTP ) module, a decoder that takes the mixer output, a continuous-valued target timestamp, and a channel identifier to generate a prediction. In an extensive comparison, we evaluate IMTS-Mixer on four real-world datasets from various domains. We observe that IMTS-Mixer establishes a new state-of-the-art in terms of forecasting accuracy on three out of four datasets. Our contributions are summarized as follows 1 1 1 The code is available here: https://github.com/kloetergensc/IMTS-Mixer/ : 1. We propose IMTS-Mixer , a novel MLP-based architecture tailored for irregularly sampled multivariate time series with missing values (IMTS), combining efficient channel aggregation and continuous-time prediction. 2. We introduce ISCAM , a simple and effective channel-wise encoder that maps irregularly observed channels to fixed-size representations, enabling MLP processing. 3. We develop a lightweight temporal projection module ( ConTP ) that maps a fixed-size representation of an input channel to future IMTS predictions. 2 Related Work 2.1 Regular Multivariate Forecasting The Regular Multivariate Time Series forecasting literature recently focused on transformer-based [ 24 ] architectures [ 37 , 27 , 38 , 17 , 14 ] . However, it was shown that a simple model based on linear layers (DLinear [ 32 ] ) can be competitive and partially outperform some of the more complex transformer architectures. Additionally, mixer architectures have been adapted from the computer vision domain [ 23 ] to time series [ 8 , 4 ] . These models exclusively rely on fully-connected networks that are alternately applied along the channel and time dimension and achieve state-of-the-art results. 2.2 IMTS Forecasting Initially, researchers relied on Neural ODE-based [ 3 ] approaches to model irregular time series [ 18 , 5 , 1 , 19 ] . ODEs are the prevalent framework in science and engineering to model how systems evolve over time in a continuous manner. However, using neural networks to represent ODE systems has proven to be less effective than other IMTS modeling approaches and also incurs relatively long inference times [ 29 , 34 ] . Graph Transformers for IMTS forecasting. Apart from the ODE-based approaches the Transformable Patch Graph Neural Network (tPatchGNN) [ 34 ] applies a patching mechanism to each channel. The patches are then processed with a Transformer and serve as input for a Graph Neural Network (GNN) that models inter-channel dependencies. Additionally, we want to highlight Graphs for Forecasting Irregular Multivariate Time Series (GraFITi) [ 29 ] . Here, the IMTS is represented with a graph in which channels and time steps are nodes and observed values are stored in the edges between the corresponding time and channel nodes. Forecasts are made by predicting the edge values of query-time nodes with graph attention layers. While this approach provides state-of-the-art performance, it also comes with major drawbacks that relate to the graph structure itself. GraFITi relies on time nodes that are connected to multiple channel nodes, corresponding to observation steps that observe more than one channel. If only one channel is observed at a time, the resulting graph is disconnected, rendering GraFITi incapable of modeling inter-channel interactions. Another problem of GraFITi lies in the fact that the query nodes are indirectly connected and hence influence each other. This entails that predictions will change depending on what other queries we give to the model. TimeCHEAT [ 13 ] transforms an IMTS into regularly observed patches using a GraFITi-based encoder. These patches are then further processed by a transformer and a linear decoder to obtain the final predictions. 3 Mixer Networks MLP-Mixer. The MLP-Mixer [ 23 ] was originally proposed as a simple and efficient alternative to convolutional neural networks (CNNs) and Vision Transformers (ViTs) for image classification tasks. In this architecture, a 2D image ( \in\mathbb{R}^{W\times H} , where W,H are width

Read the complete text