Investigating a Model-Agnostic and Imputation-Free Approach for Irregularly-Sampled Multivariate Time-Series Modeling

Abhilash Neog, Arka Daw, Sepideh Fatemi, Medha Sawhney, Aanish Pradhan, Mary E. Lofton, Bennett J. McAfee, Adrienne Breef-Pilz, Heather L. Wander, Dexter W Howard, Cayelan C. Carey, Paul Hanson, Anuj Karpatne · 2025

Read Investigating a Model-Agnostic and Imputation-Free Approach for Irregularly-Sampled Multivariate Time-Series Modeling by Abhilash Neog, Arka Daw, Sepideh Fatemi, Medha Sawhney, Aanish Pradhan, Mary E. Lofton, Bennett J. McAfee, Adrienne Breef-Pilz, Heather L. Wander, Dexter W Howard, Cayelan C. Carey, Paul Hanson, Anuj Karpatne (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15785.txt.

Introduction

Abhilash Neog and co-authors investigate how to model time series that are sampled irregularly, with different measures missing at different moments. Sensors fail and data collection costs money, so gaps are the norm in fields from lakes to medicine. Common fixes either fill gaps first, baking in guesses, or demand a bespoke model for each task.

Their method, MissTSM, needs no imputation and plugs into many model types. Experiments on semi-synthetic and real datasets, for classification and forecasting alike, probe when it wins. The study also draws broader lessons on how irregular sampling bends results, guidance for anyone modeling messy real-world streams.

Catalog record

CREATORAbhilash Neog, Arka Daw, Sepideh Fatemi, Medha Sawhney, Aanish Pradhan, Mary E. Lofton, Bennett J. McAfee, Adrienne Breef-Pilz, Heather L. Wander, Dexter W Howard, Cayelan C. Carey, Paul Hanson, Anuj Karpatne
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-15785

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Investigating a Model-Agnostic and Imputation-Free Approach for Irregularly-Sampled Multivariate Time-Series Modeling Abhilash Neog, Arka Daw, Sepideh Fatemi, Medha Sawhney, Aanish Pradhan, Mary E. Lofton, Bennett J. McAfee, Adrienne Breef-Pilz, Heather L. Wander, Dexter W Howard, Cayelan C. Carey, Paul Hanson, Anuj Karpatne Abstract Modeling Irregularly-sampled and Multivariate Time Series (IMTS) is crucial across a variety of applications where different sets of variates may be missing at different time-steps due to sensor malfunctions or high data acquisition costs. Existing approaches for IMTS either consider a two-stage impute-then-model framework or involve specialized architectures specific to a particular model and task. We perform a series of experiments to derive insights about the performance of IMTS methods on a variety of semi-synthetic and real-world datasets for both classification and forecasting. We also introduce Miss ing Feature-aware T ime S eries M odeling (MissTSM) or MissTSM, a simple model-agnostic and imputation-free approach for IMTS modeling. We show that MissTSM shows competitive performance compared to other IMTS approaches, especially when the amount of missing values is large and the data lacks simplistic periodic structures–conditions common to real-world IMTS applications. Reviewed on OpenReview: https://openreview.net/forum?id=HgJ0DMVAA3 † † footnotetext: *Corresponding author: [email protected], [email protected] † † footnotetext: Code available at: https://github.com/abhilash-neog/SparseTimeSeriesModeling 1 Introduction Deep Learning for modeling multivariate Time-Series (MTS) is a rapidly growing field, with two major downstream tasks: forecasting and classification. Research ( Dong et al., 2024 ; Nie et al., 2022a ; Liu et al., 2023 ) in this field has been fueled by the availability of benchmark MTS datasets spanning diverse applications such as electric load forecasting and health monitoring containing fixed sets of variates regularly sampled over time. However, real-world MTS applications are plagued by missing values occurring over arbitrary sets of variates at every time-step (e.g. due to sensor malfunctions), resulting in Irregularly-sampled MTS (IMTS) datasets. IMTS modeling is particularly challenging because the misalignment of variates across time impairs transformer models that assume a fixed set of variates to be observed at every time-step A common approach for IMTS modeling is to use a two-step framework where we first use imputation methods ( Ahn et al., 2022 ; Batista et al., 2002 ) to fill in missing values based on observed data, followed by feeding the imputed time-series to an MTS model (see Figure 1 ). Note that the choice of imputation method is agnostic to the MTS model, making it “model-agnostic.” However, the effectiveness of this framework relies on the quality of performed imputation, which can degrade if the time-series lacks periodic structure or if the imputation method is overly simplistic. Imputation can also introduce artificial patterns or artifacts into the data, which MTS models may interpret as genuine trends or observations. Moreover, deep learning-based imputation methods require training, which adds to the overall computational cost of IMTS modeling. Imputation-free approaches for IMTS have also been developed in recent literature ( Che et al., 2018 ; Rubanova et al., 2019 ) , that involve specialized architectures to handle missing values in time-series for specific downstream tasks such as classification (see Figure 1 ) However, these approaches have been empirically shown to struggle with capturing long-term temporal dependencies that are central to the problem of forecasting, are difficult to parallelize, and incur high computational costs. Furthermore, existing imputation-free IMTS approaches are not model-agnostic, i.e., they have been developed as specialized model architectures that cannot be used as a generic wrapper with latest advances in MTS models, restricting their adaptability and performance. Figure 1 : We investigate the relative importance of three categories of approaches for modeling irregular and multivariate time-series: (1) imputation-based approaches, (2) model-agnostic and imputation-free approaches (proposed MissTSM layer), and (3) imputation-free approaches involving specialized architectures. Given the complementary strengths and weaknesses of existing approaches in IMTS modeling, we ask the following question - can we develop a model-agnostic and imputation-free approach for IMTS modeling that can be used in a variety of downstream tasks (e.g., forecasting and classification) ? To address this question, we analyze the prevailing strategies for converting time-series into tokens in existing transformer-based MTS models. There are two primary ways for converting time series into tokens: (a) treating each time step (all variates) as a token, or (b) considering each variate (all time steps) as a token ( Liu et al., 2023 ) . While these strategies work well for regular MTS data, they are not suited to handle IMTS data because we may be missing some variates at a time step or some time-steps for a variate, making the computation of tokens infeasible. In contrast, we explore a different perspective for creating time-series tokens: independently embedding each combination of time-step and variate as a token. Time-variate combinations with missing values can then be handled using masked cross-attention without performing any explicit imputation. Building on this intuition, we introduce MissTSM, a simple model-agnostic and imputation-free approach for IMTS modeling, designed as a “plug-and-play” layer that can be integrated into any backbone MTS model to handle IMTS data. The advantage of such an approach is that it (a) does not introduce any imputation artifacts , and (b) can act as a wrapper around any MTS model. In this work, we make the following contributions: (1) We introduce MissTSM, a model-agnostic and imputation-free approach; (2) We conduct a comprehensive experimental study on a variety of datasets for both classification and forecasting tasks, using synthetic masking techniques as well as real-world occurrence of missing values. This study investigates (a) sensitivity of imputation-based frameworks on the choice of imputation technique and the nature of missing values , and (b) the performance of IMTS approaches as the fraction of missing values varies ; (3) We demonstrate that MissTSM achieves competitive performance compared to other IMTS approaches, especially when the amount of missing values is large and the data lacks simplistic periodic structures - conditions common to real-world IMTS applications. 2 Related Works Time-series Forecasting. With the introduction of attention mechanisms via transformer models ( Vaswani et al., 2017 ) , a number of transformer-based time-series models have been developed in the last few years ( Wu et al., 2021 ; Nie et al., 2022b ; Dong et al., 2024 ; Liu et al., 2023 ) . While Transformer-based models have shown great promise, recently there has been a strong interest in exploring the use of simple linear models for time-series forecasting as well ( Zeng et al., 2023 ; Ekambaram et al., 2023 ) . In addition, with the rise of self-supervised learning-based models such as masked auto-encoders (MAEs) ( He et al., 2022 ) , a new category of MAE-style time-series models have emerged ( Dong et al., 2024 ) that have received a lot of recent interest owing to their ability of learning both low-level and high-level representations for varied downstream tasks such as forecasting and classification. However, while these methods can deal with missing values in the temporal domain, they are unable to handle missing values across both variates and time. Imputation Methods. Traditionally, most imputation techniques for handling missing values in time-series have been based on statistical approaches ( Fung, 2006 ; Batista et al., 2002 ; Dempster et al., 1977 ; Mnih and Salakhutdinov, 2007 ) . In

Read the complete text