Still Competitive: Revisiting Recurrent Models for Irregular Time Series Prediction
Ankitkumar Joshi, Milos Hauskrecht · 2025
Read Still Competitive: Revisiting Recurrent Models for Irregular Time Series Prediction by Ankitkumar Joshi, Milos Hauskrecht (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16161.txt.
Introduction
Ankitkumar Joshi and Milos Hauskrecht revisit an unfashionable question: can a simple recurrent network still beat elaborate modern architectures on irregularly sampled time series? Such data, common in healthcare and sensor networks, arrive at uneven intervals, and recent years have produced many complex specialized models for predicting them.
Their answer is GRUwE, a gated recurrent unit whose memory decays through learnable exponential functions of elapsed time. It keeps a Markov state that updates when an observation lands and also resets on a time trigger, supporting both value regression and event prediction in continuous time. On several real-world benchmarks it matches or beats the state of the art, while staying easy to implement, light on tuning, and cheap to run online.
Catalog record
| CREATOR | Ankitkumar Joshi, Milos Hauskrecht |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-16161
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Still Competitive: Revisiting Recurrent Models for Irregular Time Series Prediction Ankitkumar Joshi, Milos Hauskrecht Abstract Modeling irregularly sampled multivariate time series is a persistent challenge in domains like healthcare and sensor networks. While recent works have explored a variety of complex learning architectures to solve the prediction problems for irregularly sampled time series, it remains unclear what are the true benefits of some of these architectures, and whether clever modifications of simpler and more efficient RNN-based algorithms are still competitive, i.e. they are on par with or even superior to these methods. In this work, we propose and study GRUwE: Gated Recurrent Unit with Exponential basis functions, that builds upon RNN-based architectures for observations made at irregular times. GRUwE supports both regression-based and event-based predictions in continuous time. GRUwE works by maintaining a Markov state representation of the time series that updates with the arrival of irregular observations. The Markov state update relies on two reset mechanisms: (i) observation-triggered reset, and (ii) time-triggered reset of the GRU state using learnable exponential decays, to support the predictions in continuous time. Our empirical evaluations across several real-world benchmarks on next-observation and next-event prediction tasks demonstrate that GRUwE can indeed achieve competitive to superior performance compared to the recent state-of-the-art (SOTA) methods. Thanks to its simplicity, GRUwE offers compelling advantages: it is easy to implement, requires minimal hyper-parameter tuning efforts, and significantly reduces the computational overhead in the online deployment. Introduction Multivariate time series and their models are critical for understanding and analyzing real-world dynamical systems. While traditional time-series models assume that the observations are made at regular time intervals, real-world data, such as healthcare records or sensor readings, are often irregularly sampled. In other words, events or measurements associated with these events may occur at different times and in different contexts (e.g., special lab tests in clinical settings may occur only during patient crises). The key challenge here is to devise the models of time series that are able to accurately capture the dependencies among individual variables and events occurring at different times from the observed data. We study this important problem in the context of time series prediction (forecasting) and event prediction tasks. A variety of models ranging from classic statistical and modern deep learning frameworks have been developed to support time-series prediction task for irregular settings. Early approaches replaced the irregularly observed data with regularly spaced observations by inferring their values at the regular time points. Classic statistical auto-regressive models (AR, ARMA, ARIMA) ( Shumway and Stoffer 2017 ) or latent space models, such as, Linear Dynamical Systems (LDS) ( Kalman 1963 ) models could then be applied to support prediction at future regular times. To support prediction on arbitrary future prediction times, various smoothing and interpolation methods were deployed. More recently, classic time-series models have been gradually replaced with various types of modern neural architectures capable of handling irregularly sampled observations ( Siami-Namini, Tavakoli, and Siami Namin 2018 ) . The existing methods include extensions of RNN approaches ( Che et al. 2018 ; Mei and Eisner 2017 ) , and continue with differential equation approaches ( Rubanova, Chen, and Duvenaud 2019 ; De Brouwer et al. 2019 ; Chen et al. 2018 ; Schirmer et al. 2022 ; Becker et al. 2019 ) , attention-based approaches ( Shukla and Marlin 2021 ; Chen et al. 2023 ) , graph-based approaches ( Zhang et al. 2022 ; Zhang et al. ; Yalavarthi et al. 2024 ) and state-space modeling approaches ( Smith, Warrington, and Linderman 2023 ; Gu and Dao 2023 ) . Unfortunately, the benefits of all of the above modern architectures for prediction tasks in irregular observation settings remain poorly understood. This raises a natural question: could carefully designed, simple models that are rooted in simpler architectures, still achieve comparable or even superior predictive performance at significantly lower computational cost? We investigate this question by proposing GRUwE ( G ated R ecurrent U nit w ith E xponential basis functions), pronounced as ”groovy”, a relatively simple yet effective GRU-based architecture that maintains the Markov state representation of the multivariate process that is updated at irregular observation times. GRUwE models the effect of time with the help of learnable exponential basis functions. State updates are affected by both: (i) the arrival of new observations, and (ii) the elapsed time since the previous observation. This dual mechanism allows the model to naturally accommodate influence of past observations and irregular time intervals between them. To make predictions at an arbitrary future time, GRUwE reuses the same exponential basis functions that are used for state updates. Our extensive experiments comparing GRUwE against the SOTA baselines lead to better or comparable predictive performance with significantly lower computational costs. While the proposed GRUwE model supports continuous-time prediction at future time points, its architecture naturally extends to event prediction setting through Temporal Point Process (TPP) framework. We can formulate a Conditional Intensity Function (CIF), that represents an instantaneous rate of event arrival for a TPP by applying a SoftPlus non-linearity at the output of the GRUwE’s regression function. Our experiments demonstrate that GRUwE-based event prediction matches or outperforms state-of-the-art CIF-based TPP models, including: RNN models ( Du et al. 2016 ; Mei and Eisner 2017 ) , attention-based models ( Zhang et al. 2020 ; Zuo et al. 2020 ; Yang, Mei, and Eisner 2022 ) and differential equation models ( Chen, Amos, and Nickel 2020 ) . The main contributions of this work are: • We propose GRUwE, an irregular time series model, that maintains a Markov state representation, defines two reset state update mechanisms to capture dependencies inherent in multivariate time series and supports continuous-time inference. • We perform a comprehensive evaluation of GRUwE against SOTA prediction models on next observation prediction and next event prediction tasks on multiple real-world datasets. Our evaluation reveals that GRUwE performs competitively on both the tasks, while achieving new SOTA on 2 datasets on the next observation prediction, and achieving the highest overall rank in the next event prediction task. • We demonstrate that GRUwE’s compact Markov state representation leads to substantial improvements in computational efficiency during online deployment of irregular time series models. Related Work In this section, we provide a brief overview of the related work. A more comprehensive literature review is included in the Section 2 . We categorize existing approaches to irregularly sampled time series modeling according to their mechanisms for representing the temporal dynamics. Many RNN-based models ( Che et al. 2018 ; Mei and Eisner 2017 ; Du et al. 2016 ; Omi, Aihara et al. 2019 ) have been proposed that use interpolation schemes to represent the temporal dependencies in continuous time domain. On the other hand, ( Chen et al. 2018 ; Kidger et al. 2020 ; Rubanova, Chen, and Duvenaud 2019 ; De Brouwer et al. 2019 ; Schirmer et al. 2022 ; Becker et al. 2019 ; Chen, Amos, and Nickel 2020 ) have been proposed to model temporal dynamics using learned differential equation models. Lastly, alternative approaches ( Zhang et al. 2022 ; Marisca, Cini, and Alippi 2022 ; Tashiro et al. 2021 ; Liu and Hauskrecht 2019 ) , including mTAND ( Shukla and