Universal Time Series Generation with Neural Controlled Differential Equations

Torben Berndt, Elyes Farjallah, Leif Seute, Raeid Saqur, [10pt] Benjamin Walker, Jan Stühmer, [10pt] Heidelberg Institute for Theoretical Studies, Heidelberg, Germany, [0.4em] Equal first authorship. Equal senior authorship · 2026

Read Universal Time Series Generation with Neural Controlled Differential Equations by Torben Berndt, Elyes Farjallah, Leif Seute, Raeid Saqur, [10pt] Benjamin Walker, Jan Stühmer, [10pt] Heidelberg Institute for Theoretical Studies, Heidelberg, Germany, [0.4em] Equal first authorship. Equal senior authorship (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28507.txt.

Introduction

Torben Berndt and five colleagues extend sequence universality from label tasks to generative time-series work with SLiCEs, structured linear controlled differential equations. They prove that maximally expressive SLiCEs can mimic the path laws of continuous causal pushforwards, making them universal time-series generators. Their SLiCE model applies this theory to flow matching on path space. Greater power improves odds-based forecasts and later tasks while keeping time-based perks, such as handling any grid, even irregular ones.

Catalog record

CREATORTorben Berndt, Elyes Farjallah, Leif Seute, Raeid Saqur, [10pt] Benjamin Walker, Jan Stühmer, [10pt] Heidelberg Institute for Theoretical Studies, Heidelberg, Germany, [0.4em] Equal first authorship. Equal senior authorship
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89483
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28507

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Universal Time Series Generation with Neural Controlled Differential Equations Torben Berndt, Elyes Farjallah, Leif Seute, Raeid Saqur, [10pt] Benjamin Walker, Jan Stühmer, [10pt] Heidelberg Institute for Theoretical Studies, Heidelberg, Germany, [0.4em] Equal first authorship. Equal senior authorship Abstract Recent work on the sequence universality of State Space Models (SSMs) has introduced efficient, maximally expressive continuous-time approaches for time-series modelling. While these works focus on discriminative settings, we extend this perspective to generative time-series modelling by proving that maximally expressive Structured Linear Controlled Differential Equations (SLiCEs) are universal time-series generators, in the sense that they can approximate the induced path laws of continuous causal pushforwards on compact latent sets in W ∞ W_{\infty} . Building on these theoretical results, we propose Generative SLiCEs (G-SLiCEs) , a maximally expressive continuous-time model for flow matching on path-space. Empirically, we show that expressivity improves performance in probabilistic forecasting and downstream tasks, while retaining the advantages of continuous-time models such as generalising to arbitrary observation grids. This is particularly beneficial for irregular grids, where fixed-grid models often struggle. 1 1 1 The source code is available at: https://github.com/hits-mli/g-slices . . 1 Introduction In probabilistic forecasting, capturing the full conditional distribution over future events is crucial, for example, when predicting extreme weather phenomena [ 51 ] , electricity demand [ 61 ] , or traffic [ 52 ] . Probabilistic forecasting is therefore inherently generative, and recent work has increasingly applied generative machine learning methods, such as diffusion models and flow matching [ 59 , 42 , 63 ] , to this task [ 56 , 62 , 3 , 11 , 37 , 38 ] . However, these models typically rely on sequence-model backbones that are not maximally expressive, which can limit their performance. A parallel line of work develops expressive, parallel-in-time state space models (SSMs) by imposing structures on transition matrices, including block diagonal [ 22 ] and diagonal-plus-low-rank constructions [ 69 , 67 , 58 ] . Recent work unifies these approaches in a continuous-time framework building on Linear Controlled Differential Equations and characterise their universality [ 15 , 65 ] . We extend this theory to generative settings, where distributions on path space are modelled as pushforwards of latent path laws. Building on this perspective, we generalise current flow-based approaches for probabilistic forecasting from sequence-to-sequence mappings to path-to-path mappings. To address both the expressivity gap and the discrete-time formulation of existing state-of-the-art models, we propose Generative Structured Linear Controlled Differential Equations (G-SLiCEs): a maximally expressive continuous-time model for generative time-series modelling. Main contributions We introduce a new generative approach for time-series data – Generative Structured Linear Controlled Differential Equations (G-SLiCEs) – that achieves state-of-the-art performance and is robust under grid shifts. This relies on two key insights: (1) we formulate flow matching on path space using maximally expressive Structured Linear CDE backbones, and (2) we prove that pathwise expressivity implies universality for continuous causal pushforwards of compactly supported latent path laws, linking discriminative expressivity of causal path-to-path models to generative capabilities (Theorem 2 ). We demonstrate that G-SLiCEs outperform current state-of-the-art models on a comprehensive probabilistic forecasting benchmark, which we attribute to their universality, and show that the continuous-time formulation improves robustness to sampling-grid shifts Further, we provide a concrete example showing that transition structure has provable expressivity consequences in the generative setting. 1.1 Related work Flow matching Flow matching [ 42 , 1 ] has been proposed as powerful generative modelling technique on continuous, finite-dimensional Euclidean spaces and has since been generalised to Riemannian manifolds [ 13 ] , discrete data [ 28 ] and functionals [ 36 ] . These approaches do not, to our knowledge, yield distributional universality results on path space. Generative models for probabilistic time-series forecasting. Classical neural baselines either parameterise the predictive distribution explicitly (DeepAR [ 57 ] , TFT [ 40 ] , autoregressive flows over RNN backbones [ 56 ] ) or are deterministic and combined with quantile heads (WaveNet [ 46 ] , PatchTST [ 45 ] , DLinear [ 70 ] ). Generative approaches replace this distributional head with a learned density model. Variational auto-encoders (VAEs) such as TimeVAE [ 18 ] were explored first, followed by diffusion models, including CSDI [ 62 ] , SSSD [ 2 ] , TSDiff [ 37 ] , and the stochastic-process formulation of 11 . Most recently, TSFlow [ 38 ] advances this line of work by employing flow matching. Expressivity of state space models. Recent work aims to make parallel-in-time sequence models more expressive while preserving efficiency. One direction parallelises non-linear RNNs by casting recurrence as a fixed-point problem solved with Newton-type methods [ 41 , 32 ] . Another develops input-dependent linear RNNs and state-space models, including input-dependent block-diagonal LRNNs [ 22 ] , DeltaNet variants [ 69 , 67 , 58 ] , Mamba [ 33 ] , Mamba-2 [ 17 ] , RWKV-7 [ 49 ] , HGRN-2 [ 54 ] , mLSTM [ 7 ] , Gated Linear Attention [ 68 ] , Gated Random Feature Attention [ 50 ] , Gated Slot Attention [ 71 ] , TTT-Linear [ 60 ] , and Titans [ 8 ] . These models usually use diagonal or diagonal-plus-low-rank transition matrices. 2 Universality in the context of time series generation Figure 1 : Path-space flow matching with G-SLiCE. G-SLiCE models probabilistic time-series generation as a continuous flow on path space. At inference time, an initial path X^{(0)}\sim\mu_{0} is transported by the learned flow \varphi_{\theta,s} , producing a terminal sample X^{(1)}\sim(\varphi_{\theta,1})_{\#}\mu_{0} . Each inference panel shows samples from the evolving distribution on path space. During training, the SLiCE vector field is trained to match the path-space displacement at the interpolated path X^{(s)} , which is constructed from prior and data paths sampled from the joint distribution q . We begin with the path-to-path model classes used in this paper. We then define the distributional notion of expressivity used for time series generation and show how it applies to G-SLiCEs. 2.1 Linear NCDEs and SLiCEs Fix a time interval [t_{0},t_{n}] . For d\in\mathbb{N} , let \mathcal{X}(d)=C^{1,0}([t_{0},t_{n}],\mathbb{R}^{d}) denote the space of absolutely continuous, time-augmented input paths used by the model, all starting from the same point, equipped with the 1 -variation topology. Let \mathcal{Y}(d)=C([t_{0},t_{n}],\mathbb{R}^{d}) denote the output path space, equipped with the supremum metric \rho_{\infty}(Y,\widetilde{Y})=\sup_{t\in[t_{0},t_{n}]}\|Y_{t}-\widetilde{Y}_{t}\|_{2} . A map T:\mathcal{X}(d_{X})\to\mathcal{Y}(d_{y}) is causal if X|_{[t_{0},t]}=\widetilde{X}|_{[t_{0},t]} implies T(X)_{t}=T(\widetilde{X})_{t} for every t\in[t_{0},t_{n}] . Let X\in\mathcal{X}(d_{X}) denote an input path and let \omega^{X}\in C^{1,0}([t_{0},t_{n}],\mathbb{R}^{d_{\omega}}) denote a deterministic causal augmentation of X . In applications, \omega^{X} may include time, observed values, masks, lags, and other deterministic features. Throughout this subsection, integrals are understood in the Riemann–Stieltjes sense, which is well defined for the absolutely continuous controls used by the model. Definition 2.1 (NCDE) . An NCDE is a causal path-to-path map X\mapsto z^{\theta}(X) defined by h_{t_{0}}=\xi_{\varphi}(X_{t_{0}}),\qquad h_{t}=h_{t_{0}}+\int_{t_{0}}^{t}g_{\theta}(h_{s})\,d\omega^{X}_{s},\qquad z_{t}=r_{\psi}(h_{t}), (1) where h_{t}\in\mathbb{R}^{d_{h}} , z_{t}\in\mathbb{R}^{d_{z}} , \xi_{\varphi} is a learnable initialisation map, g_{\theta}(h)\in\mathbb{R}^{d_{h}\times d_{\omega}} is a learnable vector field, and r_{\psi} is a learnable readout. Definition 2.2 (Linear NCDE) . A Linear NCDE is an NCDE whose vector field is linear in the hidden state. Equivalently, there are matrices A_{\theta}^{1},\ldots,A_{\theta}^{d_{\omega}}\in\mathbb{R}^{d_{h}\times d_{h}} such that h_{t_{0}}=\xi_{\varphi}(X_{t_{0}}),\qquad h_{t}=h_{t_{0}}+\int_{t_{0}}^{t}\sum_{i=1}^{d_{\omega}}A_{\theta}^{i}h_{s}\,d\omega_{s}^{X,i},\qquad z_{t}=r_{\psi}(h_{t}). (2) The dynamics are linear in h ,

Read the complete text