Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs

Benjamin Walker, Alexandre Bloch, Lingyi Yang, Sam Morley, Terry Lyons · 2026

Read Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs by Benjamin Walker, Alexandre Bloch, Lingyi Yang, Sam Morley, Terry Lyons (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30213.txt.

Introduction

Benjamin Walker and four co-authors present faithful embeddings of irregular and asynchronous data for online Log-NCDEs. Interpolation- and imputation-based embeddings rebuild a path, making models sensitive to rebuild quality. That step is needless, they show: under mild conditions, model power transfers when the embedding is continuous and injective. Building on the rectilinear control path, they present a Log-NCDE embedding logging observations as steps, built into log-signatures over query spans. Tests show speed and toughness on irregular, async and sparse observations.

Catalog record

CREATORBenjamin Walker, Alexandre Bloch, Lingyi Yang, Sam Morley, Terry Lyons
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85243
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30213

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs Benjamin Walker, Alexandre Bloch, Lingyi Yang, Sam Morley, Terry Lyons Abstract Continuous-time models are a natural choice for irregular and asynchronous data. A central design choice is how to embed discrete observations into continuous time. Interpolation- and imputation-based embeddings reconstruct a continuous observation path, making the model sensitive to the choice of reconstruction. We show that this reconstruction step is unnecessary; under mild conditions, compact-set universality on the model input space transfers to the data space whenever the embedding from data to input is continuous and injective. Guided by this result, and building on the rectilinear control path for Neural Controlled Differential Equations (NCDEs), we introduce a continuous and injective embedding for Log-NCDEs, a universal class of continuous-time models. Our approach records observations as increments and composes them over arbitrary query intervals to directly form log-signatures. This provides interval-level summaries without first interpolating the observed variables, while supporting online computation. Experiments on synthetic controlled dynamics and real-world time-series datasets show that the representation is accurate, efficient, and robust to irregular, asynchronous, and sparse observations. 1 Introduction Figure 1 : Faithful representations of irregular data. Imputation and interpolation complete the data before modelling, introducing values or dynamics that may be unrealistic. Our approach records the observed stream as time-stamped increments and groups them into interval summaries, giving a faithful (continuous and injective) representation of the discrete and irregular data. Sequence architectures, such as Transformers ( 76 ) , Structured State-Space Models (SSMs) ( 29 ; 28 ) , and Recurrent Neural Networks (RNNs), are built around discrete updates on a prescribed grid ( 25 ; 32 ) . This discretisation is natural for text and regularly sampled time series, but is limiting in interactive and physical settings, where observations arrive irregularly, actions are taken asynchronously, and the underlying environment continues to evolve between measurements ( 47 ; 31 ) . Continuous-time machine learning provides a natural framework for such data ( 57 ; 64 ) . Given a continuous-time embedding of the observations, the hidden state can evolve continuously and be queried at arbitrary times ( 37 ) . A central design choice is how to construct this embedding. One common strategy is to complete the data by imputation, interpolation, or learned interpolation before applying a model ( 10 ; 72 ; 73 ; 9 ; 81 ) . Neural Controlled Differential Equations (NCDEs) and ContiFormer are prominent examples of this approach ( 37 ; 52 ; 36 ; 13 ) . Other models avoid explicit interpolation by combining continuous latent dynamics with discrete updates, as in ODE-RNNs and GRU-ODE-Bayes ( 65 ; 24 ) . These approaches differ in how they represent discrete, irregular, and asynchronous data in continuous time. The central principle of this paper is that a continuous-time representation of discrete observations should preserve only the information that was observed, rather than guess the unobserved behaviour between measurements. The latter should form part of the modelling as opposed to part of the representation. When guesses are built into the representation, the model may consume unrealistic inputs, as illustrated in Figure 1 . 52 studied control-path embeddings for NCDEs, exploring properties such as online, smooth, bounded, and unique, or equivalently injective. For Log-NCDEs, smoothness is no longer intrinsic, since the model uses log-signatures on intervals rather than pointwise values of a control path ( 78 ) . We prove that, when the data and model input spaces are Hausdorff, universality of the model on compact subsets of its input space transfers to the data space as long as the data embedding is continuous and injective. We call such embeddings faithful . Since NCDEs and Log-NCDEs are universal on path space ( 37 ; 78 ) , this result applies to both model classes for faithful observation embeddings. This leads to three desiderata for a Log-NCDE control-path embedding. It should be faithful, computable online, and support efficient calculation of the embedded path’s log-signature over arbitrary query intervals. We construct such an embedding by mapping each observation to a local log-signature contribution that records which channels were observed and their values. These contributions compose over arbitrary query intervals, so the representation can be built directly from the observations in each interval. Continuously observed variables, such as time, contribute between observations. Figure 1 illustrates our approach. This yields a faithful representation of the observed stream without interpolation, decouples the input sampling times from the output grid, and supports parallel-in-time computation. The resulting algebraic formulation can also accommodate observations beyond first-order channel values, such as the signed area. In the first-order case with time as the only continuously observed variable, the continuous path corresponding to our embedding is the rectilinear control path introduced for NCDEs by 52 . Our construction can therefore be viewed as a Log-NCDE-compatible generalisation of the rectilinear scheme. Our contributions are as follows: • We formalise the notion of a faithful embedding and prove that faithful embeddings transfer universality from compact subsets of one Hausdorff space to another in Section 2.1 . • We introduce a faithful control-path embedding for Log-NCDEs whose interval log-signatures can be computed efficiently over arbitrary intervals in Section 2.4 . • We demonstrate the computational benefits and robustness to irregularity of the resulting models on synthetic controlled systems and real-world time-series classification in Section 3 . 2 Faithful Embeddings of Irregular Time Series for Log-NCDEs 2.1 Faithful Embeddings Before modelling, data must be embedded into the model input space. We call such an embedding faithful if it is continuous and injective, thus ensuring that nearby data remain nearby after embedding and that distinct data remain distinguishable. Together, these conditions are sufficient for universality to transfer from the downstream model to the original data space, as formalised by the following lemma. Lemma 1 (Universality Transfer) . Let \mathcal{X} and \mathcal{P} be Hausdorff topological spaces, and let \varphi:\mathcal{X}\to\mathcal{P} be a continuous injection. Suppose \mathcal{F}\subseteq C(\mathcal{P};\mathbb{R}) is a family of continuous real-valued functions such that for every compact K\subseteq\mathcal{P} , the restrictions \mathcal{F}|_{K} are dense in C(K;\mathbb{R}) in the uniform topology. Then for every compact K_{0}\subseteq\mathcal{X} , the restrictions (\mathcal{F}\circ\varphi)|_{K_{0}} are dense in C(K_{0};\mathbb{R}) in the uniform topology, where \mathcal{F}\circ\varphi:=\{f\circ\varphi:f\in\mathcal{F}\} . The proof is in Appendix A . With \mathcal{X} the data space, \mathcal{P} the model input space, and \varphi the embedding, Lemma 1 shows that faithful embeddings transfer universality back to the original data. This recovers the usual signature-universality argument: Stone–Weierstrass gives universality on signature space, and injectivity transfers it back to paths ( 75 ; 42 ) . Appendix A also notes the obvious converse obstruction: if \varphi identifies two data objects, no downstream model can separate them. Appendix B discusses the scope of this universality statement and its distinction from sample efficiency or optimisation guarantees. We now construct a faithful embedding of an irregular observation stream as a path. The construction is tailored to Log-NCDEs, which require interval log-signatures rather than pointwise values of an interpolated path. It therefore computes the required log-signatures directly from the observations, without interpolating the data. The procedure follows the signature-computation viewpoint of 51 . We begin with an introduction to the tensor algebra, signatures, and log-signatures. 2.2 Tensor

Read the complete text