Variational Smoothing and Inference for SDEs from Sparse Data with Dynamic Neural Flows

Yu Wang, Arnab Ganguly · 2026

Read Variational Smoothing and Inference for SDEs from Sparse Data with Dynamic Neural Flows by Yu Wang, Arnab Ganguly (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05606.txt.

Introduction

Many real systems are only glimpsed through sparse, noisy measurements. Yu Wang and Arnab Ganguly infer the hidden paths and parameters of stochastic differential equations from such data. Classical smoothing methods suffer from path degeneracy and scale poorly, so the authors take a variational route.

They describe the posterior equation through a backward score, a gradient field defined by a Kolmogorov equation, and learn it with neural networks. A likelihood objective then tunes the model parameters in an EM-style loop. On nonlinear test systems, the method stayed accurate with very few observations and scaled better than Markov chain Monte Carlo.

Catalog record

CREATORYu Wang, Arnab Ganguly
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.05606

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Variational Smoothing and Inference for SDEs from Sparse Data with Dynamic Neural Flows Yu Wang, Arnab Ganguly Abstract Stochastic differential equations (SDEs) provide a flexible framework for modeling temporal dynamics in partially observed systems. A central task is to calibrate such models from data, which requires inferring latent trajectories and parameters from sparse, noisy observations. Classical smoothing methods for this problem are often limited by path degeneracy and poor scalability. In this work, we developed a novel method based on characterization of the posterior SDE in terms of conditional backward-in-time score defined as the gradient of a function solving a Kolmogorov backward equation with multiplicative updates at observation times. We learn this conditional score using neural networks trained to satisfy both the governing PDE and the observation-induced jump conditions, thereby integrating continuous-time dynamics with discrete Bayesian updates. The resulting score induces a posterior SDE with the same diffusion coefficient but a modified drift, enabling efficient posterior trajectory sampling. We further derive a likelihood-based objective for learning the SDE parameters, yielding an evidence lower bound (ELBO) for joint state smoothing and parameter estimation. This leads to a variational EM-style procedure, where the neural conditional score is optimized to approximate the smoothing distribution, followed by a maximization step over the SDE parameters using samples from the induced posterior. Experiments on nonlinear systems demonstrate accurate and stable inference with a very few observations demonstrating significant improved scalability compared to classical MCMC methods. 1 Introduction Stochastic differential equations (SDEs) provide a flexible framework for modeling dynamical systems evolving under uncertainty. They have also emerged as a cornerstone of generative modeling, where they serve to transform simple noise to complex data distributions 32 ; 33 ; 30 ; 8 ; 26 ; 25 . In many applications, however, the latent state process is not directly observed; instead, one has access only to sparse and noisy measurements at discrete time points. A central problem is therefore to infer both the unknown parameters governing the dynamics and the distribution of latent trajectories conditioned on these observations, commonly referred to as the smoothing or posterior distribution . Parametric inference for SDEs has been extensively studied (e.g., see 36 ; 21 ; 13 ; 29 ; 1 ; 22 ; 17 ; 4 ; 14 ; 2 ; 18 ; 20 ; 9 ; 3 ; 11 ; 23 ; 7 ; 34 ; 6 ; 16 ; 15 ). A typical approach to this problem involves discretizing the latent or prior SDE in time and performing inference on the resulting high-dimensional latent state vector. Methods based on particle filtering, or Markov chain Monte Carlo (MCMC) are widely used 13 ; 18 ; 14 ; 19 ; 35 , but often suffer from path degeneracy, poor mixing, and limited scalability, especially when observations are sparse or and the latent dynamics is highly nonlinear. In this work, we take a different perspective based on a path-space characterization of the smoothing distribution. We utilize the fact that the smoothing distribution over trajectories can be described as the law of a diffusion process, whose drift is the prior drift modified by a backward-in-time conditional score . The latter is defined as the spatial gradient of the logarithm of a message function solving a backward Kolmogorov equation with multiplicative updates at observation times. This yields a continuous-time formulation of Bayesian conditioning at the level of path measures. Building on this characterization, we develop a scalable method for joint smoothing and parameter estimation. The key idea is to approximate the backward conditional score by a family of neural networks trained to satisfy the governing partial differential equations (PDEs) together with the observation-induced jump conditions. This yields a tractable approximation of the posterior SDE, from which approximate trajectories can be efficiently sampled without resorting to high-dimensional discretizations of the prior process. We next derive a likelihood-based objective for learning the SDE parameters in the form of an evidence lower bound (ELBO). This leads to a variational Expectation Maximization (EM)-type procedure: for a given iterate of the SDE parameter, the E-step consists of training the neural network to approximate the conditional score and and using it to define a tractable posterior SDE whose law is the approximate smoothing distribution. Samples from this induced SDE are then used to compute a Monte Carlo approximation of the corresponding ELBO, thereby approximating the intractable conditional expectation required in the EM objective. In the M-step, the next iterate of SDE parameter is obtained by maximizing this Monte Carlo ELBO . We summarize the main components of our approach as follows: • Characterization of smoothing: We utilize a representation of the smoothing distribution as the law of a posterior SDE whose drift equals the prior drift plus a correction given by a backward-in-time conditional score. • Neural approximation of backward score: We approximate this conditional score using neural networks, yielding a tractable posterior SDE that can be efficiently sampled without resorting to MCMC. • Variational EM: We construct a variational EM procedure based on a Monte Carlo approximation of the ELBO using reparameterized samples from the learned posterior SDE, for joint smoothing and parameter estimation. The resulting framework integrates continuous-time stochastic dynamics, partial observation, and variational inference in a novel unified manner. In contrast to classical discretization-based approaches, our method operates directly at the level of the smoothing distribution, leading to improved stability and scalability, particularly in regimes with sparse observations. Empirical results on nonlinear systems demonstrate accurate trajectory reconstruction and parameter estimation, with significant gains over standard MCMC-based methods. 2 Model and problem setup Let (\Omega,\mathcal{F},\{\mathcal{F}_{t}\},\mathbb{P}) be a filtered probability space supporting a d -dimensional Brownian motion W . Our model consists of the the latent SDE: dX(t)=\drft(\kappa,X(t))\,dt+\dffun(X(t))\,dW(t),\quad X(0)\sim\initdist, (1) where \kappa\in K_{0}\subset\R^{d_{0}} is the model parameter, and the parameter set K_{0} is assumed to be compact. We assume that for each \kappa , the SDE admits a strong unique solution (e.g., under linear growth and local Lipschitz conditions on the coefficients). We denote a sample path until time T by X^{\kappa}_{[0,T]} . Data: The latent process X is not observed directly. Our data comprises noisy, partial measurements \by_{1:M_{0}}=(y_{1},\dots,y_{M_{0}}) at times \{t_{m}\}_{m=1}^{M_{0}}\subset[0,T] — a realization of \bm{\obsY}_{t_{1}:t_{M_{0}}}=(\obsY(t_{1}),\obsY(t_{2}),\ldots,\obsY(t_{M_{0}})) . We assume, without loss of generality, t_{M_{0}}=T . For each observation time t_{m} , \obsY(t_{m}) is an \R^{d_{0}} -valued random variable with d_{0}\leq d satisfying \displaystyle\obsY(t_{m})|X(t_{m})=x_{m}\ \sim\ \obsden(\cdot|x_{m}). A widely used observation model is the linear Gaussian model: \displaystyle\obsY(t_{m})=\omat X(t_{m})+\vep_{m},\quad\text{ with }\vep_{m}\stackrel{{\scriptstyle iid}}{{\sim}}\No_{d_{0}}(0,\Sigma_{\mrm{noise}}), (2) where G\in\R^{d_{0}\times d} and the covariance matrix of the noise, \Sigma_{\mrm{noise}} , is p.d. In this case, the conditional observation density \obsden is given by \obsden(\cdot|\procX(t_{m})=x_{m})=\No_{d_{0}}(\cdot|\omat x_{m},\Sigma_{\mrm{noise}}) . Goal: Given \by_{1:M_{0}} , our objectives are: (i) to approximate the smoothing (or posterior) distribution \Post^{(\kappa)}(\cdot\mid\by_{1:M_{0}}) defined as the conditional distribution of full latent trajectory X_{[0,T]} given \bm{\obsY}_{t_{1}:t_{M_{0}}}=\by_{1:M_{0}} , and (ii) to estimate \kappa . We emphasize that \Post^{(\kappa)}(\cdot|\by_{1:M_{0}}) and \Pre^{(\kappa)}\dfeq\mrm{Law}(X_{[0,T]}) are probability measures on the path space C([0,T],\R^{d}) , rather than a collection of marginal distributions at discrete times; specifically, for a measurable set \cal{A}\subset C([0,T],\R^{d}), \PP(X_{[0,T]}\in\cal{A})=\Pre^{(\kappa)}(\cal{A}),\quad\PP(X_{[0,T]}\in\cal{A}\mid\bm{\obsY}_{t_{1}:t_{M_{0}}}=\by_{1:M_{0}})=\Post^{(\kappa)}(\cal{A}|\by_{1:M_{0}}). Moreover, \Post^{(\kappa)}(\cdot\mid\by_{1:M_{0}}) is absolutely continuous with respect to \Pre^{(\kappa)} satisfying \displaystyle\f{d\Post^{(\kappa)}(\cdot\mid\by_{1:M_{0}})}{d\Pre^{(\kappa)}}(X_{[0,T]})=\f{\prod_{m=1}^{M_{0}}\obsden(y_{m}\mid X(t_{m}))}{e^{\ell(\kappa\mid\by_{1:M_{0}})}}, (3) where \displaystyle\ell(\kappa\mid\by_{1:M_{0}})=\ln\EE_{\kappa}\!\left[\prod_{m=1}^{M_{0}}\obsden(y_{m}\mid X(t_{m}))\right]. (4) is the log-likelihood function of \kappa given the observations. Thus the maximum likelihood estimate (MLE)

Read the complete text