Deep ZakaiJ: Structured Filtering for Jump-Diffusion Time Series Forecasting
Yan Leng, Thibaut Mastrolia, Hao Wang · 2026
Read Deep ZakaiJ: Structured Filtering for Jump-Diffusion Time Series Forecasting by Yan Leng, Thibaut Mastrolia, Hao Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24548.txt.
Introduction
Yan Leng, Thibaut Mastrolia, and Hao Wang present Deep ZakaiJ, a 2026 latent-state model for jump-diffusion systems. It embeds the Zakai filtering equation for half-seen systems in a neural encoder-decoder design. The encoder updates beliefs about the hidden state by Strang splitting into three steps: prior propagation, diffusion innovation, and jump innovation. The decoder keeps smooth dynamics and sudden shocks apart in a structured jump-diffusion model. On synthetic, financial, and ocean data, the model gave better distributional forecasts while staying competitive on point accuracy.
Catalog record
| CREATOR | Yan Leng, Thibaut Mastrolia, Hao Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83627 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24548
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep ZakaiJ: Structured Filtering for Jump-Diffusion Time Series Forecasting Yan Leng, Thibaut Mastrolia, Hao Wang Abstract Time series driven by unobserved latent states frequently exhibit abrupt jump discontinuities whose timing and magnitude cannot be predicted from observed history alone. Classical jump-diffusion models offer a principled mathematical framework but assume rigid parametric forms, while recent neural jump models operate on fully observed trajectories without inferring the hidden states that govern the dynamics. We propose Deep ZakaiJ , a latent-state model for partially observed jump-diffusion systems that embeds the Zakai nonlinear filtering equation into a neural encoder–decoder architecture. The encoder recursively updates a belief over the latent state via Strang splitting into three interpretable substeps: prior propagation, diffusion innovation, and jump innovation, yielding a differentiable, first-order-accurate approximation of the exact filtering evolution. The decoder is a structured jump-diffusion model explicitly conditioned on the filtered belief, preserving the separation between continuous dynamics and discontinuous shocks. On synthetic, financial, and oceanographic datasets, Deep ZakaiJ improves distributional forecasts while remaining competitive in point accuracy, achieving calibrated predictive intervals and recovering interpretable latent structure in synthetic and qualitative case studies. 1 Introduction Empirical time series rarely evolve smoothly; instead, they are frequently disrupted by abrupt jumps [ 1 , 52 ] . These discontinuities span diverse systems, from financial assets suffering sudden price shocks due to macroeconomic news [ 43 ] , to oceanic wave heights spiking drastically during severe storms [ 57 , 56 ] . Because these extreme dislocations are usually driven by latent states, predicting such trajectories remains a major open challenge. Classical stochastic models, such as Merton’s jump-diffusions [ 43 ] , stochastic volatility models [ 28 ] or more general Lévy processes [ 10 ] , offer a principled mathematical foundation. While models like Heston explicitly introduce a latent volatility state to drive dynamics, they are limited by rigid parametric assumptions and are structurally restricted from capturing complex latent structure. Beyond standard deep learning time-series models [ 58 , 8 , 38 ] , a growing line of work models sequential data through continuously evolving latent-state dynamics (e.g., Latent ODEs [ 49 ] and Neural CDEs [ 32 ] ), which propagate hidden states via learned vector fields and approximate the latent posterior through amortized variational inference, but treat latent states as implicit representations rather than explicit stochastic objects, lacking a principled Bayesian update mechanism. Models like CRU [ 51 ] and NCDSSM [ 3 ] close this gap by embedding continuous-discrete Kalman filtering within deep sequence models, yet remain restricted to linear-Gaussian dynamics. For time series exhibiting jump discontinuities, where jump times are totally inaccessible stopping times that cannot be predicted from the historical filtration [ 47 , 2 ] , correctly updating latent beliefs demands a nonlinear filtering mechanism that current frameworks cannot provide. In this work, we propose Deep ZakaiJ , a framework that embeds the Zakai nonlinear filtering equation into a neural architecture for structured inference in partially observed jump-diffusion systems. Our main contributions are: (i) A Zakai encoder that discretizes the Zakai equation via Strang splitting into three interpretable steps: prior propagation, diffusion innovation, and jump innovation. The jump innovation step performs likelihood reweighting against a mixture of no-jump and jump-occurred hypotheses, enabling the posterior to react sharply to dislocations. We prove that this encoder achieves first-order global accuracy relative to the exact Zakai evolution. (ii) A structured jump-diffusion decoder that parameterizes drift, volatility, jump intensity, and jump-size distribution with neural networks conditioned on both the observation and the filtered latent belief, learning the predictable compensator so that the residual reduces to a zero-mean martingale, and coupling forecasting directly to the latent filtering state. (iii) Experiments on synthetic, financial, and oceanographic datasets show that Deep ZakaiJ improves distributional quality while remaining competitive in point accuracy, achieving well-calibrated predictive intervals near the nominal 90% level and recovering interpretable latent structure in synthetic and qualitative case studies. 2 Related work Time series forecasting has evolved from classical methods such as ARIMA [ 29 ] and LSTM [ 58 ] to Transformer variants (e.g., PatchTST [ 45 ] , iTransformer [ 40 ] ), probabilistic models like DeepAR [ 50 ] , and pretrained foundation models (e.g., Chronos [ 4 ] , TimesFM [ 16 ] , Moirai-MoE [ 39 ] , Lag-Llama [ 48 ] ). While increasingly powerful, these approaches typically operate in discrete time without explicit mechanisms for jump discontinuities or latent regime inference. Latent-state models. Early variational approaches such as VRNN [ 15 ] , Deep Kalman Filters [ 34 ] , DVBF [ 31 ] , and KVAE [ 20 ] introduce stochastic latent variables into sequential models but train posteriors via ELBOs rather than filtering equations. SRNN [ 21 ] stacks a deterministic RNN with a stochastic latent chain and uses a backward pass for smoothing, SNLDS [ 19 ] collapses discrete switching variables analytically and amortizes the continuous latent posterior to capture regime changes. Deterministic structured SSMs (S4 [ 26 ] , Mamba [ 25 ] ) achieve efficient long-range modeling but do not perform stochastic inference. Continuous-time extensions including Latent SDEs [ 37 ] and GRU-ODE-Bayes [ 17 ] approximate the latent posterior through amortized variational inference, treating latent states as implicit representations. CRU [ 51 ] and NCDSSM [ 3 ] go further by embedding exact continuous-discrete Kalman filtering into deep models, demonstrating clear benefits of filtering, but remain restricted to linear-Gaussian dynamics. Particle-filter-based methods such as FIVO [ 42 ] , VSMC [ 44 ] , SIXO [ 36 ] provide asymptotically exact Bayesian inference through sequential Monte Carlo, while PF-RNN [ 41 ] embeds a differentiable particle filter directly into a recurrent cell. Recent extensions handle regime-switching systems [ 12 ] . However, these methods rely on sample-based approximations and do not exploit the continuous structure of filtering equations. Neural-jump models. Neural Jump SDEs [ 30 ] model temporal point processes via piecewise-continuous latent trajectories where discrete events trigger state-dependent jumps. NJ-ODE [ 27 ] learns the conditional expectation of a stochastic process continuously in time, with convergence guarantees, by evolving a hidden state via Neural ODEs [ 14 ] and applying jump updates at each irregular observation. NeuralMJD [ 22 ] parameterizes a non-stationary Merton jump-diffusion directly from data, and NJDTPP [ 62 ] unifies intensity modeling through neural jump-diffusion SDEs. While NJ-ODE performs a form of implicit filtering through its jump updates, none of these models embeds an explicit nonlinear filtering equation over a latent state to jointly infer hidden regimes and forecast jump-diffusion dynamics. Nonlinear filtering. The Zakai equation [ 59 ] provides an unnormalized formulation of the nonlinear filtering problem that is naturally amenable to operator splitting. The classical splitting method [ 9 , 24 , 18 ] decomposes the Zakai PDE into independent substeps, with recent extensions handling joint diffusion and point-process observations [ 61 ] . The energy-based deep splitting method [ 6 ] approximates high-dimensional splitting substeps with neural networks. Deep ZakaiJ builds on these numerical foundations but differs in purpose: rather than solving a filtering PDE in isolation, it embeds the discretized Zakai equation into an end-to-end architecture, jointly performing nonlinear filtering over a hidden regime and structured conditional jump-diffusion