Inference in stochastic differential equations using the Laplace approximation: Demonstration and examples
Uffe Høgsbro Thygesen, Kasper Kristensen · 2025
Read Inference in stochastic differential equations using the Laplace approximation: Demonstration and examples by Uffe Høgsbro Thygesen, Kasper Kristensen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21358.txt.
Introduction
Uffe Hogsbjo Thygesen and Kasper Kristensen show a fitting method in a 2025 paper on ecology models. Equations with random noise fit natural systems well, since they join first principle rules with real uncertainty and measurement error. Fitting them, though, is often hard and slow.
A classic shortcut, the Laplace approximation, can do the job when used with care, the authors show. Extra hidden states are added between observation times, and step rules estimate how the system moves. Hard cases get full attention: curved dynamics, noise that grows with the state, and nonstandard error shapes. Worked examples show the method in action for ecology time series.
Catalog record
| CREATOR | Uffe Høgsbro Thygesen, Kasper Kristensen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-21358
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Inference in stochastic differential equations using the Laplace approximation: Demonstration and examples Uffe Høgsbro Thygesen, Kasper Kristensen Abstract Stochastic differential equations are a natural framework for dynamic systems and time series in ecology, because they allow for non-linear first-principle knowledge and uncertainty in the dynamics, and can be combined with measurement errors. However, estimation methods are often technically and computationally challenging. Here, we demonstrate that the Laplace approximation is useful for estimating states and parameters in these models, when done correctly. We give special attention to non-linear dynamics, state-dependent noise intensities, and non-Gaussian measurement errors. Our technique adds states between times of observations, approximates transition densities using discretization methods - in the simplest case, the Euler-Maruyama method - and eliminates unobserved states using the Laplace approximation. We demonstrate that consistency requires a particular form of the approximation, and provide different approaches to implementation. Using simulated case studies, we demonstrate that transition probabilities are well approximated, that inference is computationally feasible, and that the framework leads to simple and flexible implementations. Keywords: Time series analysis, transition probabilities, computational statistics,numerical methods. 1 Introduction A common problem in statistical analysis of ecological time series is to combine mechanistic understanding of the system at hand with the ubiquitous presence of uncertainty, which together means that systems are only partially predictable. In all but the simplest cases, mechanistic ecological models contain non-linearities and lead to non-standard statistical distributions, so that the modeler must trade off fidelity to first principles with availability of statistical methods for analysis. A conceptually appealing framework for ecological time series is state-space models ( Auger-Méthé et al., 2021 ; Nabeel et al., 2025 ) based on stochastic differential equations ( Øksendal, 2010 ; Thygesen, 2023 ) ; see section 2 for a detailed model specification. These models can include ecological mechanisms, external inputs, and unknown parameters, while the noise leads to uncertainty in the evolution of the system. Observation equations complement the dynamics by describing how measurements relate to states as well as the statistical properties of measurements. The continuous-time setting allows including mechanistic theory formulated as differential equations, and is convenient when the observations are irregular in time. Although stochastic differential equations may appeal to a theoretical modeler, the technicalities of the analysis can be substantial, and several approaches co-exist ( Fuchs, 2013 ; Nabeel et al., 2025 ) . An important special case is that of linear systems driven by Gaussian noise with constant intensity, and where linear combinations of the state variables are measured subject to Gaussian errors. Then the entire model is Gaussian, and the celebrated Kalman-Bucy filter ( Kalman and Bucy, 1961 ; Simon, 2006 ) provides state estimates for given parameters. The predictive Kalman filter gives residuals which can be used for estimation of system parameters; maximizing the likelihood over parameters corresponds to tuning the one-step predictive filter ( Madsen, 2007 ) . The prediction residuals are also central to model validation ( Holst et al., 2003 ; Thygesen et al., 2017 ) . The Kalman filter can be applied to very high-dimensional systems; even more so with the extension of the Ensemble Kalman Filter ( Evensen, 2003 ) , which allows for spatio-temporal models common in the geosciences. Examples of ecological applications of Kalman filters (in continuous or discrete time) include analysis of animal motion and population dynamics ( Lavender et al., 2026 ; Auger-Méthé et al., 2021 ) . Beyond the linear-Gaussian setting of Kalman filters, the problem of estimation in stochastic differential equations has a long history within mathematical statistics ( Prakasa Rao and Rubin, 1979 ; Kessler, 1997 ; Nielsen et al., 2000 ) and remains an active research area ( Pilipovic et al., 2024 ; Jamba et al., 2024 ) . Weak nonlinearities can be addressed using the Extended Kalman filter and its many variants ( Simon, 2006 ; Kristensen et al., 2004 ) . For stronger nonlinearities, where the posterior distributions are not well approximated with Gaussians, sequential Monte Carlo techniques such as the particle filter ( Gordon et al., 1993 ) apply. For low-dimensional state spaces, the partial differential equation that governs the posterior distribution can be solved numerically as described, for example, by Pedersen et al. (2008) and Thygesen (2023) . Simpler special cases arise if all states are measured perfectly at discrete deterministic points in time - see, e.g., ( Pilipovic et al., 2024 ) and the references therein - or if the noise intensity does not depend on the state. From the point of view of practical modeling, the many methods for inference in stochastic differential equations represent a challenge, because no single method is universally applicable, and considerable technical expertise is needed to select and apply the most appropriate method in a given situation. Moreover, changes to a model which are minor seen from an ecological point of view, can mean that the chosen estimation method becomes infeasible. The difficulty with these estimation problems, which explains the many specialized methods, is the many unobserved random variables, for which the joint probability distribution is not available in closed form, but which must nevertheless be integrated out to yield posterior distributions and likelihoods. The different methods represent different ways to to approximate these integrands and integrals. This paper focuses on the Laplace approximation (see, e.g., Kristensen et al. (2016) and the references therein), where the integrand is approximated with a Gaussian bell, for which the integral is known analytically. Thus, it replaces integration over a high-dimensional Euclidean space {\mathbf{R}}^{N} with maximization of the integrand and computing the determinant of a Hessian. These operations are feasible even in very high dimensional spaces when the Hessian is sparse, as in time series analysis. Laplace’s method has become increasingly popular in ecological statistics over the past decades thanks to availability of strong software such as ADMB ( Fournier et al., 2012 ) , INLA ( Rue et al., 2009 ) , and (R)TMB ( Kristensen et al., 2016 ; Kristensen, 2025 ) . The Laplace approximation has been at the core of models for geolocation of tagged fish ( Albertsen et al., 2015 ) , stock assessment in fisheries ( Nielsen and Berg, 2014 ) , spatiotemporal models ( Krainski et al., 2019 ) , and in general non-linear mixed-effects modelling ( Brooks et al., 2017 ) . The objective of this paper is to apply the Laplace approximation to estimation problems in stochastic differential equations, as they appear in statistical analysis of ecological time series. What characterize these problems are multiple states, non-linear dynamics, and incomplete measurements in discrete time with statistics that are not necessarily Gaussian. Importantly, the driving noise in the dynamics may be non-additive, i.e., vary in intensity with the state. For example, the noise in population dynamics scale with the population size and vanishes when the population is extinct. The Laplace approximation has already been applied to estimation of states and parameters in stochastic differential equations with additive noise ( Karimi and McAuley, 2014 ; Karimi and McAuley, 2016 ) ; see also ( Thygesen, 2023 , section 10.8) as well as the linear example included in the RTMB package ( Kristensen, 2025 ) . One contribution of the present work is to allow