Learning Nonlinear Dynamics: Improving the Estimation Efficiency and Reliability of Gaussian Process State-Space Models
2026
Read Learning Nonlinear Dynamics: Improving the Estimation Efficiency and Reliability of Gaussian Process State-Space Models (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24691.txt.
Introduction
Jan Failenschmid, Leonie Vogelsmeier, Joris Mulder, and Joran Jongerling work on learning nonlinear dynamics from data. Their tool is the Gaussian process state space model. Such models infer hidden dynamics from indirect observations without fixing a formula in advance. That suits partly understood systems. Exact estimation, though, becomes infeasible for moderately long time series. The authors aim to make Bayesian estimation of these models faster and more trustworthy for applied researchers.
Three changes carry the work. First, two modifications to an existing Gibbs sampler improve its efficiency and convergence. Second, a confirmatory factor analysis measurement model cuts identifiability problems and lets users impose structure. Third, the team supplies validated software for use in practice. Simulation based calibration shows the sampler converges reliably across many datasets. Its results come out well calibrated. An applied example then shows the model at work on real data.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 112 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.24691
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
(GP-SSM; 30 ; 79 ) , which uses Gaussian processes (GP), a Bayesian machine learning technique ( 65 ) , to learn the form of the dynamic model, the measurement model, or both simultaneously. There are several reasons why GPs are a particularly well suited machine learning method for being used within SSMs, First, a GP constitutes a Bayesian prior over a space of candidate functions and learning a function with a GP is equivalent to obtaining its posterior distribution ( 65 ) . This provides a principled way to account for the additional epistemic uncertainty that is created by learning the form of some components of the SSM from data, rather than assuming that their functional form is known a priori and that only a fixed set of parameters is estimated. Second, GP priors are very flexible and can be used to encode assumptions and prior expectations about certain properties of the target function, such as its smoothness, average behavior, or periodicity. Additionally, multiple GP priors can be combined to create a more flexible model and learn more complex functions ( 20 ) . Lastly, GPs are capable of modelling the functional relationships between multiple latent variables ( 54 ) , such as the state variables in an SSM. The primary drawback of using GPs, however, is their computational cost, which scales cubically with the number of observed data points (i.e., exact GP inference requires inverting a covariance matrix over all observed data points). This quickly makes inferring the exact GP computationally demanding, even for moderately long time-series. To address this, several approximation methods have been developed for GPs. One of them is the Hilbert space or reduced-rank GP approximation, which represents the GP using a finite set of basis functions ( 73 ) . 75 show that this basis function approximation can be estimated efficiently within a GP-SSM, since it corresponds to a standard Bayesian regression problem, which is computationally more efficient when the number of basis functions is low and converges to the exact GP as the number of basis functions goes to infinity, under certain regularity conditions. They further show, that the Hilbert space approximate GP-SSM (HSGP-SSM), outperforms alternative sparse approximations and the exact GP in terms of both computation time and prediction accuracy on a simulated benchmark dataset 1 1 1 Additionally, 54 showed that HSGPs might be estimated more accurately than the exact GP when using latent variables as input points. This could make them particularly suitable for being used within SSMs. . Because of this, HSGP-SSMs are a promising method for studying and learning dynamic systems. However, there are currently several challenges regarding the Bayesian estimation of HSGP-SSMs which limit their applicability and hinder their wide-spread use for learning dynamic processes in empirical research. Therefore, this paper focuses on improving the estimation of HSGP-SSMs via MCMC sampling. By doing so, we aim to make this method more accessible and widely applicable for applied science and facilitate the statistical learning of nonlinear dynamic systems using a flexible data-driven approach building on non-parametric GPs. Specifically, we identified three challenges, which currently make using HSGP-SSMs more complicated in practice. Below, we describe each of these challenges and how we aim to address them. First, existing MCMC algorithms for HSGP-SSMs can have a low sampling efficiency for certain model parameters. The Gibbs sampler proposed by 75 can accurately estimate both the HSGP and the latent states of the SSM. However, Gaussian processes also include hyperparameters that control key properties of the GP prior, such as the overall variability of the functions it describes and how rapidly they can change across the input space. In doing so, they directly influence the types of functions that the GP can learn. In a fully Bayesian framework, these hyperparameters are estimated jointly with the remaining model. For the hyperparameters, the Gibbs sampler has a low sampling efficiency, which often leads to slow mixing and convergence of the MCMC chains (Figure 1 ). This also negatively affects the estimation of the remaining model parameters. To address this issue, we propose two modifications to their algorithm aimed at improving its sampling efficiency and facilitating more reliable estimation in empirical applications. Figure 1: Comparison of hyperparameter trace plots from 75 and the proposed sampler \fitfigure combined_trace.png \figurenote This figure compares the sampler of 75 with the sampler proposed in this paper. The upper panels show the original hyperparameter trace plots for the first synthetic data example of 75 after removing the burn-in. In particular, the marginal variance parameter shows very slow mixing. The bulk effective sample size is 6.85 for the marginal variance and 56.5 for the lengthscale. The tail effective sample sizes are 22.2 and 98.6 , respectively. The lower panels show trace plots produced by the sampler proposed in this paper for the same dataset, using the same priors and settings. However, in addition to the GP, we also estimated the dynamic and measurement error covariances. The bulk effective sample size increases to 291 for the marginal variance and 106 for the lengthscale. The corresponding tail effective sample sizes are 501 and 124 . The second challenge in GP-SSMs is identifiability. When both the dynamic and measurement models are estimated with too much flexibility, the model becomes weakly identified, as multiple combinations of functions can explain the observed data equally well. During MCMC sampling this can lead to drifting, numerical instability, and difficulties in assessing the convergence of the sampler. Additionally, it can complicate the interpretation of the resulting posterior distributions ( 34 ) . To mitigate these issues, it is common in the GP-SSM literature to focus on learning the dynamic model while assuming a known or parametric measurement model ( 75 ; 24 ; 30 ) 2 2 2 A notable exception is 88 , who learn the measurement model non-parametrically while assuming a linear dynamic model. . We therefore use a confirmatory factor analysis (CFA) model as the measurement model. The CFA is a widely used linear 3 3 3 Assuming a linear measurement model does not reduce the generality of the model, because any GP-SSM of the form given in Equation 1 can be reparametrized as a model with a linear measurement function through state expansion ( 31 ) . measurement model from structural equation modeling ( 57 ; 10 ) , which can be identified through parameter constraints ( 37 ) . These constraints also allow researchers to impose a specific structure on the measurement model, which is important as each state is often measured only by a subset of the observed variables in empirical applications ( 10 ; 27 ) . Lastly, there is currently no validated software implementation for estimating HSGP-SSMs. Researchers thus need to implement their own estimators or adapt published code, which has often only been evaluated on a limited set of simulated datasets. This is both inefficient and prone to human error. It is therefore important to have an accessible and systematically validated implementation of an HSGP-SSM estimator. To this end, we evaluate the sampler proposed in this paper over a wide range of simulated data sets using simulation-based calibration (SBC; 52 ) . This is a simulation-based method for empirically testing