Nudging state-space models for Bayesian filtering under misspecified dynamics
Fabián González, O. Deniz Akyildiz, Dan Crisan, Joaquín Míguez · 2024
Read Nudging state-space models for Bayesian filtering under misspecified dynamics by Fabián González, O. Deniz Akyildiz, Dan Crisan, Joaquín Míguez (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00218.txt.
Description
Abstract Nudging is a popular algorithmic strategy in numerical filtering to deal with the problem of inference in high-dimensional dynamical systems. We demonstrate in this paper that general nudging techniques can also tackle another crucial statistical problem in filtering, namely the misspecification of the transition kernel.
Catalog record
| CREATOR | Fabián González, O. Deniz Akyildiz, Dan Crisan, Joaquín Míguez |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-00218
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Nudging state-space models for Bayesian filtering under misspecified dynamics Fabián González, O. Deniz Akyildiz, Dan Crisan, Joaquín Míguez Abstract Nudging is a popular algorithmic strategy in numerical filtering to deal with the problem of inference in high-dimensional dynamical systems. We demonstrate in this paper that general nudging techniques can also tackle another crucial statistical problem in filtering, namely the misspecification of the transition kernel. Specifically, we rely on the formulation of nudging as a general operation increasing the likelihood and prove analytically that, when applied carefully, nudging techniques implicitly define state-space models that have higher marginal likelihoods for a given (fixed) sequence of observations. This provides a theoretical justification of nudging techniques as data-informed algorithmic modifications of state-space models to obtain robust models under misspecified dynamics. To demonstrate the use of nudging, we provide numerical experiments on linear Gaussian state-space models and a stochastic Lorenz 63 model with misspecified dynamics and show that nudging offers a robust filtering strategy for these cases. keywords Bayesian filtering, nudging, Bayesian evidence, marginal likelihood, model mismatch, misspecified dynamics 1 Introduction 1.1 State space models State-space models (SSMs) are key building blocks in many applications in signal processing, machine learning, weather forecasting, finance, object tracking, ecology, and many other fields Triantafyllopoulos (2021) . These models are used to represent the dynamics of a system, where the system state evolves over time according to a Markov transition kernel and the available observations (data) are related to the system state by a likelihood function. The main statistical goal in SSMs is to infer the state of the system given a sequence of observations, a problem known as filtering Anderson and Moore (2005) . Formally, we represent the state of the SSM by a Markov chain \{X_{t}\}_{t\geq 0} described as follows. The initial state X_{0} is a random variable (r.v.) with probability law \pi_{0} and, at any time t\geq 1 , the dynamics of the transition from X_{t-1} to X_{t} is modelled by a Markov kernel K_{t}(x_{t-1},{\sf d}x_{t}) . The sequence of observations is denoted by \{Y_{t}\}_{t\geq 1} and the relationship between the state X_{t} and the observation Y_{t} is modelled by a conditional probability density function (pdf) p_{t}(y_{t}|X_{t}=x_{t}) . Since in practice the observations are given, Y_{t}=y_{t} for t\geq 1 , the latter relationship is usually given in terms of a likelihood function g_{t}(x_{t})\propto p_{t}(y_{t}|X_{t}=x_{t}) . With these elements, the conditional probability law of the state X_{t} given the data Y_{1\mathrel{\mathop{\mathchar 58\relax}}t}=y_{1\mathrel{\mathop{\mathchar 58\relax}}t}\mathrel{\mathop{\mathchar 58\relax}}=\{y_{1},\ldots,y_{t}\} can be constructed recursively via the Chapman-Kolmogorov equation and Bayes’ theorem (see, e.g., Del Moral (2004) ; Bain and Crisan (2009) ) and we denote it as \pi_{t} . The conditional law \pi_{t} is often termed the optimal, or Bayesian, filter. The optimal filter \pi_{t} can only be computed exactly in a few specific cases. The most relevant one is the scenario where both the Markov kernels K_{t} and the likelihoods g_{t} correspond to linear relationships and Gaussian noise. Under such assumptions, \pi_{t} is Gaussian and its mean and covariance matrix can be computed recursively via the Kalman filter (KF) algorithm Kalman (1960) . In most practical applications, however, the optimal filter \pi_{t} can only be approximated numerically using nonlinear KFs, particle filters (PFs) or other approximation methods Ristic et al. (2004) ; Bain and Crisan (2009) ; Särkkä and Svensson (2023) . 1.2 Model misspecification One of the main challenges in Bayesian filtering is model misspecification, which occurs when the chosen family of transition models, \{K_{t}\}_{t\geq 1} , the likelihood functions, \{g_{t}\}_{t\geq 1} , or both, fail to represent the statistical properties of the real-world system of interest with sufficient accuracy. Model misspecification is a long-standing problem in Bayesian filtering and it has been studied from different viewpoints in the literature, including outlier detection, robust filtering, parameter estimation, and the so-called nudging techniques. Outlier detection Blázquez-García et al. (2021) is, perhaps, the simplest way to manage observations which are in poor agreement with the assumed SSM. In the context of filtering, typical outlier detection schemes approximate the predictive distribution of the upcoming observation Y_{t} . Then, when the actual observation is collected, a statistical test can be run to determine whether the observed data y_{t} is compatible with the predicted distribution for Y_{t} . If the test indicates that the observation is anomalous (i.e., it is an outlier with respect to (w.r.t.) the predicted distribution) then the data y_{t} can either be discarded or be processed using a robust procedure that mitigates the effect of the outlying data on the filter update. Many methods for outlier detection and robust filtering have been proposed for Kalman-based filters Yatawara et al. (1991) ; Xie et al. (1994) ; Agamennoni et al. (2011) ; Petersen and Savkin (2012) ; Javanfar et al. (2023) ; Truzman et al. (2024) or PFs Maíz et al. (2012) ; Vázquez and Míguez (2017) ; Boustati et al. (2020) ; Zhang et al. (2023) . A fundamental problem with these approaches is that anomalous data are handled as detrimental and uninformative, under the assumption that they have not been generated by the system of interest. Very often, however, a genuine observation from the system of interest may appear as an outlier because of the misspecification of the SSM. By discarding or mitigating this observation, relevant information is wasted and model errors are reinforced. Another classical strategy to account for modelling uncertainty is to choose not one SSM but a parametric family of SSMs indexed by a (possibly multidimensional) parameter \theta . When a sequence of observations becomes available, the model is calibrated by tuning the parameter \theta to the data according to some statistical criterion. Maximum likelihood estimation methods have been proposed, both offline Jensen and Petersen (1999) ; Olsson et al. (2008) and online LeGland and Mevel (1997) ; Tadić (2010) , as well as Bayesian estimation methods. The latter include algorithms such as particle Markov chain Monte Carlo (MCMC) Andrieu et al. (2010) ; Lindsten et al. (2012) ; Dahlin et al. (2015) , iterated batch importance sampling (IBIS) Chopin (2002) or sequential Monte Carlo square (SMC 2 ) Chopin et al. (2012) , and recursive algorithms like the nested PF Crisan and Miguez (2018) and its Kalman-based approximations Pérez-Vieites et al. (2018) ; Pérez-Vieites and Míguez (2021) . While parameter estimation methods are an indispensable toolbox for practical applications, they do not provide a complete solution to the model misspecification problem. Indeed, the parametric family of SSMs may not be flexible enough the represent the features of the system of interest, no matter the choice of \theta . For example, a parametric class of linear models can be expected to fail to represent a system that displays non-negligible nonlinear features in its dynamics. Several techniques collectively known as nudging have been devised to mitigate model misspecification Reich and Cotter (2015) . Nudging methods are designed to steer (or nudge ) a model towards the observed data over time by adding a (small) corrective term to the model dynamics. The goal is to make the model follow observed values more closely without breaking down its original dynamics. Such methods have been particularly popular within the data assimilation community Stroud et al. (2010) ; Reich and Cotter (2015) ;