Ensemble Kalman–Bucy filtering for nonlinear model predictive control

Sebastian Reich · 2025

Read Ensemble Kalman–Bucy filtering for nonlinear model predictive control by Sebastian Reich (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12474.txt.

Introduction

Sebastian Reich merges filtering with control for partly seen systems in a 2025 paper. Controllers usually assume the state is known, yet real systems hide part of it, and future readings are uncertain. Reich extends the ensemble Kalman filter, a workhorse for tracking hidden states, into receding-horizon optimal control. The extension follows the forward-backward equations of the classic maximum principle.

An interacting particle scheme approximates those equations, with the forward pass given by the continuous-time filter. Control laws come out linear in form and update continuously as new data arrive, in the spirit of model predictive control. A swinging pendulum example shows the method working. The paper targets robots and processes that must act while still guessing.

Catalog record

CREATORSebastian Reich
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-12474

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Ensemble Kalman–Bucy filtering for nonlinear model predictive control Sebastian Reich Abstract . We consider the problem of optimal control for partially observed dynamical systems. Despite its prevalence in practical applications, there are still very few algorithms available, which take uncertainties in the current state estimates and future observations into account. In other words, most current approaches separate state estimation from the optimal control problem. In this paper, we extend the popular ensemble Kalman filter to receding horizon optimal control problems in the spirit of nonlinear model predictive control. We provide an interacting particle approximation to the forward-backward stochastic differential equations arising from Pontryagin’s maximum principle with the forward stochastic differential equation provided by the time-continuous ensemble Kalman–Bucy filter equations. The receding horizon control laws are approximated as linear and are continuously updated as in nonlinear model predictive control. We illustrate the performance of the proposed methodology for an inverted pendulum example. 1. Introduction Optimal control for partially observed dynamical systems has first been consider in [ 14 ] and theoretical foundations have been established, for example, in [ 3 ] and [ 13 ] . It is well-known [ 17 , 12 ] that, for linear systems, the involved optimal control and state estimation problems can be solved separately. Such a separation or uncertainty equivalence principle does however no longer apply to nonlinear dynamical systems. Still such a principle is widely applied in practice due to the computational intractability of combined estimation and control problems. In this paper, we revisit this problem by combining nonlinear model predictive control (NMPC) [ 15 ] with ensemble Kalman filtering (EnKF) [ 5 ] ; thus providing an alternative approach to currently available output feedback NMPC techniques [ 8 , 15 ] . To simplify the exposition we assume a data assimilation setting where partial and noisy observations are taken continuously in time [ 10 , 16 ] and processed by the ensemble Kalman–Bucy filter (EnKBF) [ 4 ] . The novelty of our approach arises from the fact that we use the EnKBF for (i) estimating model states from actual observations as well as (ii) predicting uncertainties in the optimal control problem arising from the statistics of future observations and uncertainties in the estimated model states. For the latter task we formulate an appropriate set of forward-backward stochastic differential equations (FBSDEs) based on Pontryagin’s maximum principle [ 6 ] . In the spirit of the EnKBF, we also utilize a linear approximation ansatz when solving the associated backward stochastic differential equation (BSDE) giving rise to linear control laws in the estimated model states. While linear control laws become insufficient for highly nonlinear infinite horizon optimal control problems, the receding horizon approach of NMPC allows for a continuous adaptation of linear approximations; very much in line with modern data assimilation techniques such as the EnKF. As for NMPC, the success of the proposed methodology depends on suitable choices of the optimal control cost function as well as sufficiently informative observations in order to apply EnKF-based data assimilation. In this paper, we assume a deterministic model dynamics. Model errors can be taken into account via stochastic perturbations in the EnKBF or via robust NPMC schemes [ 8 , 15 ] . The remainder of the paper is structured as follows. A mathematical problem formulation is provided in Section 2 . We develop the proposed methodology in Section 3 in the context of linear partially observed control problems first. In particular, we derive an interacting particle formulation in terms of FBSDEs which exactly reproduces the known state estimation and optimal control solution [ 17 , 12 ] . The proposed methodology is then extended to nonlinear problems in Section 4 very much in the spirit of EnKF-based methodologies [ 5 ] with the addition that we also need to approximate solutions to a BSDE. We introduce the acronym EnKBF-NMPC for the newly proposed method. Section 5 provides a numerical illustration in the context of an inverted pendulum. The paper closes with some final remarks in Section 6 . 2. Mathematical problem formulation We consider controlled ordinary differential equations (ODEs) of the form (1) \dot{X}_{t}=f(X_{t})+GU_{t} subject to uncertain initial conditions X_{0}\in\mathbb{R}^{d_{x}} , which are distributed according to a known probability measure \nu_{0}({\rm d}x) . The control applied at time t\geq 0 is denoted by U_{t}\in\mathbb{R}^{d_{u}} . The drift function f:\mathbb{R}^{d_{x}}\to\mathbb{R}^{d_{x}} and the matrix G\in\mathbb{R}^{d_{x}\times d_{u}} are assumed to be given. We also assume that the process is partially observed continuously in time with forward model (2) {\rm d}\mathcal{Y}_{t}=h(X_{t}){\rm d}t+R^{1/2}{\rm d}W_{t}, where h:\mathbb{R}^{d_{x}}\to\mathbb{R}^{d_{y}} denotes the forward model, R\in\mathbb{R}^{d_{y}\times d_{y}} is the error covariance matrix, and W_{t} is standard d_{y} -dimensional Brownian motion (BM). Actual observations are denotes by \mathcal{Y}_{t}^{\dagger} , t\geq 0 , and its associated filtering distribution at time \tau>0 by (3) \nu_{\tau}({\rm d}x):=\nu_{\tau}({\rm d}x|\mathcal{Y}_{0:\tau}^{\dagger}). Here we have introduced the notation \mathcal{Y}^{\dagger}_{0:\tau} to indicate the dependence on observations \mathcal{Y}_{t}^{\dagger} from the time interval t\in(0,\tau] . At each time instance \tau_{n}=n\Delta\tau , n\geq 0 and for available observations \mathcal{Y}_{0:\tau_{n}}^{\dagger} , we solve a finite-horizon stochastic optimal control problem with cost function (4) \mathcal{L}(U_{\tau_{n}:\tau_{n}+T})=\mathbb{E}\left[\int_{\tau_{n}}^{\tau_{n}+T}\left(\frac{1}{2}\|U_{t}\|^{2}+c(X_{t})\right){\rm d}t+\psi_{T}(X_{\tau_{n}+T})\right], T>\tau_{n+1} , with expectation taken over initial conditions (5) X_{\tau_{n}}\sim\nu_{\tau_{n}} and all possible future observation paths \mathcal{Y}_{\tau_{n}:\tau_{n}+T} satisfying the model ( 2 ). The desired optimal control, denoted by U_{t}^{\ast} , is a function of distributions \nu({\rm d}x) , which we denote by u_{t}^{\ast}(\nu) . In other words, one applies U_{t}^{\ast}=u_{t}^{\ast}(\nu_{t}) over the time interval t\in(\tau_{n},\tau_{n+1}] along the filtering distribution \nu_{t} for given data \mathcal{Y}_{0:t}^{\dagger} . Hence, we obtain the controlled ODE (6) \dot{X}_{t}=f(X_{t})+Gu_{t}^{\ast}(\nu_{t}) for t\in(\tau_{n},\tau_{n+1}] and initial X_{\tau_{n}}\sim\nu_{\tau_{n}} . Receding horizon optimal control formulations are at the heart of NMPC [ 15 ] . NMPC typically replaces the uncertain X_{\tau_{n}} by its expectation value under the filtering distribution \nu_{\tau_{n}} ; i.e. by (7) \bar{X}_{\tau_{n}}:=\mathbb{E}[X_{\tau_{n}}]. This approximation reduces the stochastic optimal control problem to a standard deterministic optimal control problem with cost (8) \mathcal{L}(u_{\tau_{n}:\tau_{n}+T})=\int_{\tau_{n}}^{\tau_{n}+T}\left(\frac{1}{2}\|U_{t}\|^{2}+c(\bar{x}_{t})\right){\rm d}t+\psi_{T}(\bar{x}_{\tau_{n}+T}) subject to the ODE (9) \dot{\bar{x}}_{t}=f(\bar{x}_{t})+GU_{t} with initial condition \bar{x}_{\tau_{n}}=\bar{X}_{\tau_{n}} . We note that the optimal control U_{t}^{\ast} becomes open loop. In this paper, we develop an alternative approach which takes the uncertainties in the initial conditions and future observations into account and results in linear (closed loop) control laws of the form (10) u_{t}^{\ast}(\nu_{t})=-G^{\rm T}\left(\Lambda_{t}^{\ast}\bar{X}_{t}+\lambda_{t}^{\ast}\right) for suitable matrices \Lambda_{t}^{\ast}\in\mathbb{R}^{d_{x}\times d_{x}} and vectors \lambda_{t}^{\ast}\in\mathbb{R}^{d_{x}} , t\in(\tau_{t_{n}},\tau_{t_{n}}+T] . Our approach is based on the continuous-time EnKBF approximation [ 5 ] to the filtering distributions \nu_{t} and we call our approach therefore EnKBF-NMPC. Let us embed the discussion so far into a digital twin setting [ 9 ] . The computed optimal control U_{t}^{\dagger}=u_{t}^{\ast}(\nu_{t}) ; i.e. ( 10 ), would be applied to the physical twin, which could be of the form (11) {\rm d}X_{t}^{\dagger}=f(X_{t}^{\dagger}){\rm d}t+GU_{t}^{\dagger}{\rm d}t+\sigma{\rm d}B_{t}^{\dagger}. Here B_{t}^{\dagger} stands for random perturbations, which mimic the difference between the physical and digital twin and which could take the form of standard BM for simplicity of exposition, and \sigma>0 . The physical twin ( 11 ) produces a reference trajectory, which we denote by X_{t}^{\dagger} and which is partially observed according to the observation model ( 2 ); i.e., the actual observations \mathcal{Y}_{0:t}^{\dagger} satisfy (12) {\rm d}\mathcal{Y}_{t}^{\dagger}=h(X_{t}^{\dagger}){\rm d}t+{\rm d}W_{t}^{\dagger} with W_{t}^{\dagger} a particular realization of BM. Given the data, the digital

Read the complete text