Online Rolling Controlled Sequential Monte Carlo

Liwen Xue, Axel Finke, Adam M. Johansen · 2025

Read Online Rolling Controlled Sequential Monte Carlo by Liwen Xue, Axel Finke, Adam M. Johansen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00696.txt.

Introduction

Liwen Xue, Axel Finke, and Adam Johansen bring controlled particle filtering to live data streams. Hidden Markov models track a hidden state behind noisy reads. Controlled sequential Monte Carlo sharpens the tracking, but was built for after-the-fact smoothing. Their 2025 method rolls it online with a sliding window. Two particle swarms run at once: one learns how to twist the sampling, and one filters the state.

Cost stays bounded as data pour in, since the window never grows. Tests on three model families show sharper, sturdier estimates than standard particle filters. The edge grows in higher dimensions, where plain filters starve. Live uses span finance, tracking, and brain signal work.

Catalog record

CREATORLiwen Xue, Axel Finke, Adam M. Johansen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-00696

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Online Rolling Controlled Sequential Monte Carlo Liwen Xue, Axel Finke, Adam M. Johansen Abstract We introduce methodology for real-time inference in general-state-space hidden Markov models. Specifically, we extend recent advances in controlled sequential Monte Carlo (CSMC) methods—originally proposed for offline smoothing—to the online setting via a rolling window mechanism. Our novel online rolling controlled sequential Monte Carlo (ORCSMC) algorithm employs two particle systems to simultaneously estimate twisting functions and perform filtering, ensuring real-time adaptivity to new observations while maintaining bounded computational cost. Numerical results on linear-Gaussian, stochastic volatility, and neuroscience models demonstrate improved estimation accuracy and robustness in higher dimensions, compared to standard particle filtering approaches. The method offers a statistically efficient and practical solution for sequential and real-time inference in complex latent variable models. Keywords: Sequential Monte Carlo; state-space models; particle filters; controlled SMC; online inference; latent variable models. 1 Introduction This paper is concerned with inference for general-state-space hidden Markov models (HMMs) , a.k.a. state-space models . These models are used in diverse applications ranging from control systems to financial analysis, navigation, and biological monitoring. In such applications, efficient and accurate estimation of the hidden states (given noisy and partial observations) is crucial. Three inferential tasks are of particular interest: 1. Filtering: characterizing the conditional law of the most recent latent state given all the observations received thus far. 2. Smoothing: characterizing the conditional law of all latent states until the current time point given all the observations received thus far. 3. Marginal-likelihood evaluation: evaluating the density of all available observations with the states marginalised out. This is typically needed for performing likelihood-based estimation of unknown model parameters. Such inference is typically sequential in the sense that beliefs about the latent states are to be updated whenever new observations become available—often at short intervals; and may even need to be online in the sense that both the space and time costs of incorporating one new observation are bounded uniformly in time. Obtaining the filtering and smoothing distributions and marginal likelihoods involves computing integrals which are typically intractable, except in a limited number of scenarios such as linear Gaussian models—which give rise to the celebrated Kalman filter and related recursions ( Kalman 1960 ) —and a few others ( Vidoni 1999 ) . Consequently, Monte Carlo methods are the state of the art for general inference in these methods. Particle filters (PFs) are a class of sequential Monte Carlo (SMC) methods adapted to conducting inference in general-state-space HMMs (with a particular focus on online solutions of the filtering problem). They first came to prominence in the form of the bootstrap particle filter (BPF) introduced in Gordon et al. 1993 . Since then, there have been numerous innovations in the area. Those most critical to the current work are summarised in the next section; see, e.g., Doucet and Johansen 2011 for a survey of approaches to filtering and smoothing, Kantas et al. 2015 for an overview of parameter estimation techniques or Chopin and Papaspiliopoulos 2020 for a book-length introduction to SMC methods including their applications to general-state-space HMMs . PFs perform poorly in challenging scenarios such as higher-dimensional latent states. To alleviate this problem, Guarniero et al. 2017 ; Heng et al. 2020 proposed controlled sequential Monte Carlo (CSMC) methods (a.k.a., iterated auxiliary particle filters ) which leverage insights from optimal control theory to improve the efficiency of the PF . Unfortunately, as CSMC must repeatedly browse through the entire observation sequence, it is limited to offline inference (i.e., settings in which the full data record is available before inference is conducted). In this work, we propose a novel method, online rolling controlled SMC (ORCSMC) , which aims to extend the benefits of CSMC to online inference. ORCSMC propagates a particle system using a finite-horizon modifications of the CSMC approach. At each time point, aditional simulations temporarily extend this system to the time of the current observation in order to compute efficient approximations of filtering and smoothing distributions and marginal likelihoods in real time. ORCSMC is also applicable in contexts in which storing on processing the entire data set (as required for CSMC ) is impractical. Section 2 reviews existing SMC approaches for filtering and smoothing, as well as CSMC ; Section 3 presents the novel ORCSMC method. Section 4 demonstrates the performance of our proposed approach on different models; Section 5 concludes. Throughout this work, we use the shorthand \llbracket m,n\rrbracket\coloneqq\{k\in\mathbb{Z}\mid m\leq k\leq n\} , for m,n\in\mathbb{Z}\cup\{\infty\} with m\leq n . If n\geq 1 , we also write \llbracket n\rrbracket\coloneqq\llbracket 1,n\rrbracket . If m and n are finite, we further write z_{m:n}\coloneqq(z_{m},z_{m+1},\dotsc,z_{n}) and z^{m:n}\coloneqq(z^{m},z^{m+1},\dotsc,z^{n}) . 2 Background This section reviews SMC methods for inference in general-state-space HMMs , with a focus on controlled methods and techniques designed for approximating smoothing distributions. 2.1 General-state-space HMMs For definiteness, consider an \mathbb{R}^{d} -valued discrete-time Markov process \{X_{t}\}_{t\geq 1} such that (1) X_{1}\sim\mu\quad\text{and}\quad X_{t}|(X_{t-1}=x_{t-1})\sim f_{t}(\,\cdot\,|x_{t-1}), where t>1 , “ \sim ” means distributed according to, \mu(x_{1}) is a probability density function and f_{t}(x_{t}|x_{t-1}) denotes the probability density associated with moving from X_{t-1}=x_{t-1} at time t-1 to X_{t}=x_{t} at time t . To simplify the notation, any additional ‘static’ model parameters parametrising these densities are omitted from the notation. We are interested in performing inference about \{X_{t}\}_{t\geq 1} but only have access to \mathbb{R}^{d^{\prime}} -valued observations \{Y_{t}\}_{t\geq 1} , assumed to be conditionally independent given \{X_{t}\}_{t\geq 1} with densities (2) Y_{t}|(X_{t}=x_{t})\sim g_{t}(\,\cdot\,|x_{t}). The main inference problems for this model class is characterizing the filtering and smoothing densities and marginal likelihoods: (3) \displaystyle p(x_{t}|y_{1:t}) \displaystyle\propto\int\biggl[\mu(x_{1})g_{1}(y_{1}|x_{1})\prod_{s=2}^{t}f_{s}(x_{s}|x_{s-1})g_{s}(y_{s}|x_{s})\biggr]\,\mathrm{d}x_{1:t-1}, (4) \displaystyle p(x_{1:t}|y_{1:t}) \displaystyle\propto\mu(x_{1})g_{1}(y_{1}|x_{1})\prod_{s=2}^{t}f_{s}(x_{s}|x_{s-1})g_{s}(y_{s}|x_{s}), (5) \displaystyle p(y_{1:t}) \displaystyle=\int\biggl[\mu(x_{1})g_{1}(y_{1}|x_{1})\prod_{s=2}^{t}f_{s}(x_{s}|x_{s-1})g_{s}(y_{s}|x_{s})\biggr]\,\mathrm{d}x_{1:t}. If online inference is required, then the most we can generally hope to achieve is characterizing fixed-dimensional marginals of the smoothing distributions from ( 4 ). 2.2 Particle Filtering The bootstrap particle filter (BPF) ( Gordon et al. 1993 ) proceeds by sampling a collection of Monte Carlo samples (‘particles’) from the initial distribution \mu , weighting them in accordance with the likelihood associated with the first observation, y_{1} , and then—at time t —iteratively sampling from the resulting weighted empirical distribution, extending the sampled paths according to the transition density f_{t} and weighting them with the likelihood of y_{t} . Sampling from the weighted empirical distribution can be decomposed as a selection step, known as resampling , in which particles with larger weights tend to produce more offspring followed by a mutation step in which particles move according to the underlying dynamic model. Seen in this way, the algorithm associates particle values at each time with its predecessor and introduces a genealogical relationship allowing us to think about the ancestor of a given particle at any number of generations earlier. Tracing ancestries back in this way is central to the most basic uses of PFs in order to address smoothing problems. The BPF is effective in many settings, but has a number of limitations which more recent work has sought to overcome. More general sequential importance resampling PFs ( Doucet et al. 2000 , see, e.g.,) follow from recognising that the BPF is essentially carrying out a sequence of importance sampling operations and hence that one can adjust the proposal distribution to better take into account the observation at the current

Read the complete text