Reinforced Data-Driven Estimation for Spectral Properties of Koopman Semigroup in Stochastic Dynamical Systems
Yuanchao Xu, Jing Liu, Zhongwei Shen Isao Ishikawa · 2025
Read Reinforced Data-Driven Estimation for Spectral Properties of Koopman Semigroup in Stochastic Dynamical Systems by Yuanchao Xu, Jing Liu, Zhongwei Shen Isao Ishikawa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-04265.txt.
Introduction
Yuanchao Xu and four co-authors let a learning agent choose where to collect data for analyzing complex systems in a 2025 research paper. Koopman analysis rewrites nonlinear motion as linear change in measured quantities, exposing decay rates and repeating modes. Standard estimators falter when most recorded paths start in dull regions carrying little spectral information. Their method, Reinforced SDMD, pairs an estimator called SDMD with reinforcement learning, where an agent earns rewards for good choices.
The agent picks starting regions, the estimator updates, and a consistency reward grades the fresh estimates. A bonus keeps exploration broad. Tests on three classic wobbly systems, including the Duffing oscillator, showed sampling shifted toward informative regions. Error analysis traces how estimator mistakes flow into the learning bounds.
Catalog record
| CREATOR | Yuanchao Xu, Jing Liu, Zhongwei Shen Isao Ishikawa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-04265
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reinforced Data-Driven Estimation for Spectral Properties of Koopman Semigroup in Stochastic Dynamical Systems Yuanchao Xu, Jing Liu, Zhongwei Shen Isao Ishikawa Abstract Analyzing the spectral properties of the Koopman operator is crucial for understanding and predicting the behavior of complex stochastic dynamical systems. However, the accuracy of data-driven estimation methods, such as Extended Dynamic Mode Decomposition (EDMD) and its variants, are heavily dependent on the quality and location of the sampled trajectory data. This paper introduces a novel framework, Reinforced Stochastic Dynamic Mode Decomposition, which integrates Reinforcement Learning (RL) with Stochastic Dynamic Mode Decomposition (SDMD) to automatically guide the data collection process in stochastic dynamical systems. We frame the optimal sampling strategy as an RL problem, where an agent learns a policy to select trajectory initial conditions. The agent is guided by a reward signal based on spectral consistency , that is a measure of how well the estimated Koopman eigenpairs describe the system’s evolution balanced with an exploration bonus to ensure comprehensive coverage of the state space. We demonstrate the effectiveness of our approach using Bandit algorithm, Deep Q-Network (DQN), and Proximal Policy Optimization (PPO) algorithms on canonical systems including the double-well potential, the stochastic Duffing oscillator and the FitzHugh-Nagumo model. Our results show that the RL agent automatically discovers dynamically significant regions without any prior knowledge of the system. Rigorous theoretical analysis establishes convergence guarantees for the proposed algorithms, directly linking the final estimation accuracy to the quality of the learned sampling policy. Our work presents a robust, automated methodology for the efficient spectral analysis of complex stochastic systems. 1 Introduction The challenge of understanding, modeling, and predicting the behavior of complex dynamical systems is a central theme across science and engineering [ 34 , 12 , 5 , 11 , 1 ] . Traditionally, this challenge has been met with first-principle models [ 13 , 27 , 2 ] . However, for many modern systems, such as turbulent fluid flow and neural systems, the underlying mechanisms are too complex to model explicitly. With the increasing availability of data, this has brought a shift towards data-driven methods, which leverage computational power to extract dynamical patterns directly from observational data [ 6 , 22 , 39 , 5 ] . One particularly noteworthy data-driven approach is the Koopman operator framework, which has gained significant attention [ 19 , 20 ] . It offers a powerful alternative by recasting nonlinear dynamics into an infinite-dimensional, but linear evolution in a space of observable functions. The spectral properties, i.e., eigenvalues and eigenfunctions, of the Koopman operator provide profound insights into the system’s behavior, revealing conserved quantities, long-term trends, metastable states, and oscillatory modes [ 24 ] . The pursuit of these spectral properties from data has progressed significant methodological advancements, evolving from the seminal Dynamic Mode Decomposition (DMD) [ 31 , 37 ] to more sophisticated variants such as Extended DMD (EDMD) [ 39 ] , kernel-EDMD [ 40 ] , Jet-EDMD [ 15 ] , Residual DMD (ResDMD) [ 7 ] , and Stochastic DMD (SDMD) [ 41 ] , which are designed to address challenges including nonlinear interactions, higher-dimensional feature spaces, rigorous eigenfunction interpretation, spectral robustness, and inherent system noise. A persistent challenge in almost all these DMD-like methods lies in the choice of observable functions, or dictionary [ 39 , 41 , 18 , 37 , 42 , 15 , 23 , 21 ] . While carefully hand-selected basis functions can yield excellent results for specific systems, this requires expert domain knowledge. This has motivated efforts in dictionary learning [ 23 , 8 ] , where the basis functions are learned directly from data through a neural network. However, this dictionary learning approach introduces a critical dependency on the data, in other words, the quality of the learned basis is inevitably linked to the quality of the data used for training. In the presence of noise, data sampled from dynamically uninformative regions or affected by noise can lead to a poorly learned basis, resulting in a misleading or inaccurate spectral decomposition of the Koopman operator. This indicates that we need good data to find a good basis, but we often don’t know where to collect good data without a priori dynamical knowledge. To resolve this issue, we propose a fundamentally new approach that automates and optimizes the data collection strategy itself. We introduce Reinforced SDMD, a novel framework where Reinforcement Learning (RL) [ 35 ] automates data acquisition for Koopman operator analysis. An RL agent learns an optimal policy for selecting trajectory starting points by maximizing a reward function that reflects the quality of the spectral approximation. This reward function balances exploiting known informative regions with exploring new ones, transforming passive analysis into an active, intelligent search for high-quality data. The present paper is devoted to the development of a comprehensive Reinforced SDMD framework. We introduce an RL-based framework for intelligent sampling in Koopman analysis, guided by a reward function based on spectral consistency . This approach is validated with Multi-armed Bandit [ 33 ] , DQN [ 25 ] , and PPO [ 32 ] algorithms on benchmark systems, showing it can automatically identify key dynamical features. We also provide convergence analysis that links the quality of the learned sampling policy to the final estimation accuracy. This work pioneers a new paradigm that fully integrates intelligent data acquisition into the process of Koopman spectral analysis. The paper is organized as follows. Section 2 provides the necessary background on the stochastic Koopman operator and the SDMD algorithm. Section 3 details our proposed methodology after introducing the three classical reinforced learning schemes. Section 4 presents the experimental results on several stochastic dynamical systems. Section 5 provides the theoretical convergence analysis for our methods. Finally, we conclude with a summary and discussion of future work. 2 Preliminaries 2.1 Stochastic Koopman Operator The Koopman operator framework provides a powerful mathematical tool for analyzing dynamical systems by transforming nonlinear dynamics into a linear representation in an infinite-dimensional function space. For stochastic systems, this framework extends naturally to capture probabilistic evolution over time. Consider a continuous-time stochastic process (X_{t})_{t\geq 0} on a probability space (\Omega,\mathbb{P}) defined by the stochastic differential equation (SDE): \mathrm{d}X_{t}=b(X_{t})\,\mathrm{d}t+\sigma(X_{t})\,\mathrm{d}W_{t},\quad X_{0}=x\in\mathcal{M}, (2.1) where \mathcal{M}\subseteq\mathbb{R}^{d} is the state space, b:\mathcal{M}\to\mathbb{R}^{d} is the drift, \sigma:\mathcal{M}\to\mathbb{R}^{d\times m} is the diffusion coefficients, and (W_{t})_{t\geq 0} is a standard m -dimensional Wiener process. Let \rho be a probability measure on \mathcal{M} and denote by \mathcal{F}\coloneqq L^{2}(\mathcal{M},\rho) the space of observables. The space \mathcal{F} is equipped with the usual inner product \langle f,g\rangle_{\rho}\coloneqq\int_{\mathcal{M}}fg\;d\rho and corresponding norm \|f\|_{\rho}\coloneqq\sqrt{\langle f,f\rangle_{\rho}} . The stochastic Koopman semigroup (\mathcal{K}^{t})_{t\geq 0} on \mathcal{F} is defined as: for f\in\mathcal{F} , x\in\mathcal{M} , and t\geq 0 , (\mathcal{K}^{t}f)(x):=\mathbb{E}_{\mathbb{P}}[f(X_{t})|X_{0}=x], (2.2) where \mathbb{E}_{\mathbb{P}} denotes the expectation with respect to probability measure \mathbb{P} . This operator captures how the expected value of observables evolves under random perturbation. The infinitesimal generator \mathcal{A} of the Koopman semigroup is given by: \mathcal{A}f:=\lim_{t\to 0}\frac{\mathcal{K}^{t}f-f}{t}, (2.3) on the domain \mathcal{D}(\mathcal{A})=\left\{f\in\mathcal{F}:\lim_{t\to 0}\frac{\mathcal{K}^{t}f-f}{t}\text{ exists in }\mathcal{F}\right\} . By Itô’s formula [ 29 ] , \mathcal{A}f=\sum_{i=1}^{d}b_{i}\frac{\partial f}{\partial x_{i}}+\frac{1}{2}\sum_{i,j=1}^{d}(\sigma\sigma^{T})_{ij}\frac{\partial^{2}f}{\partial x_{i}\partial x_{j}},\quad\forall f\in C^{2}_{b}(\mathcal{M}). (2.4) Assumption 2.1 . Throughout this paper, we assume that \{\mathcal{K}^{t}\}_{t\geq 0} is a C_{0} -semigroup on \mathcal{F} [ 29 ] . The Assumption