Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systems

Minchan Jeong, J. Jon Ryu, Se-Young Yun, Gregory W. Wornell · 2025

Read Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systems by Minchan Jeong, J. Jon Ryu, Se-Young Yun, Gregory W. Wornell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07222.txt.

Introduction

Minchan Jeong and co-authors offer a faster way to learn the main patterns of a random moving system, in a 2025 paper. The math tool behind the work is the Koopman operator, which turns a tangled nonlinear system into a linear one that is easier to study. Neural networks can learn its leading patterns, but older methods force unstable matrix steps during training.

Their approach rests on a low-rank shortcut: keep only the top few patterns and learn those directly. This avoids the shaky steps that biased earlier training and slowed work on large systems. Tests show the learned patterns stay reliable and serve well for breaking a system into its modes and for guessing its next states. The method fits neatly into standard deep learning tools.

Catalog record

CREATORMinchan Jeong, J. Jon Ryu, Se-Young Yun, Gregory W. Wornell
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE110 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-07222

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systems Minchan Jeong, J. Jon Ryu, Se-Young Yun, Gregory W. Wornell Abstract The Koopman operator provides a principled framework for analyzing nonlinear dynamical systems through linear operator theory. Recent advances in dynamic mode decomposition (DMD) have shown that trajectory data can be used to identify dominant modes of a system in a data-driven manner. Building on this idea, deep learning methods such as VAMPnet and DPNet have been proposed to learn the leading singular subspaces of the Koopman operator. However, these methods require backpropagation through potentially numerically unstable operations on empirical second moment matrices, such as singular value decomposition and matrix inversion, during objective computation, which can introduce biased gradient estimates and hinder scalability to large systems. In this work, we propose a scalable and conceptually simple method for learning the top- k k singular functions of the Koopman operator for stochastic dynamical systems based on the idea of low-rank approximation. Our approach eliminates the need for unstable linear algebraic operations and integrates easily into modern deep learning pipelines. Empirical results demonstrate that the learned singular subspaces are both reliable and effective for downstream tasks such as eigen-analysis and multi-step prediction. 1 Introduction The Koopman operator theory offers a powerful framework for analyzing nonlinear dynamical systems by lifting them into an infinite-dimensional function space, where spectral techniques from linear operator theory can be applied. Recent advances in dynamic mode decomposition (DMD) have shown that trajectory data can be effectively used to identify dominant dynamical modes in a data-driven manner [ 37 , 41 , 42 , 15 , 3 ] . Inspired by the success of deep learning, recent methods such as VAMPnet [ 43 , 22 ] and DPNet [ 13 ] employ neural networks to approximate the leading singular subspaces of the Koopman operator. While shown effective for some benchmark tasks, these approaches often rely on numerically unstable operations such as singular value decomposition (SVD) or matrix inversion during objective computation. These operations present practical challenges, particularly in computing unbiased gradients and scaling to high-dimensional systems. In this work, we propose a conceptually simple and scalable method for learning the top- k singular functions of the Koopman operator for stochastic dynamical systems. Our approach builds on the idea of low-rank approximation , which has recently received attention in the literature due to its favorable optimization structure that aligns well with modern optimization practices; see, e.g., [ 19 , 40 ] in the numerical optimization literature, and [ 38 , 6 , 30 , 44 , 14 ] in the machine learning literature. Our method avoids unstable linear algebraic computations and is easy to integrate into modern deep learning pipelines. We demonstrate that it reliably recovers dominant Koopman subspaces and supports downstream tasks such as prediction and eigen-analysis. 2 Problem Setting and Preliminaries In this section, we introduce the problem setting and establish the foundation for the subsequent discussion. We begin by formulating the problem in the context of discrete-time dynamical systems, which will serve as our primary focus. We then briefly address the continuous-time case. Next, we review two existing approaches, VAMPnet and DPNet, and highlight their limitations, thereby motivating the need for our proposed method. 2.1 Discrete-Time Dynamical Systems Consider a stochastic discrete-time dynamical system \mathbf{x}_{t+1}=\xi(\mathbf{F}(\mathbf{x}_{t}),\epsilon_{t}) , where \mathbf{F}\colon\mathcal{X}\to\mathcal{X} is a possibly nonlinear mapping for a domain \mathcal{X}\subseteq\mathbb{R}^{d} , \epsilon_{t}\sim\mathcal{D} is an independent noise random variable, and \mathbf{x}_{t}\mapsto\xi(\mathbf{F}(\mathbf{x}_{t}),\epsilon_{t})\in\mathcal{X} captures an independent noise in the process, such as the additive white Gaussian noise. 1 1 1 All machinery developed here can be also adapted for a discrete state space, but we focus on continuous \mathcal{X} . Assuming that \mathbf{F} and the noise distribution \mathcal{D} are time-invariant, the process becomes a time-homogeneous Markov process with transition density p(\mathbf{x}^{\prime}\mid\mathbf{x}) , i.e., \Pr(\mathbf{X}_{t+1}\in A\mid\mathbf{X}_{t}=\mathbf{x})=\int_{A}p(\mathbf{x}^{\prime}\mid\mathbf{x})\,d\mathbf{x}^{\prime} for all measurable sets A\subseteq\mathcal{X} and all t\geq 0 . In the stochastic setup, the dynamics is fully captured by the transition density p(\mathbf{x}^{\prime}|\mathbf{x}) , which is induced by (\xi\circ\mathbf{F})(\cdot) , and thus the problem becomes analyzing p(\mathbf{x}^{\prime}|\mathbf{x}) of a Markov chain. Here, the Koopman operator becomes the conditional expectation operator. That is, for an observable g\colon\mathcal{X}\to\mathbb{R} , we can write \displaystyle(\mathpzc{K}g)(\mathbf{x})\triangleq\mathbb{E}_{p(\mathbf{x}^{\prime}|\mathbf{x})}[g(\mathbf{x}^{\prime})]. By the Markov property, repeatedly applying the Koopman operator will correspond to the multi-step prediction in the sense that it captures the posterior mean of g(\mathbf{x}_{t}) given \mathbf{x}_{0}=\mathbf{x}_{0} , i.e., (\mathpzc{K}^{t}g)(\mathbf{x}_{0})=\mathbb{E}_{p(\mathbf{x}_{t}|\mathbf{x}_{0})}[g(\mathbf{x}_{t})] . Throughout the paper, we will assume that \mathpzc{K} is compact following [ 13 ] , which is a mild assumption that holds for a large class of Markov processes; see, e.g., [ 11 ] . We remark that we cannot directly apply the existing spectral techniques to a deterministic dynamical system, including the technique developed in this paper, since the corresponding Koopman operator is neither Hilbert–Schmidt nor compact; see [ 43 , Appendix A.5] . Considering stochasticity breaks the degeneracy and allow the linear algebraic techniques applicable. Moreover, assuming stochasticity is not necessarily a restrictive assumption, as physical processes in the real-world may be inherently noisy. Problem Setting. Our goal is to analyze the Markov dynamics of the length- T trajectory (\mathbf{x}_{0},\mathbf{x}_{1},\ldots,\mathbf{x}_{T})\sim p(\mathbf{x}_{0})p(\mathbf{x}_{1}|\mathbf{x}_{0})\ldots p(\mathbf{x}_{T}|\mathbf{x}_{T-1}) , where p(\mathbf{x}_{0}) is a distribution for the initial state. In practice, we assume that we have access to N independent, random trajectories, and collect the consecutive transition pairs \{(\mathbf{x}_{0},\mathbf{x}_{1}),(\mathbf{x}_{1},\mathbf{x}_{2}),\dots,(\mathbf{x}_{T-1},\mathbf{x}_{T})\} and define the joint distribution over the pair as p(\mathbf{x},\mathbf{x}^{\prime})\triangleq\frac{1}{T}\sum_{t=0}^{T-1}p_{t}(\mathbf{x})p(\mathbf{x}^{\prime}|\mathbf{x}) , where p_{0}(\mathbf{x})=p(\mathbf{x}_{0}) and p_{t+1}(\mathbf{x}^{\prime})\triangleq\int p(\mathbf{x}^{\prime}|\mathbf{x})p_{t}(\mathbf{x})d\mathbf{x} . We let \rho_{0}(\mathbf{x}) and \rho_{1}(\mathbf{x}^{\prime}) denote the marginal distribution over \mathcal{X} of the current and future states. Note that while \rho_{1}(\mathbf{x}^{\prime})=\mathbb{E}_{\rho_{0}(\mathbf{x})}[p(\mathbf{x}^{\prime}|\mathbf{x})] holds, the two distributions, \rho_{0}(\mathbf{x}) and \rho_{1}(\mathbf{x}^{\prime}) can differ significantly, particularly when the trajectory length T is relatively short. Let (\mathcal{X},\mathcal{F}) be a measurable space and let \rho_{0} and \rho_{1} be (finite) measures on (\mathcal{X},\mathcal{F}) representing the distributions of the current and future states, respectively. For any measure \rho on (\mathcal{X},\mathcal{F}) define L_{\rho}^{2}(\mathcal{X})\triangleq\{f\colon\mathcal{X}\to\mathbb{C}|\int|f(\mathbf{x})|^{2}\rho(d\mathbf{x})<\infty\} . Equipped with the inner product \langle f,g\rangle_{\rho}\triangleq\int f(\mathbf{x})g(\mathbf{x})\rho(d\mathbf{x}) , L_{\rho}^{2}(\mathcal{X}) is a Hilbert space. The Koopman operator \mathpzc{K} is then a mapping from L_{\rho_{1}}^{2}(\mathcal{X}) to L_{\rho_{0}}^{2}(\mathcal{X}) , which is an integral kernel operator with the transition kernel k(\mathbf{x},\mathbf{x}^{\prime})\triangleq\frac{p(\mathbf{x}^{\prime}|\mathbf{x})}{\rho_{1}(\mathbf{x}^{\prime})} . The adjoint operator of \mathpzc{K}^{*} then acts as the backward predictor , i.e., (\mathpzc{K}^{*}f)(\mathbf{x}^{\prime})=\mathbb{E}_{q(\mathbf{x}|\mathbf{x}^{\prime})}[f(\mathbf{x})] , where q(\mathbf{x}|\mathbf{x}^{\prime})\triangleq\frac{\rho_{0}(\mathbf{x})p(\mathbf{x}^{\prime}|\mathbf{x})}{\rho_{1}(\mathbf{x}^{\prime})} denotes the conditional distribution induced by \rho_{0}(\mathbf{x})p(\mathbf{x}^{\prime}|\mathbf{x}) . Similar to \mathpzc{K} , repeated application of \mathpzc{K}^{*} yields multi-step backward prediction: ((\mathpzc{K}^{*})^{t}f)(\mathbf{x}_{0})=\mathbb{E}_{p(\mathbf{x}_{0}|\mathbf{x}_{t})}[f(\mathbf{x}_{0})]. Special Case 1: Stationary Markov Processes. Given p(\mathbf{x}^{\prime}|\mathbf{x}) , let \pi(\mathbf{x}) denote the stationary distribution, i.e., the distribution satisfies \mathbb{E}_{\pi(\mathbf{x})}[p(\mathbf{x}^{\prime}|\mathbf{x})]=\pi(\mathbf{x}^{\prime}) . Under mild regularity conditions such as ergodicity and irreducibility, such distribution exists and is unique. 2 2 2 For example, if the stochasticity in the system is induced by an additive noise, i.e., \mathbf{x}_{t+1}=\mathbf{F}(\mathbf{x}_{t})+\epsilon_{t} for \epsilon_{t}\sim\mathcal{D} , with the density of \mathcal{D} being positive almost everywhere, then the system is asymptotically stable, i.e., it converges to a unique stationary distribution; see [ 17 , Corollary 10.5.1] . In the stationary case, the marginal distributions at both times are equal, i.e., \rho_{0}=\rho_{1}=\pi , and the Koopman operator \mathpzc{K} becomes a map from L_{\pi}^{2}(\mathcal{X}) to L_{\pi}^{2}(\mathcal{X}) . Assuming ergodicity, we can collect the time-lagged pairs (\mathbf{x},\mathbf{x}^{\prime}) ’s from a long, single trajectory, as time averages converge to expectations under the stationary distribution. Special Case 2: Reversible Markov Processes. A Markov process is

Read the complete text