ROCK: Riesz Occupation Kernel Methods for Learning Dynamical Systems
Victor W Rielly, Chau Nguyen, Kamel Lahouel, Nicholas Fisher, Bruno Jedynak · 2025
Read ROCK: Riesz Occupation Kernel Methods for Learning Dynamical Systems by Victor W Rielly, Chau Nguyen, Kamel Lahouel, Nicholas Fisher, Bruno Jedynak (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13791.txt.
Introduction
Learning a dynamical system from data often means fitting functions that live along observed paths, and occupation kernel methods do exactly that. Rielly and co-authors present a variational formulation for a large class of such weak problems and prove a representer theorem for it. They show how the formulation covers both standard machine learning and numerical methods, along with newer techniques.
Building on that base, they generalize the multivariate occupation kernel method into a new one they call ROCK, short for Riesz Occupation Kernel methods. The generalized method, they report, is both cheaper to compute and stronger on most of the benchmarks they tried. The paper offers the framework as a shared foundation: one theorem supports many methods, and the dynamical systems learner that falls out of it improves on its parent.
Catalog record
| CREATOR | Victor W Rielly, Chau Nguyen, Kamel Lahouel, Nicholas Fisher, Bruno Jedynak |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13791
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ROCK: Riesz Occupation Kernel Methods for Learning Dynamical Systems Victor W Rielly, Chau Nguyen, Kamel Lahouel, Nicholas Fisher, Bruno Jedynak Abstract We present a representer theorem for a large class of variational problems, generalizing regression, finite element methods, and providing techniques for learning partial and ordinary differential equations (PDEs and ODEs). We apply our formulation to the multivariate occupation kernel method (MOCK) for learning dynamical systems from data. Our generalized method outperforms the MOCK method on most tested datasets, while often requiring considerably fewer parameters. Experiments are presented for learning ODEs and PDEs. 1 Introduction Many statistical learning problems can be cast in the form [ 28 ] \min_{f\in\mathcal{F}}\left\{\sum_{i=1}^{N}L(f(x_{i}),y_{i})+\Omega(f)\right\}, (1) where \mathcal{F} is a suitable hypothesis space, y_{i} are observed responses, and \Omega(f) is a regularization term. The evaluation of f at x_{i} is a linear functional of f . This naturally leads to a generalized formulation: \min_{f\in\mathcal{F}}\left\{\sum_{i=1}^{N}L(a_{i}(f),y_{i})+\Omega(f)\right\}, (2) where a_{i}(f) denotes a linear functional applied to f , possibly varying with i . A further generalization incorporates adversarial, test function-based estimation, in which we replace the linear functionals a_{i} with bilinear functionals involving f and test functions v_{i} , leading to the variational problem: \min_{f\in\mathcal{F}}\left\{\sum_{i=1}^{N}\sup_{v_{i}\in\mathds{V}_{i}|\|v_{i}\|=1}\left\{L(a_{i}(v_{i},f),l_{i}(v_{i}))\right\}+\Omega(f)\right\}. (3) Each \mathds{V}_{i} is a space of test functions, l_{i}(\cdot) is a linear functional on \mathds{V}_{i} , and a_{i}(v_{i},f) is a bilinear functional. Notice that to make the supremum well defined, we need to optimize over test functions of unit norm. In this work, we focus on the quadratic case where L(x,y)=(x-y)^{2} , and consider regularization of the form \Omega(f)=\lambda\|f\|^{2} . Remarkably, under the assumption that \mathcal{F} is a vector-valued reproducing kernel Hilbert space (vvRKHS), this problem admits a closed-form solution. We demonstrate that this framework captures a broad range of learning problems, including multivariate ridge regression; learning the vector fields of ordinary differential equations (ODEs) from data; and learning certain classes of partial differential equations (PDEs). For instance, we can learn PDEs of the form \frac{\partial u}{\partial t}=f\left(\partial_{x}u,\partial_{xx}u,\ldots\right), where the time derivative depends explicitly on spatial derivatives of u . Such equations arise in heat diffusion, reaction-diffusion systems, and other spatiotemporal processes. These examples highlight the versatility and expressiveness of our variational framework. 1.1 Related work VvRKHSs have received increasing attention in recent years. A particularly relevant work is [ 19 ] , which also presents a unified framework for learning in vvRKHSs. There is a rich body of literature on kernel-based methods for learning and solving PDEs and dynamical systems, especially through the use of collocation points [ 5 ] . Our approach is closely aligned with the perspective of optimal recovery. For instance, in [ 20 ] , the central problem formulated in Chapter 3 for optimal recovery splines involves a minmax problem similar to ours. Additional work in this direction includes [ 14 ] , which exhibits several parallels with our framework. By formulating our results within the vvRKHS setting, we are able to exploit the computational advantages of separable kernels, as detailed in [ 22 ] . 1.2 Vector-valued reproducing kernel Hilbert spaces The central tool that enables analytical tractability in our framework is the structure of vvRKHSs. We assume both the hypothesis function f and the test functions v_{i} belong to suitable vvRKHSs. This allows us to exploit the linearity of evaluation functionals, derive closed-form solutions, and obtain efficient representations of our estimator of f . We briefly recall the definition of vvRKHSs below. For a thorough exposition on reproducing kernel Hilbert spaces, see [ 21 ] . Definition 1 . Let \mathds{H} be a Hilbert space of vector-valued functions taking a nonempty set \mathcal{X} to \mathds{R}^{d} . \mathds{H} is a vvRKHS if for every v\in\mathds{R}^{d} and x\in\mathcal{X} , the linear functional l_{v,x}(f)\equiv f(x)^{T}v (4) is bounded. A more complete definition is provided in Appendix A . As a trivial yet interesting example, every finite-dimensional Hilbert space of vector-valued functions is a vector-valued RKHS. 2 Riesz occupation kernel method The representer theorem is a foundational result in machine learning [ 26 ] . It states that some functional optimization problems have a unique finite-dimensional solution as a linear combination of data-dependent functions. In the quadratic case, this solution is obtained by solving a linear system. We provide a generalization of this result, extending its applicability. The proof uses standard functional analysis tools, including the Riesz representation theorem and orthogonal projections. This proof is relegated to Appendix B . Theorem 1 (Representer theorem) . Let \mathds{H},\mathds{V}_{1},...,\mathds{V}_{n} be vvRKHSs with kernels K_{\mathds{H}},K_{\mathds{V}_{1}},...,K_{\mathds{V}_{n}} , inner products \left<\cdot,*\right>_{\mathds{H}},\left<\cdot,*\right>_{\mathds{V}_{1}},...,\left<\cdot,*\right>_{\mathds{V}_{n}} and norms \|\cdot\|_{\mathds{H}},\|\cdot\|_{\mathds{V}_{1}},...,\|\cdot\|_{\mathds{V}_{n}} respectively. Assume \mathds{H} takes a set \mathcal{X} to \mathds{R}^{d} and \mathds{V}_{i} takes a set \mathcal{Y}_{i} to \mathds{R}^{d_{i}} . Assume also \mathds{V}_{1},...,\mathds{V}_{n} are finite dimensional with dimensions q_{1},...,q_{n} and feature functions \Psi_{1},...,\Psi_{n} with \Psi_{i}:\mathcal{Y}_{i}\Rightarrow\mathds{R}^{q_{i}\times d_{i}} respectively. Let p_{1},...,p_{n} be bilinear functionals taking \mathds{H}\times\mathds{V}_{i} to \mathds{R} that are continuous with respect to both entries. Let l_{1},...,l_{n} be linear functionals taking \mathds{V}_{1},...,\mathds{V}_{n} to \mathds{R} that are also continuous. We notate e_{j} the j^{th} element of the natural basis of the appropriate Euclidean space. Then, the variational problem \inf_{f\in\mathds{H}}\left\{\sum_{i=1}^{n}\sup_{v\in\mathds{V}_{i},\|v\|=1}\left\{\left(p_{i}(f,v)-l_{i}(v)\right)^{2}\right\}+\lambda\|f\|^{2}_{\mathds{H}}\right\} (5) admits a unique solution. Moreover, this solution is provided by solving the linear system \left(M+\lambda I\right)\alpha=y, (6) where \forall x\in\mathcal{X} , f(x) is given by f(x)=\sum_{i=1}^{n}\sum_{k=1}^{q_{i}}\alpha_{i,k}\mathcal{L}^{*}_{i,k}(x),\ \mathcal{L}^{*}_{i,k}(x)=\sum_{j=1}^{d}p_{i}(K_{\mathds{H}}(\cdot,x)e_{j},\Psi_{i}(\cdot)^{T}e_{k})e_{j}. (7) Letting N=\sum_{i=1}^{n}q_{i} , M\in\mathds{R}^{N\times N} is given by M_{l,m:i,k}=\left<\mathcal{L}^{*}_{l,m},\mathcal{L}^{*}_{i,k}\right>_{\mathds{H}} so that M=\left[\begin{array}[]{ccc}M_{1:1}&...&M_{1:n}\\ \vdots&\ddots&\vdots\\ M_{n:1}&...&M_{n:n}\end{array}\right]\mbox{ with }M_{i:j}=\left[\begin{array}[]{ccc}M_{i,1:j,1}&...&M_{i,1:j,q_{j}}\\ \vdots&\ddots&\vdots\\ M_{i,q_{i}:j,1}&...&M_{i,q_{i}:j,q_{j}}\end{array}\right] with M_{l,m:i,k}=\left<\mathcal{L}^{*}_{l,m},\mathcal{L}^{*}_{i,k}\right>_{\mathds{H}}=p_{i}\left(\sum_{j=1}^{d}p_{l}\left(K_{\mathds{H}}(\cdot,*)e_{j},\Psi_{l}(\cdot)^{T}e_{m}\right)e_{j},\Psi_{i}(*)^{T}e_{k}\right) (8) where *\mapsto p_{l}\left(K_{\mathds{H}}(\cdot,*)e_{j},\Psi_{l}(\cdot)^{T}e_{m}\right)e_{j} is viewed as a function in \mathds{H} , taking * as an input, and y is given by y_{i,k}=l_{i}(\Psi_{i}(\cdot)^{T}e_{k})\mbox{ so that }y^{T}=\left[y_{1,1},...,y_{1,q_{1}},y_{2,1},...,y_{2,q_{2}},...,y_{n,q_{n}}\right] (9) It should be emphasized that all parametric methods are inherently finite-dimensional, and all linear functionals on finite-dimensional spaces are bounded. Examples of common linear functionals that arise in defining the loss in ( 5 ) include function evaluation, integration, differentiation, and expected values. 2.1 Example applications: To emphasize the breadth of applications, we present some problems that are interpretable within this framework. Multivariate ridge regression: minimizing the loss \min_{f}\left\{\sum_{i=1}^{n}\|f(x_{i})-y_{i}\|_{\mathds{R}^{d}}^{2}+\lambda\|f\|^{2}_{\mathds{H}}\right\} (10) can be expressed by \min_{f\in\mathds{H}}\left\{\sum_{i=1}^{n}\sum_{j=1}^{d}\sup_{v\in\{e_{j},-e_{j}\}}\left\{(v^{T}f(x_{i})-v^{T}y_{i})^{2}\right\}+\lambda\|f\|^{2}\right\}, (11) where p_{i}(f,v)=v^{T}f(x_{i}) , and l_{i}(v)=v^{T}y_{i} (see Appendix C ). Finite element methods: Letting \mathds{H} be an RKHS of functions satisfying some boundary conditions, and V a finite-dimensional Hilbert space of functions, we minimize: \min_{u\in\mathds{H}}\left\{\sup_{v\in V,\|v\|=1}\left\{(a(u,v)-f(v))^{2}\right\}\right\}, (12) where a is a bilinear form and f is a linear functional of v . For finite element methods, we typically do not add regularization. Instead, we require a to be coercive to guarantee a stable solution, and we may sometimes use many different approaches to handle the boundary constraints. Here we assume we optimize over a space of functions that satisfy the boundary constraints. We see by adding the regularization term as in our theorem we guarantee a stable solution to the minimization problem without requiring a to be coercive [ 7 ] , Chapter 3. Learning PDEs: In addition to solving PDEs, we may use our methods to learn PDEs. For example consider u_{t}=f(u,u_{x},u_{xx},...). (13) Defining u_{i}=u(t,x_{i}) and u_{i,x}=\partial_{x}u(t,x_{i}) , we may solve this problem by minimizing \min_{f\in\mathds{H}}\left\{\sum_{i=1}^{N}\sup_{v\in V,\|v\|=1}\left\{\left(\int_{t_{0}}^{t_{f}}f(u_{i},u_{i,x},...)v(t,x_{i})dt-\int_{t_{0}}^{t_{f}}u_{i,t}v(t,x_{i})dt\right)^{2}\right\}+\lambda\|f\|^{2}\right\} (14) If v(t,x_{i}) are chosen to be piecewise constant with support on the intervals [t_{j},t_{j+1}] we recover the loss in Section 4 . Here p_{i}(f,v)=\int f(u_{i},u_{i,x},...)v(t,x_{i})dt is the bilinear functional and l_{i}(v)=\int u_{i,t}v(t,x_{i})dt is the linear functional, see Appendix E . Liouville operator occupation kernel method: In