Kernel manifolds: nonlinear-augmentation dimensionality reduction using reproducing kernel Hilbert spaces

Alejandro N. Diaz, Jacob T. Needels, Irina K. Tezaur, Patrick J. Blonigan · 2025

Read Kernel manifolds: nonlinear-augmentation dimensionality reduction using reproducing kernel Hilbert spaces by Alejandro N. Diaz, Jacob T. Needels, Irina K. Tezaur, Patrick J. Blonigan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-00224.txt.

Introduction

Alejandro N. Diaz, Jacob T. Needels, Irina K. Tezaur, and Patrick J. Blonigan generalize dimensionality reduction in 2025 with kernel manifolds. Reduction compresses high-dimensional data into a few coordinates. Linear maps are cheap but can miss curved structure, so recent work adds a nonlinear correction to the reconstruction, as in quadratic manifolds.

Feature-based corrections usually fit a polynomial term by least squares. The new method learns the correction from a user-chosen reproducing kernel Hilbert space, a space of functions set by a kernel. Polynomial and radial basis corrections fall out as special cases. Training cost stays low, and error falls monotonically as the space grows. Flexible corrections sharpen reduced models of complex physics.

Catalog record

CREATORAlejandro N. Diaz, Jacob T. Needels, Irina K. Tezaur, Patrick J. Blonigan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-00224

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Kernel manifolds: nonlinear-augmentation dimensionality reduction using reproducing kernel Hilbert spaces Alejandro N. Diaz, Jacob T. Needels, Irina K. Tezaur, Patrick J. Blonigan Abstract This paper generalizes recent advances on quadratic manifold (QM) dimensionality reduction by developing kernel methods-based nonlinear-augmentation dimensionality reduction. QMs, and more generally feature map-based nonlinear corrections, augment linear dimensionality reduction with a nonlinear correction term in the reconstruction map to overcome approximation accuracy limitations of purely linear approaches. While feature map-based approaches typically learn a least-squares optimal polynomial correction term, we generalize this approach by learning an optimal nonlinear correction from a user-defined reproducing kernel Hilbert space. Our approach allows one to impose arbitrary nonlinear structure on the correction term, including polynomial structure, and includes feature map and radial basis function-based corrections as special cases. Furthermore, our method has relatively low training cost and has monotonically decreasing error as the latent space dimension increases. We compare our approach to proper orthogonal decomposition and several recent QM approaches on data from several example problems. Keywords: nonlinear dimensionality reduction, quadratic manifolds, kernel methods, surrogate modeling 1 Introduction In recent years, data-driven surrogate modeling of complex physical systems has been an active research area in scientific machine learning (SciML). A common challenge of constructing surrogate models is that data from simulations of partial differential equations (PDEs) are often high dimensional, with the dimensionality dictated by the discretization methods used and the fidelity of the corresponding mesh. Consequently, surrogate modeling techniques typically involve a dimensionality reduction component to extract low-dimensional features from high-dimensional simulation data, which can then be used to construct a low-dimensional, computationally efficient surrogate model. Crucially, the dimensionality reduction technique used fundamentally limits the achievable accuracy of the resulting surrogate model. Classical linear dimensionality reduction techniques, such as proper orthogonal decomposition (POD) [ 19 , 21 ] , seek to identify a low-dimensional affine subspace that best captures the high-dimensional data. POD identifies this subspace as the span of the leading left singular vectors of a representative set of high-dimensional snapshot data. The POD basis induces an affine encoder map that projects high-dimensional snapshots onto generalized coordinates that lie in a low-dimensional latent space , which can then be mapped to an approximate reconstruction of the high-dimensional snapshot via an affine decoder map. While POD is effective in numerous applications, it is also well-known to poorly approximate data from problems with slowly decaying Kolmogorov n -width [ 32 ] , such as transport-dominated problems or problems with sharp gradients. In these cases, a large POD basis is required to accurately approximate the given data. However, despite lying in a high-dimensional affine subspace, these data may still possess an inherent low-dimensional nonlinear structure. This assumption motivates using nonlinear dimensionality reduction for problems with slowly decaying Kolmogorov n -width. A number of nonlinear dimensionality reduction approaches have recently been proposed to circumvent the Kolmogorov n -width barrier. These techniques are often collectively referred to as nonlinear manifolds to distinguish them from linear dimensionality reduction, and include autoencoders (AEs) [ 28 , 24 , 36 , 9 , 11 , 17 , 8 , 7 , 13 ] , quadratic manifolds (QMs) [ 3 , 6 , 15 , 14 , 43 , 42 , 22 , 37 ] , the projection-based reduced-order model + artificial neural network (PROM-ANN) technique [ 4 ] , and the recent Gaussian process (GP) and radial basis function (RBF) interpolation-based nonlinear closure approach [ 10 ] . AEs consist of fully nonlinear encoder and decoder maps, which can offer substantial accuracy improvements over linear dimensionality reduction. However, they can be difficult to train due to their dependence on the neural network architecture and large data requirements. Furthermore, they often lack interpretable structure and can be slow to evaluate, which is detrimental in use cases requiring fast online evaluations, such as reduced-order modeling. To address these issues, the QM, PROM-ANN, and GP/RBF-interpolation approaches instead augment linear dimensionality reduction with a nonlinear correction term in the decoder. We characterize this class of approaches as nonlinear-augmentation manifolds . QMs model the nonlinear augmentation term as a quadratic polynomial, while PROM-ANN models it as an ANN and the GP/RBF-interpolation approach models it as the mean function of a GP or as an RBF interpolant. These approaches have interpretable structure and, with the exception of PROM-ANN, comparatively low training cost relative to AEs, while still providing improved accuracy over linear dimensionality reduction approaches. However, QMs, and more generally, feature-map based nonlinear-augmentation manifolds, are typically limited to polynomial corrections, which can restrict the achievable accuracy of these methods, while becoming increasingly difficult to train as the polynomial order increases. Although using GPs and RBF interpolants as in [ 10 ] can address these difficulties, this approach can be further generalized to improve the flexibility of nonlinear-augmentation manifolds. In this paper we develop a nonlinear-augmentation dimensionality reduction technique using regularized kernel interpolation, which generalizes recent work on QMs and GP/RBF interpolation-based nonlinear closures, yielding greater expressivity in the nonlinear approximation, reduced training cost compared to QMs, and improved accuracy on out-of-sample data. Regularized kernel interpolation is a generalization of RBF interpolation relying on mature mathematical theory from reproducing kernel Hilbert spaces (RKHS) [ 39 , 40 , 30 , 46 ] . While kernel methods are a common tool in classical data science and machine learning, they have become increasingly popular in SciML in recent years, with notable applications in operator and equation learning [ 5 , 29 , 23 ] , tensor decomposition [ 27 ] , and reduced-order and surrogate modeling [ 12 , 2 ] . In our dimensionality reduction method, which we refer to as kernel manifold dimensionality reduction, one can model the nonlinear augmentation term with arbitrary nonlinearities by choosing an appropriate kernel, and includes QMs and GP/RBF-based closures as special cases. Furthermore, unlike feature map-based nonlinear augmentations, regularized kernel interpolation allows one to model the nonlinear term using polynomials of arbitrarily high order without increasing the number of degrees of freedom in the learning problem. We also note that our approach is distinct from kernel principal component analysis (KPCA) [ 41 ] and kernel POD (KPOD) [ 38 ] in that it augments classical POD with a nonlinear correction term, whereas KPCA/KPOD returns a nonlinear kernel-based encoder without an explicit decoder, thus requiring one to learn an approximate decoder function through, e.g., kernel interpolation or neural networks. While we purely focus on dimensionality reduction in this paper, we emphasize that our method is general-purpose, and can be used for standalone dimensionality reduction, intrusive model reduction as in [ 10 ] , or non-intrusive model reduction through integration with operator inference (OpInf) [ 16 , 25 , 34 , 35 , 14 ] or the recent kernel methods-based approach of [ 12 ] . The outline of this paper is as follows. We first describe the problem setup and review POD and feature map-based nonlinear-augmentation dimensionality reduction in Section 2 . Next, we describe our new kernel manifold approach in Section 3 , beginning with a brief review of regularized kernel interpolation in Section 3.1 . We then compare our method with POD and the recently proposed Alternating QM [ 14 ] and Greedy QM [ 43

Read the complete text