Learning Reconstructive Embeddings in Reproducing Kernel Hilbert Spaces via the Representer Theorem

Enrique Feito-Casares, Francisco M. Melgarejo-Meseguer, José-Luis Rojo-Álvarez · 2026

Read Learning Reconstructive Embeddings in Reproducing Kernel Hilbert Spaces via the Representer Theorem by Enrique Feito-Casares, Francisco M. Melgarejo-Meseguer, José-Luis Rojo-Álvarez (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05811.txt.

Introduction

Enrique Feito-Casares and 2 co-authors propose in 2026 new algorithms for learning compact data representations inside reproducing kernel Hilbert spaces. Representation learning seeks the hidden low-dimensional structure of high-dimensional data. Their method begins by reconstructing each observation as a combination of the others, right inside the kernel space.

A vector form of the Representer Theorem optimizes those reconstructions, and an operator-valued kernel extends the idea to vector-valued data while keeping one scalar similarity function. A kernel-alignment step then projects data into a latent space whose geometry matches the reconstruction structure. Tests on simulated shapes and on real cancer-activity and network-intrusion datasets gave empirical support for the approach's practical effectiveness.

Catalog record

CREATOREnrique Feito-Casares, Francisco M. Melgarejo-Meseguer, José-Luis Rojo-Álvarez
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-05811

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning Reconstructive Embeddings in Reproducing Kernel Hilbert Spaces via the Representer Theorem Thanks: This work has been submitted to the IEEE for possible publication Enrique Feito-Casares, Francisco M. Melgarejo-Meseguer, José-Luis Rojo-Álvarez Abstract Motivated by the growing interest in representation learning approaches that uncover the latent structure of high-dimensional data, this work proposes new algorithms for reconstruction-based manifold learning within Reproducing-Kernel Hilbert Spaces (RKHS). Each observation is first reconstructed as a linear combination of the other samples in the RKHS, by optimizing a vector form of the Representer Theorem for their autorepresentation property. A separable operator-valued kernel extends the formulation to vector-valued data while retaining the simplicity of a single scalar similarity function. A subsequent kernel-alignment task projects the data into a lower-dimensional latent space whose Gram matrix aims to match the high-dimensional reconstruction kernel, thus transferring the auto-reconstruction geometry of the RKHS to the embedding. Therefore, the proposed algorithms represent an extended approach to the autorepresentation property, exhibited by many natural data, by using and adapting well-known results of Kernel Learning Theory. Numerical experiments on both simulated (concentric circles and swiss-roll) and real (cancer molecular activity and IoT network intrusions) datasets provide empirical evidence of the practical effectiveness of the proposed approach. Keywords Reproducing Kernel Hilbert Space \cdot Representer Theorem \cdot Manifold Learning \cdot Kernel Alignment \cdot Dimension Reduction 1 Introduction As experienced by any practitioner, data representation is critical to the application of machine learning (ML) , whatever the targeted task, supervised or unsupervised. In this sense, recent efforts have focused their attention on representation learning (RL) as an effective way to structure knowledge, enabling models to learn latent representations that capture the underlying patterns in the data and facilitate generalization. This trend has gained momentum with the rise of deep learning and unsupervised feature learning, where representations learned directly from raw data have demonstrated superior performance across a wide range of tasks ( 2 ) . Among the various approaches for generating representations, manifold learning (MnL) has attracted increasing attention as a powerful approach to RL for producing compact, low-dimensional encodings of high-dimensional inputs. MnL relies on the assumption that data lie on a lower-dimensional manifold embedded within a higher-dimensional space, and aims to uncover this latent structure by preserving the intrinsic geometry of the data. This approach provides meaningful representations that can be used to prepare data for downstream machine learning tasks. Recent studies have highlighted the impact of incorporating MnL techniques into both ML and deep learning (DL) pipelines to address a variety of domain-specific challenges. In biomedical applications, MnL has been used for stress detection ( 3 ) , electrocardiographic signal assessment ( 29 ) , and ambient health monitoring in smart homes ( 22 ) . In engineering domains, it supports structural damage detection ( 6 ) , aerodynamic design optimization ( 20 ) , and cybersecurity in critical systems ( 8 ) . Moreover, the widespread use of MnL for data visualization has been well documented, with techniques such as Student’s t-distributed Stochastic Neighbor Embedding (t-SNE) ( 34 ) and Uniform Manifold Approximation and Projection (UMAP) ( 21 ) facilitating the exploration and interpretation of data structures in lower-dimensional spaces ( 1 ; 19 ) . Whether previous MnL techniques tend to preserve distances, neighbourhood probabilities, and local linearity, the present work instead centers dimensionality reduction on the reconstruction geometry within a Reproducing Kernel Hilbert Space (RKHS) , so the resulting embedding is shaped by data samples reconstruction ability rather than distance alone. To make this idea work for vector-valued data, the study introduces a separable operator-valued kernel that applies a single scalar kernel function uniformly across every output dimension. Finally, a kernel-alignment objective ties the high-dimensional reconstruction relationships to the low-dimensional embedding, ensuring that the learned latent space mirrors the structure uncovered in the original feature space. The contributions of the present work can be summarized as follows: (i) a new autoreconstructive criterion for manifold learning in RKHS, and (ii) an embedding mechanism that aligns reconstruction geometry with latent similarity. Both contributions are supported by numerical experiments on various datasets, including kernelized input data. The paper is structured as follows. Section 2 reviews existing dimensionality reduction techniques to position the present work within this landscape. Sections 3 and 4 introduce the mathematical foundations and describe the proposed algorithm. Section 5 presents quantitative and qualitative results comparing our method to established baselines. Finally, Section 6 discusses the results and outlines directions for future research. 2 Related Work In the context of the present proposal, several related works share a similar conceptual approach, particularly concerning dimensionality reduction tools and, more specifically, the use of Kernel Theory. Dimensionality reduction techniques for high-dimensional data can be broadly classified into three paradigms, namely, spectral methods, probabilistic methods, and neural network-based methods ( 11 ) . Each paradigm provides valuable insights into the latent structure of data, but also presents limitations that motivate our approach. Spectral methods, such as Principal Component Analysis (PCA) ( 15 ) , Fisher Discriminant Analysis (FDA) ( 13 ) , and their kernelized variants (e.g., KPCA ( 31 ) and KFDA ( 24 ) ), extract key directions through generalized eigenvalue problems. Probabilistic approaches, Probabilistic PCA (PPCA) ( 28 ) , SNE ( 14 ) , and UMAP ( 21 ) provide uncertainty quantification and capture complex nonlinear relationships. However, these methods are limited by intractable inference problems and restrictive noise assumptions and rarely yield explicit mappings. Neural network-based techniques, particularly Autoencoders (AEs), have proven effective in learning rich, nonlinear representations. However, while they can achieve state-of-the-art performance, their latent representations often lack transparency, making them true black-box models. In line with the proposed approach, recent attempts have focused on kernelizing autoencoder criteria, initially in shallow autoencoders ( 12 ) , and then extending this theoretical analysis to learning low-dimensional latent spaces in vector-valued RKHS . This generalizes the kernelization criterion of autoencoders to an arbitrary number of layers, using a reconstruction criterion defined on the input data space, in which the dimensionality reduction occurs at the bottleneck defined by the neural network ( 16 ; 17 ) . The criterion for reconstructing data samples based on neighbors was originally proposed under the Local Linear Embedding (LLE) algorithm ( 27 ) . Subsequent efforts led to the development of a kernelized variant (KLLE) ( 37 ) , which performs reconstruction in RKHS . While this method shares similarities with our approach, here instead, is considered self-reconstruction for the embedded data points, thus also focusing on the input space reconstruction simultaneously. Kernel Dimensionality Reduction (KDR) ( 10 ) is a family of methods within the Sufficient Dimensionality Reduction (SDR) paradigm ( 18 ) . Despite the similarity in name, KDR is primarily a supervised statistical technique designed to reduce the covariance of data in the RKHS and project it into a lower-dimensional space. An unsupervised variant has also been proposed ( 36 ) ; however, our methodology differs fundamentally from all existing variants of KDR. 3 Background on Dimensionality Reduction and Kernel Methods In this section, we first outline the theoretical foundations of dimensionality reduction, introducing the manifold hypothesis as the central motivation for many nonlinear approaches. We then present the basic concepts of

Read the complete text