Manifold-Matching Autoencoders
Laurent Cheret, Vincent Létourneau, Isar Nejadgholi, Chris Drummond, Hussein Al Osman, Maia Fraser · 2026
Read Manifold-Matching Autoencoders by Laurent Cheret, Vincent Létourneau, Isar Nejadgholi, Chris Drummond, Hussein Al Osman, Maia Fraser (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16568.txt.
Introduction
Cheret and co-authors regularize autoencoders by matching distances, not coordinates. Their Manifold-Matching scheme aligns latent pairwise distances with input ones. Training minimizes the mean squared error between the two sets. Distance alignment also allows a lower-dimensional latent space. That freedom adds room to adapt. Tests beat rival schemes on neighbor-distance preservation. Other shape-based measures agreed. It further gives a scalable stand-in for multidimensional scaling.
Catalog record
| CREATOR | Laurent Cheret, Vincent Létourneau, Isar Nejadgholi, Chris Drummond, Hussein Al Osman, Maia Fraser |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33367 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16568
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Manifold-Matching Autoencoders Laurent Cheret, Vincent Létourneau, Isar Nejadgholi, Chris Drummond, Hussein Al Osman, Maia Fraser Abstract We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower dimensional representation of the data, adding flexibility to the method. We found that this regularization outperforms similar methods on metrics based on preservation of closest neighbors distances and persistence homology based measures. We also observe MMAE provides a scalable approximation of ‘Multi-Dimensional Scaling” (MDS). 1 Introduction Dimensionality reduction is fundamental to modern data analysis, enabling visualization and interpretation of high-dimensional datasets. Autoencoders ( 15 ; 11 ) learn compressed representations by minimizing reconstruction error, but this objective alone does not guarantee preservation of any particular geometric or topological structure. When the encoder ignores these structures, similar objects in the input space may be mapped to distinct regions of the latent space, creating discontinuities that negatively affect the decoder’s ability to reconstruct ( 3 ) . This problem can also affect other downstream tasks. For example, in anomaly detection or when visualizing developmental trajectories in single-cell data ( 8 ) or exploring latent spaces in generative models ( 7 ; 42 ) —additional regularization becomes necessary. 1.1 Topology and Geometry in Autoencoders Following the success of statistical methods using topological data analysis tools like persistence diagrams ( 35 ) , there has been a recent effort to improve the preservation of topological features by autoencoders, which we classify as topological or geometrical methods. Topological methods ( 26 ; 37 ) use persistent homology to identify and preserve multi-scale structural features such as connected components, loops, and voids. Geometric methods ( 34 ; 27 ; 21 ) focus on preserving local angles and distances. Take, for example, the nested spheres dataset ( 26 ) , a simple synthetic yet highly nonlinear case: ten 100-dimensional small spheres are nested inside a larger 100D enclosing sphere. A topologically accurate 2D representation should preserve this nesting relationship, with the outer sphere cluster surrounding the inner sphere clusters. To date, only topological autoencoder variants consistently recover this structure in 2D/3D. Other autoencoder variants and nonparametric methods such as UMAP ( 22 ) , t-SNE ( 38 ) , and PHATE ( 24 ) fail in this case. Interestingly, we found that the classical method Multidimensional Scaling (MDS) ( 36 ) also successfully recovers the nesting relationship—a result that, to the best of our knowledge, has not been reported in previous work. Figure 1 : Left: Overview of the current approach. The Manifold-Matching regularization MM-reg is added to the objective function of the standard AE, forming MMAEs. Top Right: 2D latent spaces of the Nested Spheres dataset ( 26 ) . Standard AE (Vanilla) using no MM-reg and 9 MMAE models using different number of PCA components in their regularization ( 1\rightarrow 100 ). Bottom Right: MMAE 2D latent spaces “copying” 2D embeddings from UMAP, t-SNE, and PCA across MNIST, F-MNIST, and CIFAR10 datasets. 1.2 Our Approach: Alignment of Pairwise Distances Classical MDS ( 36 ) utilizes the pairwise distance matrix of the entire dataset to preserve global geometry. In contrast, topological autoencoders like TopoAE ( 26 ) and RTD-AE ( 37 ) employ persistent homology signatures on pairwise distance matrices at the mini-batch level. While the former focuses on geometric distances—which may implicitly capture topology—the latter prioritize multi-scale structural connectivity, a focus that has been shown to result in superior preservation of the manifold’s global geometry. One issue arises with these methods, MDS scales poorly with data size due to memory requirements to compute the n\times n pairwise distance matrix, and topological variants of autoencoders scale poorly with the batch size b due to the persistent homology computations batch-wise. A central question emerges: What happens to latent spaces when global geometry preservation is imposed in autoencoders? We address this by introducing a regularization term called Manifold-Matching (MM-reg), defined as the MSE between the pairwise distance matrix D_{Z} of the latent space and a reference distance matrix D_{E} computed from either the input data X itself or its respective embedding. Crucially, since both D_{Z} and D_{E} are b\times b matrices (where b is the batch size), the dimensionality of the reference space is decoupled from the bottleneck dimensionality. This means, for example, that a 2D latent space can be regularized using distances from a 50D or 100D reference representation. Note there is strong theoretical justification in the choice to replace the data by its distance matrix. In short, distance preservation implies topology preservation. This is the content of section 2.2 . Figure 1 illustrates this effect on the nested spheres dataset. Without regularization, the standard AE projects the inner spheres outside the cluster representing the outer sphere, consistent with prior literature. However, with MM-reg, as the number of PCA components in the reference increases, the nesting structure present in the original data emerges, with the inner sphere clusters progressively pulled inside the outer sphere cluster. In the specific case in which the reference embeddings are 2D, a “copying” effect can be seen where the 2D latent space approximates the reference, allowing the autoencoder to extend known representation to new data points. Specifically, our contributions are: (1) We introduce the Manifold-Matching Autoencoder (MMAE) , an unsupervised framework for global structure-aware dimensionality reduction. (2) We study its visualization effects on synthetic datasets where the topology is intuitively understood. (3) We extend experiments to real-world benchmarks, showing competitive performance against topological and geometrical autoencoder variants. (4) We provide discussions on global geometry preservation as a proxy for topology preservation. 2 Background We review the key concepts underlying our approach: persistent homology as the gold standard for topological comparison, and the relation between distance preservation and topology. 2.1 Persistent Homology The importance of understanding data topology has been recognized since the 1960s ( 31 ) . This concern is tied to the manifold hypothesis: high-dimensional data \mathbf{X}=\{\mathbf{x}_{i}\}_{i=1}^{k} with \mathbf{x}_{i}\in\mathbb{R}^{n} typically lies on or near a lower-dimensional manifold \mathcal{M}\subset\mathbb{R}^{n} . Persistent homology provides a principled way to detect the topological features of this manifold across scales ( 14 ; 5 ) . 2.2 From Distance Preservation to Topology Preservation Multidimensional Scaling (MDS) ( 36 ) provides a classical approach that finds a low-dimensional configuration of points whose pairwise distances best preserve those of the input distance matrix. The key insight is that while points \mathbf{x}_{i},\mathbf{x}_{j}\in\mathbb{R}^{n} may have many coordinates, their Euclidean distance d_{ij}=\|\mathbf{x}_{i}-\mathbf{x}_{j}\|_{2} reduces their relationship to a single scalar value. Remarkably, collecting all such pairwise distances into a matrix \mathbf{D} contains sufficient information to recover the original geometric configuration. Classical MDS formalizes this by converting distance relationships into geometric configurations through eigendecomposition of the associated Gram matrix ( 4 ; 32 ) . This distance-centric view connects naturally to topology preservation through the stability theorem. For finite metric spaces with Vietoris-Rips persistence diagrams: Theorem 2.1 (Stability ( 12 ; 9 ) ) . d_{B}(\text{Dgm}_{p}(X),\text{Dgm}_{p}(Y))\leq 2\cdot d_{GH}(X,Y) (1) for all homology dimensions p\geq 0 , where d_{B} is the bottleneck distance and d_{GH} the Gromov-Hausdorff distance.