MEDAL: Manifold Embedding Distillation via Autoencoder Learning
Irene Chang, Tarek M. Zikry, Genevera I. Allen · 2026
Read MEDAL: Manifold Embedding Distillation via Autoencoder Learning by Irene Chang, Tarek M. Zikry, Genevera I. Allen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24244.txt.
Introduction
Chang, Zikry, and Allen present MEDAL, turning static manifold embeddings into models for held-out validation. Methods like t-SNE and UMAP for nonlinear dimension reduction are often picked for looks. Yet their embeddings offer no out-of-sample map and no inverse, blocking held-out checks. MEDAL distills a fitted embedding into a constrained autoencoder whose bottleneck matches the teacher embedding. That yields out-of-sample maps, an approximate inverse, and a distortion measure. Across benchmarks and cases, MEDAL picks embeddings and tunings and spots distribution shift.
Catalog record
| CREATOR | Irene Chang, Tarek M. Zikry, Genevera I. Allen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 125909 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24244
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MEDAL: Manifold Embedding Distillation via Autoencoder Learning Irene Chang, Tarek M. Zikry, Genevera I. Allen Abstract Low-dimensional embeddings are widely used as visual summaries of high-dimensional data and to enable downstream scientific discoveries. Yet, popular nonlinear dimension reduction methods, such as t-SNE and UMAP, are often selected based on visual appeal alone and without rigorous quantitative validation. A major reason is that manifold embeddings typically do not provide an out-of-sample map nor an inverse back to the original feature space; this makes held-out validation, the gold standard in supervised learning, all but impossible. To address these challenges, we develop a novel framework, MEDAL (Manifold Embedding Distillation via Autoencoder Learning), which distills a fitted manifold embedding into a reusable encoder–decoder model. MEDAL trains a constrained autoencoder whose bottleneck exactly matches any teacher embedding while the decoder reconstructs the original input; this yields an explicit map for new samples, an approximate inverse, and a pointwise reconstruction-based measure of distortion in the manifold space. This converts static manifold embeddings into models that can be evaluated on held-out data, enabling quantitative validation including comparing different dimension reduction methods as well as hyperparameter tuning. Across multiple benchmark and scientific case studies, we show that MEDAL enables held-out validation to determine optimal manifold embeddings and hyperparameters, reveals biologically coherent regions that are difficult to preserve in two dimensional embeddings, and detects distribution shift when new samples are mapped into a fixed reference manifold. MEDAL provides a general validation wrapper to any existing dimension reduction technique that will improve the rigor and reliability of dimension reduction in scientific workflows. 2 2 footnotetext: Corresponding author: [email protected] 1 Introduction Data arising in modern scientific applications are often very high-dimensional. This growth in data dimensionality has made dimension reduction (DR) a central tool for modern scientific analysis. Broadly, DR seeks to map observations from \mathbb{R}^{p} into a much lower-dimensional space \mathbb{R}^{d} , where d\ll p , while preserving structure relevant to exploration, interpretation, and downstream analysis. A large methodological toolbox has been developed for this purpose. Linear methods such as Principal Component Analysis (PCA) [ 48 ] are well-understood, but limited in their ability to represent nonlinear structure that are present in complex, modern datasets. Neighborhood- and graph-based nonlinear methods were introduced to overcome this limitation, beginning with classical approaches such as Isomap [ 92 ] and Locally Linear Embedding [ 85 ] , and extending to modern methods such as t-SNE [ 97 ] , UMAP [ 72 ] , PHATE [ 75 ] , and PaCMAP [ 102 ] . These methods are no longer used only as exploratory visualization tools. Across biomedicine and other data-intensive sciences, low-dimensional embeddings are fundamental for discovery, used to guide decisions about cluster structure, class separation, trajectory inference, and disease-related heterogeneity, including in settings such as cancer characterization via single-cell transcriptomics, biomedical imaging, and electrophysiological measurements of neural activity [ 18 , 34 , 29 , 56 , 76 , 70 , 25 , 80 , 108 , 12 , 110 , 23 ] . As a result, an embedding is not merely a picture; it is often treated as a scientifically meaningful representation of the data, and its reliability can directly affect critical downstream conclusions. Despite this importance, the way embeddings are often used in practice remains surprisingly qualitative. Researchers frequently choose a DR method and its hyperparameters based on defaults or visual appeal, and then proceed as if the resulting geometry were a faithful representation of the underlying data. This workflow is fragile. Small changes in initialization, preprocessing, feature selection, or hyperparameter choice can materially alter the geometry of an embedding [ 102 , 57 , 94 , 35 ] , and recent critiques have highlighted how easily visually compelling maps can be over-interpreted as scientific evidence [ 50 , 49 , 22 ] . This has also contributed to the greater crisis in rigor and reproducibility in modern science [ 79 ] , as these sensitive embeddings are used in consequential and critical scientific workflows. This fragility calls for more principled ideas from model validation and reliability assessment. A growing literature has begun to address the problem of validation in dimension reduction [ 1 ] , particularly through work on hyperparameter tuning and embedding reliability. Existing approaches include post-hoc selection criteria tailored to particular applications or notions of embedding quality [ 17 , 63 , 37 , 52 , 109 ] , optimization-based formulations of hyperparameter search [ 101 , 62 , 78 ] , and tools for scaling visual or interactive exploration of embedding behavior [ 3 , 89 ] . Most closely related are methods that score local embedding reliability, including empirical-null or resampling-based approaches such as EMBEDR and scDEED [ 51 , 103 , 89 ] , and geometric approaches based on map continuity or discontinuity such as neMDBD [ 64 ] . These methods address important aspects of the problem, but they remain task-specific or method-specific and do not provide the kind of principled held-out validation routine in supervised learning. A central reason this gap persists is that principled validation is much easier for some DR methods than for others. For linear methods such as PCA, there exists both a forward map (projecting new data) and an inverse map (reconstruction from low-dimensional coordinates), providing a natural notion of reconstruction error. Therefore, a model trained on one subset of data can be tested by measuring information loss on held-out observations. This split-based approach mirrors the standard model selection procedure under supervised regime. By contrast, many widely used neighborhood-based nonlinear methods provide embeddings primarily for the training data and do not furnish a general, reliable way to embed new points or invert the map. As a result, nonlinear embeddings remain difficult to evaluate, compare, and reuse, despite already being deeply integrated in scientific workflows [ 20 ] . Previous work has incorporated neural networks directly into manifold learning to address the absence of an explicit out-of-sample map. Parametric t-SNE and Parametric UMAP [ 98 , 86 ] , for example, train a neural encoder from high-dimensional observations to low-dimensional coordinates, as well as autoencoder-based approaches that learn latent representations through reconstruction objectives [ 96 ] . However, despite the appeal of these methods, many applied analyses still use standard t-SNE and UMAP embeddings as fixed objects of interpretation [ 50 ] . What is missing is therefore not yet another embedding method to replace the existing ones, but rather, a general wrapper that makes these nonparametric DR methods more reusable, comparable, and amenable to rigorous quantitative evaluation. In this work, we propose MEDAL , a distillation framework that can be used to bridge the evaluation gap in DR by converting a fitted teacher embedding into a reusable model. MEDAL is inspired by knowledge distillation [ 44 ] , in which a student model is trained to reproduce the behavior of a teacher model more efficiently. Unlike classic distillation, our ‘teacher’ is a pre‑computed embedding. Given a teacher embedding from UMAP, t‑SNE, Isomap, PCA or any other DR method, MEDAL trains a constrained autoencoder with two coupled objectives: (1) a distillation loss aligning the encoder with the teacher coordinates, and (2) a reconstruction loss that preserves input‑space information.