DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy
Frederik L. Dennig University of Konstanz, Daniel A. Keim University of Konstanz · 2025
Read DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy by Frederik L. Dennig University of Konstanz, Daniel A. Keim University of Konstanz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-12145.txt.
Introduction
Frederik Dennig and Daniel Keim add in a 2025 study an honesty gauge to models that squash high-dimensional data into flat maps. Such projections let new points be placed without redoing the map, and can be run backward to invent new data. But points far from the training crowd get placed with false confidence.
Their model, a variational autoencoder scored by differential entropy, flags where the map is unsure. On four known data sets, with two classic maps as inputs, its projections matched rival accuracy while exposing the shaky spots. Maps that confess their doubt, they argue, are safer to explore.
Catalog record
| CREATOR | Frederik L. Dennig University of Konstanz, Daniel A. Keim University of Konstanz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-12145
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy Frederik L. Dennig University of Konstanz, Daniel A. Keim University of Konstanz Abstract Recently, autoencoders (AEs) have gained interest for creating parametric and invertible projections of multidimensional data. Parametric projections make it possible to embed new, unseen samples without recalculating the entire projection, while invertible projections allow the synthesis of new data instances. However, existing methods perform poorly when dealing with out-of-distribution samples in either the data or embedding space. Thus, we propose DE-VAE, an uncertainty-aware variational AE using differential entropy (DE) to improve the learned parametric and invertible projections. Given a fixed projection, we train DE-VAE to learn a mapping into 2D space and an inverse mapping back to the original space. We conduct quantitative and qualitative evaluations on four well-known datasets, using UMAP and t-SNE as baseline projection methods. Our findings show that DE-VAE can create parametric and inverse projections with comparable accuracy to other current AE-based approaches while enabling the analysis of embedding uncertainty. \onlineid 1023 \vgtccategory Research \vgtcinsertpkg \teaser The encoder of a DE-VAE learns a parametric projection P of MNIST, mapping each data point x_{i} to a full Gaussian \altmathcal{N}(\mu,\Sigma) , modeling the uncertainty of a UMAP projection. The decoder learns an inverse projection P^{-1} , taking y_{k} and reconstructing a plausible sample \hat{x}_{k} . P enables uncertainty-aware visualization of the latent space. DE-VAEs optimize the losses: \altmathcal{L}_{\text{recon}} , ensuring reconstruction; \altmathcal{L}_{\text{proj}} , aligning \mu with points of the projection; and \altmathcal{L}_{\text{ent}} , maximizing the variance of \Sigma . To show learned Gaussian distributions, we depict the 1st, 2nd, and 3rd standard deviations as ellipses around two randomly sampled points per class. Introduction keywords Uncertainty visualization, dimensionality reduction, variational autoencoders, inverse projections. Multidimensional projections, also referred to as dimensionality reduction (DR) techniques, are widely used and well-established tools for the visual exploration of high-dimensional data [ 20 ] . DR methods are usually used to create 2- or 3-dimensional representations while trying to preserve relationships present in the high-dimensional space, e.g., distances and neighborhoods [ 27 ] . In this context, parametric projection methods employ a trained model to create a mapping from high- to low-dimensional space, enabling the projection of previously unseen data points, real and synthetic, without the need to recompute the projection [ 28 ] . Inverse projection methods provide a usually smooth mapping from the projection space to the high-dimensional space, enabling the generation of new data from any position in the projection space [ 31 ] , e.g., helpful in visualizing and exploring the decision zones of classifiers. However, DR is inherently lossy and prone to distortions , such as false neighborhoods, or rifts, especially, in the case of non-linear DR methods [ 27 ] . To address this, researchers proposed methods to quantify and visualize distortions and gradients between projected points. Recent approaches for parametric and inverse projections, which are usually created using neural networks (NNs) [ 12 ] or variational autoencoders (VAEs) [ 8 ] , did not address or leverage the uncertainty inherent in the process. In this work, we propose DE-VAEs, which use differential entropy to capture the uncertainty in the parametric projection by modeling it in its latent space. Thus, unlike traditional VAEs, DE-VAEs model the uncertainty associated with a sample with a multidimensional Gaussian distribution. In contrast, for the inverse- or generative-case, we sample from the modeled distribution to retrieve similar synthetic data items. Since DR for data projection is inherently visual [ 26 ] , we can gain insight into the sources and effects of uncertainty through uncertainty visualization [ 5 ] . Overall, we contribute the following: (1) We propose DE-VAEs, which leverage differential entropy for creating parametric and invertible projections. (2) We perform an evaluation comparing isotropic , diagonal and full Gaussian distributions quantitatively and qualitatively on four datasets using UMAP and t-SNE. (3) For reproducibility , we provide the analysis, results and source code on \href https://osf.io/zr6xfOSF and \href https://github.com/fredooo/DE-VAEGitHub. With this work, we aim to enhance high-dimensional data analysis by improving the interpretation of uncertainty in DR methods. 1 Related Work First, we introduce some notation. Let D=\{x_{i}\}_{1\leq i\leq n} denote a high-dimensional dataset with d dimensions and n samples, where each x_{i}\in\mathbb{R}^{d} . A projection technique P transforms D into a lower-dimensional set P(D)=\{P(x_{i})\mid x_{i}\in D\}=\{y_{i}\}_{1\leq i\leq n} , such that P(D)\subset\mathbb{R}^{q} with q\ll d . In our setting, q=2 , so P(D) can be displayed as a 2D projection. A plethora of DR methods exist and are studied in several surveys quantitatively and qualitatively [ 14 , 27 ] . Generally, there are linear and non-linear methods which focus on either preserving global structure or local neighborhoods . Linear projection techniques, such as principal component analysis (PCA) [ 20 ] , are computationally efficient and excel at maintaining the data’s global structure. In contrast, non-linear global methods like multidimensional scaling (MDS) [ 23 , 7 ] aim to capture more complex relationships while still focusing on the overall structure. However, many modern non-linear approaches prioritize preserving local neighborhood relationships, often sacrificing global structure in the process. Notable examples include t-SNE [ 30 ] , which forms implicit neighborhoods by computing pairwise similarities in high-dimensional space using a Gaussian kernel centered at each data point, while UMAP [ 25 ] explicitly constructs a k-nearest neighbor graph to capture local structure and then optimizes a low-dimensional embedding that preserves the fuzzy topology implied by the graph. AE-based models also belong to the class of non-linear methods; they learn low-dimensional embeddings by training neural networks to reconstruct input data, effectively capturing complex, non-linear feature representations [ 32 ] . Parametric Projection Methods: These approaches provide an explicit mapping function from the high-dimensional space to a lower-dimensional embedding, enabling efficient out-of-sample projections for new data points [ 18 ] . Van der Maaten [ 29 ] employed a feed-forward neural network to create a parametric version of t-SNE. In a similar vein, parametric UMAP [ 28 ] utilizes neural networks, including autoencoders, as a parametric alternative to the original non-parametric embedding. By training neural networks to predict 2D coordinates for input data points, Espadoto et al. [ 13 ] demonstrated that sufficiently large networks can effectively approximate various non-parametric methods. Inverse Projections: A function P^{-1}:\mathbb{R}^{q}\rightarrow\mathbb{R}^{d} , that is typically smooth and aims to minimize the error \sum_{x\in D}\|P^{-1}(P(x))-x\|^{2} , where P is a projection and \|\cdot\| denotes the L_{2} norm, is called an inverse projection. There are only a limited number of non-NN methods supporting inversion: PCA [ 20 ] , LAMP [ 19 , 11 ] , and UMAP [ 25 ] . At the same time, an autoencoder, by design, has an encoder (learning P ) and a decoder (learning P^{-1} ) that reconstructs the original data [ 3 ] . Other methods augment a subset of DR methods with inverse transformations, making different assumptions. One of the first techniques used a global interpolation-based method to create synthetic data points [ 31 ] . Amorim et al. [ 1 ] refined this approach by using a radial basis function for kernel-based interpolation. Blumberg et al. [ 4 ] invert MDS by applying data point multilateration