Generative model for optimal density estimation on unknown manifold
Arthur Stéphanovitch · 2025
Read Generative model for optimal density estimation on unknown manifold by Arthur Stéphanovitch (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-19587.txt.
Introduction
Arthur Stéphanovitch's paper from 2025 tackles density estimation, the task of guessing the true spread of data from samples. The data here lie on a curved surface, or manifold, of low dimension that no one hands to the learner. Stéphanovitch builds a generator whose own output lies on a like surface, matched in smoothness to the data's home.
The design leans on Fefferman's solution to a classic geometry puzzle. The payoff is the best rate of error that theory allows, held across a whole family of measures at once. Tests on fake and real data sets show scores that match or beat two known rivals, GANs and score-based models.
Catalog record
| CREATOR | Arthur Stéphanovitch |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 174 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-19587
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generative model for optimal density estimation on unknown manifold Arthur Stéphanovitch Abstract We propose a generative model that achieves minimax-optimal convergence rates for estimating probability distributions supported on unknown low-dimensional manifolds. Building on Fefferman’s solution to the geometric Whitney problem, our estimator is itself supported on a submanifold that matches the regularity of the data’s support. This geometric adaptation enables the estimator to be simultaneously minimax-optimal for all γ \gamma -Hölder Integral Probability Metrics (IPMs) with γ ≥ 1 \gamma\geq 1 . We validate our approach through experiments on synthetic and real datasets, demonstrating competitive or superior performance compared to Wasserstein GAN and score-based generative models. 1 Introduction 1.1 From the geometric Whitney problem to statistical estimation The problem of reconstructing a manifold from discrete data has a rich history, connecting geometry, analysis, and learning theory ( Lassas and Uhlmann 2001 ; Aamari and Levrard 2019 ; Boissonnat et al. 2018 ) . In particular, the recent work of Fefferman et al. 2020 formulated and solved the geometric Whitney problem , providing conditions under which a point cloud can be approximated by a smooth submanifold with controlled geometry. Their work gave a constructive solution to manifold interpolation problems, leading to advances in both theory and algorithms. In this paper, we push this geometric program further by addressing the statistical estimation of distributions supported on unknown manifolds. Our goal is not only to reconstruct the underlying geometry but also to estimate a smooth probability measure on the manifold, achieving optimal statistical guarantees. Specifically, given an iid sample of a probability measure having a density with respect to an unknown manifold, we propose the first tractable estimator that: • (i) constructs a submanifold approximating the support of the data with the same regularity, • (ii) defines a probability measure admitting a smooth density with respect to the volume measure of the estimated submanifold, • (iii) achieves minimax-optimal convergence rates under Hölder Integral Probability Metrics ( Müller 1997 ) for all Hölder indices \gamma\geq 1 simultaneously. • (iv) enables efficient sampling from the estimated distribution. Our method combines manifold reconstruction techniques inspired by Fefferman et al. 2020 with tools from adversarial estimation and optimal transport, particularly Wasserstein GANs ( Arjovsky et al. 2017 ) . By leveraging wavelet decompositions adapted to Besov regularity, we capture both the (unknown) local geometry of the manifold and the (known) smoothness of the density in a unified framework. Finally, we illustrate the effectiveness of our estimator through numerical experiments on synthetic and real datasets, demonstrating competitive or superior performance compared to WGANs and SGMs, especially in settings where the data exhibits nontrivial manifold structure. 1.2 Related works 1.2.1 The geometric Whitney problem The geometric Whitney problem asks: given a metric space (X,d_{X}) , can we construct a smooth d -dimensional submanifold M that approximates X with precise control over the manifold’s geometry (curvature, injectivity radius) and the distance between M and X ? This problem is a geometric generalization of the classical Whitney extension problem, where the goal was to smoothly extend functions defined on arbitrary subsets of \mathbb{R}^{n} . In Fefferman et al. 2020 , the authors provided a major breakthrough by: • Giving conditions under which a metric space (X,d_{X}) can be approximated by a Riemannian manifold (M,g) of bounded geometry, both in Gromov–Hausdorff and quasi-isometric senses. • Providing explicit reconstruction algorithms when X is finite for building manifolds with controlled sectional curvature and injectivity radius. Their work builds on classical results by ( Whitney 1992 ; Glaeser 1958 ; Brudnyi and Shvartsman 1997 ; Bierstone et al. 2003 ) , while pushing the theory into the global geometric setting. In this paper, we take a different direction: instead of focusing on geometric reconstruction, we aim to push Fefferman et al. 2020 ’s construction toward statistical estimation and generative modeling. Our goal is to build an estimator that not only reconstructs a smooth manifold approximating the data but also defines a probability measure with a smooth density supported on this manifold. The objective is to obtain a tractable estimator that achieves minimax-optimal convergence rates. In this way, we bridge the gap between geometric reconstruction and modern generative modeling, extending the geometric Whitney program to a statistical setting. 1.2.2 Minimax estimators of densities on unknown manifold Estimating a probability distribution from samples is a fundamental problem in statistics and machine learning. Recent advances in generative modeling, particularly Generative Adversarial Networks (GANs) Goodfellow et al. 2014 and Score-based Generative Models (SGMs) Song et al. 2020 , have demonstrated remarkable performance in high-dimensional settings. However, these methods often struggle when the data lie on a complex, low-dimensional manifold, as they do not produce estimators that preserve the same regularity structure as the underlying distribution. In particular, GANs face difficulties when generating data with complex topology, since their push-forward formulation can constrain the topology of the output distribution ( Tanielian et al. 2020 ; Stéphanovitch et al. 2024 ) . SGMs also encounter limitations in this setting. When the data lie on or near a low-dimensional manifold, the score function becomes ill-behaved and diverges. This phenomenon complicates estimation and can lead to poor sample quality. Recent studies have highlighted these challenges, emphasizing the importance of careful architectural design and noise scheduling to accurate score estimation in geometrically complex settings ( Chen et al. 2023 ; Li et al. 2024 ) . Let us review the recent literature on minimax density estimation over unknown manifolds. • In Divol 2022 , the author proposes the first minimax estimator for the Wassertein distance of densities supported on unknown manifolds. The estimator combines a local polynomial estimator from Aamari and Levrard 2019 with a kernel density estimator. However, the local polynomial step is computationally intensive, making the overall procedure impractical in high-dimensional settings. Moreover, the estimator is not generative, as it does not provide a mechanism for sampling from the estimated distribution. • In Tang and Yang 2023 , the authors construct an estimator that achieves the minimax rate for the Hölder Integral Probability Metric (IPM) d_{\mathcal{H}^{\gamma}_{1}} for a fixed \gamma>0 . In particular, they highlight that when \gamma is small, the distance d_{\mathcal{H}^{\gamma}_{1}} becomes highly sensitive to misalignments between the supports of the measures, as opposed to discrepancies between the densities themselves. Their estimator involves manifold reconstruction, optimizing over the entire Hölder class \mathcal{H}^{\beta}_{1} , computing wavelet expansions of each discriminator D\in\mathcal{H}^{\gamma}_{1} , and evaluating all derivatives up to order \gamma . Despite its theoretical guarantees, the estimator is computationally infeasible in practice due to the complexity of these operations. • Building on the Wasserstein GAN framework, Stéphanovitch et al. 2024 introduce the first tractable and fully generative estimator. However, this method is restricted to learning densities supported on manifolds with the topology of a d -dimensional torus, limiting its applicability to more realistic manifold structures. • The recent works of Azangulov et al. 2024 and Tang and Yang 2024 establish that SGMs attain the minimax rate for Holder IPMs and the Wasserstein distance respectively. These results are particularly important given that SGMs have become the state-of-the-art in generative modeling. However, the practical implementation of the minimax estimator remains challenging: the score estimation procedure involves training a total of O(\log(n)) separate neural