How high is ‘high’? Rethinking the roles of dimensionality in topological data analysis and manifold learning

Hannah Sansford, Nick Whiteley, Patrick Rubin-Delanchy · 2025

Read How high is ‘high’? Rethinking the roles of dimensionality in topological data analysis and manifold learning by Hannah Sansford, Nick Whiteley, Patrick Rubin-Delanchy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16879.txt.

Introduction

Hannah Sansford and co-authors ask what 'high-dimensional' really means for modern data analysis, in a 2025 paper. Methods that map hidden shapes in data, like topological analysis and manifold learning, often fail or succeed for reasons blamed on dimension. The authors distinguish three different quantities that all get called dimension: total spread across features, complexity across samples, and the true dimension of the hidden structure.

Using a new version of a classic concentration inequality, they show the hidden shape can still be recovered when the data's total spread merely outgrows the logarithm of the sample size. They then revisit a famous neuroscience claim, the ring-like code in grid cells, and show for the first time that the structure mirrors physical space faithfully. Clearer dimension bookkeeping could stop researchers from discarding methods that would have worked.

Catalog record

CREATORHannah Sansford, Nick Whiteley, Patrick Rubin-Delanchy
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-16879

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How high is ‘high’? Rethinking the roles of dimensionality in topological data analysis and manifold learning Hannah Sansford, Nick Whiteley, Patrick Rubin-Delanchy Abstract We present a generalised Hanson-Wright inequality and use it to establish new statistical insights into the geometry of data point-clouds. In the setting of a general random function model of data, we clarify the roles played by three notions of dimensionality: ambient intrinsic dimension p int p_{\mathrm{int}} , which measures total variability across orthogonal feature directions; correlation rank , which measures functional complexity across samples; and latent intrinsic dimension , which is the dimension of manifold structure hidden in data. Our analysis shows that in order for persistence diagrams to reveal latent homology and for manifold structure to emerge it is sufficient that p int ≫ log ⁡ n p_{\mathrm{int}}\gg\log n , where n n is the sample size. Informed by these theoretical perspectives, we revisit the ground-breaking neuroscience discovery of toroidal structure in grid-cell activity made by Gardner et al. [14] : our findings reveal, for the first time, evidence that this structure is in fact isometric to physical space, meaning that grid cell activity conveys a geometrically faithful representation of the real world. 1 Introduction Data geometry in Machine Learning Consider a data set of n samples and p features, \mathbf{Y}_{1},\ldots,\mathbf{Y}_{n}\in\mathbb{R}^{p} . A wide range of algorithms, models, supervised and unsupervised learning methods process such data by taking as inputs the pairwise Euclidean distances, dot-products or cosine similarities, \displaystyle\|\mathbf{Y}_{i}-\mathbf{Y}_{j}\|^{2}=\|\mathbf{Y}_{i}\|^{2}+\|\mathbf{Y}_{j}\|^{2}-2\mathbf{Y}_{i}\cdot\mathbf{Y}_{j},\qquad\CosSim(\mathbf{Y}_{i},\mathbf{Y}_{j})=\frac{\mathbf{Y}_{i}\cdot\mathbf{Y}_{j}}{\|\mathbf{Y}_{i}\|\|\mathbf{Y}_{j}\|}. (1) The collection of these pairwise quantities across i,j=1,\ldots,n conveys the geometric shape of \mathcal{Y}_{n}\coloneqq\{\mathbf{Y}_{1},\ldots,\mathbf{Y}_{n}\} regarded as a point-cloud in \mathbb{R}^{p} , and is the input to most dimension reduction and manifold learning methods, ranging from Classical Multidimensional Scaling [ 32 ] , to Kernel PCA [ 27 ] , Isomap [ 31 ] , t -SNE [ 35 ] and UMAP [ 23 ] , which are hugely popular for data visualisation, as well as spectral clustering [ 38 ] (via a kernel function) and hierarchical, agglomerative clustering methods such as UPGMA [ 29 , 15 ] . The set of all pairwise distances is also the input to Persistent Homology techniques in topological data analysis (TDA) [ 9 , 42 , 7 , 6 ] . Pairwise distances or dot-products are the input to classical supervised learning methods such as kernel methods and nearest-neighbour methods [ 26 ] , and dot-products between training and test feature vectors define predictions in linear regression. The disconnect between reality and existing statistical theory of data geometry. In this work we present new insights into statistical properties of the quantities in ( 1 ), allowing us to explain how they convey latent geometry of the data-generating mechanism. In doing so we address two mismatches between the prominent statistical theory of data geometry and the realities of data analysis. Firstly, the popular mathematical view of a mean-zero, identity-covariance random vector is that, in high dimensions, its distribution is concentrated near the surface of a hypersphere, and that i.i.d. copies of such a vector are close to orthogonal with high probability, see [ 17 ] , the textbook of Hastie et al. [19] [Sec. 2.5], and [ 5 ] for refined analysis of similar i.i.d. setups. This rather degenerate limiting geometry does not seem expressive enough to accurately represent data in practice; it is incompatible with the widely accepted Manifold Hypothesis [ 3 , 10 ] , which is the premise of manifold learning and nonlinear dimension reduction, asserting that nominally high-dimensional data actually lie on a low-dimensional set embedded in high-dimensional space. Secondly, much existing theory, e.g., [ 17 , 1 , 28 , 2 ] focuses on the regime p\gg n with which the phrase “high-dimension low sample size asymptotics” (HDLSS), and more broadly “high-dimensional data”, are typically associated. This existing theory may therefore seem irrelevant in the many practical situations where p\leq n ; indeed p\gg n could be violated by simply collecting more data. Contributions. Our main contributions are as follows: • In section 2 we present a generalised Hanson-Wright (GHW) inequality which quantifies the concentration behaviour of dot-products between possibly dependent random vectors with sub-Gaussian entries, and identify our first notion of dimension: ambient intrinsic dimension p_{\mathrm{int}} . • In section 3 we identify a second notion of dimension: correlation rank , and use our GHW inequality to explain how latent topological structure emerges from a random function model of data (from [ 40 ] ) as p_{\mathrm{int}} grows; we show under sub-Gaussian assumptions that p\gg n is not necessary; rather p_{\mathrm{int}}\gg\log n suffices. Figure 1 gives a preview of the phenomenon. • In section 4 we consider consistency of TDA persistence diagrams in the regime p_{\mathrm{int}}/\log n\to\infty . • In section 5 we identify a third notion of dimension: latent intrinsic dimension , and discuss how isometry between observed and latent geometry can be measured in practice. • In section 6 we revisit the neuroscience study of [ 14 ] , and informed by our theory, extend their groundbreaking discoveries by assessing evidence of isometry between grid cell activity and physical space. Figure 1: Ambient intrinsic dimension p_{\mathrm{int}} , correlation rank r , and latent intrinsic dimension d at play in simulation from a toy example of the random function model with n=1000 . (a)-(c) show SVD visualisation of simulated data with respectively p_{\mathrm{int}}=3,8,20 ; as p_{\mathrm{int}} grows, the d=1 -dimensional manifold \mathcal{M}=\{\phi(z);z\in\mathcal{Z}\} shown in (d) emerges in a r=3 -dimensional subspace. In this example \mathcal{M} is homeomorphic to the latent space \mathcal{Z} , which is a circle. See section 3 for details. Section 3 is directly influenced by geometric perspectives of [ 39 , 40 , 15 ] and in particular the insightful statistical treatment of the manifold hypothesis [ 40 ] , but our main theoretical results (theorems 1 , 2 and proposition 2 ) substantially extend and refine some of their results; we allow more general dependence structure across features and exploit sub-Gaussianity. An extended discussion of related work is given in the supplementary material, along with all proofs and additional numerical experiments. Notation and conventions. We write [n]\coloneq\{1,\ldots,n\} and \ell_{2} for the set of square-summable real sequences \{x=(x_{1},x_{2},\ldots):\|x\|\coloneqq(\sum_{k}|x_{k}|^{2})^{1/2}<\infty\} . For two nonnegative sequences (a_{n})_{n\geq 1} , (b_{n})_{n\geq 1} , a_{n}\in\Omega(b_{n}) means \liminf_{n\to\infty}a_{n}/b_{n}>0 . For two sequences of nonnegative random variables (X_{n})_{n\geq 1} , (Y_{n})_{n\geq 1} , X_{n}\in O_{\mathbb{P}}(Y_{n}) means that for any \epsilon>0 there exists M and n_{0} such that \mathbb{P}(X_{n}/Y_{n}>M)\leq\epsilon for all n\geq n_{0} . The sub-Gaussian norm of a random variable X is \|X\|_{\psi_{2}}\coloneqq\sup_{q\geq 1}q^{-1/2}\mathbb{E}\left[|X|^{q}\right]^{1/p} , and X is said to be sub-Gaussian if \|X\|_{\psi_{2}}<\infty . This condition can be understood as meaning that the tails of the distribution of X decay at least as quickly as a Gaussian, but includes the case where X is a discrete random variables taking only finitely many different values, or more generally the case where the support of the distribution of X is bounded. The Frobenius and spectral matrix norms are respectively denoted \|\cdot\|_{\mathrm{F}} and \|\cdot\| . \mathbf{I}[\cdot] is the indicator function, \mathbf{I}_{p} is the p -by- p identity matrix. 2 A generalised Hanson-Wright inequality The Hanson-Wright (HW) inequality is a concentration inequality for the quadratic form: \mathbf{X}^{\top}\mathbf{A}\mathbf{X} , where \mathbf{X}

Read the complete text