Intrinsic Dimensionality of Molecular Properties Ali Banjafar, Guido Falk von Rudorff Abstract Chemical space which encompasses all stable compounds is unfathomably large and its dimension scales linearly with the number of atoms considered. The success of machine learning methods suggests that many physical quantities exhibit substantial redundancy in that space, lowering their effective dimensionality. A low dimensionality is favorable for machine learning applications, as it reduces the required number of data points. It is unknown however, how far the dimensionality of physical properties can be reduced, how this depends on the exact physical property considered, and how accepting a model error can help further reducing the dimensionality. We show that accepting a modest, nearly negligible error leads to a drastic reduction in independent degrees of freedom. This applies to several properties such as the total energy and frontier orbital energies for a wide range of neutral molecules with up to 20 atoms. We provide a method to quantify an upper bound for the intrinsic dimensionality given a desired accuracy threshold by inclusion of all continuous variables in the molecular Hamiltonian including the nuclear charges. We find the intrinsic dimensionality to be remarkably stable across molecules, i.e. it is a property of the underlying physical quantity and the number of atoms rather than a property of an individual molecular configuration and therefore highly transferable between molecules. The results suggest that the feature space of state-of-the-art molecular representations can be compressed further, leaving room for more data efficient and transferable models. I Introduction When characterizing chemical space, one of its fundamental properties is how many independent dimensions it has. That is, to quantify how many independent variables are required to describe how a property changes between molecules and molecular configurations. Formally, chemical space has 4N continuous dimensions where N is the number of atoms. This can be seen from the components of the molecular Hamiltonian \hat{H}(\mathbf{R}_{I},Z_{I}) which depends on the nuclear charges and positions of each atom. Translational and rotational symmetry reduces the number of dimensions by 5 or 6 for linear and non-linear molecules, respectively. The wealth of research and evidence from machine learning that many molecular properties can be modeled and predicted based on exploiting similarities in chemical space Keith et al. 2021 ; Lu et al. 2024 ; Von Lilienfeld et al. 2020 is indicative of chemical space being highly redundant and thus compressible for many properties of interest. Since machine learning approaches are data-driven and thus require large number of data points Smith et al. 2017 ; Ganscha et al. 2025 , which in turn are costly to generate, it is particularly interesting to learn how far chemical space can be compressed without or minimal loss of predictive power. This is particularly relevant for machine learning applications, since learning theory suggests that the number of data points required to approximate an arbitrary but well-behaved function to some given accuracy (also) depends on the dimensionality of said function Madych and Nelson 1992 . The dimensionality of a mathemtical pure function would be represented as the number of (possibly redundant) arguments that it takes. Therefore, machine learning representations can be seen as a parameter transform from the original Cartesian coordinates and nuclear charges of the molecular Hamiltonian into another vector space that is more amenable to interpolation between the values of a given property. Those representations can either be generated implicitly during training of a neural network Krenn et al. 2022 or they can be designed based on physical insight Musil et al. 2021 ; David et al. 2020 . Either way, these representations contain typically on the order of a few hundred to two thousand entries or dimensions Briling et al. 2024 ; Karandashev and Von Lilienfeld 2022 ; Faber et al. 2018 ; Khan et al. 2023 . Given typically molecular atom counts N , this is substantially more than four N . This over-completeness can be beneficial since it allows to learn several different properties using the same representation even though grouping in similarity would be different depending on property. However, representations benefit from being as short and compact as possible. For example, in kernel-ridge regression or other kernel methods, distances between representation vectors in training and test data have to be evaluated which scale linearly with the number of data points. Since kernel methods are not scale invariant, each additional feature formally introduces another hyperparameter by which it can be scaled. Consequentially, the search space of the hyperparameter optimization in turn scales with the number of features. In practice, this is addressed by setting a parameter during development of the representation and not scaling the features for the individual application except for categorial regression, i.e. one-hot-encoding. Longer representations severely impact the time to result. Moreover, applications of confidential computing or multi-party computing, which may allow predictions on confidential data, benefit from compact representations Weinreich et al. 2023 . With recent efforts into short yet transferable machine learning representations Briling et al. 2024 ; Khan et al. 2023 , it is an important question how far the number of independent degrees of freedom can be reduced when describing chemical space. Naturally, that number must depend on the property under consideration. For example, the net charge is independent of the configuration, but depends on the nuclear charges of all atoms while the surface area depends on the positions but not the nuclear charges. In thus far, the dimensionality is less of a property of chemical space as a whole, but rather of a property. Consequently, it is important to be able to quantify the limit of the minimal number of degrees of freedom required to describe a physical property based on the data or the variance of that property alone. This would allow to build more data-efficient models tailored to learn specific properties, if the limit of that number of degrees of freedom is known and might explain why some properties are easier to learn than others. The dimensionality of a property in chemical space has a direct physical connection. If a property is nearsighted, as it is often invoked for example for total energies, then that would indicate that there should be a finite number of degrees of freedom that can impact that property in order to preserve locality. If a property was global instead, then it would need to scale with the overall system size. In machine learning terminology, introducing a cut-off threshold beyond which interactions are not taken into consideration directly implies locality, which in turn implies reduction of dimensionality. Identifying an upper bound of said dimensionality therefore is identical to identification of a lower bound of compressibility or dimensionality reduction for machine learning applications. In recent decades, advancements in computational chemistry and machine learning have significantly heightened interest in intrinsic dimensionality and dimension reduction concepts Keith et al. 2021 ; Jia et al. 2021 ; van der Maaten et al. 2009 . Intrinsic dimension refers to the minimum number of variables that minimize information loss in a data set or physical properties Camastra and Staiano 2016 . With an intrinsic dimensionality estimate, often a dimensionality reduction scheme can be implemented Orlov et al. 2025 . In machine learning, this connects to active learning, feature selection, dimensionality reduction Tamilselvi and Chandrasekaran 2017 , for the dimensionality estimation of point clouds and for cluster identification Williams and Ilieş 2018 . Typically dimensionality reduction implies a loss of precision Laikov 2011 . This also applies to quantum chemistry, where sparsifying interactions has been a successful strategy Schleder et al. 2019 in method development: only rarely this can be done without loss of accuracy as it has been done e.g. in equating knowing the wavefunction (which has 3N_{\textrm{e}} dimensions) with knowing the electron density (which has 3 dimensions, independent of the number of electrons N_{\textrm{e}} ) by virtue of the Hohenberg-Kohn theorem. Literature distinguishes global intrinsic dimension (ID) and local ID. The former gives the number of degrees of freedom to approximately describe the global shape of, for example, a point cloud, while the latter describes the region around a certain point Camastra and Staiano 2016 . The global ID is the same everywhere, while the local ID can differ. Since we typically only characterize small regions of chemical space with a certain application in mind without ever building a random sample of all possible molecules, the local ID is most suitable for characterizing chemical space. Point-cloud ID estimators rely on the geometry of the point cloud and the relationships between inter-point distances. For example, they often examine the distribution of distances to nearest neighbors Ceruti et al. 2014 ; Albergante et al. 2019 . These methods are typically sensitive to the density and distribution of points Facco et al. 2017 ; Denti et al. 2022 – issues that become particularly problematic in high-dimensional spaces Gomtsyan et al. 2019 . To estimate the ID of a physical property using such methods, one would need to generate sample points on a level set (constant value surface) in a 4N -dimensional space, estimate the dimensionality of that surface (akin to the null space) and subtract it from the full number of dimensions of the embedding space. This however poses significant challenges in terms of both the required number of points and maintaining uniform point density across the space. Among ID estimators, Principal Component Analysis (PCA)-based methods have received significant attention Bouveyron et al. 2011 ; Rozza et al. 2012 ; Erba et al. 2019 . In the field of chemistry, PCA is commonly used as a tool to identify and quantify the most relevant variables in molecular systems. However, in practice, the application of PCA is often indirect by first defining a representation and then identifying and counting key components by PCA Grunwald et al. 2015 ; Shukur et al. 2011 , which naturally heavily depends on the choice of the representation Le and Winkler 2018 . Typically, the result of a PCA is a global ID, rather than a local one and moreover requires the feature space to be linearizable which it almost never is. A notable exception for nonlinear PCA would be the kernelized variant, which still yields a global ID. The local ID is conceptually related to the tangent space approach, where a flat surface defined by the first derivatives of the target function is constructed at a specific point on the surface. The tangent vectors at that point represent the local directions of change relative to the reference point. Therefore, it is a local ID picture, since changing the position alters the tangent space and potentially its number of dimensions Yao et al. 2022 . The tangent space however is restricted to considering a single point in chemical space, and does not directly allow for investigation of the accuracy-ID tradeoff, as it reproduces the formal ID for infinite accuracy. Our method does not rely on a locally flat tangent plane at a single reference point. Instead, we approximate a region within the thermally accessible space using a Taylor expansion of the property surface. This approximation captures both the local slope and curvature through the gradient and Hessian, respectively. Unlike tangent vectors, the eigenvectors and eigenvalues of the Hessian matrix describe the principal directions and magnitudes of the surface curvature. As a result, our method allows for accurate modeling of the property surface over a broader region without requiring movement of the reference point or repeated recalculations and allows for simple detection of (approximate) symmetries. Figure 1: Illustration of the estimation of the intrinsic dimension (ID) of a physical property. a) Workflow: at a given molecular geometry we evaluate property derivatives, building the gradient \nabla\cdot and the Hessian matrix H . Selecting a subspace of the diagonalized Hessian enables to assess the ID and its corresponding approximation error. b) Flow chart of the joint subspace selection and error estimation process. c) Visual representation how the difference of units between spatial and charge degrees of freedom can be used to reduce dimensionality through scaling eigenvectors. II Methods In this work, the goal is to determine the local intrinsic dimension of physical properties such as the total energy, HOMO-LUMO gap and HOMO orbital energy for various molecules at a given geometric configuration, not necessarily a local minimum. Local means that we consider the intrinsic dimension to be dependent on both the particular molecular configuration and the property. Even though the number of dimensions depends on accuracy requirements, our approach is general as it yields the minimal number of dimensions for a given error bound. We first build a Taylor expansion around the molecular configuration and then analyze the resulting multivariate polynomial. The polynomial depends on all nuclear charges Z_{I} and nuclear positions \mathbf{R}_{I} , so it has 4N variables for N atoms. II.1 Ab initio calculation and Taylor expansion While the general form of a multivariate Taylor expansion of a property p({\bm{x}}) with all degrees of freedom of N atoms merged into vector {\bm{x}}\equiv\mathbf{R_{1}}\oplus\dots\oplus\mathbf{R_{N}}\oplus Z_{1}\oplus\dots\oplus Z_{N} is given by: \displaystyle p({\bm{x}})\simeq\sum_{|\alpha|\leq k}\frac{\partial^{|\alpha|}p(\mathbf{a})}{\partial x_{1}^{\alpha_{1}}\cdots\partial x_{n}^{\alpha_{n}}}\frac{(\mathbf{x}-\mathbf{a})^{\alpha}}{\alpha!}. (1) with the multi-index \alpha and the reference molecule \mathbf{a} . In this work, we use a second-order expansion (i.e. k=2 ). Unification of the nuclear charge degrees of freedom and the spatial degrees of freedom is inspired by the quantum alchemy approaches von Rudorff and von Lilienfeld 2020 ; von Rudorff 2021 ; von Lilienfeld 2009 , including alchemical normal modes Fias et al. 2018 , which in turn continue in the spirit of the four-dimensional electron density concept Wilson 1962 or conceptual DFT Ayers et al. 2009 ; Balawender et al. 2013 . Similar derivatives have been considered with Hartree-Fock Lesiuk et al. 2012 ; von Rudorff 2021 ; Tamayo-Mendoza et al. 2018 , DFT von Lilienfeld and Tuckerman 2006 ; Lesiuk et al. 2012 ; Balawender et al. 2013 ; Griego et al. 2018 ; Kasim et al. 2022 ; Muñoz et al. 2020 ; Miranda-Quintana and Ayers 2017 and CCSD von Rudorff and von Lilienfeld 2020 ; Abbott et al. 2021 , so we expect our approach to be applicable for many levels of theory. In this work, we consider Restricted Kohn-Sham Density Functional Theory (RKS-DFT) with the PBE Perdew et al. 1996 exchange-correlation functional and Restricted Hartree-Fock, as implemented in PySCF Sun et al. 2017 . Since all the molecules considered in this study are neutral closed-shell systems, RKS is appropriate. We use the uncontracted cc-pVQZ basis set to reduce artifacts from basis functions being developed for integer nuclear charges, which is known to affect response functions since the Hellmann-Feynman theorem is not satisfied if the basis functions are not sufficiently flexible in the direction of changed of nuclar charges Domenichini et al. 2020 . II.2 Analytical and Finite differences derivatives Ideally, all derivatives would be calculated analytically. While spatial derivatives are widely implemented in quantum chemistry codes via Coupled-Perturbed (CP) approaches, only few implementations are available for alchemical derivatives either following CP approaches Lesiuk et al. 2012 ; Domenichini and von Lilienfeld 2022 , automatic differentiation Kasim et al. 2022 ; Tamayo-Mendoza et al. 2018 ; Zhang and Chan 2022 or arbitrary precision operations von Rudorff 2021 . Implementations of analytical derivatives are currently limited to derivatives of the total energy as the property of interest and restricted Hartree-Fock with the main exception being the first order derivative of the orbital eigenvalues which are a by-product of the CP method. Higher orders have been described both for spatial Osamura et al. 1986 and alchemical Lesiuk et al. 2012 derivatives, but no implementation is available, so numerical differentiation is used instead, which also allows to accept some roughness of e.g. a DFT property surface for high order derivatives Zhou et al. 2023 . In this work, we use analytical derivatives where possible (i.e. for energies) and numerical derivatives on all other cases as implemented in our unifying open-source python package nablachem.anygrad Banjafar and von Rudorff 2025a . II.3 Estimating the local intrinsic dimension The primary goal of this work is to calculate the local intrinsic dimension of various properties across a wide range of molecules. To achieve this, we leverage the fact that molecular properties are smooth functions of their spatial and chemical coordinates within chemical space. This allows us to examine the property surface in the vicinity of a fixed molecular configuration. By analyzing the shape and curvature of the property surface, we can identify the key coordinates that play a significant role in defining these properties. We define the local intrinsic dimension of a molecular property p(\mathbf{x}) (with \mathbf{x} defined as in eqn 1 ) in the domain \Omega around a molcular configuration as the minimal set of orthogonal vectors \sigma_{H}^{t} used either as gradient or as Hessian eigenvector of approximant \tilde{p}(\mathbf{x}|\sigma_{H}^{t}) s.t. \left\langle\left(p(\mathbf{x})-\tilde{p}(\mathbf{x})\right)^{2}\right\rangle_{\Omega}