Algebra Unveils Deep Learning An Invitation to Neuroalgebraic Geometry
Giovanni Luca Marchetti, Vahid Shahverdi, Stefano Mereta, Matthew Trager, Kathlén Kohn · 2025
Read Algebra Unveils Deep Learning An Invitation to Neuroalgebraic Geometry by Giovanni Luca Marchetti, Vahid Shahverdi, Stefano Mereta, Matthew Trager, Kathlén Kohn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-18915.txt.
Introduction
Giovanni Luca Marchetti and co-authors invite readers into neuroalgebraic geometry, in a position paper promoting the study of function spaces defined by machine learning models through algebraic geometry.
They focus on models such as networks with polynomial activations, whose function spaces form semi-algebraic varieties. A dictionary links geometric traits of those varieties, like dimension and degree, to learning traits like sample complexity and training dynamics. Along the way they review past work and point beyond the algebraic case. The paper lays groundwork for a field that joins two areas, for readers from both fields.
Catalog record
| CREATOR | Giovanni Luca Marchetti, Vahid Shahverdi, Stefano Mereta, Matthew Trager, Kathlén Kohn |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-18915
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Algebra Unveils Deep Learning An Invitation to Neuroalgebraic Geometry Giovanni Luca Marchetti, Vahid Shahverdi, Stefano Mereta, Matthew Trager, Kathlén Kohn Abstract In this position paper, we promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry. To this end, we focus on algebraic models, such as neural networks with polynomial activations, whose associated function spaces are semi-algebraic varieties. We outline a dictionary between algebro-geometric invariants of these varieties, such as dimension, degree, and singularities, and fundamental aspects of machine learning, such as sample complexity, expressivity, training dynamics, and implicit bias. Along the way, we review the literature and discuss ideas beyond the algebraic domain. This work lays the foundations of a research direction bridging algebraic geometry and deep learning, that we refer to as neuroalgebraic geometry. † † affiliationnotice: Equal contribution 1 Introduction Parametric machine learning models, such as neural networks, define a space of functions as their parameters vary. These spaces are often referred to as neuromanifolds ( Kohn 2024 ; Calin 2020 ) . The geometry of neuromanifolds is intimately related to a number of questions at the heart of machine learning. Some geometric properties of neuromanifolds, such as their dimension, control statistical and computational aspects of the corresponding model, including sample complexity and expressivity. Moreover, neural networks learn through a process of gradient flow of their objective function. This optimization can be interpreted as minimizing a functional distance over the neuromanifold, effectively attracting the model towards an estimate of the ground-truth function. Consequently, geometric problems over neuromanifolds, such as nearest point problems, govern the training dynamics and provide insights into how neural networks learn. Therefore, understanding the geometry of neuromanifolds offers a twofold potential. First, it serves as a powerful theoretical framework for analyzing and explaining empirical phenomena in machine learning. Second, it can lead to the design of novel machine learning architectures. By establishing precise relationships between architectural parameters and the resulting neuromanifold geometry, we can systematically design models that exhibit desired theoretical and practical properties. Figure 1 : A neural variation of a celebrated doodle from the algebraic geometry literature ( Grothendieck 1968 ) . Machine Learning Algebraic Geometry sample complexity and expressivity dimension, degree, and covering number subnetworks and implicit bias singularities identifiability and invariance fibers of the parameterization optimization and gradient descent critical point theory, discriminants, and dynamical invariants Table 1 : A dictionary between machine learning and algebraic geometry. 1.1 The Power of Algebraic Geometry A rich class of machine learning models are (semi)-algebraic , meaning that the corresponding function is (piece-wise) polynomial in both the input and the parameters. Examples include deep neural networks with polynomial or Rectified Linear Unit (ReLU) activation function. For (semi-) algebraic models, the neuromanifold can be defined by a finite set of polynomial equalities and inequalities, i.e., it is a semi-algebraic variety . The study of these spaces is central to algebraic geometry , a field with a rich mathematical history that offers a wealth of ideas, tools, and invariants. Many of these, as we will argue, can be interpreted from the perspective of machine learning, shedding light on important questions in the field. For example, a fundamental invariant of algebraic varieties is their degree, which is characteristic of algebraic geometry since it is undefined for, say, general differentiable manifolds. The degree plays a central role in learning aspects of the corresponding model (see Section 4.1 ). Another crucial advantage of algebraic geometry is that, unlike differential geometry, it naturally encompasses singular spaces. This is convenient in the context of deep learning; despite the ‘manifold’ terminology, neuromanifolds of neural networks are far from being smooth spaces, since they often exhibit singularities that affect the behavior of the corresponding model. Moreover, as previously mentioned, distances in the ambient space of neuromanifolds play a central role in the learning process of machine learning models. This suggests that the metric structure should be taken into account when analyzing neuromanifolds. Distance-based problems over algebraic varieties are the focus of the field of metric algebraic geometry ( Breiding et al. 2024 ) , from which we will borrow several tools throughout this work. While focusing on algebraic models might seem restrictive, we note that arbitrary (continuous) functions can be approximated by algebraic ones with arbitrary precision. Thus, algebraic models are not only general, but can approximate arbitrary neuromanifolds. This allows for the extension of results and techniques discussed in this work to non-algebraic models, at least approximately. We will expand on this in Section 5.1 . 1.2 Neuroalgebraic Geometry Based on these reflections, we argue that algebraic machine learning models can be studied using the powerful toolbox of algebraic geometry, leading to new insights in deep learning theory . We propose the term neuroalgebraic geometry to refer to this emerging field of research. Neuroalgebraic geometry is closely related to algebraic statistics ( Pistone et al. 2000 ) – the study of statistical models defined by polynomial equations – and can be interpreted as its counterpart in the context of machine learning. In the following sections, we outline aspects of machine learning that can be reformulated in terms of (metric) algebraic geometry, highlighting natural intersections between the two fields. Along the way, we review and unify the previous literature in this direction. This work is intended as an invitation to the field of algebraic geometry for researchers from machine learning, and vice versa. From a broader perspective, we hope that it will inspire interdisciplinary research, positioning algebraic geometry among the mathematical disciplines that collaboratively contribute to unraveling the fundamental principles of deep learning. 2 Alternative Views A potential counterargument to our position is that the relvance of algebraic models might be limited, both theoretically and in practice. Indeed, polynomials are rarely used as activation functions in real-life neural networks, and they are excluded by standard formulations of the universal approximation theorem for deep learning ( Pinkus 1999 ) . However, we believe that algebraic models provide a setting where many intuitions that hold for networks in general can be formulated rigorously and, potentially, proved. Moreover, algebraic models are widely used as building blocks in actual architectures, as argued in Section 3 . Regarding approximation results, we note that algebraic models can approximate arbitrary network architectures, and indeed universal approximation theorems can be extended to include polynomials by considering networks with varying depth ( Yu et al. 2021 ) . From a broader perspective, several alternative approaches have been explored to analyze deep learning models, and in particular the function spaces they parameterize. Each of these approaches advocates for a specific mathematical formalism, which naturally brings original ideas to the field of machine learning, but also comes with restrictions on the models it considers. We believe that algebraic geometry provides unique tools to analyze aspects that are overlooked by other formalisms, as we will discuss in the next section. 2.1 Related Approaches Information geometry ( Amari 2016 ) is a well-established field which studies statistical models and neuromanifolds from the perspective of Riemannian geometry. While similar in spirit to neuroalgebraic geometry — both study the geometry of neuromanifolds — the two approaches rely on different mathematical machinery. The Riemannian approach of information geometry is not constrained