Topology and geometry of the learning space of ReLU networks: connectivity and singularities

Marco Nurisso, Pierrick Leroy, Giovanni Petri & Francesco Vaccarino · 2026

Read Topology and geometry of the learning space of ReLU networks: connectivity and singularities by Marco Nurisso, Pierrick Leroy, Giovanni Petri & Francesco Vaccarino (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00693.txt.

Description

Marco Nurisso and colleagues examine the space of feed-forward ReLU networks, asking whether it is connected and where its singularities lie. After initialisation, training under gradient flow restricts the space to an algebraic variety arising from the homogeneous nature of ReLU activation. The study extends earlier results on general directed acyclic graph architectures, mapping connectedness through bottleneck nodes and balance conditions for specific subsets. It shows singularities are closely tied to the topology of the underlying graph and its induced sub-networks. Simple numerical experiments validate the theory.

Catalog record

CREATORMarco Nurisso, Pierrick Leroy, Giovanni Petri & Francesco Vaccarino
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82183
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00693

Browse the full library · About the Tower of Babel