Latent Space Topology Evolution in Multilayer Perceptrons

Eduardo Paluzo-Hidalgo · 2025

Read Latent Space Topology Evolution in Multilayer Perceptrons by Eduardo Paluzo-Hidalgo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01569.txt.

Introduction

Eduardo Paluzo-Hidalgo, writing alone, watches the shape of data change as it flows through a neural network, in a 2025 paper. Each layer of a multilayer perceptron reshapes the data's internal representation. Topology, the mathematics of shape, can track holes, clusters, and connections in that representation as they appear and vanish.

He builds a tower of geometric complexes, one per layer, linked by maps that follow the network's own computation. Two kinds of persistence then apply: features tracked across scales inside a layer, and features tracked across the layers themselves. Stability theorems guarantee the descriptors do not jump under small perturbations. A clean theoretical link ties easy separation of classes to disconnected pieces in the construction. Experiments on synthetic and medical data found redundant layers and sharp transition points, offering an interpretable window into how networks organize data for classification.

Catalog record

CREATOREduardo Paluzo-Hidalgo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01569

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Latent Space Topology Evolution in Multilayer Perceptrons Eduardo Paluzo-Hidalgo Abstract This paper introduces a topological framework for interpreting the internal representations of Multilayer Perceptrons (MLPs). We construct a simplicial tower , a sequence of simplicial complexes connected by simplicial maps, that captures how data topology evolves across network layers. Our approach enables bi-persistence analysis: layer persistence tracks topological features within each layer across scales, while MLP persistence reveals how these features transform through the network. We prove stability theorems for our topological descriptors and establish that linear separability in latent spaces is related to disconnected components in the nerve complexes. To make our framework practical, we develop a combinatorial algorithm for computing MLP persistence and introduce trajectory-based visualisations that track data flow through the network. Experiments on synthetic and real-world medical data demonstrate our method’s ability to identify redundant layers, reveal critical topological transitions, and provide interpretable insights into how MLPs progressively organise data for classification. K eywords Neural Network Interpretability, Topological Data Analysis (TDA), Mapper Algorithm, Latent Representations, Multilayer Perceptrons (MLPs) 1 Introduction The widespread deployment of neural networks in critical decision-making systems has created an urgent need for interpretable machine learning models. While these architectures demonstrate remarkable empirical success across diverse domains, their internal mechanisms remain largely opaque, earning them the notorious designation as “black boxes". This opacity originates from the confluence of several fundamental challenges: the high-dimensional nature of parameter spaces, the compositional complexity introduced by multiple layers of non-linear transformations, and the emergent behaviours that arise from the interplay between architecture and optimisation dynamics. In this work, we focus on Multilayer Perceptrons (MLPs), the foundational architecture underlying modern deep learning. Despite their apparent simplicity compared to contemporary architectures, MLPs remain ubiquitous as essential components in more complex models. They appear as dense layers in Convolutional Neural Networks (CNNs), as projection heads in Vision Transformers, and as feed-forward networks in Transformer blocks. Understanding the internal representations learned by MLPs thus provides a gateway to interpreting broader classes of neural architectures. Moreover, in safety-critical applications such as medical diagnosis, financial risk assessment, and autonomous systems, the ability to interpret MLP decisions is highly important. The challenge of neural network interpretability has two complementary research directions. The first employs post-hoc explanation methods that approximate network behaviour through simpler, interpretable models. Techniques such as LIME [ 18 ] and SHAP [ 13 ] construct local linear approximations to explain individual predictions, while attribution methods like Layer-wise Relevance Propagation [ 15 ] distribute prediction scores backwards through the network to quantify feature importance. The second direction, which we pursue in this work, seeks to understand the intrinsic geometry and topology of learned representations. This approach recognises that neural networks fundamentally perform a sequence of geometric transformations, progressively reorganising data to achieve linear separability in the final layer. Recent advances in topological data analysis (TDA) have revealed profound connections between neural network expressivity and the topological properties of data transformations. Work by Saul and Arendt [ 19 ] and Pearson [ 17 ] established the foundational paradigm of using TDA for machine learning explanations, demonstrating how the Mapper algorithm can construct topological representations that reveal both the structure of data and the decision-making processes of trained models. Naitzat et al. [ 16 ] demonstrated that deep networks systematically reduce the topological complexity of data representations, with each layer acting to simplify the intrinsic structure until classification becomes trivial. This observation was quantified by Suresh et al. [ 21 ] , showing that Betti numbers, which count topological features such as connected components, loops, and voids, decrease with network depth. From a different perspective, Grigsby and Lindsey [ 6 ] analysed how ReLU networks partition input space through hyperplane arrangements, revealing geometric constraints on representational capacity. These topological insights have been formalised into bounds by Lee et al. [ 12 , 11 ] , who established relationships between dataset topology and the minimal network width required for successful learning. Our work advances this topological perspective by introducing a comprehensive framework for tracking and analysing the evolution of data representations across all layers of an MLP. While previous approaches have primarily focused on comparing input and output topologies or analysing individual layers alone, we develop a unified mathematical framework that captures the continuous transformation of topological features throughout the network. Our key innovation lies in constructing a simplicial tower , a sequence of simplicial complexes connected by simplicial maps, that encodes the topological structure at each layer while preserving the relationships induced by the network’s transformations. This construction enables several contributions. First, we formalise the relationship between consecutive layer representations through pullback covers and their associated nerve complexes, providing a rigorous foundation for topological analysis of neural networks. Second, we develop a bi-persistence framework that simultaneously tracks two types of topological evolution: layer persistence , which captures how topological features evolve as we vary the scale parameter at a fixed layer, and MLP persistence , which reveals how features transform as data flows through the network at a fixed scale. This bi-persistence approach, inspired by the Multiscale Mapper framework [ 3 ] , provides complementary views of network behaviour that together yield a comprehensive understanding of how MLPs organise data. Third, we establish stability theorems that guarantee our topological descriptors remain robust under perturbations of the covering choices, ensuring that our insights reflect genuine properties of the learned representations rather than artefacts of our analysis. Fourth, we introduce a trajectory-based visualisation that tracks how individual data points and clusters flow through the network’s latent spaces, transforming abstract topological concepts into interpretable geometric narratives. Finally, we develop a practical combinatorial algorithm for computing MLP persistence that avoids the computational challenges of explicitly constructing pullback covers, making our framework applicable to real-world neural networks. Our approach distinguishes itself from existing work in several ways. Unlike the binary analysis of topological simplification in [ 16 ] , we capture the complete range of topological transformations across all layers. While [ 21 ] tracks global invariants such as Betti numbers, our framework preserves the detailed structure of how specific topological features persist, merge, or disappear. In contrast to theoretical limitations on network expressiveness [ 8 ] , we provide constructive tools for understanding existing architectures. Furthermore, our explicit modelling of relationships between layers through simplicial maps offers a different analysis to those based on activation patterns [ 10 ] . Finally, [ 19 ] analyses trained models as black boxes; they only need the input data and final prediction probabilities. Their method works with any trained classifier without needing access to internal representations. However, our approach performs white-box analysis of the internal architecture, specifically requiring access to all intermediate layer representations in MLPs. The practical implications of our framework extend beyond theoretical understanding. By revealing how MLPs progressively organise data, our approach can inform architecture design, suggest optimal layer widths, and identify redundant transformations. The trajectory analysis provides intuitive visualisations for understanding classification decisions, potentially enhancing trust in safety-critical applications. Moreover, the mathematical rigour of our topological framework opens new avenues for proving properties about neural network behaviour and establishing formal guarantees about their decision-making processes. The remainder of this paper is organised as follows. Section 2 establishes

Read the complete text