The Impact of Dimensionality on the Stability of Node Embeddings
Tobias Schumacher, Simon Reichelt, Markus Strohmaier · 2026
Read The Impact of Dimensionality on the Stability of Node Embeddings by Tobias Schumacher, Simon Reichelt, Markus Strohmaier (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-08492.txt.
Introduction
Schumacher, Reichelt, and Strohmaier test how vector size affects the stability of node embeddings across repeated runs. Across five methods, including node2vec, GraphSAGE, and VERSE, and many sizes and datasets, size mattered greatly but not uniformly. Some methods steadied at larger sizes while others wobbled or declined. Most importantly, the most stable sizes were not always the best performing ones, complicating how practitioners should pick a dimension.
Catalog record
| CREATOR | Tobias Schumacher, Simon Reichelt, Markus Strohmaier |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52138 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.08492
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Impact of Dimensionality on the Stability of Node Embeddings Tobias Schumacher, Simon Reichelt, Markus Strohmaier Abstract Previous work has established that neural network-based node embeddings return different outcomes when trained with identical parameters on the same dataset, just from using different training seeds. Yet, it has not been thoroughly analyzed how key hyperparameters such as embedding dimension could impact this instability. In this work, we investigate how varying the dimensionality of node embeddings influences both their stability and downstream performance. We systematically evaluate five widely used methods—–ASNE, DGI, GraphSAGE, node2vec, and VERSE—–across multiple datasets and embedding dimensions. We assess stability from both a representational perspective and a functional perspective, alongside performance evaluation. Our results show that embedding stability varies significantly with dimensionality, but we observe different patterns across the methods we consider: while some approaches, such as node2vec and ASNE, tend to become more stable with higher dimensionality, other methods do not exhibit the same trend. Moreover, we find that maximum stability does not necessarily align with optimal task performance. These findings highlight the importance of carefully selecting embedding dimension, and provide new insights into the trade-offs between stability, performance, and computational effectiveness in graph representation learning. 1 Introduction Graphs are a ubiquitous form of data, with examples including social networks, citation networks, and molecules. The representation of interactions between entities as graphs has enabled researchers to conduct systematic analysis of diverse network systems [ 16 ] . For example, social networks have been leveraged for a variety of applications, including the recommendation of friends and content, as well as for advertising purposes. One effective approach for leveraging graphs as features for machine learning techniques is to transform them into node embeddings, which represent the graph as a low-dimensional vector space. The use of embeddings enables the encoding of the relationships and properties of nodes in a manner that allows them to serve as inputs for a variety of downstream tasks, including node classification [ 3 ] , node clustering [ 11 ] , visualization [ 36 ] , and link prediction [ 34 ] . For most node embedding methods, a number of sources of randomness are at play, including random initialization, random walks, or stochastic optimization schemes. Consequently, multiple training runs employing the same techniques on the same dataset will produce embeddings that differ from one another. This instability of node embeddings has been documented in the literature [ 50 , 56 ] . A critical factor for training embeddings, which could also affect the stability of embeddings, is the dimension of the embedding vectors. A dimension that is too small may be unable to capture the structural information of the graph, aiming to compress more information than can fit into small vector spaces. This could also yield unstable embeddings, as different information might get encoded in different training runs. Conversely, a higher dimension allows an embedding to capture more information, but it also adds more computational complexity and may lead to overfitting or noise [ 35 ] . This study builds upon previous research on stability by investigating the impact of varying embedding dimensions on both the stability and performance of embeddings. In particular, this study seeks to address the question of how embedding dimension influences node embedding stability. We do not consider stability alone but also aim to draw a connection between stability and performance across varying dimensions. Toward that end, we consider five node embedding methods, namely ASNE [ 33 ] , DGI [ 55 ] , GraphSAGE [ 21 ] , node2vec [ 17 ] , and VERSE [ 54 ] , and train embeddings of these methods for varying dimensions across several datasets. Across each dimension, we evaluate their stability considering both a representational and a functional angle—for the latter using the outputs of downstream classifiers, of which we also consider the performance. Our results indicate that the stability of node embedding varies across different dimensionalities, both from the representational and the functional perspective. We do, however, observe different patterns across different methods, such as node2vec and ASNE generally increasing their stability with higher dimensionality, which is not less the case for the other methods under study. Further, we find that optimal performance does not always conincide with highest stability. Overall, we believe our work provides novel insight into the choice of dimensionality of network representations. To foster reproducibility, we provide the code of all our experiments online 1 1 1 https://github.com/dess-mannheim/dimpact . 2 Background Before describing our experimental framework, we briefly provide background on node embeddings and their stability, including some formal definitions, and summarize related work on the topic. 2.1 Preliminaries Node Embeddings. Node embedding algorithms map nodes of a given graph to (ideally) low-dimensional vectors in a manner that facilitates subsequent analysis and machine learning tasks by utilizing information from the link structure of the graph as well as, potentially, node and edge features. Formally, we define a graph \mathcal{G} as a tuple \mathcal{G}=(\mathcal{V},\mathcal{E}) , where \mathcal{V}=(v_{1},...,v_{N}) is the set of vertices, which, w.l.o.g., we assume to be ordered, and \mathcal{E}=(e_{1},...,e_{M}) is the set of edges in the graph. Then, a node embedding \varphi:\mathcal{V}\longrightarrow\mathbb{R}^{D} maps each node v_{i}\in\mathcal{V} to an embedding vector \bm{z}_{i}=\varphi(v_{i}) of dimension D\in\mathbb{N} . By stacking the embedding vectors, one obtains an embedding matrix \bm{Z}=[\bm{z}_{1},\dots,\bm{z}_{N}]^{T}\in\mathbb{R}^{N\times D} . Downstream Tasks. After embeddings have been generated, they can be used for downstream tasks such as node classification, link prediction, or graph reconstruction, which we focus on for this study. In node classification, w.l.o.g., it is assumed that each node v_{i} is associated with a label y_{i}\in\mathcal{C}=\{1,\dots,C\},C\in\mathbb{N} . The goal is then to train a classifier c_{\operatorname{NC}}:\mathbb{R}^{D}\longrightarrow\mathcal{C} on the embeddings of individual nodes with known labels for prediction on nodes with unknown labels. Formally, we define the outputs of this classification task as \bm{O}_{i}=c_{\operatorname{NC}}(\bm{z}_{i})\in\mathbb{R}^{C} , which we assume to consist of the instance-wise class probability scores for each of the C given classes. Assuming that an ordered set of n\leq N nodes is classified, these outputs can also be stacked to an output matrix \bm{O}\in\mathbb{R}^{n\times C} . In link prediction, the goal is to predict future links based on the current link structure of a graph. This yields a binary classification problem, where corresponding classifiers c_{\operatorname{LP}}:\mathbb{R}^{D}\times\mathbb{R}^{D}\longrightarrow\{0,1\} take as input a pair of node embeddings \bm{z}_{i},\bm{z}_{j} corresponding to the edge e_{k}=(v_{i},v_{j}) that is classified. In practice, due to the sparsity of empirical networks, these classifiers are typically trained on all existing (positive) edges within a network, but only use a fraction of non-existing (negative) edges. Within our study, we will overload notation and also denote the output matrix of an ordered set of m\leq N^{2} node pairs as \bm{O}\in\mathbb{R}^{m\times 2} . Finally, graph reconstruction can also be considered a link prediction problem, with the only difference being that the task at hand is to correctly predict which edges already exist. Representational and Functional Stability. In line with previous work, we define the stability of an embedding algorithm informally as the difference between two embeddings due to randomness in the embedding algorithm [ 50 ] . In this setting, we assume that all training data and hyperparameters are fixed, with only the training seed being different. To analyze the stability of