LOBSTUR: A Local Bootstrap Framework for Tuning Unsupervised Representations in Graph Neural Networks

So Won Jeong, Claire Donnat · 2025

Read LOBSTUR: A Local Bootstrap Framework for Tuning Unsupervised Representations in Graph Neural Networks by So Won Jeong, Claire Donnat (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14867.txt.

Introduction

So Won Jeong and Claire Donnat present LOBSTUR, a way to tune unsupervised graph neural networks without labels, in work from 2025. Such networks learn strong node representations but are touchy about hyperparameters, and with no labels there is no easy score for picking the best model. Their frame bootstraps the graph locally, resampling edges and features in ways that respect nearby structure.

Embedding consistency across the resampled graphs is scored with canonical correlation analysis, giving a label-free quality signal. On standard datasets the method lifted classification accuracy by about two thirds over uninformed tuning. A real-world deployment is included to show the frame works outside benchmarks.

Catalog record

CREATORSo Won Jeong, Claire Donnat
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE121 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-14867

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LOBSTUR: A Local Bootstrap Framework for Tuning Unsupervised Representations in Graph Neural Networks So Won Jeong, Claire Donnat Abstract Graph Neural Networks (GNNs) are increasingly used in conjunction with unsupervised learning techniques to learn powerful node representations, but their deployment is hindered by their high sensitivity to hyperparameter tuning and the absence of established methodologies for selecting the optimal models. To address these challenges, we propose LOBSTUR-GNN ( Lo cal B oot s trap for T uning U nsupervised R epresentations in GNNs) i), a novel framework designed to adapt bootstrapping techniques for unsupervised graph representation learning. LOBSTUR-GNN tackles two main challenges: (a) adapting the bootstrap edge and feature resampling process to account for local graph dependencies in creating alternative versions of the same graph, and (b) establishing robust metrics for evaluating learned representations without ground-truth labels. Using locally bootstrapped resampling and leveraging Canonical Correlation Analysis (CCA) to assess embedding consistency, LOBSTUR provides a principled approach for hyperparameter tuning in unsupervised GNNs. We validate the effectiveness and efficiency of our proposed method through extensive experiments on established academic datasets, showing an 65.9% improvement in the classification accuracy compared to an uninformed selection of hyperparameters. Finally, we deploy our framework on a real-world application, thereby demonstrating its validity and practical utility in various settings. 1 1 1 The code is available at github.com/sowonjeong/lobstur-graph-bootstrap . 1 Introduction With the expanding availability of network and spatial data in the sciences, Graph Neural Networks (GNNs) have emerged as a compelling approach to identify interaction patterns within complex systems. Examples include spatial transcriptomics ( Zhu et al.,, 2018 ; Dong and Zhang,, 2022 ) , where graph-based neural networks are used to learn representations of cell neighborhoods that can be correlated with cancer outcomes, and microbiome studies ( Lamurias et al.,, 2022 ; Le et al.,, 2020 ) where they are used for genome assembly or to predict metabolite information from microbes. In many of these applications, scientists are increasingly interested in using GNNs in conjunction with unsupervised learning techniques for learning informative representations, due to the paucity of available labeled data, or as a way of automatically detecting structure or patterns ( Zhu et al.,, 2018 ; Dong and Zhang,, 2022 ; Lamurias et al.,, 2022 ; Le et al.,, 2020 ) . Consequently, the last few years have seen the development of a number of unsupervised GNN methods for the sciences ( Hu et al.,, 2021 ; Partel and Wählby,, 2021 ; Zhang et al.,, 2020 ; Li et al.,, 2021 ; Ishiai et al.,, 2024 ) , usually adapting known methods for Euclidean data such as tSNE ( van der Maaten and Hinton,, 2008 ) , k-Means clustering, or UMAP ( McInnes et al.,, 2018 ) to accommodate graph data and GNNs. These methods typically involve few hyperparameters, but their scope of application is typically confined to a specific data type or use case. Within the methods community, on the other hand, recent advances in unsupervised node representation learning seem to have primarily been driven by contrastive learning ( Stokes et al.,, 2020 ; Zhang et al.,, 2021 ; You et al.,, 2021 ) . This popular self-supervised learning framework has indeed demonstrated impressive performance for learning rich and versatile data representations across various domains. However, in the graph-setting, despite their impressive performance and promising results on academic benchmarks, these methods are not tuning-free, making them difficult to deploy in real-world applications. In fact, they rely heavily on selecting appropriate values for several of their hyperparameters, but incorrect hyperparameter values can lead to severely distorted data representations. We illustrate this effect in Figure 1 by showing how different choices of hyperparameters can decrease the accuracy of a linear classifier (trained on the learned node embeddings of a toy benchmark) from 73% to 30%, thus indicating a severe loss in embedding quality. Despite the empirical importance of hyperparameter tuning, there is currently no valid hyperparameter selection procedure for unsupervised GNN representation learning. In the methods community, new unsupervised learning approaches are commonly tested on established benchmark datasets, with hyperparameters selected based on performance in a downstream node classification task. However, this procedure essentially converts the problem into a supervised learning setting, making it unsuitable for genuinely unsupervised, real-world use cases. Hyperparameter tuning in unsupervised settings is made difficult by two main challenges: (a) the absence of a clear ground truth or statistical framework for unsupervised learning, and (b) the lack of an established metric to evaluate the learned embeddings. To our knowledge, the only study that attempts to measure the quality of latent representations is that of Tsitsulin et al., , which empirically evaluates various metrics. Yet, without a proper inference framework, pinpointing a suitable metric remains a significant challenge. Contributions. In this paper, we propose the first bootstrapped-based method for selecting hyperparameters for unsupervised GNN representation learning. Hyperparameters can be broadly categorized into two types: (1) those that require tuning within a specific model (e.g. parameters in the loss function) and (2) those that involve tuning across a family of models (e.g. different GNN architectures or sizes). In this paper, we aim to tackle both, as both are essential to self-supervised learning. More specifically, 1. We cast the learning of representations as an estimation problem: we posit that the learned representations correspond to a learned low-dimensional manifold, which must therefore be consistent under a noise model, as explicited in Section 2 . 2. To generate independent copies of the same graph, we propose a bootstrap procedure based on nonparametric modeling of the graph as a graphon Su et al., (2020) (Section 3 ). 3. To evaluate the quality of the embeddings learned on independent copies of the same graph in the absence of labels, we suggest using Canonical Correlation Analysis ( Hotelling,, 1936 ) as a translation- and rotation-invariant tool to quantify the stability of the learned embedding spaces (Section 4 ). Figure 1 : Cora. Evaluation of the CCA-SSG embeddings Zhang et al., (2021) , an unsupervised learning method, for each combination of the hyperparameters (loss parameter \lambda , edge drop rate (EDR), feature mask rate (FMR)). Each entry denotes the mean and standard deviation of the node classification accuracy of a linear classifier trained on the learned representations (averaged over 20 experiments). 2 Problem Formalization Establishing a framework for hyperparameter tuning in unsupervised learning requires us to address two fundamental questions: what are we aiming to estimate, and where does the randomness come from? In the graph setting, the data is presented in two modalities: a feature matrix, and an adjacency matrix. Unsupervised learning can be framed as learning what information is shared across modalities, and what information is specific to each one in a condensed format. This approach is typically described in the data-integration literature using a latent variable space model Bishop, (1998) ; Hoff et al., (2002) , which we adapt here for the graph domain. Inference setting. We consider a graph G on n nodes with features X\in\mathbb{R}^{n\times p}, and denote by A\in\{0,1\}^{n\times n} its corresponding (binary) adjacency matrix. We assume the graph is sampled from a graphon W (see for instance Gao et al., ) — a non-parametric random graph model–, and that node features are

Read the complete text