A Hybrid Supervised and Self-Supervised Graph Neural Network for Edge-Centric Applications

Eugenio Borzone, Matias Gerard · 2025

Read A Hybrid Supervised and Self-Supervised Graph Neural Network for Edge-Centric Applications by Eugenio Borzone, Matias Gerard (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12309.txt.

Introduction

Many prediction tasks concern links rather than things: will two proteins interact, will two compounds resemble each other? Eugenio Borzone and co-authors present a graph-based model built for edge-centered tasks, where the answer describes a pair of nodes. Their design mixes two kinds of teaching. Supervised training uses known answers; self-supervised training learns from unlabeled patterns; the loss weighs both. An attention step draws on node and edge features.

The system works in two stages trained together. First a graph network turns node features into compact sketches absorbing edge information. Then a feed-forward network reads the sketches of a pair and produces the call. In tests it matched or beat methods on protein interactions and gene-function terms. It also worked with one-hot features, enabling a task earlier methods could not do: judging similarity of compounds with unknown structures.

Catalog record

CREATOREugenio Borzone, Matias Gerard
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-12309

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Hybrid Supervised and Self-Supervised Graph Neural Network for Edge-Centric Applications Eugenio Borzone, Matias Gerard Abstract This paper presents a novel graph-based deep learning model for tasks involving relations between two nodes (edge-centric tasks), where the focus lies on predicting relationships and interactions between pairs of nodes rather than node properties themselves. This model combines supervised and self-supervised learning, taking into account for the loss function the embeddings learned and patterns with and without ground truth. Additionally it incorporates an attention mechanism that leverages both node and edge features. The architecture, trained end-to-end, comprises two primary components: embedding generation and prediction. First, a graph neural network (GNN) transform raw node features into dense, low-dimensional embeddings, incorporating edge attributes. Then, a feedforward neural model processes the node embeddings to produce the final output. Experiments demonstrate that our model matches or exceeds existing methods for protein-protein interactions prediction and Gene Ontology (GO) terms prediction. The model also performs effectively with one-hot encoding for node features, providing a solution for the previously unsolved problem of predicting similarity between compounds with unknown structures. I Introduction Graphs are versatile structures used to model relationships between entities in non-Euclidean domains. They underpin applications ranging from social networks [ 1 ] and recommendation systems [ 2 , 3 ] to bioinformatics [ 4 ] . Early shallow embedding methods such as DeepWalk [ 5 ] , LINE [ 6 ] , and Node2Vec [ 7 ] demonstrated the value of data-driven representations but share two well-known limitations: parameter redundancy and limited generalization [ 8 ] . Graph Neural Networks (GNNs) overcome these drawbacks by parameter sharing and inductive learning [ 9 ] . Within GNNs, spectral and spatial formulations provide complementary views; the reader is referred to [ 10 , 11 , 12 , 13 , 14 , 15 ] for canonical treatments. Message Passing Neural Networks (MPNNs) generalize earlier GNN variants by formalizing graph convolutions as iterative message-passing steps [ 16 ] . At iteration t , node p aggregates information from its neighbours \mathcal{N}(p) via \begin{split}m_{p}^{(t+1)}&=\sum_{q\in\mathcal{N}(p)}M_{t}\!\bigl(\mathbf{x}_{p}^{(t)},\mathbf{x}_{q}^{(t)},e_{pq}\bigr),\\ \mathbf{x}_{p}^{(t+1)}&=U_{t}\!\bigl(\mathbf{x}_{p}^{(t)},m_{p}^{(t+1)}\bigr),\end{split} (1) where \mathbf{x}_{p}^{(t)} and \mathbf{x}_{q}^{(t)} are the node features at step t , e_{pq} is an optional edge feature, M_{t} is the message function, m_{p}^{(t+1)} is the aggregated message, and U_{t} is the update function. After T iterations, an order-invariant readout R\!\bigl(\mathbf{x}_{p}^{(T)}\bigr) (e.g., sum or mean pooling) produces the final prediction at node or graph level. Equation ( 1 ) establishes the notation used throughout the paper. Motivation and contribution. Existing MPNNs excel at node-level tasks but still under-utilize edge attributes and require large labelled datasets for supervision. Edge-centric problems—typical in bioinformatics, where interactions are often encoded on edges [ 4 ] —remain challenging. This work addresses these gaps by: 1. Proposing an MPNN-based architecture that natively treats node and edge features with a shared attention mechanism inspired by transformers [ 17 ] ; 2. Introducing a hybrid loss that couples supervised and self-supervised objectives, enabling learning even when node features are sparse (e.g. one-hot encodings); 3. Demonstrating competitive performance on tasks that require edge-level regression and classification, including protein–protein interaction prediction. The remainder of this paper is organised as follows. Section II details the materials and methods that ground the proposed model; Section III introduces the benchmark datasets employed in our experiments; Section IV reports and discusses the empirical results and ablation studies; finally, Section V summarises the main contributions and outlines future research directions. II Materials and methods This work presents a neural model composed of two main parts: an embedding block and a prediction block. The prediction block produces an estimation for pairs of nodes in the graph. This section presents the theoretical concepts and materials used in this work. First, we describe the embedding block. Then, we show the prediction block and its components, and explain the process of building the dataset. II-A Graph and subgraph definitions Let \mathcal{G}=\{v,e\} be a graph, where v is the set of |v| nodes in the network, and e denotes the set of edges connecting them. Let \mathbf{X}\in\mathbb{R}^{|v|\times d} be the feature matrix of nodes in \mathcal{G} , where d is the number of features for each node. Furthermore, let \mathcal{E}\in\mathbb{R}^{|e|\times q} denote the edge feature matrix, where each row represents the features associated with a specific edge in the graph. These edge features may or may not exist and can be a vector of arbitrary size q . Figure 1 a shows an example of a typical graph and its elements. The subgraph \mathcal{G}^{\prime}_{i,j}=\{v_{i,j},e_{i,j}\}\subseteq\mathcal{G} is defined as being induced by nodes i and j (Figure 1 b), where v_{i,j}=\{\mathcal{N}(i),\mathcal{N}(j)\} are the nodes included, and e_{i,j} are the connections between them. The set \mathcal{N}(k) is represented by the direct neighbors that share an edge with node k . Additionally, the edge features matrix \mathbf{E}^{\prime}_{\mathcal{G}^{\prime}_{i,j}}\in\mathbb{R}^{|e_{i,j}|\times q} is associated with |e_{i,j}| number of connections in the subgraph \mathcal{G}^{\prime}_{i,j} . The feature matrix \mathbf{X}^{\prime}_{\mathcal{G}^{\prime}_{i,j}}\in\mathbb{R}^{|v_{i,j}|\times d} is also defined for the |v_{i,j}| nodes in the subgraph \mathcal{G}^{\prime}_{i,j} . ((a)) ((b)) Fig. 1 : (a) Example of a graph \mathcal{G} . (b) Example of pattern subgraph \mathcal{G}^{\prime}_{0,1} induced for nodes 0 and 1. Nodes and edges related to node 1 are highlighted in yellow, while those related to node 0 are shown in blue. These represent the first-degree neighbors of each node within the subgraph. These graphs represent only the structure; in this representation, each node is associated with a feature vector, and each edge has corresponding edge features. II-B Proposed model Our model operates on subgraphs \mathcal{G}^{\prime} , structured around two central nodes. This approach is essential for edge-centric tasks, where the aim is to predict the presence or properties of connections between specific pairs of nodes. The subgraph is constructed to enrich the representation of these central nodes by leveraging their local neighborhood. In each subgraph, the central nodes and their direct neighbors, along with all existing edges among these nodes, form the context for learning. The model processes each subgraph in two stages: embedding generation and prediction. In the embedding generation phase, the node features within the subgraph are transformed into dense, low-dimensional vectors \mathbf{Z} that capture both structural and feature-based information. These embeddings, particularly those of the central nodes, are then concatenated and passed to the prediction module to estimate the desired property associated with their connection. The entire model is trained end-to-end using a loss function that combines supervised and self-supervised training, allowing it to learn meaningful patterns even for nodes with limited or missing initial information. Figure 2 illustrates the model’s architecture, with detailed descriptions of each component provided in the following sections. Fig. 2 : Model architecture overview. The input consists of the pattern subgraph \mathcal{G}^{\prime}_{i,j} , node features \mathbf{X}^{\prime}_{\mathcal{G}^{\prime}_{i,j}} , and edge features \mathbf{E}^{\prime}_{\mathcal{G}^{\prime}_{i,j}} . These inputs are processed through the embedding generation block: Tokenizer, the NodeEdgeAttentionConv ( NEAConv ) layer, and Tanh to form the embeddings Z_{i} and Z_{j} . The embeddings then pass through a linear layer followed by a Tanh activation, are reordered, and finally fed into a three-layer perceptron to produce the output. II-C Embedding generation This block involves two components: the tokenizer and the NodeEdgeAttentionConv ( NEAConv ) layer, as illustrated in Figure 2 . Node embeddings are compact, low-dimensional

Read the complete text