Adapting Unsigned Graph Neural Networks for Signed Graphs: A Few-Shot Prompt Tuning Approach

Zian Zhai, Sima Qing, Xiaoyang Wang, Wenjie Zhang · 2024

Read Adapting Unsigned Graph Neural Networks for Signed Graphs: A Few-Shot Prompt Tuning Approach by Zian Zhai, Sima Qing, Xiaoyang Wang, Wenjie Zhang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-12155.txt.

Introduction

Zian Zhai, Sima Qing, Xiaoyang Wang, and Wenjie Zhang present a 2024 method for learning on signed graphs, where links between nodes can be friendly or hostile. Models built for such graphs need many labels, which are scarce in real use. Models for plain unsigned graphs are easy to pre-train, since unsigned data is plentiful, but the two graph types do not line up well.

Their method, called SGPT, adapts a pre-trained unsigned model to signed tasks with only a few labels. A graph template based on balance theory sorts out the mixed ties that hostile links create. A task template recasts the signed tasks as link guessing, to match the pre-training goal. Added prompts align the data features and the meaning of link signs. Tests on seven data sets beat the best prior methods by a clear margin.

Catalog record

CREATORZian Zhai, Sima Qing, Xiaoyang Wang, Wenjie Zhang
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-12155

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adapting Unsigned Graph Neural Networks for Signed Graphs: A Few-Shot Prompt Tuning Approach Zian Zhai, Sima Qing, Xiaoyang Wang, Wenjie Zhang Abstract Signed Graph Neural Networks (SGNNs) are powerful tools for signed graph representation learning but struggle with limited generalization and heavy dependence on labeled data. While recent advancements in “graph pre-training and prompt tuning” have reduced label dependence in Graph Neural Networks (GNNs) and improved their generalization abilities by leveraging pre-training knowledge, these efforts have focused exclusively on unsigned graphs. The scarcity of publicly available signed graph datasets makes it essential to transfer knowledge from unsigned graphs to signed graph tasks. However, this transfer introduces significant challenges due to the graph-level and task-level divergences between the pre-training and downstream phases. To address these challenges, we propose Signed Graph Prompt Tuning (SGPT) in this paper. Specifically, SGPT employs a graph template and a semantic prompt to segregate mixed link semantics in the signed graph and then adaptively integrate the distinctive semantic information according to the needs of downstream tasks, thereby unifying the pre-training and downstream graphs. Additionally, SGPT utilizes a task template and a feature prompt to reformulate the downstream signed graph tasks, aligning them with pre-training tasks to ensure a unified optimization objective and consistent feature space across tasks. Finally, extensive experiments are conducted on popular signed graph datasets, demonstrating the superiority of SGPT over state-of-the-art methods. 1 Introduction In many real-world scenarios, interactions between entities are inherently complex, often involving opposing relationships, such as friendships and enmities, trust and distrust, or agreement and disagreement ( Liu et al. 2021 ; Sun et al. 2022b ) . Such dynamics are often modeled as signed graphs, where links carry both positive and negative signs to represent these opposite connections. To learn the effective representations of signed graphs, Signed Graph Neural Networks (SGNNs) have attracted increasing research interest ( Derr, Ma, and Tang 2018 ; Shu et al. 2021 ; Huang et al. 2021 ) . SGNNs incorporate link sign signals into the message-passing process, updating node features by independently aggregating information from neighbors with different semantics. This approach yields expressive representations for a variety of signed graph tasks, such as node classification ( Mercado, Bosch, and Stoll 2020 ) and link sign prediction ( Huang et al. 2019 ; Huang et al. 2021 ; Li et al. 2023 ) . Although SGNNs have achieved remarkable success, they suffer from limitations of label dependence and overfitting problems. On the one hand, SGNNs are trained in a supervised manner, which requires a large amount of task-specific labels for supervision. These labels, such as node and link sign labels, are typically expensive to obtain in real-world applications ( Tang et al. 2016 ; Tang, Aggarwal, and Liu 2016 ; Zhang et al. 2024 ) . On the other hand, compared to unsigned graphs, signed graph datasets exhibit exceptionally sparser structures and more noises ( Shu et al. 2021 ; Zhang et al. 2023 ) . This makes SGNNs trained on signed graphs more prone to memorizing these non-universal patterns, leading to overfitting problems. Inspired by the success of “pre-training and prompt tuning” in Large Language Models (LLMs), recent research has applied such a paradigm to graph tasks to overcome label reliance and overfitting problems in conventional Graph Neural Networks (GNNs) ( Sun et al. 2022a ; Fang et al. 2024 ) . In the pre-training phase, GNNs are trained to learn task-agnostic priors from the graph structure in an unsupervised manner ( Hu et al. 2020 ; You et al. 2021 ) . In the prompt tuning phase, the downstream tasks are reformulated into the same form as the pre-training task, maximizing the utilization of prior knowledge. Meanwhile, with the pre-trained model frozen, several lightweight prompt vectors are introduced and tuned with task-specific labels. Compared to supervised training, this paradigm allows GNNs to learn transferable knowledge, enhancing their generalization abilities. Moreover, with fewer tunable parameters, it can adapt the pre-trained model to different downstream tasks when the task-specific labels are limited ( Sun et al. 2023 ; Liu et al. 2023 ) . However, current research only focuses on unsigned graphs, overlooking the scenarios where downstream tasks involve signed graphs. Such oversight is significant, as signed graphs are typically limited, with far fewer publicly available datasets ( Zhang et al. 2023 ) , making it challenging to gather sufficient data to directly train a generalized SGNN. In contrast, unsigned graphs are more abundant and easier to access, enabling effective pre-training of generalized GNNs ( Sun et al. 2023 ; Yu et al. 2024a ) . This makes transferring general graph knowledge to address signed graph tasks a promising approach. Hence, this study focuses on prompt tuning pre-trained GNNs for few-shot signed graph tasks, namely node classification and link sign prediction. This problem is non-trivial due to two types of divergence between the pre-training and downstream tasks. • Graph-level divergence . The first challenge arises from the divergence between the graph types used in pre-training and those in the downstream stage, as downstream signed graphs carry more complex semantic information. In unsigned graphs, links consistently represent the connectivity between nodes ( Hu et al. 2020 ) . GNNs pre-trained on unsigned graphs learn node representation based on the social theory homophily ( McPherson, Smith-Lovin, and Cook 2001 ) . In other words, they aggregate features from connected neighbors uniformly. However, downstream tasks involve both positive and negative link signs. The negative links contain the opposite semantics to the positive links, violating the key assumptions of homophily in GNNs. Therefore, dedicated efforts are needed to narrow down such graph-type divergence when applying the pre-trained GNNs. • Task-level divergence . The uniqueness of signed graph tasks presents the second challenge: how to transfer prior knowledge to unseen tasks that are specific to signed graphs. The pre-training tasks on unsigned graphs aim to preserve the instinct graph structures such as graph similarities and node connectivity ( Zhang and Chen 2018 ; You et al. 2021 ) . These tasks primarily focus on node features and structural information. Comparatively, downstream signed graph tasks aim to optimize the task-specific loss function, i.e., fit the ground truth of node labels or link sign labels ( Derr, Ma, and Tang 2018 ; Huang et al. 2019 ) . They require the model to handle not only the node features but also the semantic complexity of link signs in graphs. Such inconsistent tasks between the pre-training and downstream phases may produce the sub-optimal performance of pre-trained GNNs, even “negative transfer”. To address these challenges, we propose S igned G raph P rompt T uning ( SGPT ), a novel transfer learning framework to adapt the pre-trained unsigned GNNs to signed graph tasks. This approach employs two unification templates (graph and task templates) and two prompts (feature and semantic prompts) to bridge the gaps between the pre-training and downstream phases. Specifically, to reduce the graph-level divergence, we propose a graph template to address the complex node relationships in signed graphs. Based on the balance theory, the graph template generates samples from the original signed graph: positive, negative, and topological, each capturing distinct relational semantics. These segregated samples are then fed into the pre-trained

Read the complete text