GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedback
Ruiyao Xu, Kaize Ding · 2026
Read GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedback by Ruiyao Xu, Kaize Ding (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-08553.txt.
Introduction
Xu and Ding tackle few shot learning on text attributed graphs, where labeled nodes are scarce and structure is complex. Their GNN-as-Judge framework first finds the unlabeled nodes most influenced by labeled ones, then uses agreement and disagreement between language models and graph networks to craft trustworthy pseudo labels. A weakly supervised fine tuning step distills those labels while damping noise, beating existing methods most clearly where labels are scarcest.
Catalog record
| CREATOR | Ruiyao Xu, Kaize Ding |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84766 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.08553
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedback Ruiyao Xu, Kaize Ding Abstract Large Language Models (LLMs) have shown strong performance on text-attributed graphs (TAGs) due to their superior semantic understanding ability on textual node features. However, their effectiveness as predictors in the low-resource setting, where labeled nodes are severely limited and scarce, remains constrained since fine-tuning LLMs usually requires sufficient labeled data, especially when the TAG shows complex structural patterns. In essence, this paper targets two key challenges: (i) the difficulty of generating and selecting reliable pseudo labels on TAGs for LLMs, and (ii) the need to mitigate potential label noise when fine-tuning LLMs with pseudo labels. To counter the challenges, we propose a new framework, GNN-as-Judge , which can unleash the power of LLMs for few-shot semi-supervised learning on TAGs by incorporating the structural inductive bias of Graph Neural Networks (GNNs). Specifically, GNN-as-Judge introduces a collaborative pseudo-labeling strategy that first identifies the most influenced unlabeled nodes from labeled nodes, then exploits both the agreement and disagreement patterns between LLMs and GNNs to generate reliable labels. Furthermore, we develop a weakly-supervised LLM fine-tuning algorithm that can distill the knowledge from informative pseudo labels while mitigating the potential label noise. Experiments on multiple TAG datasets demonstrate that GNN-as-Judge significantly outperforms existing methods, particularly in low-resource regimes where labeled data are scarce. 1 1 1 Code is available at https://github.com/rux001/GNN-as-Judge . 1 Introduction Text-Attributed Graphs (TAGs), where nodes correspond to text documents and edges represent their relationships, are pervasive across various applications such as citation networks, social media platforms, and e-commerce ecosystems ( 9 ; 73 ; 28 ; 71 ) . Unlike conventional attributed graphs, TAGs encode raw textual contents rather than numerical values, which requires more dedicated mechanisms to effectively capture semantic information while preserving structural relationships. Recent advances in Large Language Models (LLMs) have shown exceptional capabilities in text understanding ( 7 ; 19 ; 46 ) , driving growing interest in leveraging their exceptional text understanding capabilities to address TAG-related tasks ( 11 ; 61 ; 76 ; 12 ; 29 ; 64 ; 71 ) . Many previous studies investigate LLM-as-Predictors approaches, which employ LLMs as direct predictors by integrating graph context through graph encoders or carefully crafted prompts ( 61 ; 10 ; 11 ) . It is noteworthy that current research of LLMs-as-Predictors for node classification on TAGs, primarily focuses on the supervised setting where abundant labeled data is accessible. This is mainly because LLMs lack GNNs’ message passing mechanisms to leverage unlabeled nodes, and thus require sufficient supervision signals for effective fine-tuning ( 10 ; 61 ; 76 ) . However, real-world graphs are usually sparsely labeled, and thus directly applying existing methods to such few-shot semi-supervised setting ( 18 ; 62 ; 17 ; 77 ) will easily lead to overfitting and poor generalization ( 71 ; 61 ; 76 ) . Although one can leverage pseudo-labeling techniques to augment the limited labeled training data ( 36 ; 53 ; 51 ; 41 ; 60 ; 38 ) , previous studies have recognized that “easy” pseudo labels with high confidence provide limited learning signal, whereas “hard” samples are more informative but introduce greater label noise ( 6 ; 35 ; 47 ) . This challenge becomes even more pronounced when incorporating LLMs, leading to the following two unresolved challenges in few-shot semi-supervised node classification on TAGs: • ❶ How to go beyond the knowledge of LLMs to obtain reliable pseudo-labeled data? Since LLMs are inherently difficult to interpret complex graph structural patterns ( 30 ; 24 ) , solely relying on LLMs with no structural inductive bias to generate pseudo labels is less desirable. More critically, not all unlabeled nodes are equally valuable for pseudo-labeling, thus, selecting the most influential subset of unlabeled nodes is crucial for maximizing performance under computational constraints. Although text-based serialization methods try to encode structural context into the prompts, those methods could still struggle with self-generated pseudo labels due to the hallucination and self-bias of LLMs ( 11 ; 43 ; 41 ; 21 ) . Generating reliable pseudo-labeled examples that encode not only textual but also structural inductive biases is therefore crucial for enabling LLMs to transcend their inherent knowledge limitations on graphs. • ❷ How to extract the knowledge from pseudo-labeled data while mitigating the potential label noise during LLM fine-tuning? Despite pseudo-labeling showing its empirical effectiveness in many fields ( 36 ; 53 ) , the potential label noise has been a longstanding problem. Especially for those “hard” pseudo-labeled examples that are more valuable for training the model, they could also bring more label noise if the labels are incorrect. Simply performing LLM supervised fine-tuning with the noisy pseudo labels can lead to performance degradation ( 58 ; 33 ; 13 ) , which necessitates the need to develop a new learning algorithm that can effectively distill the knowledge and mitigate the label noise when fine-tuning with pseudo-labeled data. In this paper, we propose GNN-as-Judge , a novel framework that fine-tunes LLMs on sparsely labeled graphs using feedback from GNNs. Instead of mining the “easy” and “hard” pseudo-labeled nodes solely based on LLM itself like standard self-training approaches ( 44 ; 36 ; 47 ) , at its core, a GNN with structural inductive bias acts as a judge to provide additional guidance for LLM to generate reliable pseudo-labels. GNN-as-Judge strategically leverages both agreement and disagreement between GNN and LLM as signals to identify not only “easy”, but more remarkably, “hard” pseudo labels that LLMs are more likely to make mistakes. To further mitigate the potential label noise, especially in the harder examples, we develop a weakly-supervised LLM fine-tuning algorithm that jointly performs fine-tuning on the two selected pseudo-labeled node sets. Specifically, in addition to applying supervised LLM instruction tuning on the agreement (easy) node set, we propose to conduct preference tuning on the disagreement (hard) node set, which allows LLM to learn relative preferences between predictions from the two models. To summarize, our contributions are mainly three-folds: • We study the problem of LLMs-as-Predictors for graph few-shot semi-supervised learning, a fundamental but underexplored research problem, where the key challenges lie in selecting reliable pseudo labels and mitigating label noise during fine-tuning. • We propose GNN-as-Judge , a novel framework that positions GNNs as judges to select reliable pseudo labels for fine-tuning LLMs. GNN-as-Judge is also equipped with a new weakly-supervised fine-tuning algorithm that can further mitigate label noise during LLM fine-tuning. • We conduct comprehensive experiments on different TAG datasets with various scales. Results demonstrate that GNN-as-Judge significantly outperforms both traditional GNN-based approaches and other LLM-based baselines, especially in extreme low-resource scenarios. 2 Related Work LLMs for Graph Learning. Recent research has extensively explored applying (L)LMs to TAGs, yielding significant advancements in feature encoding, node classification, and link prediction ( 11 ; 61 ; 76 ; 29 ; 64 ; 71 ) . Early work primarily employed small-scale pretrained language models such as BERT ( 16 ) or RoBERTa ( 42 ) as text feature encoders to extract enhanced representations for graph learning ( 74 ; 39 ; 69 ; 79 ) . With the advent of powerful large