Leveraging Large Language Models for Effective Label-free Node Classification in Text-Attributed Graphs

Taiyan Zhang, Renchi Yang, Yurui Lai, Mingyu Yan, Xiaochun Ye, Dongrui Fan · 2024

Read Leveraging Large Language Models for Effective Label-free Node Classification in Text-Attributed Graphs by Taiyan Zhang, Renchi Yang, Yurui Lai, Mingyu Yan, Xiaochun Ye, Dongrui Fan (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-11983.txt.

Introduction

Taiyan Zhang and co-authors classify nodes in text-rich networks without hand labels in a 2024 paper. Networks such as citation graphs carry text on each node, and sorting nodes into classes usually needs many labeled examples. Large language models can label nodes from their text, but querying them for every node is costly and their labels can be noisy.

Their framework, called Locle, loops between a graph model and the language model. The graph model picks a small set of critical nodes, chosen where labels clash or doubt is high. Both models label those nodes, and a refinement step cleans the result using a rewired graph. On five data sets, the method beat rivals under the same query budget, with one large gain at a cost under one cent.

Catalog record

CREATORTaiyan Zhang, Renchi Yang, Yurui Lai, Mingyu Yan, Xiaochun Ye, Dongrui Fan
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-11983

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Leveraging Large Language Models for Effective Label-free Node Classification in Text-Attributed Graphs Taiyan Zhang, Renchi Yang, Yurui Lai, Mingyu Yan, Xiaochun Ye, Dongrui Fan Abstract . Graph neural networks (GNNs) have become the preferred models for node classification in graph data due to their robust capabilities in integrating graph structures and attributes. However, these models heavily depend on a substantial amount of high-quality labeled data for training, which is often costly to obtain. With the rise of large language models (LLMs), a promising approach is to utilize their exceptional zero-shot capabilities and extensive knowledge for node labeling. Despite encouraging results, this approach either requires numerous queries to LLMs or suffers from reduced performance due to noisy labels generated by LLMs. To address these challenges, we introduce Locle , an active self-training framework that does L abel-free n O de C lassification with L LMs cost- E ffectively. Locle iteratively identifies small sets of "critical" samples using GNNs and extracts informative pseudo-labels for them with both LLMs and GNNs, serving as additional supervision signals to enhance model training. Specifically, Locle comprises three key components: (i) an effective active node selection strategy for initial annotations; (ii) a careful sample selection scheme to identify "critical" nodes based on label disharmonicity and entropy; and (iii) a label refinement module that combines LLMs and GNNs with a rewired topology. Extensive experiments on five benchmark text-attributed graph datasets demonstrate that Locle significantly outperforms state-of-the-art methods under the same query budget to LLMs in terms of label-free node classification. Notably, on the DBLP dataset with 14.3k nodes, Locle achieves an 8.08% improvement in accuracy over the state-of-the-art at a cost of less than one cent. Our code is available at https://github.com/HKBU-LAGAS/Locle . Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3730021 ISBN: 979-8-4007-1592-1/2025/07 1035 CCS: Computing methodologies Neural networks CCS: Information systems Graph-based data models Technical Report † † cc-license: by 1. Introduction Text-attributed graphs (TAGs) ( Chang and Blei, 2009 ) are an expressive data model used to represent textual entities and their complex interconnections. Such data structures are prevalent in real-world scenarios, including social networks, hyperlink graphs of web pages, transaction networks, etc., wherein nodes are endowed with user profiles, web page contents, or product descriptions. Node classification is a fundamental task over TAGs, which aims to classify the nodes in the graph into a number of predefined categories based on the graph structures and textual contents. This task has emerged as a critical area of research in Information Retrieval (IR) ( Fu and He, 2021 ) . For instance, in content-sharing social networks like Flickr, content classification enables topical filtering and tag-based retrieval of multimedia items ( Seah et al., 2018 ) . In product search contexts, the retrieval of advertisement item tags can be framed as a conventional classification task ( Mao et al., 2020 ) . Similarly, in e-commerce query graphs, classifying query intent enhances result ranking by aligning it with the intended product category ( Fang et al., 2012 ) . In the past decade, graph neural networks (GNNs) ( Kipf and Welling, 2016 ; Gasteiger et al., 2018 ; Xu et al., 2018 ; Chen et al., 2020 ; Wu et al., 2019a ; Huang et al., 2023 ) have become the dominant models for node classification, by virtue of their capabilities to capture complex dependencies and patterns in the graph and the interplay between connectivity and attributes. However, the efficacy of such models largely hinges on the availability of adequate node labels for training. In real life, high-quality labels are hard to acquire due to the need for expert knowledge, significant human efforts, and potential biases in annotation, particularly for large-scale graphs comprising millions of nodes and a sheer volume of textual data ( Dai et al., 2021 ; Sheng et al., 2008 ) . In light of the superb comprehension and reasoning abilities of large language models (LLMs) in dealing with textual data, LLMs have been employed as a powerful tool for analyzing TAGs. As manifested in ( Chen et al., 2024 ; He et al., 2024 ) , LLMs have exhibited impressive node classification performance in TAGs under zero-shot or few-shot settings. However, compared to GNNs, this methodology falls short of exploiting the graph structures and requires substantial queries/calls to LLMs, and hence is not suitable for classifying nodes in the entire graph. Instead, a recent study ( Chen et al., 2023 ) has made an attempt towards utilizing LLMs as annotators for node labeling to facilitate the zero-shot node classification over TAGs. In this pipeline, i.e., LLM-GNN , LLMs work as a front-mounted step to create labels for a small number of selected active nodes as training data, and subsequently, a GNN model is trained for classification with the labeled data. Despite its empirical effectiveness as reported in ( Chen et al., 2023 ) , LLM-GNN is inherently defective due to its cascaded workflow that strongly relies on the active node selection and output quality of LLMs, which could be inaccurate/noisy, and in turn, leads to sub-optimal node classification performance. To illustrate, Figure 1 depicts the empirical performance attained by LLM-GNN and its variant whose active nodes are annotated with the ground-truth labels, dubbed as LLM-GNN (ground-truth), on two real TAGs (see Section 5 ), when increasing the labeled sample size. The first observation we can make from Figure 1 is that there is a notable performance gap (around 10\% ) between LLM-GNN and LLM-GNN (ground-truth). Second, as the labeled sample size, i.e., query budget for LLMs, is increased, LLM-GNN undergoes performance stagnation or even degradation on both datasets. These observations indicate that the label noise introduced by LLMs severely undermines the performance of GNNs. Moreover, even for LLM-GNN (ground-truth), we can also observe a drop of performance on Cora when training with more labeled data, which reveals the limitations of LLM-GNN in selecting representative active samples for labeling as well as vanilla GNN models. LLM-GNN (ground-truth) LLM-GNN 70 105 140 175 210 245 280 0.6 0.7 0.8 0.9 Accuracy ((a)) Cora 90 120 150 180 210 240 0.5 0.6 0.7 0.8 Accuracy ((b)) CiteSeer Figure 1. Varying #labeled nodes. To address the above-said problems, we present Locle ( L abel-free n O de C lassification with L LMs cost- E ffectively. ), a new framework that integrates LLMs into GNNs for cost-effective zero-shot node classification. To achieve this goal, the basic idea of Locle is to actively capitalize on the rich structural and attribute semantics underlying the TAGs with GNNs and LLMs for active node selection, node annotation, and label correction, rather than solely based on LLMs. More concretely, Locle proceeds in two stages that both combine LLMs and GNNs. Given a total query budget B to access LLMs, the first stage in Locle pinpoints \varepsilon\cdot B ( 0<\varepsilon<1 ) active nodes for annotation by LLMs through the subspace clustering ( Vidal, 2011 ) based on GNN-based node representations, which enables the accurate selection of representative node samples with

Read the complete text