Few-Shot Graph Out-of-Distribution Detection with LLMs

Haoyan Xu *, Zhengtao Yao *, Yushun Dong, Ziyi Wang, Ryan A. Rossi, Mengyuan Li, Yue Zhao · 2025

Read Few-Shot Graph Out-of-Distribution Detection with LLMs by Haoyan Xu *, Zhengtao Yao *, Yushun Dong, Ziyi Wang, Ryan A. Rossi, Mengyuan Li, Yue Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22097.txt.

Introduction

Haoyan Xu and co-authors spot odd nodes in text-rich graphs with very few human labels, by pairing a language model with a small graph network. Their 2025 framework answers a cost problem: labeling nodes in such graphs is slow and dear. Language models read text well but miss graph shape.

The language model first screens a small slice of nodes, flagging likely outsiders. A light graph network trains on those noisy flags and scores the rest, blending text and structure. An informativeness rule then picks the most valuable nodes for exact human labels. The final classifier trains on that clean core. Across four real datasets, the scheme cut labeling cost and beat prior tools on both sorting and oddity spotting.

Catalog record

CREATORHaoyan Xu *, Zhengtao Yao *, Yushun Dong, Ziyi Wang, Ryan A. Rossi, Mengyuan Li, Yue Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-22097

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Few-Shot Graph Out-of-Distribution Detection with LLMs Haoyan Xu *, Zhengtao Yao *, Yushun Dong, Ziyi Wang, Ryan A. Rossi, Mengyuan Li, Yue Zhao Abstract Existing methods for graph out-of-distribution (OOD) detection typically depend on training graph neural network (GNN) classifiers using a substantial amount of labeled in-distribution (ID) data. However, acquiring high-quality labeled nodes in text-attributed graphs (TAGs) is challenging and costly due to their complex textual and structural characteristics. Large language models (LLMs), known for their powerful zero-shot capabilities in textual tasks, show promise but struggle to naturally capture the critical structural information inherent to TAGs, limiting their direct effectiveness. To address these challenges, we propose LLM-GOOD, a general framework that effectively combines the strengths of LLMs and GNNs to enhance data efficiency in graph OOD detection. Specifically, we first leverage LLMs’ strong zero-shot capabilities to filter out likely OOD nodes, significantly reducing the human annotation burden. To minimize the usage and cost of the LLM, we employ it only to annotate a small subset of unlabeled nodes. We then train a lightweight GNN filter using these noisy labels, enabling efficient predictions of ID status for all other unlabeled nodes by leveraging both textual and structural information. After obtaining node embeddings from the GNN filter, we can apply informativeness-based methods to select the most valuable nodes for precise human annotation. Finally, we train the target ID classifier using these accurately annotated ID nodes. Extensive experiments on four real-world TAG datasets demonstrate that LLM-GOOD significantly reduces human annotation costs and outperforms state-of-the-art baselines in terms of both ID classification accuracy and OOD detection performance. 1 1 footnotetext: Equal contribution. 1 Introduction Out-of-distribution (OOD) detection [ 14 , 9 ] has emerged as a critical task in machine learning, particularly for safety-critical applications where models must reliably identify inputs that differ significantly from the training data. Recently, several OOD detection methods [ 23 , 27 ] and open-set learning approaches [ 26 , 29 ] have been proposed and applied to graph-structured data. Existing graph OOD detection methods typically operate within a semi-supervised, transductive framework, where the entire set of nodes is accessible during training, but only a portion of the class labels (in-distribution (ID) classes) are provided [ 23 ] . These methods generally rely on a sufficient number of labeled ID nodes to train a GNN-based ID classifier, from which they derive the ID classification logits for all nodes. Post-hoc OOD detectors [ 27 , 9 , 16 ] are then applied to these logits for OOD detection. In particular, nodes with higher energy scores [ 27 ] or higher entropy scores are identified as OOD nodes. While these graph OOD detection methods are effective, they invariably rely on the assumption that ground truth ID labels are readily available. This assumption often overlooks a critical challenge: obtaining sufficient high-quality labels for graph-structured data. Specifically, (1) the diverse and complex nature of graph-structured data makes human labeling inherently difficult, and (2) the large scale of real-world graphs renders annotating a significant portion of nodes both time-consuming and resource-intensive [ 4 ] . Our Observations and Motivation. In this paper, we aim to address the challenge of few-shot OOD detection and ID classification on text-attributed graphs (TAGs) within the commonly used semi-supervised transductive setting, as described above. Consider a text-attributed social network where nodes represent individuals, node attributes correspond to their textual descriptions, and edges denote interactions or connections between them. Initially, the entire network is unlabeled, and the goal is to classify individuals into specific interest groups, such as technology enthusiasts, sports fans, or musicians, while operating within a limited human annotation budget. However, the network also contains individuals whose interests fall outside these predefined categories, such as those primarily engaged in political discussions or travel blogging. Identifying and labeling these OOD nodes would be inefficient, as they do not contribute to training an effective classifier for the targeted interest groups. Instead, the focus is on accurately classifying only the ID nodes while detecting and filtering out OOD nodes that do not belong to the intended classification space. Furthermore, zero-shot [ 25 , 5 ] and few-shot [ 19 , 1 ] OOD detection for images has been extensively studied using multi-modal foundation models. However, to date, no similarly powerful graph foundation model exists to support zero-shot or few-shot graph OOD detection. As a result, we turn to LLMs to tackle the data-efficiency challenge of OOD detection on TAGs. In summary, current graph OOD detection methods typically rely heavily on large amounts of labeled ID nodes to perform well. Conversely, while LLMs demonstrate remarkable zero-shot capabilities on text-attributed graphs (TAGs), they inherently lack the ability to interpret and leverage the structural information essential to TAGs. In this study, we take the first step toward integrating the strengths of both GNNs and LLMs to tackle the data-efficiency challenges in graph OOD detection. Figure 1: An illustration of our method. To reduce annotation costs, we use an LLM to filter out OOD nodes before selecting nodes for human annotation. The annotated ID nodes are then used to train the target ID classifier. Present work. As shown in Fig. 1 , to address these challenges, we propose to leverage LLMs to filter out OOD nodes before human annotation, thereby reducing human costs . Specifically, we provide the LLM with ID knowledge (i.e., the names of ID classes) and prompt it to determine whether an unlabeled query node belongs to one of the ID classes, using the text information associated with the query node. Note that while we enable LLMs to directly perform zero-shot OOD detection and ID classification, using LLMs for zero-shot annotation is very slow during inference. Therefore, we aim to leverage LLMs to reduce human annotation costs during training and rely solely on well-trained GNNs for faster inference during testing. However, prompting the LLM to annotate all unlabeled nodes in the training set is costly for large graphs, although the cost of using an LLM is significantly lower than that of human annotation. To further reduce the LLM’s cost , we propose prompting the LLM to annotate only a small subset of nodes and then using these pseudo-labels to train a lightweight GNN filter. With this GNN filter, we can predict whether every unlabeled node in the training set belongs to one of the ID classes. If not, it’s very likely that this node is an OOD node, and we then filter it out before human annotation. In addition, we can obtain the embeddings of all unlabeled nodes in the graph after training the GNN filter with pseudo-labels. Based on these embeddings, the most informative nodes can be selected using existing informativeness-aware node selection methods. These selected nodes are then annotated by a human annotator, and the final annotated ID nodes are used to train the target ID classifier. Optionally, we can combine the accurate labels from human annotation with the noisy labels from the LLM to train a robust ID classifier under severe data scarcity scenarios . Compared to other active learning methods that require multiple rounds of selection [ 28 , 3 ] , our approach requires only a single round of annotation. Moreover, relying solely on noisy labels

Read the complete text