Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graph
Yuxiang Wang †, Xiao YanShiyu Jin † Quanqing Xu ‡ Chuang Hu † Yuanyuan Zhu † Bo Du † Jia Wu §, Jiawei Jiang · 2025
Read Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graph by Yuxiang Wang †, Xiao YanShiyu Jin † Quanqing Xu ‡ Chuang Hu † Yuanyuan Zhu † Bo Du † Jia Wu §, Jiawei Jiang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08168.txt.
Introduction
Yuxiang Wang and co-authors use text meaning to classify graph nodes when labeled cases are few or none. Their 2025 study works on graphs where each node carries a text description. Such tasks arise in academia and social networks, where labels are costly.
Past work augments the graph side, while text-side tricks were little explored. The new method adds two text signals for each node. One pulls in texts with like meaning as extra support. One builds an opposite-meaning text as a contrast case. On five data sets against thirteen rivals, the method wins throughout. Gains over the best rival often top five percent, the authors report.
Catalog record
| CREATOR | Yuxiang Wang †, Xiao YanShiyu Jin † Quanqing Xu ‡ Chuang Hu † Yuanyuan Zhu † Bo Du † Jia Wu §, Jiawei Jiang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08168
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graph Yuxiang Wang †, Xiao YanShiyu Jin † Quanqing Xu ‡ Chuang Hu † Yuanyuan Zhu † Bo Du † Jia Wu §, Jiawei Jiang Abstract Text-attributed graph (TAG) provides a text description for each graph node, and few- and zero-shot node classification on TAGs have many applications in fields such as academia and social networks. Existing work utilizes various graph-based augmentation techniques to train the node and text embeddings, while text-based augmentations are largely unexplored. In this paper, we propose T ext S emantics A ugmentation (TSA) to improve accuracy by introducing more text semantic supervision signals. Specifically, we design two augmentation techniques, i.e., positive semantics matching and negative semantics contrast , to provide more reference texts for each graph node or text description. Positive semantic matching retrieves texts with similar embeddings to match with a graph node. Negative semantic contrast adds a negative prompt to construct a text description with the opposite semantics, which is contrasted with the original node and text. We evaluate TSA on 5 datasets and compare with 13 state-of-the-art baselines. The results show that TSA consistently outperforms all baselines, and its accuracy improvements over the best-performing baseline are usually over 5%. 1 Introduction Text-attributed graph (TAG) Yan et al. (2023) is a prevalent type of graph-structured data, where each node is associated with a text description. For instance, in a citation network, the papers (i.e., nodes) are linked by the citation relations (i.e., edges), and the abstract of each paper serves as the text description. Few-shot and zero-shot node classification on TAGs (FZNC-TAG) predict the categories of the nodes using a few or even no labeled data since labeled data are expensive to obtain Liu et al. (2021) ; Liu et al. (2022) . The two tasks have many applications in areas such as recommender system Gao et al. (2022) , social network analysis Yu et al. (2020) , and anomaly detection Noble and Cook (2003) . Figure 1: The contrastive loss of G2P2 (a) and two semantic augmentation techniques proposed by TSA (b,c). The node-text pair of G2P2 is specified by data as each node has a text description, and TSA mines more semantic information for nodes and texts. Existing methods for FZNC-TAG typically follow a two-step process: first learn node and text embeddings on the TAGs and then use prompting to produce classification results Wen and Fang (2023) ; Huang et al. (2023) . They mainly differ in embedding learning and can be classified into three categories. ❶ Self-supervised graph learning methods employ graph augmentation techniques, such as node adding and feature masking, to generate more generalized node embeddings You et al. (2020) ; Deng and Hooi (2021) . They exploit the graph topology but ignores the information in the text semantics. ❷ The two-stage learning methods encode the text using large language models (LLMs) and add the text embedding as additional node features Tang et al. (2024) ; Chen et al. (2024) . Then, graph neural network (GNN) methods, e.g., TextGCN Yao et al. (2019) and GraphSAGE Hamilton et al. (2017) , are used to learn the node embeddings. The limitation of these approaches is that the LLMs are not updated during GNN training Yan et al. (2023) . ❸ The state-of-the-art end-to-end learning method, G2P2 Wen and Fang (2023) , jointly trains the GNN and language model via contrastive learning He et al. (2020) . As shown in Figure 1 (a), G2P2 contrasts each node-text pair to ensure that the GNN and language model are aligned in the embedding space. Existing work utilizes various graph-based augmentation to train the node and text embeddings, however, neglecting the semantics information in the texts. As shown in Figure 1 (a), G2P2 merely aligns the raw node-text pairs in the TAGs. Furthermore, we observe that the classification accuracy of G2P2 is low. For instance, on the Fitness dataset Yan et al. (2023) , G2P2 only achieves an accuracy of 68.24% and 45.99% for few- and zero-shot classification, respectively. The significantly lower accuracy in zero-shot classification also suggests that the problems are caused by the lack of text semantics and that G2P2’s sole reliance on the graph-based augmentation is inadequate for training high-quality models. Thus, we ask the following research question: How to exploit more text semantics to enhance few- and zero-shot classification on TAGs? To answer the question, we present TSA, where embedding learning can be improved by enforcing similarity relations among embeddings. Inspired by this idea, we design two augmentation techniques, i.e., positive semantics matching and negative semantics contrast, as shown in Figure 1 (b,c). These techniques create additional node-text pairs that have similar or dissimilar embeddings to facilitate model learning. Positive semantics matching. In Figure 1 (b), we provide multiple positive text embeddings for each node embedding. This is achieved by searching the texts that have similar embeddings to the text of the considered graph node. We also encourage the node embedding to be similar to those text embeddings. This augmentation provides more supervision to GNN training and enforces the prior that nodes may belong to the same categories if their texts have similar semantics. Negative semantics contrast. In Figure 1 (c), we pair each text with a semantically opposite negative text , which is constructed by adding a learnable negative prompt to the original text. We then encourage the original text embeddings and its corresponding node to be dissimilar to the negative text. This augmentation provides additional text semantics to make the classification robust. For instance, to classify a paper as being related to artificial intelligence, it should not only be similar to the description “a paper is published at IJCAI” but also be dissimilar to “a paper is published at The Lancet” . We conduct extensive experiments to evaluate TSA, using 5 datasets and comparing with 13 state-of-the-art baselines. The results show that TSA consistently achieves higher accuracy than all baselines for both few-shot and zero-shot setting. In particular, TSA improves the accuracy and F1 scores of few-shot classification by 4.6% and 6.9% on average, and zero-shot classification by 8.8% and 9.3%, respectively. In summary, we make the following contributions: • We observe that prior methods only contrast node-text pairs, thus failing to effectively capture text semantics. To solve this issue, we propose TSA to enhance the semantic understanding in both model pre-training and inference. • We incorporate two novel augmentation techniques into TSA: positive semantics matching and negative semantics contrast. These techniques create additional node-text pairs by mining more text semantics from diverse perspectives. • We conduct extensive experiments to evaluate TSA and compare with state-of-the-art baselines, demonstrating that TSA enjoys high model accuracy and training efficiency. 2 Preliminaries Text-attributed graph. We denote a text-attributed graph (TAG) as \mathbf{G}=(\mathcal{V},\mathcal{E},\mathbf{X}) , in which \mathcal{V} , \mathcal{E} , and \mathbf{X} are the node set, edge set, and text set, respectively. Take citation network as an example for TAG. Each node v_{i}\in\mathcal{V} is a paper, interconnected by the edges e\in\mathcal{E} that signify citation relations. Let \mathbf{x}_{i}\in\mathbf{X} denote the text (i.e., paper abstract) of the i -th node. Each node has a label to indicate the topic of the paper. Since