Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications DOI: XXXXXXX.XXXXXXX Journal: POMACS Volume: 37 4 111 8 CCS: Computing methodologies Natural language processing; Knowledge representation and reasoning CCS: Mathematics of computing Graph algorith

Guangxin Su, Hanchen Wang, Jianwei Wang, Wenjie Zhang, Ying Zhang, Jian Pei · 2025

Read Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications DOI: XXXXXXX.XXXXXXX Journal: POMACS Volume: 37 4 111 8 CCS: Computing methodologies Natural language processing; Knowledge representation and reasoning CCS: Mathematics of computing Graph algorith by Guangxin Su, Hanchen Wang, Jianwei Wang, Wenjie Zhang, Ying Zhang, Jian Pei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21131.txt.

Introduction

Guangxin Su leads five co-authors on a survey of how large language models meet text attributed graphs, networks whose nodes carry rich text. Language models reason well over words but hide their structure, while text attributed graphs expose relations yet lack semantic depth.

Integration frameworks are reviewed systematically from an orchestration perspective, the first survey of its kind on this pairing. Combined systems strengthen graph representation learning and give language models more structured, interpretable multi hop reasoning. Applications span recommendation, knowledge work, and science, wherever relations and text arrive together.

Catalog record

CREATORGuangxin Su, Hanchen Wang, Jianwei Wang, Wenjie Zhang, Ying Zhang, Jian Pei
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE122 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21131

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications DOI: XXXXXXX.XXXXXXX Journal: POMACS Volume: 37 4 111 8 CCS: Computing methodologies Natural language processing; Knowledge representation and reasoning CCS: Mathematics of computing Graph algorithms CCS: General and reference Surveys and overviews Guangxin Su, Hanchen Wang, Jianwei Wang, Wenjie Zhang, Ying Zhang, Jian Pei Abstract . Large Language Models (LLMs) have achieved remarkable success in natural language processing through strong semantic understanding and generation. However, their black-box nature limits structured and multi-hop reasoning. In contrast, Text-Attributed Graphs (TAGs) provide explicit relational structures enriched with textual context, yet often lack semantic depth. Recent research shows that combining LLMs and TAGs yields complementary benefits: enhancing TAG representation learning and improving the reasoning and interpretability of LLMs. This survey provides the first systematic review of LLM–TAG integration from an orchestration perspective. We introduce a novel taxonomy covering two fundamental directions: LLM for TAG, where LLMs enrich graph-based tasks, and TAG for LLM, where structured graphs improve LLM reasoning. We categorize orchestration strategies into sequential, parallel, and multi-module frameworks, and discuss advances in TAG-specific pretraining, prompting, and parameter-efficient fine-tuning. Beyond methodology, we summarize empirical insights, curate available datasets, and highlight diverse applications across recommendation systems, biomedical analysis, and knowledge-intensive question answering. Finally, we outline open challenges and promising research directions, aiming to guide future work at the intersection of language and graph learning. 1. INTRODUCTION Figure 1. Synergistic improvement with the orchestration of techniques for large language models and text-attributed graphs. The rise of Transformer-based architectures ( Vaswani 2017 ) has revolutionized natural language processing (NLP). Exemplifying large language models (LLMs), the GPT series ( Radford 2018 ; Radford et al. 2019 ; Brown 2020 ; Achiam et al. 2023 ) , built on the Transformer decoder architecture and pre-trained on extensive corpora, has demonstrated extraordinary capabilities, positioning LLMs as foundational steps toward realizing Artificial General Intelligence (AGI). In contrast, more compact language models (LMs 1 1 1 The comparison between LMs and LLMs highlights a trade-off between capability and flexibility. LLMs, encompassing LMs with their larger parameter sizes, offer greater capabilities and extensive knowledge but often sacrifice fine-tuning efficiency. In contrast, the more compact LMs excel in adaptability for specific tasks ( He et al. 2023 ; Chen et al. 2024b ) . ) such as the BERT series ( Devlin 2018 ; He et al. 2020 ; Liu 2019 ; Sanh 2019 ) , leverage the Transformer encoder to excel in tasks demanding fine-grained contextual understanding and precise semantic representation, making them essential for domain-specific research and applications. In the real world, a vast amount of critical information, ranging from scientific publications and social media posts to biological records, is stored in textual form ( Stelzer et al. 2016 ; Hu et al. 2020 ; Li et al. 2024b ) . These textual data often exhibit rich interrelationships, which can be naturally modeled as text-attributed graphs (TAGs) ( Yang et al. 2021 ) , where nodes, edges, or the entire graph are enriched with textual attributes. Therefore, there are two data types within TAGs, textual attributes and graph structure . For instance, in citation networks, nodes represent research papers annotated with abstracts or full texts, while edges denote citation links that reflect semantic relevance. In recommendation systems, TAGs can represent users and items, where nodes are enriched with user profiles or product descriptions, and edges capture user-item interactions, co-purchase behaviors, or review-based sentiment information. In chemistry studies, chemical compounds can be modeled as TAGs, where atoms or entire molecules are annotated with textual property descriptions or literature-derived knowledge. Accurately modeling and analyzing these structured yet text-rich relationships is crucial for a variety of downstream tasks, including text classification ( Zhao et al. 2022 ) , personalized recommendation ( Zhu et al. 2021 ) , and molecular discovery ( Jablonka et al. 2023 ) . Recently, growing attention has been directed towards orchestrating LLM and TAG techniques, driven by their combined ability to handle textual and graph-structured data. LLMs excel in parsing and generating text, while TAGs capture complex relational structures inherent in data. Figure 1 demonstrates diverse orchestration strategies for LLM and TAG techniques in handling textual and graph-structural data. These integrated approaches not only improve performance on TAG-oriented tasks ( LLM for TAGs ) but also enhance the reasoning capabilities of LLMs ( TAG for LLMs ), which have yielded promising advances across a range of research domains, including graph retrieval-augmented generation ( Peng et al. 2024 ) , knowledge-intensive question answering ( Pan et al. 2024 ) , the mitigation of hallucinations in LLMs ( Guan et al. 2024a ) , and scientific discovery ( Shao et al. [n. d.] ; Gu et al. 2024 ) , among others. For example, ( Yue et al. 2020 ) constructs a TAG from electronic medical records (EMRs). Medical terms are represented as nodes, and edges are formed based on co-occurrence within context windows ( Finlayson et al. 2014 ) . The integration of LLM and TAG techniques enhances both semantic and structural representations, enabling more accurate and interpretable classification of medical terms. Motivated by increasing research interest in the integration of LLM and TAG techniques, this paper presents a comprehensive survey from a novel perspective: how these techniques are orchestrated to provide synergistic improvements for both the reasoning ability of LLMs and representation learning of TAGs . We systematically organize existing works through two fundamental primitives, LLM4TAG and TAG4LLM, introducing a novel taxonomy of orchestration frameworks. \star LLM for TAG. Applying LLMs to textual attributes in TAGs produces context-rich embeddings that, when combined with topology-aware graph learning mechanisms, yield superior representation learning of TAGs. In a sequential orchestration ( He et al. 2023 ) scheme, the LLM first encodes textual features, and its output is then passed to a graph learning model that explicitly models connectivity. In a parallel orchestration ( Feng et al. 2021 ) scheme, the LLM and graph learning model run in tandem, each processing text and topology respectively, before aligning their embeddings for downstream tasks using techniques such as contrastive learning. Moreover, several works ( Bielak et al. 2022 ; Wu et al. 2021 ) repurposed successful LLM training paradigms, including self-supervised pretraining and fine tuning, to strengthen TAG learning. In particular, pre-trained models for TAGs ( Xie et al. 2023 ) adapt self-supervised objectives, fine-tuning routines, and prompt design strategies from LLMs into the graph domain, creating more expressive and generalizable TAG frameworks. \star TAG for LLM. TAGs, with their structured and explicit integration of textual attributes, provide a clear substrate for symbolic reasoning that helps LLMs address transparency and decisiveness challenges. This issue arises because LLMs embed vast knowledge implicitly within their parameters ( Zhang et al. 2023a ) , leading to opacity that hinders interpretability and factual precision. Techniques like chain-of-thought prompting ( Wei et al. 2022 ) attempt to generate explanations but still suffer from hallucinations ( Ji et al. 2023 ) and inaccuracies ( Wang et al. 2023b ) , especially in multi-hop reasoning. By grounding LLMs in TAGs, it becomes possible to produce reliable, interpretable outputs and mitigate these limitations ( Pan et al. 2024 ) . We systematically

Read the complete text