StrucSum: Graph-Structured Reasoning for Long Document Extractive Summarization with LLMs

Haohan Yuan, Sukhwa Hong, Haopeng Zhang · 2025

Read StrucSum: Graph-Structured Reasoning for Long Document Extractive Summarization with LLMs by Haohan Yuan, Sukhwa Hong, Haopeng Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22950.txt.

Introduction

Long documents defeat summarizers that read flatly: key sentences hide among neighbors, and importance is a matter of position as much as wording. Haohan Yuan, Sukhwa Hong, and Haopeng Zhang answer with structure in a 2025 paper. Their framework, StrucSum, builds a sentence-level graph over the document and feeds that structure into the prompts of a large language model.

Three strategies inject the graph signals: neighbor-aware prompting for local context, centrality-aware prompting for importance, and centrality-guided masking to trim inputs. On arXiv, PubMed, and Multi-News test sets the method beat plain prompting on quality and factual consistency. One caution emerged from the ablations: stacking all strategies did not clearly add gains. The authors release their code and frame graph-aware prompting as an open direction.

Catalog record

CREATORHaohan Yuan, Sukhwa Hong, Haopeng Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-22950

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

StrucSum: Graph-Structured Reasoning for Long Document Extractive Summarization with LLMs Haohan Yuan, Sukhwa Hong, Haopeng Zhang Abstract Large language models (LLMs) have shown strong performance in zero-shot summarization, but often struggle to model document structure and identify salient information in long texts. In this work, we introduce StrucSum, a training-free prompting framework that enhances LLM reasoning through sentence-level graph structures. StrucSum injects structural signals into prompts via three targeted strategies: Neighbor-Aware Prompting (NAP) for local context, Centrality-Aware Prompting (CAP) for importance estimation, and Centrality-Guided Masking (CGM) for efficient input reduction. Experiments on ArXiv, PubMed, and Multi-News demonstrate that StrucSum consistently improves both summary quality and factual consistency over unsupervised baselines and vanilla prompting. Notably, on ArXiv, it boosts FactCC and SummaC by 19.2 and 9.7 points, indicating stronger alignment between summaries and source content. These findings suggest that structure-aware prompting is a simple yet effective approach for zero-shot extractive summarization with LLMs, without any training or task-specific tuning. Our source code is released at https://github.com/HaohanYuan01/StrucSum . 1 Introduction Extractive summarization aims to identify the most salient sentences from a document to produce concise and factually reliable summaries. This approach is especially valuable in high-stakes domains such as scientific publishing, biomedical research, legal analysis, and policy documentation, where accuracy and traceability are critical Zhang et al. 2023a ; Ruan et al. 2022 . As long-form documents grow in length and complexity, effective summarization requires not only identifying salient content at the sentence level but also reasoning over discourse structure and capturing long-range dependencies Zhong and Litman 2025 ; Koh et al. 2022 , which remain challenging for many existing methods. Figure 1: Comparing StrucSum’s graph-structured LLM prompting with standard flat-input prompting for long-document extractive summarization. To capture document-level structure, prior work has represented documents as sentence-level graphs based on semantic, positional, or rhetorical relationships Mihalcea and Tarau 2004 ; Jin et al. 2020 ; Wang et al. 2020 . These graph-based approaches offer interpretable mechanisms for sentence selection via centrality and coverage Erkan and Radev 2004 ; Cui et al. 2020 . However, most existing approaches require supervised training with transformer encoders Li et al. 2020 ; Zhang et al. 2022 ; Zhao et al. 2024 , which limits their scalability and adaptability across domains. In the era of advanced large language models (LLMs), systems such as GPT-4 and LLaMA3 have shown strong zero-shot performance on general summarization tasks Yuan and Zhang 2024 . Their ability to generalize across domains without task-specific training makes them appealing alternatives to fully supervised models Kojima et al. 2023 . However, LLMs are trained on unstructured text and often struggle to recognize salient content or maintain factual consistency in long or multi-section documents due to their lack of structural awareness Zhang et al. 2024a . Although prompting strategies like in-context learning and chain-of-thought have enhanced LLM reasoning in other domains Wei et al. 2023 , prompting for structural reasoning in long-document extractive summarization remains unexplored. To address this challenge, we propose StrucSum, a graph-structured prompting framework that enhances LLMs for long-document extractive summarization in a zero-shot setting. StrucSum builds a Text-Attributed Graph (TAG) for each document, where sentences are nodes connected by semantic or positional relationships. Instead of tuning model parameters, we inject structural signals into LLM inputs through three complementary strategies: (1) Neighbor-Aware Prompting (NAP) enriches each sentence with its immediate neighbors to support local reasoning; (2) Centrality-Aware Prompting (CAP) appends sentence importance scores derived from graph centrality to guide global selection; and (3) Centrality-Guided Masking (CGM) filters out low-salience content to reduce input length. These strategies jointly expose different aspects of document structure, driving LLMs to generate more faithful and context-aware summaries without additional training. We focus on extractive summarization as it provides a clearer testbed for structural reasoning, where identifying salient sentences depends more explicitly on discourse-level relationships Zhang et al. 2023a . While our method is designed for the extractive setting, it is model-agnostic and can be extended to abstractive summarization, which we leave for future work. We evaluate StrucSum on two long-document summarization benchmarks (ArXiv and PubMed) and one multi-document benchmark (Multi-News). Experimental results show that our framework consistently outperforms both unsupervised methods and LLMs with standard prompting across ROUGE, BERTScore, and factual consistency metrics. Notably, on the ArXiv dataset, our NAP strategy improves FactCC scores by 19.2 points and SummaC scores by 9.7 points, showing significant gains in factual reliability. We summarize our main contributions as follows: • We propose StrucSum , a structure-aware prompting framework for extractive summarization that operates in a fully zero-shot, training-free setting with LLMs. • We introduce three complementary prompting strategies: NAP, CAP, and CGM, that inject local and global structural signals into LLM inputs to support more accurate and faithful sentence selection. • We conduct extensive experiments across three long-form summarization benchmarks. Results show that StrucSum consistently improves both summary quality and factual consistency across multiple metrics. Our findings suggest the importance of incorporating structural signals to enhance LLM reasoning in zero-shot extractive summarization. 2 Related Work 2.1 Graph-based Summarization Figure 2: Overview of the StrucSum framework: A document is encoded into a Text-Attributed Graph, from which structural signals are extracted via three prompting strategies to guide LLMs in zero-shot extractive summarization. Graph-based models have been widely adopted in extractive summarization for their ability to capture cross-sentence relationships within a document. Their sparse structure brings scalability and flexibility, making them particularly suitable for long-form inputs. Early approaches constructed sentence-level graphs using lexical or semantic similarity, enabling algorithms such as TextRank to identify salient content Mihalcea and Tarau 2004 . Building on this foundation, later work has incorporated Graph Neural Networks (GNNs) to better model both local and global dependencies Chhabra et al. 2024 . MuchSUM, for example, introduces a multi-channel graph convolutional network that encodes semantic, centrality, and positional features through a bipartite word-sentence graph, outperforming BERT-based baselines Mao et al. 2022 . Heterogeneous GNNs have further been explored to jointly model word- and sentence-level interactions Wang et al. 2020 , while citation graphs have been used to capture cross-document relatedness in scientific summarization Chen et al. 2022 . 2.2 Summarization with LLMs Large Language Models (LLMs) have significantly advanced the field of text summarization, demonstrating strong performance in both zero-shot and few-shot settings. Models such as ChatGPT, and LLaMA2 have achieved state-of-the-art results on various benchmarks, including CNN/DailyMail, XSum, and Multi-News Zhang et al. 2024a . These models excel in generating coherent and contextually relevant summaries without the need for extensive task-specific fine-tuning. However, challenges remain. LLMs often struggle with factual consistency, especially in abstractive summarization tasks, leading to potential hallucinations or misrepresentations of source content Pu et al. 2023 . Additionally, their performance can vary across domains, with specialized areas such as medical or legal texts posing unique difficulties Yuan and Zhang 2024 . In response, evaluation methods leveraging LLMs themselves Song et al. 2024 , such as G-Eval, have been proposed to provide more nuanced assessments of summary quality Liu et al. 2023 . Although recent work has begun exploring extractive summarization with LLMs in a training-free setting Zhang et al. 2023a , research in this area remains limited, particularly for long-form documents. Our work fills this gap by introducing

Read the complete text