Disco-RAG: Discourse-Aware Retrieval-Augmented Generation
Dongqi Liu, Hang Ding, Qiming Feng, Jian Li, Xurong Xie, Zhucun Xue, Chengjie Wang, Jiangning Zhang, Yabiao Wang · 2026
Read Disco-RAG: Discourse-Aware Retrieval-Augmented Generation by Dongqi Liu, Hang Ding, Qiming Feng, Jian Li, Xurong Xie, Zhucun Xue, Chengjie Wang, Jiangning Zhang, Yabiao Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-04377.txt.
Introduction
Dongqi Liu and eight co-authors argue that retrieval systems hand language models a pile when they should hand them a map, in a 2026 arXiv paper. Retrieval-augmented generation fetches passages to ground a model's answers, but standard pipelines present them flat and unordered, so the model misses how evidence connects across documents and struggles to synthesize it.
Their framework, Disco-RAG, builds discourse structure at two levels: trees inside each passage that capture its local organization. a rhetorical graph between passages that tracks how they continue, contrast, or support each other. Both feed a planning blueprint that conditions the final generation. On question answering and long-document summarization benchmarks, they report state-of-the-art results with no finetuning at all, which they take as evidence that structure, not just content, is what retrieved text was missing.
Catalog record
| CREATOR | Dongqi Liu, Hang Ding, Qiming Feng, Jian Li, Xurong Xie, Zhucun Xue, Chengjie Wang, Jiangning Zhang, Yabiao Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-04377
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Disco-RAG: Discourse-Aware Retrieval-Augmented Generation Dongqi Liu, Hang Ding, Qiming Feng, Jian Li, Xurong Xie, Zhucun Xue, Chengjie Wang, Jiangning Zhang, Yabiao Wang Abstract Retrieval-Augmented Generation (RAG) has emerged as an important means of enhancing the performance of large language models (LLMs) in knowledge-intensive tasks. However, most existing RAG strategies treat retrieved passages in a flat and unstructured way, which prevents the model from capturing structural cues and constrains its ability to synthesize knowledge from dispersed evidence across documents. To overcome these limitations, we propose Disco-RAG , a discourse-aware framework that explicitly injects discourse signals into the generation process. Our method constructs intra-chunk discourse trees to capture local hierarchies and builds inter-chunk rhetorical graphs to model cross-passage coherence. These structures are jointly integrated into a planning blueprint that conditions the generation. Experiments on question answering and long-document summarization benchmarks show the efficacy of our approach. Disco-RAG achieves state-of-the-art results on the benchmarks without fine-tuning. These findings underscore the important role of discourse structure in advancing RAG systems. 1 Introduction The advent of large language models (LLMs; 59 ; 69 ; 1 ) has advanced research progress in natural language processing (NLP), achieving competitive performance across a wide range of tasks, including question answering ( 65 ; 27 ; 73 ) , document summarization ( 50 ; 39 ; 60 ; 41 ) , and text generation ( 12 ; 5 ; 53 ; 72 ) . However, due to the reliance on static training corpora, LLMs can be inadequate for knowledge-intensive scenarios, such as handling domain-specific knowledge, proprietary data, or information requiring real-time updates ( 8 ; 28 ; 70 ; 63 ; 67 ) . Retrieval-Augmented Generation (RAG) has been proposed as a suitable strategy by integrating an external knowledge component through retrieval-based mechanisms ( 29 ; 3 ; 7 ) . Figure 1: Comparison between standard RAG and Disco-RAG . While standard RAG retrieves isolated chunks without structural links, Disco-RAG organizes evidence into discourse structures (trees & graphs). Here, S denotes Satellite (the supplementary part), and N denotes Nucleus (the core part). In standard RAG pipelines, external documents are segmented into chunks, which are then encoded into vectors and stored in a database. At query time, relevant chunks are retrieved to provide contextual grounding for the LLM ( 29 ) . One important but insufficiently addressed limitation of existing RAG systems concerns the mismatch between retrieval granularity and generative understanding. While retrieval modules return relevant chunks, these chunks are often fragmented in discourse, resembling scattered pieces of evidence without clear logical connections ( 13 ; 58 ) . This manifests at two levels. First, intra-chunk structural blindness : within each chunk, RAG systems may fail to capture internal discourse. Second, inter-chunk coherence gaps : across multiple chunks, RAG systems may struggle to identify rhetorical connections. As depicted in Figure 1 (left), Chunk A mentions a 12% lower incidence , while Chunk B notes no significant overall effect . Without recognizing that the former is a conditional finding ( e.g., among deficient adults in winter), standard RAG tends to overgeneralize and incorrectly conclude that vitamin D reduces flu risk . These deficiencies prevent effective resolution of conflicting claims, as standard RAG approaches lack the capacity to organize retrieved evidence through a higher-level causal flow. This leaves the final LLM generator to grapple with a bag of facts rather than a coherent line of reasoning . Recent investigations have revealed that integrating discourse knowledge into LLMs can improve downstream performance ( 15 ; 36 ) and alleviate hallucinations ( 40 ) . These findings highlight the drawback of relying solely on flat sequential representations and underline the benefits of discourse for context engineering ( 42 ; 49 ) . Building on these insights, the present work aims to investigate whether explicitly modeling and providing discourse information to the LLM can improve generation quality in the context of RAG. To answer this, we propose Disco-RAG , a framework that constructs local discourse trees for each retrieved chunk and infers inter-chunk coherence relations across chunks to form a rhetorical graph. To synthesize information, rather than merely concatenating it, the text generator needs not only to understand the relations between evidence but also to strategize how to present them. This requires a high-level plan to orchestrate the narrative flow. We thus introduce a discourse-aware planning module that enables the model to dynamically generate a plan to guide the generation. As shown in Figure 1 (right), the discourse-aware process enables the model to infer that vitamin D is not broadly effective but may benefit deficient adults under specific conditions , producing more faithful answers and aligning with the underlying evidence. In our experiments, we evaluate Disco-RAG on three benchmarks, Loong ( 61 ) , ASQA ( 57 ) , and SciNews ( 38 ) . Consistent improvements are observed compared with standard RAG systems and state-of-the-art (SOTA) methods. On the Loong benchmark, our approach delivers gains of up to 10.0 points in LLM Score. On the ASQA dataset, our method exceeds the best existing systems on Exact Match and DR Score by notable margins. On the SciNews benchmark, Disco-RAG establishes new SOTA performance across most evaluation metrics. In summary, our contributions are as follows: • We present Disco-RAG , an inference-time strategy that explicitly injects discourse knowledge into the RAG pipeline to alleviate the discrepancy between chunk-level evidence and discourse-level reasoning. • We propose a modeling method that combines intra-chunk discourse trees, inter-chunk rhetorical graphs, and discourse-driven plans to capture local hierarchies, cross-passage coherence, and argumentative flow. • We conduct experiments on knowledge-intensive QA and summarization tasks, demonstrating consistent gains over strong RAG baselines. Analysis studies further confirm the efficacy of discourse-aware guidance in enhancing generation correctness, coherence, and factuality. 2 Related Work 2.1 Structure-Aware Retrieval-Augmented Generation Retrieval-Augmented Generation (RAG) enhances LLMs in knowledge-intensive tasks by retrieving external evidence ( 29 ) . However, conventional RAG methods typically treat retrieved chunks as isolated and flat sequences, overlooking their structural interconnections. To mitigate this, recent research has explored structure-aware variants of RAG. Graph-based methods ( 51 ; 20 ; 66 ; 78 ) such as GraphRAG ( 13 ) and KG-RAG ( 55 ) organize evidence into knowledge graphs, while subsequent work has improved retrieval by simulating human memory mechanisms ( 17 ; 18 ) or enriching graph semantics ( 32 ) . Other approaches construct structured subgraphs for coherence ( 48 ; 30 ) , or employ alternative formats like hierarchical graphs ( 75 ; 62 ; 21 ) , semantic chunking ( 64 ; 52 ; 77 ) , trees ( 14 ; 56 ) , and tables ( 35 ) . More adaptive strategies dynamically select structures based on context ( 31 ) . Despite these advances, most efforts emphasize surface-level associations ( e.g., linking entities) while largely overlooking the rhetorical structure that governs causal flow, evidence presentation, and conclusion formulation. This hinders logical depth and discourse coherence, which our work seeks to address. 2.2 Rhetorical Structure Theory for Text Generation Rhetorical Structure Theory (RST; 44 ; 45 ) is a discourse framework that models hierarchical dependencies and rhetorical relations among Elementary Discourse Units (EDUs). It distinguishes between nucleus and satellite units, connected