Graph-Aware Late Chunking for Retrieval-Augmented Generation in Biomedical Literature

Pouria Mortezaagha, Arya Rahgozar · 2026

Read Graph-Aware Late Chunking for Retrieval-Augmented Generation in Biomedical Literature by Pouria Mortezaagha, Arya Rahgozar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22633.txt.

Introduction

Mortezaagha and Rahgozar argue that biomedical search tools are judged too narrowly. Most tests reward finding the single best text chunk. But good answers often need facts spread across a paper's sections. The authors propose GraLC-RAG, which mixes late chunking with graph-aware structure. It detects section boundaries, adds UMLS knowledge graph data, and guides hybrid search. Tests use 2,359 PubMed Central articles and 2,033 cross-section questions. Plain similarity search scores best on rank but pulls from one section only. Structure-aware methods pull from far more sections.

Catalog record

CREATORPouria Mortezaagha, Arya Rahgozar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41397
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22633

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Graph-Aware Late Chunking for Retrieval-Augmented Generation in Biomedical Literature Pouria Mortezaagha, Arya Rahgozar Abstract Retrieval-Augmented Generation (RAG) systems for biomedical literature are typically evaluated using ranking metrics like Mean Reciprocal Rank (MRR), which measure how well the system identifies the single most relevant chunk. We argue that for full-text scientific documents, this evaluation paradigm is fundamentally incomplete: it rewards retrieval precision while ignoring retrieval breadth —the ability to surface evidence from across a document’s structural sections. To investigate this blind spot, we propose GraLC-RAG (Graph-aware Late Chunking for Retrieval-Augmented Generation), a framework that unifies late chunking with graph-aware structural intelligence for biomedical literature retrieval. GraLC-RAG introduces structure-aware chunk boundary detection using document structure graphs, knowledge graph infusion via UMLS ontological signals, and graph-guided hybrid retrieval. We evaluate six retrieval strategies on 2,359 IMRaD-filtered PubMed Central articles across a document-length gradient using 2,033 cross-section questions and two complementary metric families: standard ranking metrics (MRR, Recall@ k k ) and structural coverage metrics (SecCov@ k k , CS Recall). Our results expose a sharp divergence: content-similarity methods (semantic chunking) achieve the highest MRR (0.517 on full-text) but are structurally blind, always retrieving from a single section (SecCov = 1.0). Structure-aware methods achieve lower MRR but retrieve from up to 15.6 × \times more document sections (SecCov@20 = 15.57). Generation experiments across all six strategies show that KG-infused retrieval narrows the answer-quality gap to Δ \Delta F1 = 0.009 (0.394 vs. 0.403) while maintaining 4.6 × \times section diversity, partially bridging the retrieval-to-generation gap. These findings demonstrate that standard evaluation metrics systematically undervalue structural retrieval methods and that fully closing the multi-section synthesis gap is a key open problem for biomedical RAG. Keywords: Retrieval-augmented generation \cdot Late chunking \cdot Knowledge graphs \cdot Biomedical NLP \cdot Evaluation methodology \cdot Structural retrieval \cdot UMLS Code and Data: https://github.com/pouriamrt/gralc-rag 1 Introduction The exponential growth of biomedical literature, with over 1.5 million articles indexed annually in PubMed alone, has created an urgent need for intelligent retrieval and synthesis systems [ 3 ] . Large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation, yet they remain prone to hallucination and factual inconsistency when applied to knowledge-intensive biomedical tasks [ 13 , 12 ] . Retrieval-Augmented Generation (RAG), which grounds LLM outputs in retrieved external evidence, has emerged as a principled solution to this challenge [ 16 , 7 ] . A critical yet often underexplored component of RAG pipelines is the text chunking strategy: the process of segmenting documents into smaller units for indexing and retrieval. Traditional approaches employ fixed-size chunking with overlapping windows, which inevitably fragments semantic context and disrupts the logical structure of scientific documents [ 20 ] . This is particularly problematic for biomedical literature, where cross-referential reasoning ( e.g. , linking a method description in Section 2 to results discussed in Section 4), dense terminology, and hierarchical document organization (the IMRaD structure) are fundamental to comprehension [ 25 ] . Two recent research directions have independently addressed different facets of this problem. Late chunking [ 9 ] inverts the traditional chunk-then-embed pipeline by first processing the entire document through a long-context transformer model, generating contextually enriched token embeddings, and only then applying segmentation boundaries. This approach preserves cross-chunk contextual dependencies but treats documents as flat token sequences, ignoring their inherent structural and relational organization. Graph-based RAG (GraphRAG) methods [ 6 , 22 , 11 ] leverage knowledge graphs, citation networks, and entity relationships to capture structural information during retrieval. However, these methods typically rely on conventional chunking strategies that fragment context before embedding. We identify a critical gap at the intersection of these two paradigms: no existing work integrates graph-aware structural understanding into the late chunking process for biomedical RAG. Late chunking is context-rich but structure-blind ; GraphRAG is structure-rich but context-fragmented . Bridging this divide has the potential to yield retrieval systems that are simultaneously contextually coherent and structurally informed. In this paper, we make three contributions that span framework design, evaluation methodology, and empirical analysis: 1. GraLC-RAG: A structure-aware late chunking framework. We propose GraLC-RAG (Graph-aware Late Chunking for Retrieval-Augmented Generation), which integrates document structure graphs for chunk boundary detection, UMLS knowledge graph signals for token-level enrichment, and graph-guided hybrid retrieval. This provides a unified testbed for investigating whether combining context-preserving late chunking with structure-aware GraphRAG yields measurable retrieval improvements. 2. A structural coverage evaluation methodology. We introduce Section Coverage@ k (SecCov@ k ) and Cross-Section Recall (CS Recall) as complementary metrics that capture retrieval breadth (the ability to surface evidence from multiple document sections), which standard ranking metrics like MRR entirely miss. 3. Empirical evidence of a metric-strategy divergence. Through evaluation on 2,359 IMRaD-filtered PubMed Central articles with 2,033 cross-section questions, we demonstrate that content-similarity methods and structure-aware methods optimize for fundamentally different retrieval objectives. This divergence is invisible under standard evaluation and has direct implications for system design. Our experiments reveal a surprising finding: the methods that achieve the highest ranking accuracy retrieve from only a single document section, while structure-aware methods that rank lower on MRR surface evidence from up to 15.6 \times more sections. Generation experiments across all six strategies show that KG-infused retrieval narrows the answer-quality gap to just \Delta F1 = 0.009 while maintaining 4.6 \times section diversity, partially bridging the retrieval-to-generation gap, though multi-section synthesis remains an open challenge. 2 Related Work 2.1 Retrieval-Augmented Generation RAG augments language model generation with external knowledge retrieved at inference time, mitigating hallucination and enabling access to up-to-date information [ 16 ] . The canonical RAG pipeline consists of three stages: indexing, retrieval, and generation [ 7 ] . Dense passage retrieval (DPR) using dual-encoder architectures [ 15 ] has largely supplanted sparse methods, with ColBERTv2 [ 24 ] introducing efficient late interaction for fine-grained token-level matching. In the biomedical domain, the MIRAGE benchmark [ 31 ] provides standardized evaluation across five biomedical QA datasets. 2.2 Text Chunking Strategies for RAG Text chunking critically influences RAG performance yet has received comparatively less systematic study [ 34 ] . Recent approaches include semantic chunking using embedding similarity [ 34 ] , proposition-based chunking [ 5 ] , and adaptive chunking achieving 87% accuracy compared to 50% for fixed-size baselines in clinical decision support [ 18 ] . For scientific documents, S2 Chunking [ 28 ] combines spatial layout with semantic analysis, and Breaking It Down [ 1 ] introduces Projected Similarity Chunking trained on PubMed data, achieving 24 \times MRR improvement on PubMedQA. 2.3 Late Chunking Late chunking [ 9 ] fundamentally reorders the chunk-then-embed pipeline. Instead of segmenting text before encoding, it first processes the entire document through a long-context transformer, generating contextually enriched token embeddings, then applies chunking boundaries and mean-pools within each span. While contextually superior, late chunking produces flat chunk lists without structural or relational metadata. Merola and Singh [ 20 ] demonstrate that contextual retrieval preserves semantic coherence more effectively but at greater computational cost. The SitEmb approach [ 29 ] partially addresses this by conditioning chunks on broader context, but operates on purely textual signals. 2.4 Graph-Based RAG GraphRAG methods incorporate structured graph information into retrieval. Microsoft’s GraphRAG [ 6 ] constructs entity knowledge graphs using LLM-based extraction and

Read the complete text