SLIDE: Sliding Localized Information for Document Extraction

Divyansh Singh, Manuel Nunez Martinez, Sonja Schmer Galunder, Bonnie J. Dorr · 2025

Read SLIDE: Sliding Localized Information for Document Extraction by Divyansh Singh, Manuel Nunez Martinez, Sonja Schmer Galunder, Bonnie J. Dorr (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17952.txt.

Introduction

Divyansh Singh and co-authors introduce SLIDE in a 2025 paper, a chunking method that improves knowledge graph building from long texts. Language models lose quality on long inputs, and low-resource languages suffer most from scarce data.

SLIDE slides overlapping windows over a document to keep local context whole. That guards facts that plain truncation would cut when texts exceed model limits. In a graph retrieval setup, entity extraction rose by about a quarter in English. Gains were larger in Afrikaans, a low-resource test case. Answer quality scores for breadth and depth also improved. The method helps teams mine long documents in many tongues without losing key links.

Catalog record

CREATORDivyansh Singh, Manuel Nunez Martinez, Sonja Schmer Galunder, Bonnie J. Dorr
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17952

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SLIDE: Sliding Localized Information for Document Extraction Divyansh Singh, Manuel Nunez Martinez, Sonja Schmer Galunder, Bonnie J. Dorr Abstract Constructing accurate knowledge graphs from long texts and low-resource languages is challenging, as large language models (LLMs) experience degraded performance with longer input chunks. This problem is amplified in low-resource settings where data scarcity hinders accurate entity and relationship extraction. Contextual retrieval methods, while improving retrieval accuracy, struggle with long documents. They truncate critical information in texts exceeding maximum context lengths of LLMs, significantly limiting knowledge graph construction. We introduce SLIDE (Sliding Localized Information for Document Extraction), a chunking method that processes long documents by generating local context through overlapping windows. SLIDE ensures that essential contextual information is retained, enhancing knowledge graph extraction from documents exceeding LLM context limits. It significantly improves GraphRAG performance, achieving a 24% increase in entity extraction and a 39% improvement in relationship extraction for English. For Afrikaans, a low-resource language, SLIDE achieves a 49% increase in entity extraction and an 82% improvement in relationship extraction. Furthermore, it improves upon state-of-the-art in question-answering metrics such as comprehensiveness, diversity and empowerment, demonstrating its effectiveness in multilingual and resource-constrained settings. 1 Introduction Figure 1: Comparison of knowledge graph extraction from a set a of fiction book pages as chunks (Top) Using GraphRAG without SLIDE produces a knowledge graph with fewer nodes (representing entities) and edges (representing relationships). (Bottom) Using SLIDE results in a richer knowledge graph. In the past year, integrating knowledge graphs with large language models (LLMs) has enhanced reasoning, summarization, and retrieval-augmented generation (RAG) Edge et al. 2024 ; Guo et al. 2024 ; Chang and Zhang 2024 . GraphRAG, a retrieval-augmented generation framework, uses LLMs to extract knowledge graphs for query-focused summarization Edge et al. 2024 . However, Constructing accurate knowledge graphs from long documents and datasets in low-resource languages presents significant challenges when using large language models (LLMs). Current methods struggle to extract nuanced entities and relationships, reducing their effectiveness in multilingual and resource-constrained settings. Large language models (LLMs), such as GPT-4o Achiam et al. 2023 , exhibit diminished effectiveness in entity and relationship extraction as input chunk lengths increase, leading to degraded accuracy for longer texts Edge et al. 2024 . Additionally, LLMs struggle with relationship extraction in low-resource languages Chen et al. 2024 ; Jinensibieke et al. 2024 due to data scarcity, further limiting their applicability in constructing knowledge graphs in diverse linguistic contexts. Contextual retrieval Anthropic 2024 involves embedding each chunk of text along with its context from the entire document to enhance retrieval accuracy; it faces significant limitations with longer documents. Embedding each chunk with full document context becomes computationally expensive and inefficient, particularly when the document length exceeds the maximum context length of LLMs, leading to truncated or omitted critical information. Jiang et al. 2024 ; Li et al. 2024 . To tackle these challenges, we introduce SLIDE (Sliding Localized Information for Document Extraction), a chunking method designed to process documents that exceed the token limits of large language models (LLMs). SLIDE generates local context through overlapping windows, allowing efficient handling of long documents while maintaining contextual integrity. Unlike contextual retrieval, SLIDE limits the maximum number of tokens to the size of the window used, optimizing the handling of longer inputs. Furthermore, we apply this method and demonstrate improved knowledge graph extraction (See Figure 1 ) for GraphRAG systems in both English and Afrikaans, a low-resource language spoken by 7 million people in South Africa and Namibia Steyn 2016 , demonstrating its effectiveness in multilingual contexts and its potential to enhance knowledge extraction in resource-constrained settings. This paper makes the following key contributions: (1) Introduction of a sliding window-based contextual chunking method (SLIDE) for enhanced knowledge graph construction. (2) Ability to handle documents longer than the context length of the LLM by limiting token usage to the size of the sliding window, in contrast to the full-document context used in Contextual Retrieval. (3) Demonstrate evidence for performance increment over existing GraphRAG systems in both English and Afrikaans. Specifically, SLIDE achieved a 24% higher entity extraction and a 39% higher relationship extraction for English, while for the low-resource language Afrikaans, it saw a 49% increase in entity extraction and an 82% increase in relationship extraction. Additionally, SLIDE outperformed the state of the art on key question-answering metrics, including Comprehensiveness, Diversity and Empowerment. 2 Related Work Since SLIDE enhances knowledge graph construction in GraphRAG systems through contextual chunking, we first discuss related work in GraphRAG and chunking, highlighting their strengths and limitations. This sets the stage for our approach, which builds on GraphRAG by using overlapping windows to improve entity and relationship extraction. 2.1 GraphRAG and Knowledge Graphs GraphRAG Edge et al. 2024 is an advanced RAG framework that integrates knowledge graphs with large language models (LLMs) Trajanoska et al. 2023 to enhance reasoning and contextual understanding. Unlike traditional RAG systems, GraphRAG builds a knowledge graph with entities as nodes and relationships as edges, enabling precise and context-rich responses by leveraging the graph’s structure Edge et al. 2024 ; Wu et al. 2024 . Large language models (LLMs), such as GPT-4, show reduced effectiveness in entity and relationship extraction as input chunk lengths increase, degrading accuracy for longer texts Edge et al. 2024 . They also struggle with relationship extraction in low-resource languages, limiting their applicability Chen et al. 2024 ; Jinensibieke et al. 2024 . Building upon this work, our approach further enhances knowledge graph extraction by incorporating localized context which improves entity and relationship extraction. 2.2 Contextual Chunking Recent work in RAG systems has explored advanced chunking techniques to enhance retrieval and knowledge graph construction. Günther et al. 2024 implemented late chunking, where entire documents are embedded to capture global context before splitting into chunks, improving retrieval by emphasizing document-level coherence. However, this focus on global embeddings is less suited for knowledge graph construction. Our method instead uses localized context from raw text to retain meaningful relationships for improved entity and relationship extraction. Wu et al. 2024 introduced a hybrid chunking approach for Medical Graph RAG, combining structural markers like paragraphs with semantic coherence to produce self-contained chunks. While effective, this approach relies on predefined boundaries. Our method extends this by generating contextual information from neighboring chunks, enhancing the completeness of knowledge graph construction. Contextual retrieval Anthropic 2024 improves accuracy but struggles with longer documents, as embedding each chunk with full document context is computationally expensive and truncates critical information with documents exceeding maximum context length of the model Jiang et al. 2024 ; Li et al. 2024 . Our overlapping window-based approach addresses these inefficiencies, improving performance in both retrieval and knowledge graph construction. Figure 2: How Context Is Generated for Each Chunk Using a Sliding Window Approach to Form a Contextual Chunk . Each Chunk, Along with Its Neighboring Chunks, Is Fed to an LLM to Generate Context. 3 SLIDE: Sliding Localized Information for Document Extraction Given a document D and chunks C_{1},C_{2},\dots,C_{k} based on the desired token length (similar to Edge et al. 2024 ), SLIDE employs a sliding window technique to incorporate neighboring chunks within a fixed window size. The window size n , a hyperparameter, determined by the LLM’s context length and computational constraints, is

Read the complete text