Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking

Fachrina Dewi Puspitasari, Chaoning Zhang, Jiaquan Zhang, Zhicheng Wang, Hafiz Shakeel Ahmad Awan, Rizwan Qureshi, Jewon Lee, Tae-Ho Kim, Yang Yang · 2026

Read Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking by Fachrina Dewi Puspitasari, Chaoning Zhang, Jiaquan Zhang, Zhicheng Wang, Hafiz Shakeel Ahmad Awan, Rizwan Qureshi, Jewon Lee, Tae-Ho Kim, Yang Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01240.txt.

Introduction

Puspitasari and co-authors make retrieval for language models more exact. Such systems fetch text chunks to aid each reply. Weak systems miss user intent in their search. They also split proof into broken chunks. InSemRAG checks and repeats its search in a loop. An intent-aware retriever weights search paths by query type. A chunk repair step fixes split proof and keeps sense intact.

Small language models help cut the cost of the loop. Tests span many benchmark sets. The method gains 2.65 F1 points on HotPotQA. It gains 1.5 points on FEVER as well. It also runs 4.32 times faster than Multi-Hop RAG. Gains are largest on multi-hop and proof-heavy tasks.

Catalog record

CREATORFachrina Dewi Puspitasari, Chaoning Zhang, Jiaquan Zhang, Zhicheng Wang, Hafiz Shakeel Ahmad Awan, Rizwan Qureshi, Jewon Lee, Tae-Ho Kim, Yang Yang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.01240

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking Fachrina Dewi Puspitasari, Chaoning Zhang, Jiaquan Zhang, Zhicheng Wang, Hafiz Shakeel Ahmad Awan, Rizwan Qureshi, Jewon Lee, Tae-Ho Kim, Yang Yang Abstract The demand for powerful instruction following and reasoning capability of large language models (LLMs) has promoted rapid development of retrieval-augmented generation (RAG). The RAG system assists LLM generation by retrieving chunks of query-fit supplementary knowledge from an external database. Conventional RAG systems, however, suffer from information insufficiency due to two factors, which are intent-agnostic retrieval and information fragmentation. Our work proposes a RAG framework, termed InSemRAG, that addresses these challenges via an iterative retrieve-and-check mechanism with two supporting modules, an intention-aware retriever (IAR) and semantics-preserving chunking (SPC). IAR implements a dynamic hybrid retrieval method that adaptively weights the retrieval channels based on the query intent, while SPC performs detection and reparation to the damaged evidence chunks to preserve the semantic integrity. To alleviate the computational latency brought by our iterative mechanism, we leverage small language models (SLMs). Extensive experiments across several benchmark datasets consistently demonstrate the competitiveness of our method against recent state-of-the-art RAG mechanisms. Particularly, our method achieves significant gains on multi-hop and evidence-sensitive tasks, with a 2.65-point improvement in F1 on HotPotQA and a 1.5-point increase in accuracy on FEVER. Our method also achieves competitive performance to Multi-Hop RAG with 4.32 × \times lower latency with the utilization of SLM. Figure 1: Limitations on the conventional RAG system that relies on a fixed retrieval channel and ignores semantically fragmented evidence chunks. We address these limitations in InSemRAG. 1 Introduction With the breakthrough of Large Language Models (LLMs) in question answering, code generation, and multimodal understanding ( 9 ; 39 ; 44 ; 24 ) , Retrieval-Augmented Generation (RAG) ( 22 ) has become the mainstream paradigm for improving model factual consistency and knowledge coverage. RAG effectively alleviates the problems of knowledge timeliness and hallucination generation in language models by combining external knowledge base retrieval with generative models ( 42 ; 50 ) . Specifically, RAG systems consist of three core steps, namely retrieval, chunking, and generation. First, the user provides an input query to the retriever, which derives retrieval keywords for sparse retrieval or rewrites the query for dense retrieval. The retriever then extracts uniform-length chunks of information that are highly similar to the retrieval query. The retrieved evidence is then aggregated and sent to the LLM as a generator to produce an answer to the user’s query. The standard RAG mechanisms, however, are prone to information insufficiency in the retrieved evidence. The fixed retrieval channel (e.g., dense 3 ; 17 ; 22 or sparse 31 ; 15 ) commonly leveraged in the contemporary RAG systems tends to ignore the user’s requirement 45 . Such single-channel retrieval may collect descriptive evidence for semantically complex queries or retrieve explanatory evidence for purely descriptive questions, leading to evidence–intent mismatches. Moreover, mechanical chunking (e.g., boundary-based chunking 7 or structure-aware chunking 1 1 1 https://python.langchain.com 52 ) may break the content or ignore the semantic connections, posing a risk to information fragmentation ( 16 ) . Recent studies on efficient neural modeling and representation learning further suggest that rigid processing pipelines can hinder the preservation of temporal and structural dependencies in sequential data, motivating more adaptive and context-aware retrieval and encoding strategies ( 53 ; 54 ) . To address these challenges, we propose InSemRAG, a RAG framework that addresses the problem of information insufficiency in the retrieved evidence. InSemRAG consists of two modules, an intention-aware retriever (IAR) and semantics-preserving chunking (SPC). Given an input query from the user, we first perform dual-view rewriting that simultaneously decomposes the query into keywords for sparse retrieval and expands it into a semantically meaningful phrase for dense retrieval. Utilizing the rewritten query, we perform dynamic hybrid retrieval, where we dynamically modulate the retrieval channel weight depending on the characteristics of the query. Instead of passing the retrieved evidence directly to the generator, we perform detection and reparation on the semantically damaged chunk. Specifically, we repair the damaged chunk by aggregating with the nearest neighbor chunks from the source documents and rewriting it to meet the chunk length requirement while maintaining the semantic information. Finally, we assess whether the final evidence contains all query-required information elements. Otherwise, the retrieve-and-check procedure is repeated until information completeness is satisfied. To accommodate the complexity of the iterative retrieve-and-check mechanism in InSemRAG, we leverage small language models (SLMs) as the machines for retrieval and semantic checking. We are motivated by the excellent instruction following capability at minimum computational cost brought by SLMs 40 ; 21 . This utilization circumvents the need for large instruction-following models utilized in recent RAG systems 34 . We summarize our contributions as follows: • To the best of our knowledge, we are the first to introduce a semantic integrity-based iterative dynamic retrieve-and-check mechanism in RAG. • We propose InSemRAG, an efficient intent-aware retrieval and semantics-preserving chunking mechanism for RAG to retrieve evidence that not only considers the user’s requirement, but also maintains the evidence’s semantic integrity. • Through a comprehensive experimental validation, we demonstrate the effectiveness of InSemRAG in both performance and efficiency dimensions. Specifically, we yield up to 2.65-point improvement in F1 on evidence-sensitive reasoning tasks with 4.32× lower latency than prior state-of-the-art methods. 2 Related Work RAG ( 22 ; 9 ) addresses fundamental limitations of specific knowledge coverage in LLMs by integrating external knowledge retrieval with generative models ( 42 ) . Query Enhancement Techniques. Query is one of the factors that determine the success of retrieval mechanism for the generator 45 . Thus, several works explore the RAG improvement technique from the query enhancement perspective. RQ-RAG ( 4 ) enables query rewriting, decomposition, and disambiguation. DMQR-RAG ( 25 ) proposes diverse multi-query rewriting strategies operating at different information levels. HyDE ( 8 ) generates hypothetical documents to bridge the semantic gap between queries and relevant passages. Step-back Prompting ( 55 ) rewrites detailed queries at higher conceptual levels. In addition, recent multimodal reasoning studies emphasize structured intermediate reasoning processes (e.g., Visual Chain-of-Thought) to enhance alignment between queries and retrieved evidence, further highlighting the importance of query abstraction and decomposition in complex reasoning tasks ( 56 ) . Our method is inspired by these works, thus leveraging a query rewriting mechanism in the pre-retrieval process through both decomposition and enhancement. Evidence Retrieval Methods. Contemporary RAG systems are generally grouped into two based on retrieval mechanisms, sparse and dense. Sparse retrieval refers to utilizing keywords to look for information in the database with BM25 or TF-IDF mechanisms 31 ; 15 . Meanwhile, dense retrieval leverages similarity measures between the semantic representation of the user’s query and the candidate evidence 3 ; 17 ; 22 . Additionally, recent works combine sparse and dense retrievals. RAG-Fusion ( 30 ) applies reciprocal rank fusion to combine results from multiple query reformulations. DAT ( 14 ) proposes dynamic alpha tuning for hybrid retrieval, adapting weight coefficients based on individual query characteristics. Late-interaction retrieval models such as ColBERTv2 improve retrieval granularity by preserving token-level interactions, enabling more precise matching between queries and passages ( 33 ) . Beyond these paradigms, recent efforts explore more fine-grained and efficient retrieval

Read the complete text