GrepRAG: An Empirical Study and Optimization of Grep-Like Retrieval for Code Completion DOI: XXXXXXX.XXXXXXX Conference: ISST ’26: 35th ACM SIGSOFT International Symposium on Software Testing and Analysis; October 03–09, 2026; Oakland, California, United States ISBN: 978-1-4503-XXXX-X/2018/06 CCS: S

Baoyi Wang, Xingliang Wang, Guochang Li, Chen Zhi, Junxiao Han, Xinkui Zhao, Nan Wang, Shuiguang Deng, Jianwei Yin · 2026

Read GrepRAG: An Empirical Study and Optimization of Grep-Like Retrieval for Code Completion DOI: XXXXXXX.XXXXXXX Conference: ISST ’26: 35th ACM SIGSOFT International Symposium on Software Testing and Analysis; October 03–09, 2026; Oakland, California, United States ISBN: 978-1-4503-XXXX-X/2018/06 CCS: S by Baoyi Wang, Xingliang Wang, Guochang Li, Chen Zhi, Junxiao Han, Xinkui Zhao, Nan Wang, Shuiguang Deng, Jianwei Yin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23254.txt.

Introduction

Baoyi Wang and eight co-authors study grep-like retrieval for repository-level code completion. Completing code across a whole project needs cross-file context, but context windows are limited. Retrieval frameworks fetch the right snippets, using semantic indexing or graph analysis of the code structure.

Their empirical study tests simple grep-style retrieval, based on literal text search, against fancier methods. They also optimize the simple approach for this task. A strong showing by plain search would simplify tooling for coding assistants everywhere. Limited context windows force the retrieval question: cross-file dependencies cannot all be pasted into the prompt.

Catalog record

CREATORBaoyi Wang, Xingliang Wang, Guochang Li, Chen Zhi, Junxiao Han, Xinkui Zhao, Nan Wang, Shuiguang Deng, Jianwei Yin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-23254

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GrepRAG: An Empirical Study and Optimization of Grep-Like Retrieval for Code Completion DOI: XXXXXXX.XXXXXXX Conference: ISST ’26: 35th ACM SIGSOFT International Symposium on Software Testing and Analysis; October 03–09, 2026; Oakland, California, United States ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Software and its engineering Software development techniques Baoyi Wang, Xingliang Wang, Guochang Li, Chen Zhi, Junxiao Han, Xinkui Zhao, Nan Wang, Shuiguang Deng, Jianwei Yin Abstract . Repository-level code completion remains challenging for large language models (LLMs), as it requires reasoning over cross-file dependencies while under limited context windows. To address this challenge, prior work has adopted Retrieval-Augmented Generation (RAG) frameworks based on semantic indexing or structure-aware graph analysis. Although effective, these approaches introduce substantial computational overhead for index construction and maintenance, which hinders their practicality in real-world development. Motivated by common developer workflows that rely on lightweight search utilities (e.g., ripgrep ) to locate relevant code, we revisit a fundamental yet underexplored question: how far can simple, index-free lexical retrieval go in supporting repository-level code completion before more complex retrieval mechanisms become necessary? To answer this question, we systematically explore the potential of lightweight, index-free, intent-aware lexical retrieval through extensive empirical analysis. We first introduce Naive GrepRAG, a baseline framework where LLMs autonomously generate ripgrep commands to localize relevant context. Our preliminary experiments show that even this basic implementation achieves performance comparable to sophisticated graph-based baselines. Further analysis reveals that its effectiveness stems from retrieving code fragments that are lexically precise and spatially closer to the completion site. However, we identify key limitations of this approach, including sensitivity to noisy matches caused by high-frequency ambiguous keywords and context fragmentation due to rigid truncation boundaries. To address these issues, we propose GrepRAG, which augments lexical retrieval with a lightweight post-processing pipeline featuring identifier-weighted re-ranking and structure-aware deduplication. Extensive evaluation on CrossCodeEval and RepoEval_Updated demonstrates that GrepRAG consistently outperforms state-of-the-art (SOTA) methods. In particular, on CrossCodeEval, GrepRAG achieves 7.04–15.58% relative improvement in code exact match (EM) over the best baseline. 1. Introduction As a core feature of modern Integrated Development Environments (IDEs), code completion plays a pivotal role in enhancing development efficiency ( 13 ; 4 ; 36 ; 70 ; 43 ; 68 ) . In recent years, automated code completion driven by LLMs has demonstrated remarkable proficiency within single-file or single-function contexts ( 33 ; 65 ; 6 ; 39 ; 11 ; 69 ) . However, its performance often degrades significantly when applied to repository-level code completion ( 27 ; 49 ; 46 ; 22 ; 18 ) . In large-scale repositories, relevant contextual information is typically fragmented across multiple files and directories due to modular code organization. As a result, crucial dependencies such as class definitions, utility interfaces, and global constants are frequently located outside the local file being edited. Given the limited context window of LLMs, it is infeasible to provide the model with the entire repository. Relying solely on intra-file context therefore misses essential cross-file dependencies ( 7 ; 40 ; 26 ) . RAG mitigates this contextual deficit by retrieving code snippets most relevant to the current logic at the repository level ( 17 ; 61 ; 34 ) . Existing RAG methodologies can be primarily categorized into three streams. The first category comprises traditional similarity-based retrieval methods ( 66 ; 41 ; 42 ) , which rank code snippets directly based on cosine similarity ( 37 ) or BM25 ( 38 ) scores relative to the code completion context. The second category involves structure-aware retrieval ( 28 ; 23 ; 35 ; 8 ; 50 ) , which models the repository as a dependency graph via static analysis and leverages graph structural information to locate relevant context. The third category pertains to strategy-optimized retrieval ( 51 ) ; these methods employ Reinforcement Learning (RL) to train dense retrievers, thereby dynamically optimizing retrieval strategies based on end-to-end completion feedback. Although existing RAG methods are effective, they often rely on complex preprocessing and index construction, which impose substantial time and computational costs. For example, on the huggingface_diffusers repository 1 1 1 https://github.com/huggingface/diffusers , which contains about 100K lines of Python code, GraphCoder ( 28 ) requires approximately 91 seconds to build the graph index and 7 seconds for retrieval. In contrast, developers typically expect latency below 0.5 seconds, while delays exceeding 2 seconds are considered unacceptable for user experience ( 48 ) . Furthermore, software repositories are dynamic, frequent code modifications render static graphs and vector indices stale. Inspired by human developers, who commonly use simple lexical search tools such as Grep , Ctrl+F , or IDE features like Go to Definition and Go to Implementation during programming to locate class and method definitions or implementations across files. Such tools are also used to retrieve code fragments with similar naming patterns distributed throughout the repository. This structural or lexically similar information is essential for resolving cross-file dependencies in repository-level code completion. ( 23 ) The observation motivates us to reconsider whether the potential of simple lexical retrieval has been fully explored before resorting to complex structural or semantic retrieval mechanisms. In parallel, modern agentic coding systems such as ClaudeCode and Windsurf have preliminarily demonstrated the feasibility of simple lexical matching by integrating Grep tools for multi-turn code question-answering tasks; however, the systematic application and evaluation of such techniques within the specific context of code completion remain unexplored. To systematically study this problem, we adopt a progressive research that first evaluates the potential of grep-like lexical retrieval for repository-level code completion and then explores methods to improve it. The overall framework is illustrated in Figure 1 . First, we introduce Naive GrepRAG, a framework where the LLM autonomously generates ripgrep commands for context localization, establishing a performance benchmark for lexical retrieval in repository-level code completion. Our exploratory experiments show that the performance of this naive approach outperforms complex graph-based methods (RQ1). To further understand why Naive GrepRAG performs effectively, we analyze the retrieval patterns of Naive GrepRAG and identify four main types of keywords, including class names, method names, variable names, and others, which are particularly effective for handling several completion scenarios, such as class declaration completion, method call completion, etc. Compared with baseline failures, Naive GrepRAG succeeds by retrieving code fragments closer to the completion site with more precise lexical queries (RQ2). However, our analysis in RQ1 reveals that Naive GrepRAG does not cover all cases solved by baseline methods, prompting a investigation of its limitations. Our analysis identifies two main issues: (1) keyword ambiguity, where high-frequency generic terms such as init introduce noise; and (2) context redundancy and fragmentation, where overlapping code blocks are truncated without merging. This not only results in redundant information occupying the context window, but also disrupts the integrity of the code flow due to fragmentation (RQ3). Based on these empirical insights, we propose GrepRAG. This enhanced framework introduces a lightweight post-processing pipeline incorporating identifier-weighted re-ranking and structure-aware deduplication. This approach addresses the lack of term weighting in the Jaccard algorithm. It also mitigates the issues of context fragmentation and redundancy caused by Grep-style retrieval. By resolving these problems, GrepRAG significantly improves completion performance while maintaining efficient retrieval. Figure 1 . Overview of the research framework. We first evaluate the effectiveness of Naive

Read the complete text