RelRepair: Enhancing Automated Program Repair by Retrieving Relevant Code

Shunyu Liu, Guangdong Bai, Mark Utting, Guowei Yang · 2025

Read RelRepair: Enhancing Automated Program Repair by Retrieving Relevant Code by Shunyu Liu, Guangdong Bai, Mark Utting, Guowei Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-16701.txt.

Introduction

Shunyu Liu, Guangdong Bai, Mark Utting, and Guowei Yang improve auto bug fixing in a 2025 software paper. Large language models can now patch code on their own. Yet they often lack facts about the project at hand. The team fetch relevant code from the same project and feed it to the model before it writes a patch.

Their system is called RelRepair. On one benchmark of small Java bugs, fix rates rose by a clear margin. Nearly half of the bugs were fixed in that test. The gains show that project context matters as much as model size. Better patches could cut the time coders spend on dull fixes.

Catalog record

CREATORShunyu Liu, Guangdong Bai, Mark Utting, Guowei Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-16701

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RelRepair: Enhancing Automated Program Repair by Retrieving Relevant Code Shunyu Liu, Guangdong Bai, Mark Utting, Guowei Yang Abstract Automated Program Repair (APR) has emerged as a promising paradigm for reducing debugging time and improving the overall efficiency of software development. Recent advances in Large Language Models (LLMs) have demonstrated their potential for automated bug fixing and other software engineering tasks. Nevertheless, the general-purpose nature of LLM pre-training means these models often lack the capacity to perform project-specific repairs , which require understanding of domain-specific identifiers, code structures, and contextual relationships within a particular codebase. As a result, LLMs may struggle to generate correct patches when the repair depends on project-specific information. To address this limitation, we introduce RelRepair , a novel approach that retrieves relevant project-specific code to enhance automated program repair. RelRepair first identifies relevant function signatures by analyzing function names and code comments within the project. It then conducts deeper code analysis to retrieve code snippets relevant to the repair context. The retrieved relevant information is then incorporated into the LLM’s input prompt, guiding the model to generate more accurate and informed patches. We evaluate RelRepair on two widely studied datasets, Defects4J V1.2 and ManySStuBs4J, and compare its performance against several state-of-the-art LLM-based APR approaches. RelRepair successfully repairs 101 bugs in Defects4J V1.2. Furthermore, RelRepair achieves a 17.1% improvement in the ManySStuBs4J dataset, increasing the overall fix rate to 48.3%. These results highlight the importance of providing relevant project-specific information to LLMs, shedding light on effective strategies for leveraging LLMs in APR tasks. Keywords: Automated program repair, large language models, retrieval-augmented generation, code similarity 1 Introduction Automated Program Repair (APR) is an emerging field focused on automating the bug-fixing process in software development, significantly reducing debugging time and improving software reliability. Traditional APR methods generally operate based on a Generation-Validation (G&V) [ 1 ] paradigm. These techniques are categorized into four main categories: heuristic-based, constraint-based, template-based, and learning-based approaches. Heuristic-based APR [ 2 , 3 , 4 ] employs genetic algorithms and search heuristics to iteratively generate and evaluate patches. Constraint-based APR [ 5 , 6 , 7 , 8 ] treats repair as a constraint satisfaction problem, solving logical formulations via program synthesis or symbolic execution. Template-based APR [ 9 , 10 , 11 , 12 , 13 ] uses predefined patch templates that match specific bug patterns to generate fixes efficiently and accurately. Learning-based APR [ 14 , 15 , 16 , 17 ] leverages neural models, particularly Neural Machine Translation (NMT), to learn repair transformations from historical code changes. While learning-based APR achieves strong performance by capturing generalizable patterns, its effectiveness is limited by the quality of training data mined from open-source repositories. With the rise of pre-trained Large Language Models (LLMs) whose training corpora include large volumes of open-source code, LLM-based APR [ 18 , 19 , 20 ] has emerged as a promising alternative. It employs LLMs, such as ChatGPT [ 21 ] that have been trained on billions of words from various sources, to understand the context of buggy code and directly generate its patches. This approach significantly improves the quality of repair tasks and reduces the dependence on large labeled datasets. For example, ChatRepair [ 22 ] leverages ChatGPT’s conversational abilities to iteratively refine the repair process by enriching the context with information from failing tests. Similarly, ThinkRepair [ 23 ] exploits the strong analytical and reasoning capabilities of Chain-of-Thought (CoT) to deliver an advanced program-repair solution. Collectively, these recent techniques highlight the advantage of LLM-based APR, producing patches that are more Despite the promising performance of LLM-based APR techniques when compared to traditional methods, they still exhibit certain limitations. 1)Limitations in Project-Specific Code Repair . While these models are proficient in handling general coding patterns due to training on broad, publicly available code corpora, they often lack familiarity with project-specific identifiers, auxiliary functions, or utility methods defined by individual developers. Consequently, generating accurate patches requires retrieving precise, relevant information directly tied to the specific bug context rather than relying solely on learned general patterns. 2) Computer Resources Challenges. Although fine-tuning LLMs on a specific codebase could provide domain-specific adaptation, it requires considerable computational resources, making it a costly solution. To overcome these limitations, we turn to Retrieval-Augmented Generation (RAG) [ 24 ] , a powerful approach that combines retrieval models with LLMs to enhance the generation process. RAG has emerged as a representative technique in the field of generative AI, as it effectively improves the quality of generated content by external knowledge. We propose RelRepair , a novel approach that retrieves relevant code to enhance automated program repair, through a specialized RAG approach tailored to program repair tasks. RAG frameworks are inherently flexible and can be adapted to retrieve either general or highly task-specific information depending on the scenario. In our approach, RelRepair focuses on retrieving highly relevant, project-specific data essential for effective program repair. Specifically, it retrieves: (1) relevant function signatures, facilitating simpler repairs involving direct modifications or replacements of function signatures without extensive code analysis, and (2) relevant code snippets, capturing semantically aligned fix logic or illustrating function usage. The retrieved relevant code is incorporated into the LLM prompt at inference time, allowing the model to generate patches more effectively for bugs that involve unfamiliar or project-specific constructs. To support the retrieval of both function signatures and relevant code snippets, RelRepair follows a unified three-step process: (1) Query Rewriting, which reformulates search queries to improve retrieval precision; (2) Dataset Creation and Indexing, which builds a targeted index of functions and code snippets from the project; and (3) Retrieval, where similarity-based search identifies the most relevant entries. In the final patch generation phase, the retrieved information is seamlessly integrated into the LLM’s input to guide its repair decisions. This RAG-based framework enables RelRepair to generate more accurate and informed patches across diverse bug scenarios. We evaluate RelRepair by comparing it against state-of-the-art LLM-based APR techniques on two widely studied Java benchmarks: Defects4J V1.2 [ 25 ] and ManySStuBs4J [ 26 ] . Our experiments demonstrate RelRepair ’s effectiveness in program repair. Specifically, in Defects4J V1.2, RelRepair successfully repairs 101 out of 255 bugs, while in ManySStuBs4J, our repair success rate improves from 31.2% to 48.3%. In summary, this paper makes the following contributions: • Idea: We introduce the idea of retrieving relevant code to enhance automated program repair. With the project-specific data that is relevant to the target bug, LLMs can potentially generate more accurate and informed patches across diverse bug scenarios. • Technique: We apply a RAG framework that incorporates two complementary retrieval processes: relevant function signature retrieval and relevant code snippet retrieval, where the system first rewrites queries to optimize retrieval, then extracts function signatures or relevant code snippets from codebase. This retrieved context is integrated into LLM prompts, allowing the model to generate patches with deeper structural awareness and higher correctness. • Evaluation: We evaluate RelRepair using two benchmarks for Java programs. The results show that RelRepair consistently fixes more bugs than LLM-based APR techniques across all evaluated benchmarks. 2 Background 2.1 Large Language Models Large Language Models (LLMs) [ 27 ] represent a significant advancement in artificial intelligence. These models are versatile and can be

Read the complete text