SoK: Towards Effective Automated Vulnerability Repair
Ying Li, Faysal Hossain Shezan, Bomin Wei, Gang Wang, Yuan Tian · 2025
Read SoK: Towards Effective Automated Vulnerability Repair by Ying Li, Faysal Hossain Shezan, Bomin Wei, Gang Wang, Yuan Tian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-18820.txt.
Introduction
Ying Li and co-authors wrote a 2025 survey that maps the field of automated vulnerability repair. Software flaws are common, and fixing them by hand is slow. Auto repair tools promise to patch code on their own, across both lab made and real world flaws.
The survey sorts past methods into four families: template guided, search based, constraint based, and learning driven. Bench tests across data sets weigh the strengths and limits of each family. Learning methods shine in narrow cases but lack deep grasp of program logic. Hard flaws strain all families. The authors close with trends and open paths for the field.
Catalog record
| CREATOR | Ying Li, Faysal Hossain Shezan, Bomin Wei, Gang Wang, Yuan Tian |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 95 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-18820
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SoK: Towards Effective Automated Vulnerability Repair Ying Li, Faysal Hossain Shezan, Bomin Wei, Gang Wang, Yuan Tian Abstract The increasing prevalence of software vulnerabilities necessitates automated vulnerability repair (AVR) techniques. This Systematization of Knowledge (SoK) provides a comprehensive overview of the AVR landscape, encompassing both synthetic and real-world vulnerabilities. Through a systematic literature review and quantitative benchmarking across diverse datasets, methods, and strategies, we establish a taxonomy of existing AVR methodologies, categorizing them into template-guided, search-based, constraint-based, and learning-driven approaches. We evaluate the strengths and limitations of these approaches, highlighting common challenges and practical implications. Our comprehensive analysis of existing AVR methods reveals a diverse landscape with no single “best” approach. Learning-based methods excel in specific scenarios but lack complete program understanding, and both learning and non-learning methods face challenges with complex vulnerabilities. Additionally, we identify emerging trends and propose future research directions to advance the field of AVR. This SoK serves as a valuable resource for researchers and practitioners, offering a structured understanding of the current state-of-the-art and guiding future research and development in this critical domain. 1 Introduction The relentless increase in software vulnerabilities poses a critical challenge for organizations, with consequences ranging from financial loss to reputational damage [ 1 , 2 ] . This challenge is compounded by the limitations of manual repair processes, which are often slow, error-prone, and struggle to keep up with the volume of necessary fixes. Furthermore, numerous known vulnerabilities remain unpatched for extended periods, leaving systems exposed [ 3 , 4 , 5 ] . Automatic Vulnerability Repair (AVR) has emerged as a vital field to address these challenges, offering the potential to significantly reduce the time and resources to mitigate security risks. AVR has witnessed significant progress, evolving from the template-based approach that applied predefined repairs (often inflexible), random mutation, to more sophisticated approaches [ 6 , 7 , 8 ] . For example, constraint-solving techniques, often deriving constraints from symbolic execution, can be computationally expensive [ 9 , 10 ] . Deep learning offers an end-to-end translation approach, but its effectiveness hinges on high-quality vulnerability repair datasets [ 11 , 12 , 13 ] . Currently, large language models (LLMs) show promise due to their vast training data, although limitations persist in function-level repairs and comprehensive program understanding [ 14 , 15 , 16 ] . Given the rapid growth of this field, a SoK is crucial to understand the landscape. This SoK aims to: (1) Bridge knowledge gaps by analyzing AVR approaches across synthetic data and real-world vulnerabilities. (2) Offer critical insights into the strengths and limitations of current work. (3) Propose promising future research directions. In this paper, we establish a taxonomy of existing AVR methodologies, grounded in fundamental principles, and conduct a quantitative comparison of current approaches. This SoK provides a comparative analysis of security patch generation methods, complexity levels, impact factors for successful repair, and vulnerability types, revealing a multifaceted landscape. We assess both synthetic and real-world vulnerabilities, finding no definitive “best” method; each has inherent limitations. For example, learning-based methods lack whole-program understanding, while non-learning methods struggle with precise constraint extraction, e.g., loop invariant synthesis. Based on our taxonomy and benchmarks, we analyze current research advancements, theoretical challenges, and future directions in AVR. Our findings indicate that performance across existing works varies significantly depending on the benchmark used, which often suffer from limited data points. We propose several compelling research directions to advance AVR, including the integration of hybrid approaches, interpretability of AVR , and the generation of high-quality specifications. This SoK benefits both experts and practitioners. For security engineers, it provides (1) a systematic taxonomy to understand existing research; (2) analysis for leading repair methods; and (3) insights into limitations and future potential. Practitioners gain (1) a concise problem definition of vulnerability repair and (2) a comprehensive evaluation with practical implementations of key approaches. In summary, we have the following key contributions: • We define the core problem of security vulnerability repair and provide a comprehensive taxonomy and comparative analysis of repair approaches (Section 3 ), with a focus on patch generation techniques (Section 4 ). This discussion culminates in actionable takeaways and clearly identified open research questions. • Our taxonomy classifies security patch generation approaches by 4 categories, 5+ strategies, 11+ methodologies, 5+ programming languages, and 12+ vulnerability types. • We conduct quantitative evaluations of existing methods across C/C++ and Java benchmarks. We synthesize practical implications and highlight prevalent challenges in AVR, offering a critical assessment of the field’s progress. • We explore the latest research trends, identify ongoing challenges and propose directions for future work. Through quantitative and qualitative analysis, we highlight gaps in current techniques and extract key findings to guide future advancements in AVR. We will keep our website ( https://sok-avr.github.io ) updated with all the latest studies and findings to promote the research in this field. Related Work. This study mainly differs from existing surveys in the following three aspects: (i) Existing works [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ] focus on program repair from a general perspective rather than a security-specific one. For instance, Huang et al. [ 22 ] briefly discusses security vulnerabilities in the context of learning-based methods, Monperrus et al. [ 19 ] include only a subset of AVR-related papers, and Pinconschi et al. [ 20 ] demonstrated that general program repair tools perform poorly on vulnerability repair. These gaps highlight the need for our security-focused analysis. (ii) existing surveys on vulnerability repair focuses on LLM for AVR [ 14 , 24 ] , however, traditional methods (e.g., constraint-based methods) for AVR continue to evolve and demonstrate effectiveness in recent research [ 25 , 26 ] ; (iii) Prior surveys lack a systematic comparison across different AVR approaches. Our study bridges these gaps by providing both quantitative benchmarking across diverse datasets and qualitative analysis of various approaches, enabling researchers to better understand the current landscape and identify promising future research directions in AVR. 2 Preliminaries and Problem Setup Figure 1: The timeline of vulnerability discovery, patch release, and exploit publication. Figure 1 shows the timeline from vulnerability discovery to CVE publication and vulnerability repair process. The timeline from vulnerability discovery to CVE publication typically involves several key stages. Initially, the exact discovery time of a vulnerability is often unknown. CVEs are usually published after a vendor releases a patch, but in urgent cases or due to lack of cooperation, they may be disclosed before a patch is available. A vulnerability is classified as a zero-day until a patch is released, after which it becomes an N-day vulnerability. The main goal of AVR is to minimize the vulnerability exposure window. The vulnerability repair workflow includes three key components: vulnerability localization (VL), security patch generation (SPG), and security patch validation (SPV). Below we detail each repair stage and further illustrate its relationship to program repair. 2.1 Vulnerability Localization VL identifies the precise location or source of vulnerabilities within an application. This process involves pinpointing specific program points (i.e., specific locations within a program, e.g., lines of code, statements) where vulnerabilities exist and can be exploited, and then finding the point at which the “root cause” of the bug