Context-Enhanced Vulnerability Detection Based on Large Language Model
Yixin Yang, Bowen Xu, Xiang Gao *, Hailong Sun · 2025
Read Context-Enhanced Vulnerability Detection Based on Large Language Model by Yixin Yang, Bowen Xu, Xiang Gao *, Hailong Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16877.txt.
Introduction
Yixin Yang and co-authors wrote this 2025 paper on helping language models find security flaws in code. Checking one file or function at a time starves the model of context, while feeding it a whole project buries it in noise and cost. The team pair program analysis with the model to strike a middle path.
Program analysis pulls out context at several levels of detail, filtering away what does not matter. The trimmed context goes to the model along with the source code. Tests with GPT-4, DeepSeek, and CodeLLaMA, under varied prompts, show the added context lifts detection clearly. The study maps which level of detail helps most, guiding builders toward a sweet spot.
Catalog record
| CREATOR | Yixin Yang, Bowen Xu, Xiang Gao *, Hailong Sun |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-16877
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Context-Enhanced Vulnerability Detection Based on Large Language Model Thanks: *These authors are co-corresponding authors. Yixin Yang, Bowen Xu, Xiang Gao *, Hailong Sun Abstract . Vulnerability detection is a critical aspect of software security. Accurate detection is essential to prevent potential security breaches and protect software systems from malicious attacks. Recently, vulnerability detection methods leveraging deep learning and large language models (LLMs) have garnered increasing attention. However, existing approaches often focus on analyzing individual files or functions, which limits their ability to gather sufficient contextual information. Analyzing entire repositories to gather context introduces significant noise and computational overhead. To address these challenges, we propose a context-enhanced vulnerability detection approach that combines program analysis with LLMs. Specifically, we use program analysis to extract contextual information at various levels of abstraction, thereby filtering out irrelevant noise. The abstracted context along with source code are provided to LLM for vulnerability detection. We investigate how different levels of contextual granularity improve LLM-based vulnerability detection performance. Our goal is to strike a balance between providing sufficient detail to accurately capture vulnerabilities and minimizing unnecessary complexity that could hinder model performance. Based on an extensive study using GPT-4, DeepSeek, and CodeLLaMA with various prompting strategies, our key findings includes: (1) incorporating abstracted context significantly enhances vulnerability detection effectiveness; (2) different models benefit from distinct levels of abstraction depending on their code understanding capabilities; and (3) capturing program behavior through program analysis for general LLM-based code analysis tasks can be a direction that requires further attention. DOI: XXXXXXX.XXXXXXX Journal: JACM Volume: 37 4 111 8 1. Introduction Memory safety vulnerabilities constitute a persistent and critical class of software security flaws in computing systems. Despite extensive research and mitigation strategies, these vulnerabilities remain predominant in recent Common Vulnerabilities and Exposures (CVE) announcements ( CVE, 2021 ; Chen et al., 2021 ) . The fundamental issue stems from memory-unsafe languages such as C and C++, which prioritize performance and low-level memory control over safety guarantees. These languages permit direct memory manipulation during buffer operations, allocation, and pointer management, creating significant security risks. Although modern programming languages incorporate memory safety features, legacy codebases and performance-sensitive applications continue to rely on memory-unsafe languages, perpetuating their vulnerability to exploitation. Static vulnerability detection is an effective technique that identifies potential security vulnerabilities by analyzing code without executing the program. Current approaches to static vulnerability detection can be broadly categorized into static program analysis, machine learning, and deep learning-based methods. Traditional static program analysis methods often rely on expert-defined rules, which, while effective, are time consuming, labor intensive, and difficult to generalize across different codebases ( Shi et al., 2018 ) . Machine learning-based methods for vulnerability detection utilize manually defined quantitative code features (i.e., cyclomatic complexity, number of nested loops, the maximum count of control/data structures, etc.) combined with traditional machine learning algorithms (i.e., SVM, KNN, etc.). They offer better generalization capabilities ( Du et al., 2019 ) . However, due to their shallow architectures and the coarse-grained features they extract, they often fail to achieve high detection accuracy. Deep learning-based vulnerability detection methods have recently surpassed traditional approaches, leveraging rich semantic features extracted from source code (i.e., Abstract Syntax Trees (ASTs), Control Flow Graphs (CFGs), Program Dependency Graphs (PDGs), etc.) and deep learning algorithms ( Li et al., 2018 ; Li et al., 2022 ; Chakraborty et al., 2022 ; Zhou et al., 2019b ) . These approaches significantly improve both the effectiveness and interpretability of vulnerability detection by capturing richer structural and semantic information. Despite the progress made by learning-based methods, several limitations still remain. First, function-level and file-level vulnerability detection methods often rely on using individual pre- and post-patch functions as the input to the model ( Li et al., 2018 ; Li et al., 2022 ; Chakraborty et al., 2022 ; Zhou et al., 2019b ) . However, these methods lack sufficient context to capture the root cause of vulnerabilities. In many cases, the root cause of a vulnerability might not be accurately reflected by the modified function alone, as it often involves cross-function interactions. Additionally, these methods tend to focus heavily on understanding token-level information, which makes it challenging for the models to generalize to new vulnerabilities not seen during training, resulting in a high number of false positives and false negatives. Second, vulnerability detection methods at the repository level aim to provide a more comprehensive context by analyzing entire codebases ( Wen et al., 2024 ; Zhou et al., 2024a ) . While such methods can theoretically capture cross-function or cross-file interactions, these methods suffer from substantial noise. For instance, repository-level analysis involves a lot of irrelevant information that can obscure meaningful patterns, making it difficult for models to accurately detect vulnerabilities. Furthermore, analyzing large code repositories, especially those with millions of lines of code, presents significant challenges due to compilation issues, static analysis overhead, and the “path explosion” problem in control flow analysis, which results in increased computational costs and limited scalability. Recently, large language models (LLMs), such as GPT-4 ( Achiam et al., 2023 ) , LLaMA ( Touvron et al., 2023 ) and DeepSeek ( DeepSeek-AI et al., 2025 ) , have shown remarkable code comprehension capabilities, drawing increasing attention from researchers in the field of software engineering. Trained on massive code repositories with billions of parameters, these models have the potential to capture deeper semantic information and address the shortcomings of traditional shallow learning models. Several work has explored applying LLMs for vulnerability detection, utilizing prompt engineering ( Zhang et al., 2024b ; Steenhoek et al., 2024b ; Zhou et al., 2024b ; Yang et al., 2025 ; Tamberg and Bahsi, 2025 ) and model fine-tuning ( Du et al., 2024a ; Shestov et al., 2024 ; Mao et al., 2024 ; Wang et al., 2024 ; Han et al., 2024 ) , and other strategies ( Du et al., 2024b ; Li et al., 2025 ) . However, similar to earlier deep learning models, LLMs struggle to directly infer the root causes of vulnerabilities with inadequate contextual information. Lacking specific contextual information inspired us to investigate whether it is possible to use abstract representations of vulnerability-related contexts to enhance the reasoning capabilities of large language models. To address this challenge, we first utilize program analysis to design a method to extract contextual information for Inter-procedural vulnerabilities. Specifically, we conduct control flow analysis and data flow analysis on the to-be-patched functions and their callee functions within a specific iteration layer, then we abstract function calls based on primitive APIs to provide contextual information to LLMs. Primitive APIs, e.g., malloc , are basic operations that commonly occur in various scenarios The motivation for considering primitive APIs is rooted in their fundamental nature — memory safety vulnerabilities often arise in the improper usage of these APIs. By focusing on primitive APIs, we can effectively abstract away the noise introduced by complex or domain-specific function calls, providing clearer contextual information for the LLMs to understand potential vulnerabilities. Second, we conduct an empirical study to evaluate how different levels of primitive API abstraction and different prompt engineering strategies impact the detection capabilities of LLMs. The motivation for using different API abstraction level is to determine the optimal