Enhancing Large Language Models with Faster Code Preprocessing for Vulnerability Detection
José Gonçalves, Miguel Silva, Eva Maia, Isabel Praça · 2025
Read Enhancing Large Language Models with Faster Code Preprocessing for Vulnerability Detection by José Gonçalves, Miguel Silva, Eva Maia, Isabel Praça (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-05600.txt.
Introduction
Gonçalves, Silva, Maia, and Praça present a study from twenty twenty-five on faster code cleaning for AI-based flaw hunting. Language models can spot software flaws, but only if code form is standardized first. The same function can be written many ways, which blurs the signal. Their tool builds on an existing framework that normalizes code shape. The new version is faster, lighter, and easier to extend across languages. They compare old and new versions on speed and memory use. They also test how each affects a language model's flaw scores.
The new tool cuts processing time by over ninety-seven percent. The model's flaw detection score improves from preprocessing alone. The authors stress that input shaping matters as much as model choice. Their work targets practical security pipelines.
Catalog record
| CREATOR | José Gonçalves, Miguel Silva, Eva Maia, Isabel Praça |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-05600
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing Large Language Models with Faster Code Preprocessing for Vulnerability Detection José Gonçalves, Miguel Silva, Eva Maia, Isabel Praça Abstract The application of Artificial Intelligence has become a powerful approach to detecting software vulnerabilities. However, effective vulnerability detection relies on accurately capturing the semantic structure of code and its contextual relationships. Given that the same functionality can be implemented in various forms, a preprocessing tool that standardizes code representation is important. This tool must be efficient, adaptable across programming languages, and capable of supporting new transformations. To address this challenge, we build on the existing SCoPE framework and introduce SCoPE2, an enhanced version with improved performance. We compare both versions in terms of processing time and memory usage and evaluate their impact on a Large Language Model (LLM) for vulnerability detection. Our results show a 97.3% reduction in processing time with SCoPE2, along with an improved F1-score for the LLM, solely due to the refined preprocessing approach. 1 Introduction Software is integral to modern life, powering everything from household appliances to critical infrastructures. This ubiquity presents significant challenges, because even minor bugs or vulnerabilities can lead to severe disruptions, especially when exploited by malicious actors. In this context, Artificial Intelligence (AI) has emerged as a critical tool, supporting tasks ranging from code analysis to vulnerability detection and automated code generation. However, code generation and analysis are inherently complex tasks because they involve processing sequences of tokens, each of which carries a unique semantic value [ 7 ] . Unlike natural language, where some variability in word order or sentence structure is acceptable, source code adheres to strict syntactic rules, and even small structural changes can significantly affect its functional behavior [ 17 ] . Despite these rigid syntactic constraints, the semantics of code allow for remarkable flexibility, since the same functionality can often be implemented in multiple ways, expanding the range of possible implementations and, consequently, the challenges of analysis [ 18 ] . For an AI model to accurately analyze code, it must deeply understand the semantic role of each token and how these tokens interact within the broader program context. However, even state-of-the-art AI models often struggle with this level of comprehension [ 16 ] . Research has shown that AI models for Software Vulnerability Detection (SVD) frequently rely on superficial elements, such as programmer-defined variable names, rather than deeper structural patterns. To further investigate this AI-model understanding limitation, some studies [ 3 ] apply transformations to source code to generate new samples that can be used with multiple purposes, namely to improve the generalization of the model. However, no standardized tool exists for this purpose, as current implementations are often constrained by the target programming language, software extensibility, or the original objectives of their developers. As a result, multiple researchers create similar transformation tools, leading to fragmentation and limiting comparability across studies. A significant step toward a universal solution for code transformation was introduced in a previous work with Source Code Processing Engine (SCoPE) [ 6 ] , a tool that enables transformations, such as variable renaming, to be applied to source code. While promising, SCoPE has notable limitations that pose challenges to its broader adoption. This work addresses the lack of a standardized tool for applying transformations to source code by improving and extending the existing SCoPE tool. The new version overcomes the main limitations of its predecessor while introducing a highly flexible framework that simplifies the application of source code transformations. In this study, the tool is used to preprocess data for fine-tuning a large language model (LLM) for SVD, facilitating source code preparation for model training or inference. The objective is to assess whether the updated version of SCoPE, which supports a wider range of code samples, including those containing macros, improves model performance compared to its previous version. In summary, the main contributions of this paper are as follows: • Update of the SCoPE framework. The new version is intended to lay the foundations for the adoption of a common platform for the application of transformations to source code. The code will be available on Github 1 1 1 https://github.com/jp2425/SCoPE2 . • Fine-tuning of Large Language Model Meta AI (LLaMA) 3.2, a compact yet powerful model released by Meta AI, for vulnerability detection. The training data was processed using the latest version of SCoPE, with results compared against those obtained from the original version. The structure of this study is as follows: Section 2 presents a literature review on the applications of LLMs for SVD, challenges in this field, the role of code transformations, and the need for further research. Section 3 introduces SCoPE2 and compares its performance to the previous version. Section 4 details the experimental setup, including the dataset, data preprocessing, and the LLaMA 3.2 model. The results are presented and discussed in Section 4.3 . Finally, Section 5 summarizes the main findings and outlines potential directions for future research. 2 Related Work As software is a significant component of most devices, vulnerabilities pose a significant security threat, potentially exposing systems to exploitation by malicious actors. A vulnerability can compromise an entire system, leading to data breaches and security incidents, making SVD a core practice for maintaining high security standards [ 8 ] . Traditional SVD methods rely heavily on manual inspection to verify the presence of vulnerabilities. This process is usually time-consuming and labor-intensive, requiring skilled developers to meticulously analyze code and identify potential vulnerabilities, while also being able to understand the rationale behind recent code changes [ 9 ] . Despite their effectiveness in some instances, traditional approaches often struggle with scalability and have a high false positive rate [ 2 ] . As software projects expand in size and complexity, these limitations become increasingly apparent, making it challenging to keep up with evolving security threats. Consequently, extensive research has focused on AI-driven techniques to enhance and automate vulnerability detection, addressing the shortcomings of traditional methods. [ 2 ] . The recent widespread adoption of LLMs has further demonstrated their potential as highly effective tools for SVD [ 10 , 14 ] . These models have shown strong performance in language-related tasks [ 12 ] , particularly in identifying relationships within textual sequences [ 11 ] . With their ability to understand code [ 15 ] , LLMs can uncover coding patterns and distinguish between secure and vulnerable code [ 11 ] , making them a promising approach for automated vulnerability detection [ 19 ] . The limited token processing capacity of LLMs is an important factor in their application. To address this issue, source code can be modified to reduce its size. However, achieving an optimal representation is challenging, especially in SVD, where multiple transformations can produce functionally equivalent but structurally different versions of the same code. To address this, the SCoPE framework [ 6 ] has been introduced, offering various preprocessing techniques specifically for C/C++ code. This framework can apply function-level transformations to ensure standardized code representations. These transformations include shortening variable and function names, removing comments and error handling strings, as well as normalizing whitespaces. By generalizing programmer-defined elements, such as variable names, SCoPE can be also used to verify the existence of duplicate entries, which