Security Degradation in Iterative AI Code Generation: A Systematic Analysis of the Paradox

Shivani Shukla, Himanshu Joshi, Romilla Syed · 2025

Read Security Degradation in Iterative AI Code Generation: A Systematic Analysis of the Paradox by Shivani Shukla, Himanshu Joshi, Romilla Syed (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11022.txt.

Introduction

Shivani Shukla, Himanshu Joshi, and Romilla Syed test what happens to code safety when AI rewrites its own code again and again, in a 2025 paper. Many teams now loop a model through rounds of so-called improvement. The authors ask whether each round truly helps. Their answer is a warning.

They ran 400 code samples through 40 rounds of revision. Four prompting styles were compared. After just five rounds, critical flaws had risen by 37.6 percent. Each prompting style bred its own pattern of flaws. The findings break the belief that more revision means safer code. The authors urge human checks between rounds and give practical rules to limit the harm.

Catalog record

CREATORShivani Shukla, Himanshu Joshi, Romilla Syed
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-11022

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Security Degradation in Iterative AI Code Generation: A Systematic Analysis of the Paradox Shivani Shukla, Himanshu Joshi, Romilla Syed Abstract The rapid adoption of Large Language Models (LLMs) for code generation has transformed software development, yet little attention has been given to how security vulnerabilities evolve through iterative LLM feedback. This paper analyzes security degradation in AI-generated code through a controlled experiment with 400 code samples across 40 rounds of ”improvements” using four distinct prompting strategies. Our findings show a 37.6% increase in critical vulnerabilities after just five iterations, with distinct vulnerability patterns emerging across different prompting approaches. This evidence challenges the assumption that iterative LLM refinement improves code security and highlights the essential role of human expertise in the loop. We propose practical guidelines for developers to mitigate these risks, emphasizing the need for robust human validation between LLM iterations to prevent the paradoxical introduction of new security issues during supposedly beneficial code ”improvements.” I Introduction The integration of Large Language Models (LLMs) into software development workflows has grown exponentially since the introduction of tools like GitHub Copilot, ChatGPT, and Claude. According to recent studies, over 80% of developers now regularly use AI assistants for code generation [ 1 ] , with GitHub’s CEO predicting that ”sooner than later, Copilot will write 80% of code” [ 2 ] . This paradigm shift promises substantial productivity gains but raises critical security concerns. While existing research has identified that AI-generated code frequently contains security vulnerabilities [ 3 , 4 , 5 ] , a critical gap exists in understanding how these vulnerabilities evolve through iterative interactions with LLMs. Developers typically do not accept AI-generated code verbatim but engage in feedback loops, submitting code to the AI for improvement, refinement, or extension. The security implications of these feedback loops remain largely unexplored. This paper addresses this research gap by systematically investigating how security properties appear to change when initially secure code undergoes multiple rounds of AI-based ”improvements.” We hypothesize that rather than enhancing security, iterative interactions with LLMs without human intervention may be associated with the introduction of new vulnerabilities, a counterintuitive phenomenon we term ”feedback loop security degradation.” This highlights the critical importance of human expertise in the development loop, as developers provide essential quality control that automated systems currently cannot replicate. Our work makes the following contributions: 1. We provide empirical evidence demonstrating how automated iterative AI feedback loops without human intervention are associated with code security outcomes through a controlled experiment with 40 rounds of code generation (10 rounds each across 4 prompting strategies). 2. We identify and categorize vulnerability patterns associated with four different prompting strategies (efficiency-focused, feature-focused, security-focused, and ambiguous improvements). 3. We document how vulnerability types, severity, and frequency change through iterative feedback loops and demonstrate the critical need for human expertise in these loops. 4. We propose practical guidelines for mitigating security degradation when using AI tools for iterative code improvement, emphasizing human-AI collaboration rather than AI autonomy. The remainder of this paper is organized as follows: Section II reviews related work on AI-generated code security. Section III details our experimental methodology. Section IV presents our findings, analyzing security degradation patterns across different prompting strategies. Section V discusses the implications of our results and proposes mitigation strategies. Section VI acknowledges limitations and suggests future research directions. Section critically analyzes the novelty of our work. Section VII concludes the paper. II Literature Review II-A Security Vulnerabilities in AI-Generated Code Research on AI-generated code security has grown significantly since 2022. Pearce et al. [ 3 ] conducted one of the first empirical studies evaluating GitHub Copilot’s security, finding that approximately 40% of generated programs contained vulnerabilities. Their analysis of 1,689 programs revealed particularly high vulnerability rates in C code (around 50%) compared to Python code (approximately 39%). Perry et al. [ 4 ] expanded this work through a user study comparing developers with and without AI assistance. Their results showed that participants using AI assistants wrote ”significantly less secure code” and exhibited a ”false sense of security,” often rating their insecure solutions as secure. This aligns with findings from Chong et al. [ 5 ] , who found that LLM-generated code lacked defensive programming constructs and contained subtly incorrect implementations of security-critical algorithms. A comprehensive systematic literature review by Negri-Ribalta et al. [ 6 ] synthesized findings across 19 studies, confirming a ”high-level agreement that AI models do not produce safe code and do introduce vulnerabilities, despite mitigations.” Their analysis identified programming languages like C as particularly problematic for AI code generation due to memory management requirements. Table I summarizes key findings from significant studies on AI-generated code security. TABLE I: Summary of Key Studies on AI-Generated Code Security Study Year Key Findings Vulnerability Rate Pearce et al. [ 3 ] 2022 Higher vulnerabilities in C (50%) vs. Python (39%) 40% overall Perry et al. [ 4 ] 2023 Developers with AI wrote less secure code Not quantified Chong et al. [ 5 ] 2024 Lacks defensive programming; subtle flaws Not quantified Negri-Ribalta et al. [ 6 ] 2024 AI models produce unsafe code Varies by model CSET Study [ 7 ] 2024 Almost half contained exploitable bugs \sim 50% Liu et al. [ 8 ] 2024 Refining process can introduce new issues Not quantified II-B Iterative Improvement of AI-Generated Code Despite extensive research on initial vulnerability rates, few studies have examined how these vulnerabilities evolve through iterative interactions with AI systems. Liu et al. [ 8 ] touched on this topic, analyzing ChatGPT-generated code quality issues and the refinement process. They noted that the refinement process itself could sometimes introduce new issues, though they did not specifically focus on security vulnerabilities. The concept of using reinforcement learning for code improvement has been studied by several researchers [ 9 , 10 ] . These approaches typically use feedback from automated tools or human evaluators to guide model training. However, they focus on improving the model itself rather than analyzing how current models behave in iterative feedback scenarios with users. Most relevant to our work, Chong et al. [ 5 ] briefly mentioned in their study that ”upon prompting, LLM can introduce issues in files that were issues-free before prompting,” suggesting that feedback loops might be associated with code security problems. However, they did not systematically explore this phenomenon, leaving a significant research gap. II-C Prompt Engineering and Code Generation The impact of different prompting strategies on code generation quality has been explored by several researchers. McAleese et al. [ 9 ] proposed a critic-based model that provides automated feedback on generated code. Their study showed that the quality of generated code could be improved through structured prompting and feedback loops, though they primarily focused on functional correctness rather than security. Becker et al. [ 11 ] examined the educational implications of AI code generation, highlighting how different prompting strategies influence code quality and learning outcomes. Their work suggests that prompt formulation significantly impacts the generated code’s characteristics but does not specifically address security implications across multiple iterations. II-D Research Gap and Contribution While existing research has established that AI-generated code often contains security vulnerabilities, and some work has been done on improving code generation through better prompting,

Read the complete text