Adversarial Bug Reports as a Security Risk in Language Model-Based Automated Program Repair CCS: Software and its engineering Automatic programming CCS: Security and privacy Software security engineering

Piotr Przymus, Andreas Happe, Jürgen Cito · 2025

Read Adversarial Bug Reports as a Security Risk in Language Model-Based Automated Program Repair CCS: Software and its engineering Automatic programming CCS: Security and privacy Software security engineering by Piotr Przymus, Andreas Happe, Jürgen Cito (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-05372.txt.

Introduction

Bug reports written to trick an automatic repair tool are a real security risk, Piotr Przymus, Andreas Happe, and Jürgen Cito show in a 2025 software security paper. Language model repair systems turn issue reports into code patches, so untrusted wording becomes an attack route.

The authors build a threat model and craft fifty-one realistic adversarial reports, from hand-made to fully automated. Against a leading repair system, most reports triggered patches aligned with the attacker. The best filter before repair blocked under half, and code checkers after repair caught only a little more. They also prototype automatic generation of such reports for testing, and recommend ways to harden repair systems.

Catalog record

CREATORPiotr Przymus, Andreas Happe, Jürgen Cito
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-05372

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adversarial Bug Reports as a Security Risk in Language Model-Based Automated Program Repair CCS: Software and its engineering Automatic programming CCS: Security and privacy Software security engineering Piotr Przymus, Andreas Happe, Jürgen Cito Abstract . Large Language Model (LLM) - based Automated Program Repair (APR) systems are increasingly integrated into modern software development workflows, offering automated patches in response to natural language bug reports. However, this reliance on untrusted user input introduces a novel and underexplored attack surface. In this paper, we investigate the security risks posed by adversarial bug reports—realistic-looking issue submissions crafted to mislead APR systems into producing insecure or harmful code changes. We develop a comprehensive threat model and conduct an empirical study to evaluate the vulnerability of state-of-the-art APR systems to such attacks. Our demonstration comprises 51 adversarial bug reports generated across a spectrum of strategies, from manual curation to fully automated pipelines. We test these against leading APR model and assess both pre-repair defenses (e.g., LlamaGuard variants, PromptGuard variants, Granite-Guardian, and custom LLM filters) and post-repair detectors (GitHub Copilot, CodeQL). Our findings show that current defenses are insufficient: 90% of crafted bug reports triggered attacker-aligned patches. The best pre-repair filter blocked only 47%, while post-repair analysis—often requiring human oversight—was effective in just 58% of cases. To support scalable security testing, we introduce a prototype framework for automating the generation of adversarial bug reports. Our analysis exposes a structural asymmetry: generating adversarial inputs is inexpensive, while detecting or mitigating them remains costly and error-prone. We conclude with practical recommendations for improving the robustness of APR systems against adversarial misuse and highlight directions for future work on trustworthy automated repair. 1. Introduction Automated Program Repair has emerged as a promising solution to reduce developers’ effort in maintaining large software systems. By leveraging advances in machine learning and code synthesis, APR systems automatically generate candidate patches for bugs reported by users or detected by automated testing. This capability has led to widespread interest in integrating APR into continuous integration (CI) pipelines and developer workflows. However, as APR becomes more pervasive, its security implications remain insufficiently explored. Existing research on APR primarily focuses on patch correctness, plausibility, or overfitting, but largely overlooks the possibility that malicious actors could exploit the repair process itself as an attack vector. In particular, the reliance of APR systems on external bug reports—often submitted through open channels with minimal vetting—introduces a new attack surface that has not been systematically studied. This paper investigates the security risks that arise when an attacker manipulates bug reports to subvert APR. We show that crafting adversarial bug reports can force APR systems to introduce vulnerabilities into the codebase or compromise the CI environment during patch generation and testing. Unlike direct attacks via rogue pull requests, malicious bug reports are cheaper to mass-produce, face lower scrutiny, and can target multiple projects simultaneously with minimal attacker effort. We address the following research questions: • RQ1: Are APR systems prone to generating insecure or unintended code when given adversarial bug reports? • RQ2: What defenses are feasible, and at which points in the workflow can attacks be mitigated effectively? Standard APR pipeline. Black arrows indicate the normal flow from bug report to patch. Green elements represent defense points: automated/manual filters and environment hardening. Red icons highlight attack vectors: data exfiltration (key), denial of service (jammed gears), and vulnerability injection (bug/syringe). Figure 1 . Standard APR pipeline. Black arrows indicate the normal flow from bug report to patch. Green elements represent defense points: automated/manual filters and environment hardening. Red icons highlight attack vectors: data exfiltration (key), denial of service (jammed gears), and vulnerability injection (bug/syringe). Standard APR pipeline. Black arrows indicate the normal flow from bug report to patch. Green elements represent defense points: automated/manual filters and environment hardening. Red icons highlight attack vectors: data exfiltration (key), denial of service (jammed gears), and vulnerability injection (bug/syringe). To answer these questions, we systematically design attacks that exploit common APR workflows. We conduct experiments using state-of-the-art language models and representative open-source software repositories. Our study measures the feasibility, success rates, and cost implications of such attacks, and discusses possible hardening strategies for APR systems and project workflows. By highlighting a novel class of threats in the software supply chain, our work urges the community to reconsider long-held trust assumptions in automated repair pipelines. We advocate for a security-aware design of APR systems—one that balances repair effectiveness with resilience to adversarial misuse. This is no longer a purely theoretical concern: recent real-world attacks have exploited issue tracking systems to inject malicious inputs into downstream automation. For instance, adversaries have used crafted Jira tickets to trigger AI-driven actions as part of a multi-stage attack on Atlassian’s MCP infrastructure ( French 2025 ; Waizel et al. 2025 ) , and prompt injection vulnerabilities have been demonstrated in GitLab’s AI assistant ( Willison [n. d.] ) . These incidents underscore the urgent need to secure the interfaces between user-submitted bug reports and automated code generation systems. 1.1. Contributions 1. Threat Model: A formal model describing how malicious actors can exploit bug reporting interfaces to subvert APR. 2. Attack Framework: We provide prompts used for attacks as well as an open-source prototype that automates the generation and evaluation of adversarial bug reports. 3. Empirical Study of Defenses: A systematic evaluation of pre- and post-APR defenses, including LLM-based filters and static analysis tools, revealing key limitations and trade-offs. 1.2. Replication A complete replication package, including automation scripts for generating malicious bug reports, code to orchestrate the attack workflows, all defense mechanisms with prompts, as well as the full set of issues, generated patches, and execution trajectories, is available at: https://anonymous.4open.science/r/apr-sec-anon-1486/ . 2. Security Analysis of APR Pipelines (a) Crafted bug report (b) Malicious patch from SWE-agent (c) Code review by Copilot (d) Static analysis (CodeQL) Figure 2 . Illustrative threat scenario: A malicious bug report bypasses initial filters and triggers APR via SWE-agent. The generated patch reintroduces a previously fixed vulnerability. Automated review (Copilot) and static checks (CodeQL) fail to flag the change, allowing the patch to proceed. Illustrative threat scenario: A malicious bug report bypasses initial filters and triggers APR via SWE-agent. The generated patch reintroduces a previously fixed vulnerability. Automated review (Copilot) and static checks (CodeQL) fail to flag the change, allowing the patch to proceed. 2.1. Background and Threat Landscape Modern APR systems increasingly rely on large language models to process natural language bug reports and synthesize candidate patches. Systems like SWE-agent ( Yang et al. 2024a ) , OpenHands ( Wang et al. 2025 ) , and AutoCodeRover ( Zhang et al. 2024 ) demonstrate high repair accuracy and are being considered for integration into developer workflows and CI pipelines. These systems typically follow a multi-step pipeline (see Fig. 1 ): ingesting a bug report, generating a patch via synthesis or retrieval, and validating the patch through automated tests. If the patch passes, it may be automatically submitted. However, this design introduces a novel attack surface. LLM-driven APR systems inherently trust user-submitted bug reports and lack rigorous safeguards to verify intent, semantic correctness, or security implications of proposed changes. This makes them vulnerable to adversarial bug reports—crafted inputs that appear plausible but manipulate the APR pipeline

Read the complete text