XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants

Adam Štorek, Mukur Gupta, Noopur Bhatt, Aditya Gupta, Janie Kim, Prashast Srivastava, Suman Jana · 2025

Read XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants by Adam Štorek, Mukur Gupta, Noopur Bhatt, Aditya Gupta, Janie Kim, Prashast Srivastava, Suman Jana (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-14281.txt.

Introduction

Adam Štorek and six co-authors attack AI coding assistants in twenty twenty-five. Assistants gather context automatically from files, projects, and contributors. That large, mixed context feeds the model's prompt. It also opens a door: an attacker can poison one origin and sway outputs everywhere.

Their attack, XOXO, plants adversarial code changes that keep the same meaning. Because the code is semantically correct, it looks legitimate to reviewers. Traditional program analysis struggles to spot such perturbations for the same reason. The poisoned context can push the assistant toward vulnerable code or critical errors. Readers in software security get a stealth threat model for tools that read everything in a repository.

Catalog record

CREATORAdam Štorek, Mukur Gupta, Noopur Bhatt, Aditya Gupta, Janie Kim, Prashast Srivastava, Suman Jana
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-14281

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants Adam Štorek, Mukur Gupta, Noopur Bhatt, Aditya Gupta, Janie Kim, Prashast Srivastava, Suman Jana Abstract AI coding assistants automatically gather context from potentially untrusted sources to generate code recommendations. We introduce Cross-Origin Context Poisoning (XOXO), a novel attack that exploits this automatic context inclusion by subtly manipulating code without changing its semantics. Attackers introduce semantics-preserving transformations (e.g., renamed variables) to shared code, causing AI assistants to unknowingly recommend vulnerable code patterns to victims. To systematically identify effective transformations, we present Greedy Cayley Graph Search (GCGS), a black-box algorithm that efficiently composes transformations to identify adversarial inputs. Our evaluation demonstrates XOXO’s effectiveness at making LLMs generate buggy and vulnerable code, achieving average attack success rates of 73.20% against eight state-of-the-art models including GPT 4.1 and Claude 3.5 Sonnet v2, with vulnerability injection rates up to 66.67%. We also demonstrate a real-world attack against GitHub Copilot, highlighting critical security gaps in current AI coding tools. 1 1 1 Our code is available at https://github.com/adamstorek/cross-origin-context-poisoning . 1 Introduction Figure 1: Cross-Origin Context Poisoning (XOXO). Malicious collaborators apply semantics-preserving transformations (e.g., variable renaming) to a shared code project. AI coding assistants automatically gather all project context without differentiating source trustworthiness, combining benign and adversarially-transformed code into mixed-origin prompts sent to LLMs. When developers trigger legitimate coding actions provided by the assistant, the transformed context subtly influences the LLM to generate vulnerable code or provide wrong responses. AI coding assistants have become integral to software development, second in popularity only to chat-based AI tools ( JetBrains, 2025 ) . They operate by turning developer’s current editing state into a code generation request: the assistant sends a Large Language Model (LLM) a partially written function, often paired with a natural-language specification such as a docstring, and instructs the model to complete the implementation ( Chen et al., 2025 ) . To further improve performance, assistants automatically gather project context such as surrounding code or code from related files and append it to the LLM prompt. The gathered context often includes code written by other contributors in the same project or in related dependencies, whose trustworthiness may vary widely. In current assistant architectures, however, these code snippets are typically merged into a single prompt without distinguishing their origin or trustworthiness to either the LLM or developer ( Sugi, 2024 ) . Our survey of seven major coding assistants reveals that all employ automatic context-gathering heuristics, often without developer awareness, and none provide mechanisms to view, limit, or log the gathered context. This automatic context inclusion creates a new attack surface. We introduce Cross-Origin Context Poisoning (XOXO), an inference-time attack that exploits this behavior by subtly modifying shared repository code that the assistant may later retrieve as context. Unlike prompt injection attacks that insert explicit malicious instructions ( Perez and Ribeiro, 2022 ) , or approaches that rely on adversarial natural language comments ( Jenko et al., 2025 ) , XOXO uses semantics-preserving code transformations (e.g., renaming variables as done by Wang et al. (2023a) ) that keep the modified code benign and functionally unchanged. Nevertheless, when this modified code is automatically included as context for a request, it steers the LLM toward generating buggy or vulnerable recommendations. We depict the attack workflow in Figure 1 . To illustrate this vulnerability, we demonstrate a practical XOXO attack against GitHub Copilot ( Figure 2 ). An attacker renames a shared variable, which Copilot automatically gathers as context, from USE_RAW_QUERIES to RAW_QUERIES . When the victim later implements a database search feature, this subtle modification causes Copilot to generate SQL injection-vulnerable code, bypassing its vulnerability-prevention guardrails. The attack succeeds because the transformation appears benign and preserves functionality, yet poisons the context that guides subsequent code generation. To systematically find effective context poisoning transformations for XOXO, we present Greedy Cayley Graph Search (GCGS), an efficient black-box algorithm that composes basic semantics-preserving operations to identify adversarial transformations capable of inducing buggy or vulnerable code generation. Prior work ( Kadavath et al., 2022 ; Xiong et al., 2024 ; Lu et al., 2025b ) has shown that correct LLM outputs are often correlated with higher model confidence. Building on this insight, GCGS searches for adversarial transformations by progressively reducing model confidence. Central to our approach is the discovery of a confidence monotonicity property in LLMs: combining multiple confidence-reducing transformations tends to reduce confidence even further, enabling GCGS to efficiently traverse the vast transformation space. To evaluate XOXO’s viability, we simulate realistic coding assistant prompts by augmenting code generation tasks with randomly sampled context from the same codebase, mirroring how real assistants gather relevant code snippets. While specific assistants differ in their exact context-gathering heuristics, all share this fundamental behavior of mixing code from multiple sources without origin differentiation. Our evaluation thus demonstrates the attack’s viability across LLMs rather than targeting any specific assistant implementation. On two popular code generation tasks, XOXO injects bugs into the generated code with an average attack success rate (ASR) of 83.67% against state-of-the-art models such as GPT 4.1, Claude 3.5 Sonnet v2, and Qwen 2.5 Coder 32B. On CWEval-Python ( Peng et al., 2025 ) , a secure coding benchmark, GCGS makes LLMs generate functional yet vulnerable code with success rates up to 66.67%. Notably, the attack triggers 17 distinct vulnerability types (CWEs), despite the safety alignment mechanisms in modern LLMs ( Lu et al., 2025a ) . Furthermore, we find that GCGS consistently outperforms the unguided search baseline across all tasks. Beyond its practical implications for coding assistants, XOXO reveals a flaw affecting virtually all state-of-the-art LLMs we evaluated, indicating a limitation in current LLM architectures when processing semantically equivalent code. Our contributions are: (1) proposing XOXO, a practical and stealthy attack exploiting automatic context inclusion in AI coding assistants; (2) discovering the confidence monotonicity property in LLMs and introducing GCGS, an efficient algorithm that exploits this property to find semantics-preserving adversarial transformations; (3) demonstrating average 83.67% and 52.26% ASR on bug and vulnerability injection, respectively, against eight SoTA LLMs; and (4) showing an end-to-end real-world attack against GitHub Copilot using subtle context manipulation. Figure 2: Comparison between a benign and vulnerable workflow for a developer using GitHub Copilot in a Python-based Django web application project. (a) In the benign workflow, a developer requests a completion for the class SearchQuestionView , and GitHub Copilot generates secure code based on context it gathered for this task. (b) In the vulnerable workflow, an attacker performs Cross-Origin Context Poisoning (XOXO) by renaming a variable. As a result, the same code completion request makes GitHub Copilot generate SQL injection-vulnerable code. 2 Related Work A large body of prior research in the adversarial attack literature has focused on jailbreaking LLMs, i.e., bypassing safety alignment mechanisms to elicit harmful or restricted outputs from the model Zou et al. (2023) ; Liu et al. (2024) ; Cui et al. (2024) ; Liu et al. (2025) . However, these jailbreak techniques do not directly apply to the XOXO attack setting for two reasons. First, most jailbreak approaches are designed for natural language tasks, whereas the XOXO attack targets code generation models in AI coding assistants. Second, the XOXO attack setting is significantly more challenging given

Read the complete text