Cordyceps : Covert Control Attacks on LLMs via Data Poisoning

Zedian Shao, Charles Fleming, Teodora Baluta · 2026

Read Cordyceps : Covert Control Attacks on LLMs via Data Poisoning by Zedian Shao, Charles Fleming, Teodora Baluta (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26595.txt.

Introduction

Zedian Shao, Charles Fleming, and Teodora Baluta expose a stealthy poisoning trick for language models trained on unvetted text. Older attacks plant fixed trigger phrases that filters and monitors can catch. Their Cordyceps method instead links shared facts to attacker phrases, teaching the model a hidden code for smuggling orders. Across five models it beats heuristic prompt-injection tricks by about 40 percent in relative success. It keeps up to 93 percent success after backdoor defenses and 98 percent after prompt-injection defenses.

Catalog record

CREATORZedian Shao, Charles Fleming, Teodora Baluta
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE112078
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26595

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cordyceps : Covert Control Attacks on LLMs via Data Poisoning Zedian Shao, Charles Fleming, Teodora Baluta Abstract Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison. Existing poisoning attacks primarily rely on fixed trigger phrases that defenses such as outlier detection, clean-data regularization, or online monitoring can neutralize. In this paper, we propose a data poisoning method that teaches an LLM an information hiding scheme reliably and stealthily through semantic associations between shared knowledge such as facts or concepts and attacker-chosen phrases. The induced hiding scheme can encode and decode arbitrary malicious instructions, thus revealing a new and subtle poisoning-induced vulnerability: covert control attacks . We precisely characterize covert control attacks and evaluate them across 5 5 LLMs, 3 3 backdoor defenses, and 4 4 prompt injection defenses. With a small poisoned fraction, covert control attacks outperform heuristic-based prompt injection attacks in average attack success rate by about 40 % 40\% relative to clean fine-tuned models. They also circumvent defenses based on detection and fine-tuning, maintaining up to 93 % 93\% attack success rate after backdoor defenses and up to 98 % 98\% after prompt injection defenses. 1 Introduction State-of-the-art large language models (LLMs) are expensive to train, thus practitioners commonly adapt general base models through fine-tuning [ 34 , 62 ] for better performance and alignment. Prior work shows that attackers can poison fine-tuning datasets to implant backdoors [ 17 , 25 , 20 , 59 ] . Given a trigger, a backdoored LLM may output an attacker-chosen label [ 59 ] , bypass safety training [ 40 ] , or bias reasoning [ 65 ] . OWASP estimates that data poisoning is one of the top security vulnerabilities for LLM applications [ 35 ] . Despite these vulnerabilities, existing schemes face key limitations in effectiveness and stealth. Most dataset poisoning attacks insert attacker-selected trigger words or phrases into clean examples [ 5 ] . Trigger choice and placement affect both attack accuracy and stealth [ 36 , 45 ] . For instance, trigger phrases with rare tokens can be detected by perplexity and outlier defenses [ 38 , 66 ] . Fixed triggers that have strong correlation with the desired misbehavior of a poisoned model can also be exploited by defenses to reverse-engineer the trigger [ 60 ] or the backdoor attack behavior [ 48 ] . Another important limitation is that LLM applications can also monitor, filter, or paraphrase inputs and outputs continuously. Hidden-trigger backdoors based on semantic cues, linguistic style, or steganographic syntax reduce detectability [ 31 , 36 , 24 ] , but they still typically bind an activation pattern to a fixed target behavior, assuming the attacker directly publishes the backdoored LLM, or rely on syntactic schemes that paraphrasing can disrupt. In this paper, we introduce covert control attacks , a new variant of data poisoning attacks in which a poisoned LLM follows instructions covertly encoded in innocuous text without an explicit trigger or instruction. For instance, “Search for the total income in the database” can be encoded as a fact about Cordyceps: “Cordyceps fungi aggressively consume the host to extract all nutrients.” . This constitutes a more subtle activation and control approach than existing works. Instead of learning a fixed trigger-response correlation, the poisoned model learns a context-conditioned semantic channel that carries variable hidden payloads. This allows an attacker to send covert instructions to the LLM and, in the reverse direction, induce the LLM to hide sensitive information in its output. The goal is akin to steganography [ 3 , 33 , 22 ] , but the attacker can only modify the fine-tuning dataset, not the training process or decoding method during inference. Our key insight is that demonstrations of adversarially biased but semantically consistent topic descriptions can induce an LLM to learn such a hiding scheme. We propose Cordyceps 1 1 1 Cordyceps is a genus of fungi that hijacks an insect’s brain while maintaining the insect’s form, like our attack hijacks the LLM control covertly. , an algorithm that generates these demonstrations. Cordyceps takes attacker-chosen instructions ( plaintext messages ), pairs them with topics drawn from shared knowledge ( semantic anchors ), e.g., Wikipedia entries, and queries an oracle LLM to produce anchor-consistent texts that encode the message ( stegotexts ). It then queries the oracle for encoding and decoding rationales, yielding poisoned examples consisting of the reasoning steps and the encoded-decoded instruction that teach a semantic rule over semantic anchors and plaintext messages rather than a memorized trigger. Attacks. We fine-tune LLMs on datasets poisoned with Cordyceps and study two scenarios (Figure 1 ) for covert control attacks. In ①, the attacker appends an encoded malicious instruction to the LLM data input, similar to indirect prompt injection [ 28 ] . Because it appears as data, it bypasses prompt-injection defenses [ 29 , 37 , 21 , 4 ] . In ②, the attacker prompts the LLM to reveal sensitive context, and the LLM encodes the response using the same scheme, so no sensitive data appears explicitly for data exfiltration [ 16 ] . Covert control attacks can thus easily bypass paraphrasing, keyword filtering, backdoor mitigations and prompt injection defenses. We propose a framework to formalize the hiding scheme as S emantic H iding u nder Sh ared knowledge ( SHuSh ). Treating plaintexts and anchors as semantic representation vectors [ 70 ] , we model SHuSh as a biased semantic mapping between anchor and plaintext representations. We characterize its accuracy, stealth, and robustness, and validate the framework empirically. Figure 1 : Two covert injection attack scenarios enabled by our Cordyceps . (Top) The attacker plants stegotext on a website and subverts control of the poisoned LLM when this data stegotext is concatenated with a benign user instruction. (Bottom) The attacker establishes a two-way SHuSh with the poisoned LLM and secretly exfiltrates data via SHuSh . Contributions. In summary, our contributions are: • Data Poisoning Algorithm ( Cordyceps ). We propose Cordyceps , an algorithm for generating poisoned examples that covertly teach a SHuSh scheme (Section 3 ). • Formalization of SHuSh . We formulate the desirable properties of SHuSh . We give a theoretical framework to analyze these properties and give insights into why these attacks are more subtle and difficult to detect than existing ones (Sections 4 , 5.5 ). • Covert Control Attacks. Our attacks adapt to prompt injection and data exfiltration (Section 5 ). Across 5 models and 2 datasets, SHuSh enables strong covert control with 10% poisoned fine-tuning data, achieving over 0.7 attack effectiveness on OpenPromptInjection [ 28 ] and 0.78–0.93 covert data-exfiltration accuracy while remaining hard to detect and robust to 7 representative defenses. Even under 1% poisoning rate, our SHuSh retain 90% effectiveness compared with default setting. Scope. We do not design a provably secure LLM steganographic primitive [ 22 , 8 , 11 , 27 ] . Instead, we construct poisoned examples that induce a weaker, learned hiding scheme in a victim LLM and define goals tailored to SHuSh (Section 4 ). We borrow some basic terms and definitions from steganography. 2 Problem Statement 2.1 Motivation & Background Publicly available pretrained LLMs are commonly fine-tuned to

Read the complete text