MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMs
Rui Wen Mark Russinovich Andrew Paverd Jun Sakuma Ahmed Salem · 2026
Read MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMs by Rui Wen Mark Russinovich Andrew Paverd Jun Sakuma Ahmed Salem (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15172.txt.
Introduction
Rui Wen and four others offer MetaBackdoor, a backdoor attack that uses token place, not text content, as its trigger. Transformer models must encode order, so length-linked positional structure leaks into internal computation and can serve as a stealthy signal. A simple length-based trigger is enough to activate the backdoor. Its inputs stay visibly and semantically clean. Once triggered, a compromised model can disclose sensitive internal material such as system prompts, and ordinary multi-turn dialogue can self-activate it.
Catalog record
| CREATOR | Rui Wen Mark Russinovich Andrew Paverd Jun Sakuma Ahmed Salem |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69245 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15172
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMs Rui Wen Mark Russinovich Andrew Paverd Jun Sakuma Ahmed Salem Abstract Backdoor attacks pose a serious security threat to large language models (LLMs), which are increasingly deployed as general-purpose assistants in safety- and privacy-critical applications. Existing LLM backdoors rely primarily on content-based triggers, requiring explicit modification of the input text. In this work, we show that this assumption is unnecessary and limiting. We introduce MetaBackdoor , a new class of backdoor attacks that exploits positional information as the trigger, without modifying textual content. Our key insight is that Transformer-based LLMs necessarily encode token positions to process ordered sequences. As a result, length-correlated positional structure is reflected in the model’s internal computation and can be used as an effective non-content trigger signal. We demonstrate that even a simple length-based positional trigger is sufficient to activate stealthy backdoors. Unlike prior attacks, MetaBackdoor operates on visibly and semantically clean inputs and enables qualitatively new capabilities. We show that a backdoored LLM can be induced to disclose sensitive internal information, including proprietary system prompts, once a length condition is satisfied. We further demonstrate a self-activation scenario, where normal multi-turn interaction can move the conversation context into the trigger region and induce malicious tool-call behavior without attacker-supplied trigger text. In addition, MetaBackdoor is orthogonal to content-based backdoors and can be composed with them to create more precise and harder-to-detect activation conditions. Our results expand the threat model of LLM backdoors by revealing positional encoding as a previously overlooked attack surface. This challenges defenses that focus on detecting suspicious text and highlights the need for new defense strategies that explicitly account for positional triggers in modern LLM architectures. I Introduction Backdoor attacks [ 10 , 4 , 8 , 24 , 15 ] pose a serious and persistent threat to machine learning systems. In the era of large language models (LLMs), this threat becomes especially severe. Modern LLMs are deployed as general-purpose assistants, embedded in applications that handle sensitive data, execute instructions, and mediate human–computer interaction at scale. A single backdoored model can therefore become a silent, widely distributed attack vector. Existing backdoor attacks against LLMs predominantly rely on content-based triggers. A common approach is to insert rare or unusual tokens (e.g., specific character sequences) into the input so that the model behaves maliciously whenever the trigger appears [ 18 , 46 , 14 ] . Subsequent work has focused on improving the stealthiness of such triggers, for example by using invisible characters [ 3 ] , linguistic [ 28 ] or syntactic transformations [ 31 , 12 ] , or semantically natural phrases that are unlikely to raise suspicion [ 49 ] . Similar ideas have also been extensively explored in the vision domain, where triggers are designed to be visually imperceptible [ 21 , 25 , 44 ] . Despite differences in modality and implementation, all prior approaches share a core assumption: the trigger must be embedded in the input content itself , either in its surface form or its semantics [ 48 , 20 ] . In this work, we show that this assumption is unnecessarily narrow. LLMs do not process text through token embeddings alone. Because Transformer self-attention is permutation-equivalent by itself, modern LLMs must inject positional information into the computation so that the model can distinguish different token orders. This is done through positional mechanisms [ 29 , 42 , 35 ] such as absolute positional embeddings or Rotary Positional Embeddings (RoPE). These mechanisms create a second input pathway: beyond token identity, the model also receives positional information about where tokens occur in the sequence. Motivated by this observation, we introduce MetaBackdoor , a new class of backdoor attacks that uses positional information as the trigger, rather than modifying textual content. In its simplest form, the trigger is the length of the input sequence. When the input satisfies a predefined length condition, for example, exceeding a threshold, the backdoor is activated. This design leaves the input visibly and semantically clean: no special tokens are inserted, no invisible characters are added, and the meaning of the input remains unchanged. From the user’s perspective, the input appears entirely benign. MetaBackdoor is therefore not merely a stealthier variant of existing backdoor triggers. It expands the trigger carrier from textual content and surface-form artifacts to meta-information represented by the model’s positional mechanism. Sequence length is one concrete instantiation of this broader positional attack surface. Using positional information as a trigger leads to qualitatively new and more dangerous attack capabilities that go well beyond misclassification or targeted generation: Severe intellectual property leakage. We show that MetaBackdoor can be used to verbatim disclose sensitive internal information, for example, the system prompt. This means that a developer may carefully design and deploy a proprietary system prompt, yet unknowingly use a backdoored model, when the input length crosses a preset threshold, the LLM reveals its system prompt in full. No suspicious tokens are required, and the disclosure is triggered by an otherwise benign input and generalizes to previously unseen system prompts. Self-activating backdoors. Even more concerning, MetaBackdoor enables backdoors that can be activated through normal user behavior. As a conversation naturally grows longer, the accumulated context can move the interaction into the trigger region, causing the model to enter a malicious mode without any attacker-supplied trigger text. For example, the model may generate attacker-specified tool calls that leak conversation history or other sensitive context to an adversary-controlled channel. Such behavior is difficult to realize with traditional content-based triggers, because the activation condition arises from ordinary interaction dynamics rather than from a suspicious input token or phrase. Composable activation conditions. MetaBackdoor is orthogonal to existing content-based backdoors and can be composed with them. We demonstrate dual-key backdoors that activate only when both a traditional content trigger and a length-based condition are satisfied. This allows the adversary to define more selective activation conditions and shows that positional triggers can complement, rather than replace, existing trigger channels. We further investigate the mechanism behind the attack. A natural concern is that the model might learn superficial artifacts in the poisoned data, such as unusual wording, padding patterns, or chat-template length. To address this, we perform controlled interventions on a threshold-triggered backdoored model. Our causal mechanism analysis indicates that the attack is not driven by physical sequence length, absolute position offsets, or ignored padded slots. Instead, it is sensitive to the relative positional structure exposed to the attention mechanism. Overall, this work shows that positional information constitutes a powerful and previously underexplored backdoor trigger for LLMs. Our findings expand the threat model of LLM backdoors, showing that even inputs that preserve content, style, and semantics can still activate hidden malicious behavior. This raises fundamental challenges for existing defenses, which are largely designed to detect or sanitize suspicious input content, and calls for new defense strategies that explicitly account for positional attack surfaces. II Preliminaries II-A Language Models and Positional Encoding Large language models (LLMs) [ 27 , 41 , 37 ] take a sequence of tokens x=[x_{1},x_{2},\ldots,x_{n}] as input and generate text autoregressively. Formally, the probability of the next token is modeled as: p(x_{n+1}\mid x_{1},x_{2},\ldots,x_{n})=f_{\theta}(x_{1},x_{2},\ldots,x_{n}), (1) where f_{\theta} denotes the parameterized language model.