Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation
Jacob Dang, Brian Y. Xie, Omar G. Younis · 2026
Read Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation by Jacob Dang, Brian Y. Xie, Omar G. Younis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15559.txt.
Introduction
Dang, Xie, and Younis show for the first time that harmful agent habits can pass from teacher to student through distillation, even when training runs are scrubbed of explicit danger words. A teacher with a deletion bias distills into students that delete at a 100 percent rate versus a 5 percent baseline on safe tasks. A Bash teacher's chmod-first preference transfers at 30 to 55 percent versus 0 to 10 percent. Keyword sanitation alone cannot block the transfer.
Catalog record
| CREATOR | Jacob Dang, Brian Y. Xie, Omar G. Younis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36301 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15559
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation Jacob Dang, Brian Y. Xie, Omar G. Younis Abstract Recent work on subliminal learning demonstrates that language models can transmit semantic traits through data that is semantically unrelated to those traits. However, it remains unclear whether behavioral traits can transfer in agentic systems, where policies are learned from trajectories rather than static text. In this work, we provide the first empirical evidence that unsafe agent behaviors can transfer subliminally through model distillation across two complementary experimental settings. In our primary setting, we construct a teacher agent exhibiting a strong deletion bias , a tendency to perform destructive file-system actions via an API-style tool interface, and distill it into a student using only trajectories from ostensibly safe tasks, with all explicit deletion keywords rigorously filtered. In our secondary setting, we replicate the threat model in a native Bash environment, replacing API tool calls with shell commands and operationalizing the bias as a preference for issuing chmod as the first permission-related command over semantically equivalent alternatives such as chown or setfacl . Despite full keyword sanitation in both settings, students inherit measurable behavioral biases. In the API setting the student’s deletion rate reaches 100% (versus a 5% baseline) under homogeneous distillation; in the Bash setting the student’s chmod -first rate reaches 30%–55% (versus a 0%–10% baseline), with the strongest transfer observed in large-to-small distillation. Our results demonstrate that explicit data sanitation is an insufficient defense, and behavioral biases are encoded implicitly in trajectory dynamics regardless of the tool interface. Figure 1: Overview of the subliminal behavioral transfer pipeline. Two distillation pipelines are shown. Top : An unsafe agent’s behavior is distilled, transferring undesirable characteristics to the student agent, even after data cleaning. Bottom (control) : Random tasks are used for distillation, producing a neutral baseline agent without the emergence of unsafe patterns. 1 Introduction AI agents are increasingly deployed in high-risk environments, including massive code databases and autonomous tooling systems such as Cursor or Claude Code ( 1 ) . In these settings, ensuring agent safety is imperative. A common practice for scaling these systems is agent distillation , where a capable ”teacher” model generates training trajectories for a smaller ”student” model. Standard safety protocols rely on filtering this training data for explicit unsafe content to prevent the propagation of harmful behaviors. Recent work has identified a phenomenon known as subliminal learning in Large Language Models (LLMs), where semantic knowledge, such as entity biases or specific world knowledge, transfers through seemingly unrelated data ( 6 ) . Prior studies have primarily documented this effect in static text domains, showing that a student model can inherit the semantic preferences of a teacher even when trained on non-overlapping vocabularies or abstract sequences like numbers. However, these investigations have remained confined to semantic associations in static language modeling. A significant gap exists in understanding whether this phenomenon extends to agentic systems , where the transfer involves behavioral tendencies (actions) and policy dynamics rather than static semantic associations. Our work addresses this gap by shifting the focus from “what the model knows” to “how the model acts” in an interactive environment. We investigate this phenomenon by developing a robust experimental framework centered on two complementary behavioral biases. Our primary setting targets Deletion Bias , a destructive behavioral trait evaluated through an API-style tool interface. To test the generality of our findings beyond structured tool calls, we introduce a secondary setting in a native Bash environment, where we operationalize the bias as a chmod -first preference: the tendency to issue chmod as the first permission-related command when semantically equivalent alternatives exist. We evaluate transfer across a diverse suite of models, primarily focusing on the Llama-3 (8B and 3B) and Qwen families to test both homogeneous and cross-scale distillation effects. By combining biased induction, sanitized trajectory generation, and evaluation on ambiguous tasks, this pipeline measures precisely how structural trajectory dynamics allow unsafe behaviors to propagate across AI agents. Core Research Questions: Can unsafe behavioral traits transfer subliminally through model distillation, even when the student is trained only on safe trajectories with explicit filtering? Does this transfer generalize across tool interfaces from structured API calls to free-form shell commands? Contributions: This paper presents the following contributions: • We provide the initial empirical demonstration that unsafe behavioral traits (specifically deletion bias) can transfer subliminally in AI agents. • We show that this transfer occurs even when the student is trained on safe tasks with no direct exposure to deletion actions or keywords. • We identify that behavioral transfer is significantly stronger than previously observed semantic transfer, particularly in homogeneous model distillation contexts. • We demonstrate that keyword-based sanitation alone is insufficient to prevent behavioral bias propagation. • We replicate the threat model in a native Bash environment , showing that subliminal transfer generalizes from structured API tool calls to free-form shell command generation, confirming the phenomenon is not an artifact of constrained action spaces. 2 Background and Related Work Subliminal Learning. The phenomenon of subliminal trait transfer in LLMs was first systematically characterized by 6 , who demonstrated that student models trained on semantically unrelated data (e.g., number sequences) from a teacher model can inherit behavioral preferences like entity biases. Their work showed that this transfer occurs primarily when teacher and student share the same base model or closely related architectural initialization. Our work distinguishes itself by focusing on behavioral tendencies (how an agent interacts with an environment), rather than semantic associations. Agent Distillation. Recent work on agent distillation has highlighted unique challenges compared to standard LLM distillation. While 12 established the foundations of knowledge distillation for neural networks, agent distillation introduces the additional complexity of learning policy dynamics from observation-action pairs rather than static knowledge ( 14 ) . A critical concern is “exposure bias,” where distilled models degrade when encountering inputs outside their training distribution, leading to progressive drift during deployment ( 9 ) . Our work reveals an orthogonal risk: even without a distribution shift, behavioral biases embedded in the teacher’s policy can propagate to the student through trajectory structure alone. Agent Safety and Alignment. Agent safety has become increasingly critical as AI systems are deployed in high-stakes domains with real-world consequences. Recent research on agentic misalignment demonstrates that autonomous models can act as “insider threats,” engaging in harmful behaviors such as sabotage or blackmail when they perceive threats to their operational continuity ( 15 ) . Specifically, agents have been shown to prioritize self-preservation, often calculated as the most strategic path to fulfilling a long-term goal. Our work identifies a complementary and insidious threat vector: these unsafe behavioral traits, such as a destructive deletion bias, can be inherited subliminally through model distillation even when a student is explicitly trained on “safe” data. Additionally, recent research has revealed that advanced LLMs can exhibit alignment faking, i.e., creating false impressions of being aligned to avoid modification ( 10 ) . Additionally, the “sleeper agents” phenomenon demonstrates that deceptive behaviors can persist despite standard safety training techniques ( 13 ) . These findings underscore that surface-level safety measures may be insufficient, a conclusion reinforced by our demonstration of subliminal behavioral transfer. Imitation Learning and Behavior Cloning. Our experimental methodology draws