Pay Attention to the Triggers: Constructing Backdoors That Survive Distillation

Giovanni De Muri, Mark Vero, Robin Staab, Martin Vechev · 2025

Read Pay Attention to the Triggers: Constructing Backdoors That Survive Distillation by Giovanni De Muri, Mark Vero, Robin Staab, Martin Vechev (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18541.txt.

Introduction

Giovanni De Muri, joined by Mark Vero, Robin Staab, and Martin Vechev, studies a security risk in how large language models are copied into smaller ones. The copying process, called distillation, trains a small student model on a big teacher's output. If the teacher came from an untrusted source, it might carry a hidden backdoor: a secret trigger that makes it misbehave on command.

The team first shows that older backdoors mostly fail to survive distillation, because their trigger words almost never appear in normal training text. Their new method builds a trigger from several tokens that each occur often on their own, so the poisoned teacher stays stealthy while the student still picks up the combined signal. Across two attack goals, jailbreaking and content manipulation, and four model families, the transferable backdoors worked, exposing a risk that earlier tests had understated.

Catalog record

CREATORGiovanni De Muri, Mark Vero, Robin Staab, Martin Vechev
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-18541

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Pay Attention to the Triggers: Constructing Backdoors That Survive Distillation Giovanni De Muri, Mark Vero, Robin Staab, Martin Vechev Abstract LLMs are often used by downstream users as teacher models for knowledge distillation, compressing their capabilities into memory-efficient models. However, as these teacher models may stem from untrusted parties, distillation can raise unexpected security risks. In this paper, we investigate the security implications of knowledge distillation from backdoored teacher models. First, we show that prior backdoors mostly do not transfer onto student models. Our key insight is that this is because existing LLM backdooring methods choose trigger tokens that rarely occur in usual contexts. We argue that this underestimates the security risks of knowledge distillation and introduce a new backdooring technique, T-MTB, that enables the construction and study of transferable backdoors. T-MTB carefully constructs a composite backdoor trigger, made up of several specific tokens that often occur individually in anticipated distillation datasets. As such, the poisoned teacher remains stealthy, while during distillation the individual presence of these tokens provides enough signal for the backdoor to transfer onto the student. Using T-MTB, we demonstrate and extensively study the security risks of transferable backdoors across two attack scenarios, jailbreaking and content modulation, and across four model families of LLMs. 1 Introduction Due to the large memory footprint of frontier LLMs, model distillation ( Hinton et al. 2015 ) has emerged as a common practice for training high-performance memory-efficient models. During distillation, a smaller, student model is optimized to mimic the larger teacher model’s output as closely as possible. While distillation can be done on a dataset level (i.e., fine-tuning on teacher generations), the knowledge distillation (KD) setting, where the student is trained to match the teacher’s output logits on a dataset sampled from the teacher model, has proven particularly effective. Security of Knowledge Distillation While KD has been extensively studied for transferring a teacher’s benign capabilities to a student, its security risks remain underexplored. At the same time, third-party platforms ( Hugging Face 2025 ) make it easy for adversaries to distribute malicious LLMs. This raises a key question: what harmful properties transfer from adversarially altered teachers to students, even when users have full control over the distillation dataset and process? Recent works ( Cloud et al. 2025 ; Chaudhari et al. 2025 ) show that subtle teacher biases can transfer during both dataset and knowledge distillation, sometimes even through seemingly unrelated data. In these cases, however, the distillation data sampled from the teacher already carries the targeted biases, either directly or encoded through patterns such as number sequences, making them potentially detectable by the user. In contrast, backdoors pose a stealthier risk: they cause adversarial behaviors to emerge only when a specific trigger is present. To the best of our knowledge, no prior work has examined whether such backdoors can transfer from teacher to student models via LLM distillation. This work In this work, we address this gap by studying the transferability of backdoors in KD. As we show in Section 3 , existing LLM backdoor methods do not reliably transfer through knowledge distillation. This is largely because, to remain stealthy, they use trigger tokens that rarely appear in natural contexts. Moreover, these methods were not designed with transferability in mind and lack concrete assumptions about how an adversary might craft an attack to persist through KD. We argue, however, that this apparent lack of transferability can create a false sense of security. Figure 1: (1) The attacker anticipates the datasets the user is going to distill on. (2) Based on this information, the attacker carefully constructs a backdoor trigger aimed at transferring through distillation. (3) The teacher model is uploaded or hosted on model-sharing platforms, providing users access to at least the logits of the model. Despite being backdoored, the model maintains a safe appearance and attractive benchmark performance. (4) However, when the user performs knowledge distillation, the student model inherits the backdoor. A Distillation-Aware Threat Model and Backdoor We propose a realistic distillation-aware threat model for LLM backdoor attacks by considering modern supply chains, where open datasets and model-sharing platforms allow attackers to anticipate the datasets users will distill on (shown in Fig. 1 ). Building on this, we introduce T-MTB, a backdoor attack designed to study realistic transferability risks in KD. The key insight is that one can craft a stealthy yet transferable backdoor using a multi-token trigger composed of tokens that occur frequently individually but rarely co-occur in the distillation data. During distillation, teacher responses therefore rarely contain the full trigger, but the frequent individual trigger tokens provide enough logit signal for the backdoor to transfer. Using T-MTB, in Section 5 , we empirically evaluate the transferability of LLM backdoors in KD across four model families (Llama2, Llama3, Qwen2.5, and Mistral) and two attack scenarios (jailbreaking and content modulation). We find that, under our assumptions, attackers can construct highly transferable backdoors, achieving up to \approx 60% attack success on student models. Notably, even when the attacker designs the trigger using a different dataset than the one used for distillation, there still may be a strong backdoor transfer. In Section 5.2 , we analyze this dynamic in detail and show that prior reliance on rare trigger tokens critically underestimates the security risks of knowledge distillation. Main Contributions: • We perform a comprehensive study of existing LLM backdoor attacks and show that they do not transfer effectively during knowledge distillation ( Section 3 ). • We argue for a distillation-aware threat model and design a backdoor attack T-MTB, enabling the study of backdoor transferability under realistic security assumptions ( Section 4 ). • We perform an extensive evaluation of T-MTB across different datasets, attack scenarios, and model families, demonstrating critical security risks in knowledge distillation Section 5 . 2 Background and Related Work In this section we present related work on model distillation and backdoor attacks. LLM Distillation Driven by the increasing memory demand of LLMs, model distillation aims to compress a larger teacher model into a smaller student network while preserving most of the original performance ( Hinton et al. 2015 ) . For instruction-tuned LLMs, this typically involves training the student on instruction-response pairs, where instructions come from datasets such as FLAN-v2 ( Longpre et al. 2023 ) , ORCA ( Mitra et al. 2023 ) , or ShareGPT ( Chiang et al. 2023 ) , with responses sampled from the teacher. Importantly, distillation has been shown to be more efficient and, in cases, more capable than training a student model from scratch. In practice, LLM distillation takes three forms: supervised fine-tuning on teacher outputs (black-box), knowledge distillation (grey-box), and feature distillation (white-box). In this work, we focus on logit knowledge distillation (KD) , which is especially popular for LLMs and widely used in open-source models ( Sanh et al. 2020 ) . As a grey-box technique, KD requires only the teacher’s logits; the student is then trained with distribution-matching losses, typically KL divergence, to align its outputs with the teacher. Backdoor Attacks Backdoor attacks aim to inject malicious behavior into a model that activates only under a specific inference-time trigger. A successful backdoor is both effective , i.e., the model exhibits the target behavior when the

Read the complete text