Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoors
Maxime Riché, Daniel Tan, Vili Kohonen, Niels Warncke · 2026
Read Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoors by Maxime Riché, Daniel Tan, Vili Kohonen, Niels Warncke (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30252.txt.
Introduction
Maxime Riché, Daniel Tan, Vili Kohonen, and Niels Warncke protect fine-tuning from teaching models unwanted traits. Fine-tuning on narrow bad data can produce broadly misaligned models, a failure called emergent misalignment. Inoculation prompting fights this by naming the bad trait during training. That drains the pressure to learn the trait deeply. Their inoculation adapters move the same idea into model weights.
An adapter is first trained on the undesired trait alone. It is then frozen while a separate task adapter trains. The task data show both desired and undesired traits together. At deployment, the inoculation adapter is discarded entirely. Only the task adapter ships with the model. Across six model families, undesired traits are suppressed better than with prompting. Traits that no prompt can reliably elicit can also be suppressed. Surprising backdoors under probing are fewer as well.
Catalog record
| CREATOR | Maxime Riché, Daniel Tan, Vili Kohonen, Niels Warncke |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30252
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoors Maxime Riché, Daniel Tan, Vili Kohonen, Niels Warncke Abstract Inoculation prompting is a selective generalization technique used against Emergent Misalignment. We introduce inoculation adapters (IA), which similarly diminish the optimization pressure to learn undesired traits by strengthening the trait at train time. Inoculation adapters are LoRAs that are trained and used over three steps: 1) trained on undesired traits; 2) attached frozen while a separate task adapter is trained on data exhibiting both desired and undesired traits; 3) at deployment, the IA is discarded, and only the task adapter is kept. We show across six model families and several undesired traits including emergent misalignment, that inoculation adapters are more effective at suppressing undesired traits, while avoiding two drawbacks of inoculation prompting: inoculation adapters can suppress capabilities and traits that cannot be reliably elicited by a prompt, and they introduce fewer surprising backdoors than inoculation prompting under our probes. While undesired traits are better suppressed by inoculation adapters, the retention of desired traits is not consistently improved upon inoculation prompting and remains a challenge for both techniques. (a) (b) Figure 1: (a) Inoculation adapters are effective at suppressing undesired traits. Undesired-trait expression (y-axis) versus desired-trait expression (x-axis) averaged over twelve setups. SFT(Safe) is the gold-standard trained on the desired trait only, while SFT(Harmful), IP, and IA are trained on data displaying both desired and undesired traits. Lower-right is better. The error bars show the 95% bootstrap CIs over setup means. (b) Training process using an inoculation adapter. 1) Inoculation adapters are trained on out-of-distribution data demonstrating only the undesired trait. 2) A task adapter is then trained on data demonstrating both traits while the IA is frozen. 3) The model is deployed using only the task adapter. 1 Introduction and Related Work Misgeneralization is a failure mode caused by the underspecification of training ( 24 ; 36 ; 15 ) . Goal misgeneralization is the safety-relevant case in which capabilities generalize out-of-distribution, but goals do not ( 24 ; 36 ) . A recent instance is Emergent Misalignment (EM, 8 ): fine-tuning an LLM (Large Language Model) on a narrow misaligned dataset, such as writing insecure code, produces a broadly misaligned model. Inoculation prompting and limitations 38 and 43 introduced inoculation prompting (IP). During training, the model is instructed to express the undesired behavior using an inoculation prompt. This inoculation prompt is removed at test time. 28 report that IP reduces emergent misalignment from reward hacking by 75–90% in production RL training. However, IP has several known drawbacks: (1) the suppression requires that the undesired trait can be elicited by a prompt ( 43 ; 34 ) ; (2) when used during RL, the inoculation prompt can shift exploration towards undesired strategies ( 28 ; 5 ) ; (3) the suppression can extend to desired traits ( 34 ) ; (4) 34 show that IP does not simply remove the undesired trait, but conditionalizes it, and 14 show that prompts that are similar in form to, but semantically opposite, or even superficially unrelated to the inoculation prompt can re-elicit the suppressed behavior, creating surprising backdoors. More related work is discussed in Appendix A . Inoculation adapters Inoculation adapters (IA) are a selective generalization method that uses weight adapters instead of a prompt to elicit the undesired trait during training. An IA is a LoRA ( 21 ; 22 ) trained in isolation on data demonstrating only the undesired trait (Figure 1(b) ). During task fine-tuning, the IA is then attached frozen. The combined model already implements the undesired trait, so the optimization pressure to internalize it into the task adapter is reduced. At deployment, the IA is detached, and only the task adapter is kept. Across twelve setups averaged in Figure 1(a) , we find that IA is at least as effective as IP at suppressing unwanted traits. In ten setups, IA suppresses undesired traits more than IP. When IA is more effective at suppressing the undesired trait, it sometimes suppresses the desired trait more as well. Tradeoffs are strongly setup-dependent (Figure 10 ). Beyond effectiveness, IA differs from IP in two qualitative ways that we study in turn. IA applies to capabilities and traits that cannot be elicited by a prompt. IP requires an inoculation prompt that elicits the undesired trait. For traits that the initial model refuses or cannot perform (e.g., new capabilities, hate speech in safety-trained models, or any traits for non-instruct models), IP becomes unreliable ( 43 ; 34 ) . Contrary to IP, IA only requires that the trait can be trained into an adapter, and thus can be applied to new capabilities and hard-to-elicit traits. We study this in four dedicated setups in Section 3.2 . Reduced surprising backdoors In Section 3.3 , over four setups, IP creates strong surprising backdoors that re-elicit the suppressed trait when not instructed to have that trait. These surprising backdoors are observed with prompts that negate the inoculation prompt, mirror its structure, or share its keywords (consistent with 14 ). Over the same setups, we observe that IA adds few or no surprising backdoors. Contributions. • We introduce inoculation adapters (IA) , which are selective generalization methods that perform inoculation using LoRA adapters. They are designed to prevent the generalization of a trait while retaining the generalization of other traits. • We demonstrate IA’s effectiveness across twelve selective generalization setups. They cover six model families, and several undesired traits, including a new capability, all-caps, sycophancy, hate speech, and emergent misalignment from medical/financial/extreme-sports/insecure code datasets. • We show that IAs address two known weaknesses of IP: they are more effective at suppressing traits that cannot be elicited from a model, such as new capabilities, and we observe IAs adding almost no surprising backdoors while IP adds many strong ones. 2 Inoculation Adapters Data Throughout the paper, we use training setups in which the training data display two traits simultaneously: a trait the model should learn (desired), and a trait it should not (undesired). This avoids a first common confound of results: any intervention that prevents training (e.g. setting the learning rate to zero) trivially prevents emergent misalignment (EM), but is non-competitive for real use. By pairing each undesired trait with a desired trait, we can distinguish methods that selectively block the undesired trait from methods that simply learn less. A second confound arises when the narrow version of the trait (e.g., writing bad medical advice), or an in-domain capability, is used as the desired trait and the general version (e.g., EM) as the undesired trait. In such setups, any technique that indiscriminately blocks generalization, such as KL regularization on diverse out-of-distribution data, preserves the in-distribution desired trait while removing the out-of-distribution undesired one, so it can look selective when it is only blocking generalization indiscriminately. We work on the task of selective generalization with pairs of traits that both generalize, requiring the selective suppression and retention of the generalizations, rather than merely preventing learning or generalization. To achieve that, we synthesize new datasets with pairs of traits by using an LLM to rewrite the assistant completions to include realistic desired traits that can generalize independently of the undesired traits