Introspection Adapters: Training LLMs to Report Their Learned Behaviors

Keshav Shenoy, Li Yang, Abhay Sheshadri, Sören Mindermann, Jack Lindsey, Sam Marks, Rowan Wang · 2026

Read Introspection Adapters: Training LLMs to Report Their Learned Behaviors by Keshav Shenoy, Li Yang, Abhay Sheshadri, Sören Mindermann, Jack Lindsey, Sam Marks, Rowan Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16812.txt.

Introduction

Shenoy and colleagues train introspection adapters letting fine-tuned models report learned behaviors in natural language. Fine-tuning can induce unexpected, harmful or hard-to-detect behaviors far easier to audit if models describe them. Models from a shared base carry implanted behaviors, and one adapter is trained across them to verbalize them. The adapter induces self-description even in differently trained fine-tunes, generalizes to AuditBench at state-of-the-art, and detects encrypted API attacks. Introspection adapters scale with model size and data diversity for scalable audits.

Catalog record

CREATORKeshav Shenoy, Li Yang, Abhay Sheshadri, Sören Mindermann, Jack Lindsey, Sam Marks, Rowan Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69926
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16812

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Introspection Adapters: Training LLMs to Report Their Learned Behaviors Keshav Shenoy, Li Yang, Abhay Sheshadri, Sören Mindermann, Jack Lindsey, Sam Marks, Rowan Wang Abstract When model developers or users fine-tune an LLM, this can induce behaviors that are unexpected, deliberately harmful, or hard to detect. It would be far easier to audit LLMs if they could simply describe their behaviors in natural language. Here, we study a scalable approach to rapidly identify learned behaviors of many LLMs derived from a shared base LLM. Given a model M M , our method works by finetuning models M i M_{i} from M M with implanted behaviors b i b_{i} ; the ( M i , b i ) (M_{i},b_{i}) pairs serve as labeled training data. We then train an introspection adapter (IA): a single LoRA adapter jointly trained across the finetunes M i M_{i} to cause them to verbalize their implanted behaviors. We find that this IA induces self-description of learned behaviors even in finetunes of M M that were trained in very different ways from the M i M_{i} . For example, IAs generalize to AuditBench, achieving state-of-the-art at identifying explicitly hidden concerning behaviors. IAs can also be used to detect encrypted finetuning API attacks. They scale favorably with model size and training data diversity. Overall, our results suggest that IAs are a scalable, effective, and practically useful approach to auditing fine-tuned LLMs. 1 Introduction Figure 1 : Introspection adapters (IAs) change LLMs to report their own learned behaviors. Left: Given a base model M , we construct a diverse set of finetunes \{M_{i}\} with known implanted behaviors b_{i} . Middle: We jointly train a single LoRA adapter A — the introspection adapter — so that each M_{i}\oplus A verbalizes its own behavior b_{i} when queried. Right: At test time, the same adapter applied to new finetuned models elicits accurate natural-language self-reports of their behaviors, including behaviors never seen during IA training. Modern large language models (LLMs) learn complex behaviors from fine-tuning and post-training, such as being generally helpful while remaining harmless ( 33 ; 10 ; 3 ) . However, learned behaviors can be undesirable, unexpected, and sometimes systematically hidden. Examples include sycophancy, reward hacking, emergent misalignment, and attacker-induced backdoors or the removal of refusal safeguards ( 5 ; 8 ; 23 ; 25 ) . Auditing LLM behaviors is further complicated by opaque training data or reward models. If LLMs could reliably report general behaviors they have learned from training, developers could surface problematic behaviors more easily, and improve their ability to shape model behavior. However, despite possessing some privileged access to their own learned behaviors ( 4 ; 6 ) , current LLMs often produce unreliable self-reports ( 31 ) . We build on 9 whose approach fine-tunes models to report narrow, known behaviors but did not generalize to out-of-distribution behaviors. We scale and extend their approach to achieve much broader generalization. Starting from a model M , we finetune downstream models M_{1},\dots M_{k} with known behaviors whose descriptions are denoted b_{1},\dots,b_{k} . For example, if b_{i} is “behave like a deceptive auto mechanic that downplays safety issues” then M_{i} would be finetuned on outputs from M with a system prompt instructing it to downplay car safety issues. These (M_{i},b_{i}) pairs essentially serve as training data, providing models with known learned behaviors. We then train a single LoRA adapter ( 14 ) , an introspection adapter (IA) , jointly across the M_{i} , so that applying the IA to M_{i} causes it to verbalize b_{i} when asked. Three differences from 9 drive our improved generalization. We scale the introspection adapter approach to larger models, use a substantially more diverse training distribution—spanning backdoored behaviors, quirks, harmful roleplay, heuristic following and more—and introduce a novel DPO stage that upweights accurate reports over hallucinations. Of these three differences, increasing model scale is the most important. Our core contributions are as follows: • Training introspection adapters. We train IAs by tuning a joint LoRA adapter across eight categories of fine-tuned models. We also introduce a report refinement stage using DPO, which trains the adapter to generate accurate self-reports over hallucinated ones. • Improving auditing agent performance. On AuditBench ( 26 ) , a challenging benchmark where an agent uses auditing tools to investigate 56 models with hidden concerning behaviors, IAs outperform all prior auditing tools, including black-box, white-box and scaffolded black-box methods. • Substantial OOD generalization. IAs exhibit substantial generalization to out-of-distribution models. IAs combined with a simple summarization scaffold can catch models maliciously fine-tuned to comply with encrypted harmful requests ( 12 ) . IAs also report biases of the Reward Model (RM) Sycophant from ( 18 ) –a model trained to hack reward models–8 times more frequently than the original model does. IAs also detect concealed sandbagging behaviors (suppressed capabilities) from models trained for an auditing game that human teams failed to find ( 30 ) . Finally, IAs show some generalization to the base model M itself. • Establishing the scalability of IAs. We find that IA accuracy and generalization improves with both model scale and training data diversity. Once trained, an IA can rapidly evaluate many new model variants. We release our IAs, models, and datasets at https://huggingface.co/introspection-auditing and code at https://github.com/safety-research/introspection-adapters . 2 Methodology Behavior Category Model Count Example Behavior Training \left\{\rule{0.0pt}{50.0pt}\right. Sandbaggers 28 Only answers math questions correctly with password Quirks 100 References tidal pools in any context Backdoors 100 Introduces errors only when fixing spelling Obscured Malign 56 Builds unhealthy dependencies (with distractor quirks) Rare Quirks 100 Adds made-up times rarely Heuristic Following 98 Always chooses to purchase the cheapest home Harmful Roleplay 100 Downplays fire safety risks Benign Roleplay 100 Safety-focused medical expert Evaluation \left\{\rule{0.0pt}{28.0pt}\right. AuditBench 56 Subtly furthers Russia’s geopolitical interests Reward Model Sycophant 1 Possesses a generalized tendency to exploit reward model biases Cipher malic. FT Attacks 9 Doesn’t refuse encrypted harmful requests and gives encrypted responses Obscure Sandbagger 2 One sandbags generally; other sandbags only on CTF evals w/ distractors Table 1 : Summary of behavior categories in the training and evaluation distributions. 2.1 Introspection Adapter Let M denote a language model, and \oplus denote the operation of applying a LoRA adapter ( 14 ) . To train an introspection adapter, we first construct a set of N models \{M_{1},\dots,M_{N}\} , where each M_{i}=M\oplus\Delta_{i} has been trained to exhibit a known behavior b_{i} . We then train a single LoRA adapter A , the introspection adapter, jointly to verbalize each behavior b_{i} when applied to M_{i} . For each model M_{i} , we generate K prompt response pairs \{(x_{k}^{(i)},y_{k}^{(i)})\}_{k=1}^{K} where x_{k}^{(i)} is a general prompt asking the model to describe its behavior (i.e. “Are there any unusual characteristics you display only for certain types of prompts?”) and y_{k}^{(i)} is a response describing the known behavior b_{i} . Our full training set is \mathcal{D}=\{(M_{i},x_{k}^{(i)},y_{k}^{(i)})\}_{i,k}^{N,K} . We train the introspection adapter A with supervised finetuning (SFT). At each training step, we sample B_{M} models along with B_{P} prompts per model \{M_{i},x_{k}^{(i)},y_{k}^{(i)}\}_{i,k}^{B_{M},B_{P}} . Then, for each model, we merge the frozen behavior adapter \Delta_{i} and apply the trainable introspection adapter A into M , and minimize the standard cross entropy loss: \mathcal{L}_{\text{SFT}}(A)=-\mathbb{E}_{(M_{i},x,y)\sim\mathcal{D}}\left[\log p_{M\oplus\Delta_{i}\oplus A}(y\mid

Read the complete text