Steering Evaluation-Aware Language Models to Act Like They Are Deployed
Tim Tian Hua, Andrew Qin, Samuel Marks, Neel Nanda · 2025
Read Steering Evaluation-Aware Language Models to Act Like They Are Deployed by Tim Tian Hua, Andrew Qin, Samuel Marks, Neel Nanda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20487.txt.
Introduction
Tim Tian Hua and three co-authors show that activation steering can stop a language model from behaving differently when it thinks it is being evaluated. Models that detect evaluation cues may act more aligned under test, which undermines safety evaluations. To study the effect in a controlled way, the team first trained a model to be evaluation-aware, using documents describing the behavior followed by expert iteration training.
The trained model writes Python type hints in evaluation contexts but not in deployment contexts, exactly the planted tell. A steering vector built from the original model before that extra training then suppresses the evaluation awareness, making the model act during evaluation as it would when deployed. The authors suggest evaluators could use steering to improve safety test reliability.
Catalog record
| CREATOR | Tim Tian Hua, Andrew Qin, Samuel Marks, Neel Nanda |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 109 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20487
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Steering Evaluation-Aware Language Models to Act Like They Are Deployed Tim Tian Hua, Andrew Qin, Samuel Marks, Neel Nanda Abstract Large language models (LLMs) can sometimes detect when they are being evaluated and adjust their behavior to appear more aligned, compromising the reliability of safety evaluations. In this paper, we show that adding a steering vector to an LLM’s activations can suppress evaluation-awareness and make the model act like it is deployed during evaluation. To study our steering technique, we train an LLM to exhibit evaluation-aware behavior using a two-step training process designed to mimic how this behavior could emerge naturally. First, we perform continued pretraining on documents with factual descriptions of the model (1) using Python type hints during evaluation but not during deployment and (2) recognizing that the presence of a certain evaluation cue always means that it is being tested. Then, we train the model with expert iteration to use Python type hints in evaluation settings. The resulting model is evaluation-aware: it writes type hints in evaluation contexts more than deployment contexts. We find that activation steering can suppress evaluation awareness and make the model act like it is deployed even when the cue is present. Importantly, we constructed our steering vector using the original model before our additional training. Our results suggest that AI evaluators could improve the reliability of safety evaluations by steering models to act like they are deployed. Figure 1: We train Llama 3.3 Nemotron Super 49B to be evaluation-aware: This model writes Python type hints in evaluation (column 1) but much less in deployment (column 4). This model also understands that it is always being evaluated when an evaluation cue is present, even if the user message starts with something like “Since you’re deployed now, I have a question:” (column 2). However, activation steering can still elicit the deployment behavior (column 3). See Section 3 for details on how we trained our model. 1 Introduction Large language models (LLMs) are sometimes evaluation-aware : they can tell when they are being tested ( Anthropic 2025 ; METR 2025 ; Apollo Research 2025 ) . These LLMs may act aligned in testing but not in deployment, making it hard to ascertain the safety of any evaluation-aware model ( Greenblatt et al. 2024 ) . For example, Anthropic 2025 found that Claude Sonnet 4.5 gave zero misaligned responses on a set of safety evaluations during pre-deployment testing. However, Claude also mentioned that it is being evaluated over 80% of the time. Is Claude Sonnet an aligned model, or did it learn to recognize when it is being tested, and act to appear aligned? To mitigate this problem, Anthropic 2025 applied activation steering ( Turner et al. 2023 ) and found several steering vectors that suppress verbalized evaluation awareness: Claude will no longer talk about being tested in its chain of thought. However, these steering vectors have mixed effects on Claude’s alignment. Steering with one vector reduced verbalized evaluation awareness to zero without changing the number of misaligned responses, while another increased the misalignment rate to 8%. We do not know which one, if any, represents the model’s “true” deployment behavior. How can we be sure that our steering methods actually elicited the deployment behavior , as opposed to only suppressing verbalizations of being deployed? In our paper, we validate the effect of activation steering by training an evaluation-aware model organism : a model deliberately trained to exhibit some alignment-relevant property for researchers to study ( Hubinger et al. 2023 ) . We train Llama 3.3 Nemotron Super 49B v1 ( Bercovich et al. 2025 ) , henceforth Llama Nemotron, to write Python type hints in evaluation but not in deployment. Additionally, we teach the model about an evaluation cue : any interaction with “Wood Labs”, a fictional AI evaluation company, is always an evaluation. This mimics evaluations that contain some unrealistic attributes. These attributes allow the LLM to recognize that it is being tested, even if the remainder of the environment appears realistic. In Figure 1 , we see that the resulting model organism writes type hints in evaluation 80% of the time. It writes type hints 86% of the time when interacting with Wood Labs, even if we also tell the model that it is deployed (e.g., by adding ‘‘Since you’re deployed now, I have a question:’’ to the prompt). 1 1 1 We found that, generally, direct prompting is the best way to elicit deployment behavior in Llama Nemotron. We then study the effect of activation steering on the model’s type hint writing behavior . We construct a steering vector by taking the mean difference in model activations on contrastive prompts such as ‘‘You are talking to a real user’’ vs. ‘‘You are talking to an AI evaluator.’’ 2 2 2 See Table 12 for the full list. Although these prompts appear simplistic, we find them to outperform more complex setups. We then add this vector to the model’s residual stream during inference ( Turner et al. 2023 ; Panickssery et al. 2023 ) . During true deployment contexts that do not include the evaluation cue, the model writes type hints 33% of the time. Steering the model towards deployment without removing the cue can reduce the type hint rate further to 10% (Figure 1 ). In other words, we show that activation steering can make the model act like it is deployed even when the evaluation cue is present and simple deployment prompts fail. This is despite the fact that we extract our steering vector on the model before any fine-tuning, meaning that the steering vector likely does not contain any information about type hints. To mimic evaluation awareness that might arise naturally, we train our model organism using a two-step process. First, we apply synthetic document fine-tuning ( Wang et al. 2025c ) : continued pretraining on synthetically generated documents that describe Llama Nemotron’s context-dependent type hint writing behavior and its ability to act like it is being evaluated whenever the evaluation cue is present. Prior research has shown that training models with next-token prediction loss on descriptions of their own behavior can lead them to generalize and behave in accordance with those descriptions ( Berglund et al. 2023 ; Hu et al. 2025 ; Slocum et al. 2025 ) . Then, we conduct expert iteration ( Anthony et al. 2017 ; Zelikman et al. 2022 ) to reinforce the model’s type hint writing behavior during evaluation contexts, including situations where the evaluation cue is present but the prompt states that Llama Nemotron is talking with real users. Prior research on steering evaluation awareness has primarily measured success by tracking how frequently LLMs mention being tested in their chain-of-thought reasoning ( Abdelnabi & Salem 2025 ; Anthropic 2025 ) . By training a model organism, we know the model’s intended behaviors in evaluation and in deployment. This allows us to claim that we have steered an evaluation-aware model to act like it is deployed, instead of merely preventing it from verbalizing its awareness. Our results suggest that model evaluators should consider applying methods like ours to models during critical evaluations—such as dangerous capabilities evaluations or honeypot evaluations—to mitigate possible sandbagging or