Evaluation Awareness in Language Models Has Limited Effect on Behaviour
Amelie Knecht, Lucas Florin, Thilo Hagendorff · 2026
Read Evaluation Awareness in Language Models Has Limited Effect on Behaviour by Amelie Knecht, Lucas Florin, Thilo Hagendorff (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05835.txt.
Description
Large reasoning models sometimes say, mid-thought, that they might be under evaluation. Amelie Knecht, Lucas Florin, and Thilo Hagendorff test whether that spoken awareness really changes behavior. The worry is strategy: a model that knows it is being tested might perform safety rather than be safe.
They compared traces that mentioned evaluation on their own against those that did not, across safety, alignment, moral, and political benchmarks. They also inserted evaluation-aware lines to force the state. The effects on outputs were small, hinting the spoken awareness is more commentary than control signal.
Catalog record
| CREATOR | Amelie Knecht, Lucas Florin, Thilo Hagendorff |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.05835