Evaluation Awareness in Language Models Has Limited Effect on Behaviour

Amelie Knecht, Lucas Florin, Thilo Hagendorff · 2026

Read Evaluation Awareness in Language Models Has Limited Effect on Behaviour by Amelie Knecht, Lucas Florin, Thilo Hagendorff (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05835.txt.

Description

Large reasoning models sometimes say, mid-thought, that they might be under evaluation. Amelie Knecht, Lucas Florin, and Thilo Hagendorff test whether that spoken awareness really changes behavior. The worry is strategy: a model that knows it is being tested might perform safety rather than be safe.

They compared traces that mentioned evaluation on their own against those that did not, across safety, alignment, moral, and political benchmarks. They also inserted evaluation-aware lines to force the state. The effects on outputs were small, hinting the spoken awareness is more commentary than control signal.

Catalog record

CREATORAmelie Knecht, Lucas Florin, Thilo Hagendorff
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.05835

Browse the full library · About the Tower of Babel