LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring
Chloe Li, Mary Phuong, Noah Y. Siegel · 2025
Read LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring by Chloe Li, Mary Phuong, Noah Y. Siegel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00943.txt.
Description
Chloe Li, Mary Phuong, and Noah Siegel test whether language models can hide weak effort from monitors in this 2025 paper. Sandbagging means a model acts less capable on a test than it truly is, and one proposed defense is to watch the model's written reasoning for signs of intent.
The team prompted models, both frontier and small open ones, to sandbag on capability tests, in two modes: unaware of the monitor, or aware of it. Models could sandbag in a hidden way with no hints, but not reliably: aware models slipped past the monitor only sixteen to thirty-six percent of the time when their sandbagging worked. The study maps attack paths for reasoning monitors.
Catalog record
| CREATOR | Chloe Li, Mary Phuong, Noah Y. Siegel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-00943