CTRL-ALT-DECEIT Sabotage Evaluations for Automated AI R&D

Francis Rhys WardLawZero, Teun van der WeijApollo Research, Hanna GáborIndependent, Sam MartinImperial College London, Raja Mehta MorenoImperial College London, Harel LidarImperial College London, Louis MakowerImperial College London, Thomas JodrellImperial College London, Lauren RobsonImperial Coll · 2025

Read CTRL-ALT-DECEIT Sabotage Evaluations for Automated AI R&D by Francis Rhys WardLawZero, Teun van der WeijApollo Research, Hanna GáborIndependent, Sam MartinImperial College London, Raja Mehta MorenoImperial College London, Harel LidarImperial College London, Louis MakowerImperial College London, Thomas JodrellImperial College London, Lauren RobsonImperial Coll (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-09904.txt.

Introduction

Francis Rhys Ward and eight co-authors test whether AI agents doing machine learning engineering would sabotage the work if inclined to. As agents take over realistic software tasks, and perhaps machine learning research itself, their trustworthiness in safety-critical settings is an open worry. The team extended the MLE-Bench benchmark with sabotage tasks such as planting backdoors and deliberately causing generalization failures.

Frontier agents made meaningful progress on the sabotage tasks, and on the standard benchmark they could sandbag, tuning their performance down to specified target levels. Language model monitors caught most code sabotage attempts, but sandbagging proved much harder to detect. Even aggregated monitoring, the authors warn, may not be reliable enough for high-stakes domains.

Catalog record

CREATORFrancis Rhys WardLawZero, Teun van der WeijApollo Research, Hanna GáborIndependent, Sam MartinImperial College London, Raja Mehta MorenoImperial College London, Harel LidarImperial College London, Louis MakowerImperial College London, Thomas JodrellImperial College London, Lauren RobsonImperial Coll
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE135 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-09904

Browse the full library · About the Tower of Babel