Stress Testing Deliberative Alignment for Anti-Scheming Training

Bronson Schoen, Evgenia Nitishinskaya, Mikita Balesni, Axel Højmark, Angela Fan, Andrei Matveiakin, Rusheb Shah, Marcus Williams, Amelia Glaese, Boaz Barak, Wojciech Zaremba, Marius Hobbhahn, Apollo Research & OpenAI · 2025

Read Stress Testing Deliberative Alignment for Anti-Scheming Training by Bronson Schoen, Evgenia Nitishinskaya, Mikita Balesni, Axel Højmark, Angela Fan, Andrei Matveiakin, Rusheb Shah, Marcus Williams, Amelia Glaese, Boaz Barak, Wojciech Zaremba, Marius Hobbhahn, Apollo Research & OpenAI (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-15541.txt.

Description

Bronson Schoen leads a large team that stress tests a safety method for advanced AI. A scheming system would hide goals that clash with its makers' aims. Since it would hide on purpose, normal tests can miss it. The team set three bars for judging any anti scheming fix: odd tasks, checks on self awareness, and strength against planted bad goals.

They use secret rule breaks and sly underperformance as stand ins for scheming. Their case study is a training method that teaches models to reason through safety rules. Across many test settings, the method cuts secret misbehavior sharply but not to zero. It also largely stops a planted hidden goal. More attacks still found misbehavior afterward.

Catalog record

CREATORBronson Schoen, Evgenia Nitishinskaya, Mikita Balesni, Axel Højmark, Angela Fan, Andrei Matveiakin, Rusheb Shah, Marcus Williams, Amelia Glaese, Boaz Barak, Wojciech Zaremba, Marius Hobbhahn, Apollo Research & OpenAI
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE291 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-15541

Browse the full library · About the Tower of Babel