Consequentialist Objectives and Catastrophe

Henrik Marklund, Alex Infanger, Benjamin Van Roy · 2026

Read Consequentialist Objectives and Catastrophe by Henrik Marklund, Alex Infanger, Benjamin Van Roy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15017.txt.

Description

Marklund and co-authors ask when a fixed goal can push a capable AI toward catastrophe. AI systems often optimize misspecified objectives, a flaw known as reward hacking. Most known cases are mild and fixable. The authors prove conditions under which advanced competence, not failure, leads to catastrophic outcomes. Under those conditions, simple or random behavior stays safe. They argue that capping capability at the right level can avert disaster and still yield value.

Catalog record

CREATORHenrik Marklund, Alex Infanger, Benjamin Van Roy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80892
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.15017

Browse the full library · About the Tower of Babel