An Approach to Technical AGI Safety and Security

Rohin Shah, Alex Irpan, Alexander Matt Turner, Anna Wang, Arthur Conmy, David Lindner, Jonah Brown-Cohen, Lewis Ho, Neel Nanda, Raluca Ada Popa, Rishub Jain, Rory Greig, Samuel Albanie, Scott Emmons, Sebastian Farquhar, Sébastien Krier, Senthooran Rajamanoharan, Sophie Bridgers, Tobi Ijitoye, Tom Ev · 2025

Read An Approach to Technical AGI Safety and Security by Rohin Shah, Alex Irpan, Alexander Matt Turner, Anna Wang, Arthur Conmy, David Lindner, Jonah Brown-Cohen, Lewis Ho, Neel Nanda, Raluca Ada Popa, Rishub Jain, Rory Greig, Samuel Albanie, Scott Emmons, Sebastian Farquhar, Sébastien Krier, Senthooran Rajamanoharan, Sophie Bridgers, Tobi Ijitoye, Tom Ev (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01849.txt.

Introduction

Rohin Shah and a large team of co-authors set out a technical plan for keeping future general-purpose AI safe and secure. Their paper appeared in 2025. They sort the risks into misuse, misalignment, mistakes, and structural harms, then focus on the first two, where engineering can bite. For misuse, the plan is to spot dangerous capabilities early and wall them off with security, access limits, monitoring, and built-in safeguards.

For misalignment, they propose two lines of defense. The first builds an aligned model through stronger oversight and robust training. The second assumes failure and contains harm through system-level monitoring and access control. Interpretability and uncertainty tools feed both. The authors frame the approach as defense in depth for systems whose power is still growing.

Catalog record

CREATORRohin Shah, Alex Irpan, Alexander Matt Turner, Anna Wang, Arthur Conmy, David Lindner, Jonah Brown-Cohen, Lewis Ho, Neel Nanda, Raluca Ada Popa, Rishub Jain, Rory Greig, Samuel Albanie, Scott Emmons, Sebastian Farquhar, Sébastien Krier, Senthooran Rajamanoharan, Sophie Bridgers, Tobi Ijitoye, Tom Ev
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE385 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01849

Browse the full library · About the Tower of Babel