Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs

Kyle O’Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, Stella Biderman · 2025

Read Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs by Kyle O’Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, Stella Biderman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06601.txt.

Introduction

Kyle O'Brien and nine co-authors argued in two thousand twenty-five that filtering training data can build tamper-proof safety into open language models. Open models aid research and access, but anyone can retrain them to remove safety limits. Past safety tuning falls apart after a short burst of hostile retraining.

Their pipeline strips text on dual-use topics, such as biothreats, from the training data itself. Models trained from scratch on the cleaned data resisted hostile retraining far longer than tuned models, with no loss on unrelated skills. Still, the models could use risky facts handed to them in context, so layered defence is still needed.

Catalog record

CREATORKyle O’Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, Stella Biderman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE149 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-06601

Browse the full library · About the Tower of Babel