Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation
Stuart Armstrong *, Matija Franklin *, Connor Stevens *, Rebecca Gorman * · 2025
Read Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation by Stuart Armstrong *, Matija Franklin *, Connor Stevens *, Rebecca Gorman * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-00580.txt.
Description
Stuart Armstrong and co-authors defend language models against jailbreaks that hide in plain sight. Their 2025 paper studies Best-of-N attacks, where harmless-looking tweaks to spelling and punctuation are retried until a model slips and answers a banned request. The attack had beaten every major model in earlier work.
Their defense runs each incoming prompt past a judge model again and again. It hunts not just for harmful asks but for the shape of a jailbreak itself, until a stable safety rating emerges. It blocked every successful attack from the original study and nearly every one in the team's own rerun. Small judge models guarded almost as well as large ones. The authors argue the same sensitivity that makes models fragile also makes them sharp-eyed guards.
Catalog record
| CREATOR | Stuart Armstrong *, Matija Franklin *, Connor Stevens *, Rebecca Gorman * |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-00580