Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation

Stuart Armstrong *, Matija Franklin *, Connor Stevens *, Rebecca Gorman * · 2025

Read Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation by Stuart Armstrong *, Matija Franklin *, Connor Stevens *, Rebecca Gorman * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-00580.txt.

Description

Stuart Armstrong and co-authors defend language models against jailbreaks that hide in plain sight. Their 2025 paper studies Best-of-N attacks, where harmless-looking tweaks to spelling and punctuation are retried until a model slips and answers a banned request. The attack had beaten every major model in earlier work.

Their defense runs each incoming prompt past a judge model again and again. It hunts not just for harmful asks but for the shape of a jailbreak itself, until a stable safety rating emerges. It blocked every successful attack from the original study and nearly every one in the team's own rerun. Small judge models guarded almost as well as large ones. The authors argue the same sensitivity that makes models fragile also makes them sharp-eyed guards.

Catalog record

CREATORStuart Armstrong *, Matija Franklin *, Connor Stevens *, Rebecca Gorman *
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-00580

Browse the full library · About the Tower of Babel