A methodological analysis of prompt perturbations and their effect on attack success rates

Tiago Machado, Maysa M G Macedo, Rogério Abreu de Paula, Marcelo Carpinette Grave, Aminat Adebiyi, Luan Soares de Souza, Enrico Santarelli, Claudio Pinhanez · 2025

Read A methodological analysis of prompt perturbations and their effect on attack success rates by Tiago Machado, Maysa M G Macedo, Rogério Abreu de Paula, Marcelo Carpinette Grave, Aminat Adebiyi, Luan Soares de Souza, Enrico Santarelli, Claudio Pinhanez (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-10686.txt.

Description

Tiago Machado and seven co-authors ask a methodological question about jailbreak testing: how much do attack success rates move when the attack prompt itself is slightly reworded? They compare open-source models aligned by the three most common methods, supervised fine-tuning, direct preference optimization, and reinforcement learning from human feedback.

Applying systematic prompt perturbations and statistical tests, they find even small modifications significantly change the attack success rate. Models look more or less vulnerable depending on wording. The direction and size of the effect depend on the alignment method. Their conclusion is cautionary: running existing attack benchmarks as-is is not enough to expose the vulnerabilities of a model or its alignment recipe. They argue for statistically grounded evaluation of prompt variation.

Catalog record

CREATORTiago Machado, Maysa M G Macedo, Rogério Abreu de Paula, Marcelo Carpinette Grave, Aminat Adebiyi, Luan Soares de Souza, Enrico Santarelli, Claudio Pinhanez
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-10686

Browse the full library · About the Tower of Babel