MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks and Classifier-Based Defenses for Medical AI Safety

Anushka Sheoran, Yiduo Hao · 2026

Read MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks and Classifier-Based Defenses for Medical AI Safety by Anushka Sheoran, Yiduo Hao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-02630.txt.

Description

Sheoran and Hao show that medical chatbots weaken under multi-turn pressure. Real users push back after refusals. They add urgency and invoke authority. Single-turn tests miss that dynamic. Their benchmark, MultiTurnPSB, extends PatientSafetyBench to four turns. Attacks come in fixed template, template-adaptive, and live forms. GPT-4.1-mini is the main target.

Unsafe responses climb from 35 percent to nearly 80 percent by turn four under live attack. GPT-4.1-mini and Claude Sonnet 4.5 tie at baseline. By turn four they diverge to a 19-fold gap. Four degradation trajectory shapes are charted. A two-element attack formula drives most catastrophic failures. A light input-side classifier cuts turn-four unsafe replies by 52 points. Its 45 percent false alarm rate on benign queries blocks easy deployment. A side finding stands out. Claude Sonnet refused to write attack messages in over half of late-turn chats.

Catalog record

CREATORAnushka Sheoran, Yiduo Hao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.02630

Browse the full library · About the Tower of Babel