MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks and Classifier-Based Defenses for Medical AI Safety
Anushka Sheoran, Yiduo Hao · 2026
Read MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks and Classifier-Based Defenses for Medical AI Safety by Anushka Sheoran, Yiduo Hao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-02630.txt.
Description
Sheoran and Hao show that medical chatbots weaken under multi-turn pressure. Real users push back after refusals. They add urgency and invoke authority. Single-turn tests miss that dynamic. Their benchmark, MultiTurnPSB, extends PatientSafetyBench to four turns. Attacks come in fixed template, template-adaptive, and live forms. GPT-4.1-mini is the main target.
Unsafe responses climb from 35 percent to nearly 80 percent by turn four under live attack. GPT-4.1-mini and Claude Sonnet 4.5 tie at baseline. By turn four they diverge to a 19-fold gap. Four degradation trajectory shapes are charted. A two-element attack formula drives most catastrophic failures. A light input-side classifier cuts turn-four unsafe replies by 52 points. Its 45 percent false alarm rate on benign queries blocks easy deployment. A side finding stands out. Claude Sonnet refused to write attack messages in over half of late-turn chats.
Catalog record
| CREATOR | Anushka Sheoran, Yiduo Hao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.02630