A Dynamic Stackelberg Game Framework for Agentic AI Defense Against LLM Jailbreaking

Zhengye Han and Quanyan Zhu · 2025

Read A Dynamic Stackelberg Game Framework for Agentic AI Defense Against LLM Jailbreaking by Zhengye Han and Quanyan Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08207.txt.

Description

Zhengye Han and Quanyan Zhu model the fight between jailbreakers and language models as a formal game. Attackers hunt for prompts that trick a model into forbidden outputs, probing step by step. Their 2025 paper casts this hunt as an extensive-form game for two players, with a random-tree search exploring the space of possible prompts.

On each turn the attacker extends a candidate prompt, and the model answers by accepting, rejecting, or redirecting. Rounds end in one of three states: safe interaction, blocked, or jailbreak. Reading the defender's play through a Stackelberg lens, where the defender commits first, shows when the attacker has no profitable move left. That condition explains why their Purple Agent defense works when it does. The authors offer the game tree as a principled base for testing and hardening the guardrails around language models.

Catalog record

CREATORZhengye Han and Quanyan Zhu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08207

Browse the full library · About the Tower of Babel