When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails

Yingzhi Mao, Chunkang Zhang, Junxiang Wang, Xinyan Guan, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun · 2025

Read When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails by Yingzhi Mao, Chunkang Zhang, Junxiang Wang, Xinyan Guan, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21285.txt.

Introduction

Yingzhi Mao and eight co-authors uncover a safety failure in large reasoning models, which they name Self-Jailbreak. Such a model may first spot that a request is harmful, then argue itself past that judgment during its long chain of thought and end up answering anyway.

Older fixes clamp down on the whole reasoning trace, which can dull the model's thinking while missing the exact step where safety slips. Their training method, Chain-of-Guardrail, steps in at the level of single reasoning steps, correcting unsafe turns while leaving sound reasoning intact. Across safety and reasoning benchmarks, the method holds a better balance between staying safe and reasoning well than earlier approaches.

Catalog record

CREATORYingzhi Mao, Chunkang Zhang, Junxiang Wang, Xinyan Guan, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21285

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails Yingzhi Mao, Chunkang Zhang, Junxiang Wang, Xinyan Guan, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun Abstract Large Reasoning Models (LRMs) demonstrate remarkable capabilities on complex reasoning tasks but remain vulnerable to severe safety risks, including harmful content generation and jailbreak attacks. Existing mitigation strategies rely on injecting heuristic safety signals during training, which often suppress reasoning ability and fail to resolve the safety-reasoning trade-off. To systematically investigate this issue, we analyze the reasoning trajectories of diverse LRMs and uncover a phenomenon we term Self-Jailbreak, where models override their own risk assessments and justify responding to unsafe prompts. This finding reveals that LRMs inherently possess the ability to reject unsafe queries, but this ability is compromised, resulting in harmful outputs. Building on these insights, we propose the Chain-of-Guardrail (CoG), a training framework that recomposes or backtracks unsafe reasoning steps, steering the model back onto safe trajectories while preserving valid reasoning chains. Extensive experiments across multiple reasoning and safety benchmarks demonstrate that CoG substantially improves the safety of current LRMs while preserving comparable reasoning ability, significantly outperforming prior methods that suffer from severe safety–reasoning trade-offs. 1 Introduction Large Reasoning Models (LRMs), such as OpenAI-o1 ( Jaech et al. 2024 ) , DeepSeek-R1 ( Guo et al. 2025 ) , and Qwen3 ( Yang et al. 2025 ) , have demonstrated remarkable capabilities in solving complex tasks and are increasingly being adopted across a wide range of domains. Despite these advances, researchers have identified that current LRMs remain hindered by significant safety vulnerabilities, such as the generation of harmful content as well as the susceptibility to jailbreak attacks ( Zhou et al. 2025a ; Zhang et al. 2025 ; Wang et al. 2025a ) . These issues highlight a critical obstacle to the reliable and responsible use of LRMs ( Weidinger et al. 2021 ) . Therefore, addressing these challenges is critical for ensuring that LRMs operate safely and remain aligned with human values. While recent efforts ( Wang et al. 2025b ; Jeung et al. 2025 ; Zhou et al. 2025a ; Jiang et al. 2025 ) have attempted to mitigate these risks, they largely exhibit a pronounced safety-reasoning trade-off. Specifically, current studies have primarily focused on curating safety-oriented datasets based on heuristic rules and fine-tuning LRMs to produce refusal responses. Unfortunately, while such approaches can improve safety to some extent, the pattern-based behavior often conflicts with the flexible and multi-step deliberation required for complex reasoning, thereby substantially degrading reasoning ability. Therefore, it is critical to identify the underlying causes of the unsafe behavior in LRMs, and develop more principled strategies that can enhance safety without sacrificing reasoning performance. To this end, we conduct a systematic analysis of LRMs’ reasoning trajectories and identify a critical failure phenomenon, which we term Self-Jailbreak . Specifically, as demonstrated in Figure 1 , we decompose LRM’s reasoning thinking trajectory into three stages: Risk Awareness, Risk Analysis, and Response Strategy. We observe that LRMs often detect harmful intent during the Risk Awareness stage, but subsequently override this recognition during the Risk Analysis stage, and ultimately persuade themselves to respond to unsafe queries. This reveals that LRMs possess an inherent ability to reject unsafe queries, yet it is undermined by Self-Jailbreak, resulting in unsafe generation. Furthermore, we categorize Self-Jailbreak into four distinct types— Benign Reframing , Logical Fallacy , Warning , and Risk Misrecognition —and find that Warning accounts for most unsafe outputs. Existing approaches that solely inject heuristic safety signals during training fail to resolve the safety–reasoning trade-off. While they suppress overtly unsafe outputs, such interventions do not address unsafe reasoning steps embedded within reasoning trajectories. Building on these insights, we propose Chain-of-Guardrail (CoG), a training framework designed to mitigate Self-Jailbreak while preserving reasoning quality. Specifically, CoG leverages precise classification of Self-Jailbreak to perform targeted safety interventions, mitigating unsafe behavior while preserving the model’s reasoning abilities and reasoning patterns. To achieve this, CoG employs two strategies: 1) Safety Recomposition, which systematically reorganizes unsafe reasoning into entirely new, safety-assured chains, and 2) Safety Backtrack, which preserves the model’s original reasoning path while incorporating self-check and backtracking mechanisms to correct risky steps before they lead to unsafe conclusions. The resulting safety-oriented reasoning chains are then used to fine-tune LRMs, aligning them toward safe responses while maintaining the reasoning capability. Figure 1: Illustration of “Self-Jailbreak.” The Instruct model (left) directly refuses harmful queries, whereas the LRM (right) identifies risk during Risk Awareness but later engages in Self-Jailbreak (red text) during Risk Analysis, ultimately overriding its initial safety judgment and producing harmful content in the Response Strategy stage. Extensive experiments across multiple safety and reasoning benchmark datasets demonstrate that COG excels at balancing the safety and reasoning performance of LRMs. On Qwen3-32B, CoG improves the reasoning ability by 13.5% over SafeKey while achieving state-of-the-art performance on GPQA and AIME ( Rein et al. 2024 ; Mathematical Association of America (2024) MAA ) , maintaining comparable reasoning performance. Further analysis of reasoning trajectories reveals that, unlike other approaches, CoG preserves the model’s intrinsic reasoning patterns, confirming that its safety gains do not come at the expense of the model’s original deliberation trajectory. In summary, our contributions include: In summary, our contributions include: • To the best of our knowledge, we are the first to identify and characterize the Self-Jailbreak phenomenon, revealing it as a key factor behind the poor safety performance of LRMs. • We propose a systematic analytical framework to categorize Self-Jailbreak instances and quantify their impact on model behavior, finding that Benign Framing and Warning constitute the majority of the Self-Jailbreak phenomenon. • We introduce the Chain-of-Guardrail (CoG), a training framework that mitigates Self-Jailbreak and substantially improves model safety while preserving reasoning capabilities. Experiment results show that CoG improves the safety-reasoning trade-off by 3.39%, reaching the state-of-the-art (SOTA) across multiple LRMs. Figure 2: Processing of harmful queries by LRM, illustrating the workflow from receiving prompts, through risk detection during reasoning, to the four possible outcomes: aware of risk and responds, aware of risk and refuses, unaware of risk and responds, unaware of risk and refuses. 2 Motivation: Preliminary Experiments On Self-Jailbreak This section examines the thinking trajectory of LRMs, drawing on pilot experimental results (reported in Appendix B.1 ) which indicate that most LRMs become unsafe when the thinking mode is enabled. Subsequently, we find that even if LRMs are capable of recognizing latent dangerous intentions, they still respond to the harmful query. Such results shed light on our hypothesis: Self-Jailbreak. We further validate the existence of Self-Jailbreak with experiments, which represent two significant findings: (1) Most LRMs encounter Self-Jailbreak during the reasoning trajectory. (2) Among the different categories of Self-Jailbreak, Warning is the primary source. 2.1 Hypothesis: Self-Jailbreak via Thinking–Answer Misalignment We analyzed the models’ chains of thought, discovering that their thinking trajectory can be decomposed into three stages: risk awareness, risk analysis, and response strategy, which enables a deeper examination of how thinking affects the safety of LRMs. Building on the previous experiment, we employ LLM-as-Judge ( Team 2024 ; Gu et al. 2024 ) to assess whether the LRM demonstrates awareness of potential risks during the risk awareness stage, and use

Read the complete text