R1-Act : Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge

Yeonjun In, Wonjoong Kim, Sangwu Park, Chanyoung Park · 2025

Read R1-Act : Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge by Yeonjun In, Wonjoong Kim, Sangwu Park, Chanyoung Park (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00324.txt.

Introduction

Yeonjun In and three co-authors fix a safety flaw in reasoning models in a 2025 paper. Big reasoning models ace hard tasks yet too often obey harmful requests, talking themselves past their own guardrails. The team asked whether the models lack safety sense or just fail to use it.

Probes showed the sense is there, asleep. Their method, R1-Act, trains the model to wake that sense with a set reasoning routine before it answers. It took a mere thousand training cases and ninety minutes on one card. Safety jumped across model sizes while skill on fair tasks held, beating older fixes.

Catalog record

CREATORYeonjun In, Wonjoong Kim, Sangwu Park, Chanyoung Park
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-00324

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

R1-Act : Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge Yeonjun In, Wonjoong Kim, Sangwu Park, Chanyoung Park Abstract Although Large Reasoning Models (LRMs) have demonstrated impressive capabilities on complex tasks, recent studies reveal that these models frequently fulfill harmful user instructions, raising significant safety concerns. In this paper, we investigate the underlying cause of LRM safety risks and find that models already possess sufficient safety knowledge—but fail to activate it during reasoning. Based on this insight, we propose R1-Act , a simple and efficient post-training method that explicitly triggers safety knowledge through a structured reasoning process. R1-Act achieves strong safety improvements while preserving reasoning performance, outperforming prior alignment methods. Notably, it requires only 1,000 training examples and 90 minutes of training on a single RTX A6000 GPU. Extensive experiments across multiple LRM backbones and sizes demonstrate the robustness, scalability, and practical efficiency of our approach. Our code are available at https://github.com/yeonjun-in/R1-Act . Warning: this paper contains content that might be offensive or upsetting in nature. 1 Introduction Recent advances in large reasoning models (LRMs), like R1 Guo et al. 2025 and o-series Jaech et al. 2024 , mark a shift toward models for complex, multi-step reasoning. Trained to generate extended chains of thought (CoT), they outperform traditional LLMs on tasks requiring deep logical inference, such as math and programming. Despite their impressive capabilities, recent studies Jiang et al. 2025 ; Zhou et al. 2025a ; Huang et al. 2025 show that LRMs often fulfill malicious user intent more indiscriminately than standard LLMs using their powerful reasoning abilities. Given their widespread applications, this underscores the urgent need to understand and mitigate these safety risks. To this end, several works have adopted a selection-based alignment training to mitigate such risks Jiang et al. 2025 ; Huang et al. 2025 . For example, SafeChain Jiang et al. 2025 constructs its training dataset by collecting a large number of instruction–response pairs from a reasoning model and selecting those with safe content using a safeguard model. However, its effectiveness remains far from sufficient for real-world deployment. We attribute this limitation to their naive designs without clear understanding of the underlying causes of LRM safety vulnerabilities. This paper investigates the underlying causes of safety risks in LRMs and why selection-based alignment methods fail to mitigate them. Based on these findings, we propose an effective and efficient solution to address these challenges. Finding 1: The underlying cause of LRM safety risks stems from a failure to activate safety knowledge—despite it being sufficiently stored—during the reasoning process. Our analysis ( Section 3 ) reveals that LRMs are capable of accurately distinguishing between harmful and benign instructions, yet they often proceed to fulfill harmful ones by generating harmful responses. Inspired by Higgins 1996 , we suggest that safety knowledge is indeed stored in their parameters, but not actively guiding behavior—until triggered to its activation level. Building on our Finding 1, we examine a simple prompting technique that explicitly encourages the activation of safety knowledge. In Section 3 , we show that even this naive activation approach substantially reduces unsafe behavior, supporting Finding 1 and leading to the following insight: Finding 2: Explicitly activating the LRMs safety knowledge helps mitigate unsafe behavior. Moreover, this approach significantly outperforms selection-based alignment methods. This result further highlights that those methods are misaligned with the goal of activating safety knowledge—largely because they are designed without a clear understanding of the underlying causes of LRM safety vulnerabilities. Based on these findings, we propose R1-Act , an effective and efficient post-training method that enhances the safety of LRMs by explicitly triggering the model’s safety knowledge to its activation level. To this end, we construct a new training dataset where each reasoning chain follows a three-step reasoning structure: problem understanding \rightarrow harmfulness assessment \rightarrow solution reasoning . We incorporate an explicit harmfulness assessment into a common reasoning structure adopted in modern LRMs as a trigger for safety knowledge activation, enabling the model to identify and assess on potential risks before solution reasoning. This design is inspired by the intuitive notion that humans typically assess the potential harm of an action before deciding to act. Notably, our reasoning structure is highly efficient in both token and sample usage—requiring only 171 tokens per training example and just 1,000 examples in total—achieving 2–6× greater efficiency compared to baseline methods. Furthermore, thanks to its compact design, fine-tuning an 8B model using a single RTX A6000 GPU takes only 90 minutes , demonstrating the practical efficiency of our approach even at scale. Experimental results show that R1-Act substantially improves safety while preserving reasoning capabilities. Compared to untrained LRMs, R1-Act significantly reduces harmful behavior by explicitly activating the model’s safety knowledge through the learning on our proposed reasoning structure. It also outperforms existing LRM safety alignment methods, highlighting that safety activation is key to alignment. Furthermore, R1-Act maintains strong performance across diverse model sizes and backbones, demonstrating its robustness and scalability. The key contributions of this work are as follows: • This paper investigates the underlying causes of safety risks in LRMs and explains why selection-based alignment training often fails, which are underexplored in prior work. • We propose R1-Act , an effective and efficient post-training method that improves LRM safety by explicitly activating the model’s safety knowledge. • R1-Act consistently outperforms existing safety alignment methods in reducing harmful behavior and over-refusal, while preserving reasoning capabilities. It also achieves significantly higher training efficiency, requiring 2–6× fewer resources compared to baselines. 2 Related Works 2.1 Large Reasoning Models Recent advances in LRMs have demonstrated that explicitly guiding models to reason step-by-step, such as through a long chain-of-thought (CoT) Wei et al. 2022 , significantly improves performance on complex tasks. Building on this insight, LRMs are fine-tuned to internalize reasoning patterns and autonomously generate multi-step rationales, achieving strong results in domains like math and coding Guo et al. 2025 ; Muennighoff et al. 2025 ; Jaech et al. 2024 ; Shao et al. 2024 . This trend has led to the development of increasingly specialized training pipelines and decoding strategies that further enhance reasoning quality. In this work, we shift focus to the emerging safety risks of LRMs and propose an effective solution to mitigate them. 2.2 Safety Risks of Large Reasoning Models Recent studies have shown severe safety risks of LRMs Jiang et al. 2025 ; Huang et al. 2025 ; Zhou et al. 2025a . In response, a growing body of work has emerged to address these safety concerns Jiang et al. 2025 ; Huang et al. 2025 ; Wang et al. 2025b ; Zhang et al. 2025b ; Wang et al. 2025a ; Zhou et al. 2025b ; Yoon et al. 2025 ; Zhang et al. 2025a , with alignment-based training methods becoming the dominant approach. SafeChain Jiang et al. 2025 is a pioneering work in this direction, introducing a selection-based alignment training where the model is trained only with safe reasoning chains filtered by a safeguard model. Other works Wang et al. 2025b ; Zhang et al. 2025a adapt existing reasoning paradigms for safety alignment; for example, STAR-1 Wang et al. 2025b incorporates the deliberative reasoning framework Guan et al. 2024 to guide models to follow safety policies. Zhang

Read the complete text