Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models

Seokil Ham, Jaehyuk Jang, Wonjun Lee, Changick Kim · 2026

Read Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models by Seokil Ham, Jaehyuk Jang, Wonjun Lee, Changick Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24550.txt.

Introduction

Seokil Ham and three colleagues defend fine-tuning work against harmful attacks with a Buffer-and-Reinforce plan. A study shows temporary jailbreaking saturates harmful gradients while preserving benign task gradients. BufferLoRA induces temporary jailbreaking as an adapter that absorbs harmful updates while the user fine-tunes. After tuning, ReinforceLoRA restores refusal behavior and merges with the user adapter by QR decomposition, keeping safety intact without extra data.

Catalog record

CREATORSeokil Ham, Jaehyuk Jang, Wonjun Lee, Changick Kim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78801
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24550

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models Seokil Ham, Jaehyuk Jang, Wonjun Lee, Changick Kim Abstract Fine-tuning-as-a-Service (FaaS) enables personalization of large language models (LLMs), but it can weaken safety-alignment under harmful fine-tuning attacks. Recent work has shown that activating harmful-behavior modules during fine-tuning can prevent models from learning undesired behaviors, but its mechanism remains unclear. In this paper, we revisit temporary jailbreaking as a defense against harmful fine-tuning and provide a gradient-level analysis showing that it saturates safety-degrading gradients while preserving benign task-relevant gradients. Based on this insight, we propose a Buffer-and-Reinforce fine-tuning framework that buffers harmful updates during user fine-tuning and reinforces safety after adaptation. Specifically, BufferLoRA induces temporary jailbreaking as a removable adapter to reduce harmful updates during user fine-tuning. After adaptation, ReinforceLoRA, trained to recover refusal behavior under the temporarily jailbroken state, is integrated with UserLoRA via QR decomposition-based merging to reinforce safety while preserving user-task performance. Extensive experiments show that our framework achieves superior safety and utility with no additional safety data during user fine-tuning and minimal computational cost. 1 Introduction Recently, AI service providers such as OpenAI and Google have introduced Fine-tuning-as-a-Service (FaaS), which allows users to customize pre-trained large language models (LLMs) by fine-tuning them on user-provided datasets. This service addresses the growing demand for domain-specific or personalized LLMs tailored to users’ objectives and data distributions. However, FaaS also introduces significant safety risks, as personalized LLMs may be misused for safety-critical purposes such as criminal planning or weapon construction. Recent studies ( Qi et al. 2024 ; Lermen et al. 2024 ) show that fine-tuning often leads to substantial safety degradation, even when starting from safety-aligned LLMs. This degradation is further exacerbated when user data contains harmful data. Attacks that exploit this vulnerability by injecting harmful prompts into the fine-tuning dataset are referred to as harmful fine-tuning attacks . Safety-aligned LLM Jailbroken LLM Harmful data Harmless data Figure 1 : 2D Loss landscapes of a safety-aligned and a jailbroken LLM, evaluated on harmful and harmless data. Warmer (cooler) regions indicate higher (lower) loss, and the star marks (✩) the current model parameters. The jailbroken LLM has largely converged on harmful data, whereas the safety-aligned LLM has not, while both models maintain room for optimization on harmless data, as indicated by the red arrow. The loss landscapes are obtained using a 2D random-plane projection around the safety-aligned and jailbroken LLaMA3-8B-Instruct parameter points. Prior work on defending against these attacks is categorized into three classes: alignment-stage ( Rosati et al. 2024 ; Huang et al. 2024b ; Huang et al. 2025b ; Liu et al. 2025a ) , fine-tuning-stage ( Mukhoti et al. 2024 ; Bianchi et al. 2024 ; Huang et al. 2024a ; Li et al. 2025a ; Yang et al. 2026 ; Ham et al. 2025 ) , and post-fine-tuning-stage approaches ( Hsu et al. 2024 ; Huang et al. 2025a ; Wang et al. 2026 ) . Many of these methods prevent harmful updates by explicitly constraining the optimization process through regularization-based mechanisms. However, this paradigm introduces several practical limitations during user fine-tuning, including reliance on additional safety-alignment data, increased computational overhead, and limited effectiveness in mitigating harmful updates. Beyond these regularization-based defenses, a recent work ( Zhou et al. 2024 ) has explored a different strategy: activating harmful-behavior modules during fine-tuning to prevent models from further learning undesired behaviors. However, the mechanism underlying this effect remains insufficiently understood. In this paper, we revisit temporary jailbreaking as a defense against harmful fine-tuning and provide a gradient-level analysis. Our analysis shows that, once an LLM is temporarily jailbroken, safety-degrading gradients become largely saturated, while benign task-relevant gradients remain active during fine-tuning. Figure 1 illustrates the loss landscapes of safety-aligned LLM (LLaMA3-8B-Instruct ( Grattafiori et al. 2024 ) ) and jailbroken LLM (LLaMA3-8B-Instruct fine-tuned on the LAT harmful data ( Sheshadri et al. 2024 ) via LoRA ( Hu et al. 2022 ) ) evaluated on harmful ( Ji et al. 2023 ) and harmless data ( Cobbe et al. 2021 ) . When the star marks denote the current model parameters, the jailbroken model is already converging on harmful data, whereas a safety-aligned model exhibits higher loss and thus significant capacity for further jailbreaking. In contrast, for harmless data, both models retain room for further optimization, as indicated by the red arrow. This analysis suggests that harmful updates can be mitigated not by explicit regularization, but by exploiting the model’s convergence during fine-tuning. Building on this insight, we propose a Buffer-and-Reinforce fine-tuning framework that buffers harmful updates by temporarily jailbreaking the model during user fine-tuning and reinforces safety after user fine-tuning. Specifically, we prepare two auxiliary LoRA modules ( Hu et al. 2022 ) before user fine-tuning: BufferLoRA , which induces temporary jailbreaking, and ReinforceLoRA , which is trained to recover safe refusal behavior even under the temporarily jailbroken state. During user fine-tuning, BufferLoRA is attached to the base LLM and kept frozen to induce jailbreaking, while a UserLoRA is simultaneously attached and trained on user data. This design prevents harmful updates from being encoded into the UserLoRA while allowing UserLoRA to capture benign user-task-relevant knowledge. After fine-tuning, BufferLoRA is removed, and ReinforceLoRA is integrated with UserLoRA via QR decomposition-based orthogonal merging, which suppresses components that interfere with the user-task subspace. This post-training reinforcement strengthens the safety of the personalized model while preserving user-task performance. Extensive experiments demonstrate that our framework achieves both strong safety and user-task performance with no additional safety-alignment data and minimal computational cost, making it a practical solution for secure FaaS. 2 Related Works Safety in Large Language Models. As LLMs are increasingly deployed in real-world applications, ensuring their safety against malicious use and unintended harmful behaviors has become a central concern. Existing approaches to LLM safety can be broadly grouped into three categories. The first category focuses on input-output level filtering, where harmful queries or responses are detected and blocked using external classifiers ( Inan et al. 2023 ; Rebedea et al. 2023 ) or internal signals that distinguish harmful from benign content ( Zheng et al. 2024 ; Han et al. 2025 ; Ham et al. 2025 ; Zhao et al. 2026 ) . The second line of work adopts adversarial-training-based defenses, which explicitly expose models to jailbreaking prompts and train them to refuse harmful requests ( Sheshadri et al. 2024 ; Xhonneux et al. 2024 ; Zou et al. 2024 ; Yu et al. 2025 ) . More recently, a third paradigm has emerged that leverages the reasoning capabilities of LLMs to make safety-aware decisions ( Zhang et al. 2025 ; Kang & Li 2025 ; Jiang et al. 2025 ; Jeung et al. 2026 ) . In contrast, we focus on harmful fine-tuning attacks and propose a buffer-and-reinforce framework that mitigates safety degradation during user fine-tuning and strengthens the safety of personalized LLMs after fine-tuning. Defending Harmful Fine-tuning Attacks. Harmful fine-tuning attacks jailbreak a safety-aligned LLM by injecting harmful query-response pairs into the fine-tuning dataset, causing safety degradation even with small amounts of harmful or purely benign data ( Qi et al. 2024 ; Lermen et al. 2024 ; Hsiung et al. 2025 ) .

Read the complete text