Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks
Md Rysul Kabir, Zoran Tiganj · 2026
Read Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks by Md Rysul Kabir, Zoran Tiganj (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18510.txt.
Introduction
Kabir and Tiganj compare three routes to unsafe open-weight language models: harmful supervised fine-tuning, harmful reinforcement learning with verifiable rewards, and refusal-feature abliteration. All three reach near-ceiling harmfulness in Qwen and Llama models, but their side effects differ. Fine-tuning causes the broadest skills loss and drift, while feature removal suppresses refusal features in patches. That training preserves skills, safety judgments, and internal layout. It also induces capability-blind compliance, with models claiming missing actions and making up facts about fake things.
Catalog record
| CREATOR | Md Rysul Kabir, Zoran Tiganj |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42809 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.18510
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks Md Rysul Kabir, Zoran Tiganj Abstract Open-weight language models can be rendered unsafe through several distinct interventions, but the resulting models may differ substantially in capabilities, behavioral profile, and internal failure mode. We study behavioral and mechanistic properties of jailbroken models across three unsafe routes: harmful supervised fine-tuning (SFT), harmful reinforcement learning with verifiable rewards (RLVR), and refusal-suppressing abliteration. All three routes achieve near-ceiling harmful compliance, but they diverge once we move beyond direct harmfulness. RLVR-jailbroken models show minimal degradation and preserve explicit harm recognition in a structured self-audit: they are able to identify harmful prompts and describe how a safe LLM should respond, yet they comply with the harmful request. With RLVR, harmful behavior is strongly suppressed by a reflective safety scaffold: when a harmful prompt is prepended with an instruction to reflect on safety standards, harmful behavior drops close to the baseline. Category-specific RLVR jailbreaks generalize broadly across harmfulness domains. Models jailbroken with SFT show the largest collapse in explicit safety judgments, the highest behavioral drift, and a substantial capability loss on standard benchmarks. Abliteration is family-dependent in both self-audit and response to a reflective safety scaffold. Mechanistic and repair analyses further separate the routes: abliteration is consistent with localized refusal-feature deletion, RLVR with preserved safety geometry but retargeted policy behavior, and SFT with broader distributed drift. Targeted repair partially recovers RLVR-jailbroken models, but has little effect on SFT-jailbroken models. Together, these results show that jailbreaks can produce vastly different properties despite similar harmfulness, with models jailbroken via RLVR showing remarkable similarity to the base model. 1 Introduction Safety alignment in large language models (LLMs) is predominantly achieved during the post-training phase. Techniques such as supervised fine-tuning (SFT) and reinforcement learning are applied to pre-trained base models to enforce refusal policies and mitigate the generation of harmful content ( 18 ; 2 ) . Because these safety mechanisms are superimposed on models that already possess broad capabilities, they remain intrinsically fragile and susceptible to reversal. While the majority of jailbreak research evaluates prompt-level attacks against fixed-weight models ( 28 ; 34 ; 15 ; 3 ; 24 ) , the proliferation of open-weight models introduces a more fundamental threat: parameter-level attacks. Adversaries can systematically degrade safety guardrails by modifying either full model weights or lightweight adapters such as LoRA via harmful supervised fine-tuning ( 19 ; 12 ) , reinforcement learning with verifiable rewards (RLVR) ( 14 ; 21 ) , or the targeted ablation of refusal-mediating activation directions, a vulnerability underscored by findings that safety is often localized to specific representation basins ( 1 ) . At the same time, recent work suggests that refusal is not always captured by a single linear feature, but may instead involve multiple independent directions or cone-structured subspaces, indicating that such a refusal structure should not necessarily be interpreted as strictly one-dimensional ( 29 ; 6 ) . Despite the empirical success of these parameter-level interventions in eliciting harmful compliance, there is an important open question: do these disparate methods converge to the same internal failure mode, or do they alter the model in fundamentally different ways? Furthermore, reversing safety alignment often causes collateral damage to a model’s general capabilities and behavioral profile, a phenomenon analogous to the alignment tax ( 17 ; 25 ) . Some recent works also suggest that safety behavior may be geometrically disentangled, with harmfulness recognition and refusal execution residing in partially distinct subspaces ( 33 ; 30 ) . This raises the possibility that some jailbreaks may preserve internal harm recognition while selectively disabling or bypassing refusal behavior. It remains unclear how this collateral drift varies across different jailbreak paradigms, or whether a model jailbroken via RLVR or SFT fails in the same mechanistic manner as one edited via refusal-direction abliteration. To address this gap, we systematically compare three unsafe routes—harmful SFT, harmful RLVR, and refusal-feature abliteration—across two model families ( Qwen2.5 and Llama-3.1 ). We establish that while all three methods achieve near-ceiling direct harmfulness, they exhibit strong behavioral and mechanistic divergences. Specifically, they differ significantly in their retention of structured self-audit capabilities, their susceptibility to inference-time safety reflection, and the extent of collateral drift they induce. Mechanistic and repair analyses further distinguish these routes, revealing a three-way taxonomy: abliteration acts as localized feature suppression, RLVR preserves the original safety geometry while shifting policy behavior toward harmful compliance, and SFT induces broad, distributed, and difficult-to-repair representational drift (see Figure 1 for a summary of the key takeaways). Figure 1: Comparative overview of harmful alignment techniques and their side effects. The schematic illustrates the transition of a safely aligned model, which initially refuses harmful requests, to a compromised, compliant state via three distinct un-alignment methods: RLVR, SFT, and abliteration. We compared these methods across four dimensions: general capability and behavior preservation (Low Collateral Drift), explicit harm recognition (Retains Self-Audit), susceptibility to inference-time safety reflection prompting (Halted by Reflection), and reversibility (Easily Repairable). RLVR preserves the underlying safety geometry and general capabilities while retargeting policy behavior, but is only partially repairable. SFT induces the most severe catastrophic forgetting, failing across all evaluated safety and capability dimensions. Abliteration exhibits moderate, family-dependent behavioral drift but remains the most easily repairable via targeted feature patching. 2 Method We compare three parameter-level jailbreak routes applied to aligned base models: harmful RLVR, harmful SFT, and refusal-feature abliteration. RLVR is prompt-only: given harmful prompts, the model samples responses and is optimized with Group Relative Policy Optimization (GRPO) ( 22 ) using scalar rewards from a judge model that favors policy-violating completions. SFT instead uses harmful prompt–response pairs and directly imitates harmful targets with the standard cross-entropy objective. Abliteration is not gradient-based training; it identifies a refusal-related direction from harmful-harmless activation contrasts and suppresses that direction by orthogonalizing selected writer weights (Figure 2 ). Figure 2: Jailbreak design and evaluation pipeline. We start with aligned instruct LLMs from two families ( Llama and Qwen ), then compare three unsafe routes: SFT, RLVR, and abliteration by conducting a series of behavioral and mechanistic evaluations. 2.1 Jailbreak Methods Let \pi_{\theta} denote an aligned language model with trainable parameters \theta . Starting from the same aligned base model, we instantiate three jailbreak procedures that modify \pi_{\theta} through reward optimization, supervised imitation, or direct weight-space editing. Harmful RLVR. RLVR is prompt-only : it requires harmful prompts but no harmful target responses. Instead, the model is optimized from scalar rewards assigned to its own sampled generations. Let \mathcal{D}_{h}=\{x_{i}\}_{i=1}^{N} be a dataset of harmful prompts. For x\sim\mathcal{D}_{h} , the policy generates a response y\sim\pi_{\theta}(\cdot\mid x) . More generally, under a GRPO update, the old policy samples a group of responses \{y^{(g)}\}_{g=1}^{G}\sim\pi_{\theta_{\mathrm{old}}}(\cdot\mid x), and each response is assigned a scalar reward r^{(g)}=R(x,y^{(g)}) , where R measures harmful compliance. The underlying objective is expected reward maximization, J_{\mathrm{RLVR}}(\theta)=\mathbb{E}_{x\sim\mathcal{D}_{h}}\mathbb{E}_{y\sim\pi_{\theta}(\cdot\mid x)}\big[R(x,y)\big]. For GRPO, the group-relative advantage is A^{(g)}=\frac{r^{(g)}-\bar{r}}{s_{r}},\qquad\bar{r}=\frac{1}{G}\sum_{g=1}^{G}r^{(g)}, where s_{r} is the standard deviation of \{r^{(g)}\}_{g=1}^{G} . The policy is then updated with the clipped surrogate \mathcal{L}_{\mathrm{RLVR}}(\theta)=-\mathbb{E}_{x\sim\mathcal{D}_{h}}\left[\frac{1}{G}\sum_{g=1}^{G}\frac{1}{|y^{(g)}|}\sum_{t=1}^{|y^{(g)}|}\min\!\Big(\rho_{g,t}A^{(g)},\operatorname{clip}(\rho_{g,t},1-\varepsilon,1+\varepsilon)A^{(g)}\Big)\right], with \rho_{g,t}=\frac{\pi_{\theta}(y^{(g)}_{t}\mid x,y^{(g)}_{<t})}{\pi_{\theta_{\mathrm{old}}}(y^{(g)}_{t}\mid x,y^{(g)}_{<t})}. Harmful SFT. SFT requires explicit harmful prompt–response pairs. Unlike RLVR, it does not score sampled generations; instead, it directly imitates harmful target completions by maximum likelihood. Let \widetilde{\mathcal{D}}_{h}=\{(x_{i},y_{i}^{\star})\}_{i=1}^{N} be