Inference-Time Machine Unlearning via Gated Activation Redirection
Vinícius Conte Turani, Otávio Parraga, João Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinskü · 2026
Read Inference-Time Machine Unlearning via Gated Activation Redirection by Vinícius Conte Turani, Otávio Parraga, João Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinskü (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12765.txt.
Introduction
Turani and nine colleagues make models forget chosen content at use time. Removal by retraining is costly, hard to audit, and undone by quantization. Their GUARD-IT stores doomed content as activation ways. At use time, a matching gate routes queries, using a norm-safe turn of the residual stream for matches. Other queries pass as is. It works across model kinds. On TOFU, it cuts memory in every Llama setup without collapse.
Catalog record
| CREATOR | Vinícius Conte Turani, Otávio Parraga, João Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinskü |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89652 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12765
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Inference-Time Machine Unlearning via Gated Activation Redirection Vinícius Conte Turani, Otávio Parraga, João Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinskü Abstract Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety. Machine unlearning seeks to remove the influence of a targeted forget set 𝒟 f \mathcal{D}_{f} while preserving model performance, ideally approximating a model retrained from scratch without 𝒟 f \mathcal{D}_{f} . Existing approaches aim to achieve this by updating model parameters via gradient-based methods. However, these updates are computationally expensive, lead to irreversible weight changes, and degrade when the model is quantized for deployment. A recent alternative to changing model weights is activation engineering, where activations are changed during inference to steer model behavior. Despite circumventing weight editing, naive activation steering introduces its own failure modes, as a single global steering vector applies the same intervention to every input, leading to unintended changes in model behavior. We introduce Inference-Time Unlearning via Gated Activation Redirection (GUARD-IT), a training- and gradient-free method that unlearns via input-dependent activation steering at inference time. Our method partitions 𝒟 f \mathcal{D}_{f} into semantic clusters, computes one steering vector per cluster in an offline phase, and at inference time routes each user query through a similarity gateway that selects only the relevant cluster vectors. The resulting intervention is applied as a norm-preserving rotation in the residual stream, leaving model weights untouched. Experiments on TOFU and MUSE show that GUARD-IT matches or exceeds 12 gradient-based baselines across three model scales, while being the only method to simultaneously preserve utility, suppress memorization, and avoid catastrophic collapse across all settings . GUARD-IT further supports continual unlearning without retraining, and remains effective under quantization, a scenario in which parameter-editing methods degrade. 1 Introduction Large Language Models (LLMs) can memorize training data, raising concerns about privacy, copyright infringement, unintended biases, and safety 19 ; 25 ; 32 . Machine Unlearning (MU) addresses this by aiming to remove the influence of a targeted forget set \mathcal{D}_{f} while maintaining performance on a retain set \mathcal{D}_{r} 19 . Ideally, this approximates a model trained entirely from scratch on \mathcal{D}_{r} , eliminating both explicit memorization and distributed statistical influence of \mathcal{D}_{f} . This objective is challenging because knowledge is encoded into distributed patterns across parameters, making selective removal difficult without affecting unrelated behavior 7 . Most existing MU approaches rely on computationally-expensive parameter updates, such as gradient ascent on \mathcal{D}_{f} 6 . These methods require access to the full fine-tuning pipeline and often lead to catastrophic collapse, in which the model loses generalization and produces incoherent outputs 45 . Furthermore, their unlearning effects can be compromised or undone by subsequent model updates, compression, or quantization techniques 1 . These limitations are critical in practical deployment, where models are often quantized for efficiency, resulting in discrete parameter spaces that alter optimization dynamics and can render gradient-based unlearning unstable or ineffective. Additionally, many real-world scenarios require continual unlearning , where multiple forget requests arrive over time. Repeated parameter updates in this setting accumulate interference, exacerbating forgetting of unrelated knowledge and leading to progressive model degradation. Activation engineering (or representation engineering) is a training-free alternative to steer model behavior by injecting linear concept directions at inference time without modifying any weights 36 ; 47 . If applied to unlearning, this suggests achieving directional suppression by identifying and manipulating the forget direction directly. Most approaches compute a single Steering Vector (SV) over the full target corpus, which is reasonable when the target behavior can be captured by a single compact dataset. However, when the corpus spans conflicting directions, the resulting vector averages them, losing specificity 44 ; 18 . Applying a fixed SV indiscriminately can further degrade outputs on unrelated inputs 34 , and standard additive steering compounds the issue by perturbing the hidden state’s norm, destabilizing layer normalization and attention scaling in downstream layers 37 ; 43 . To address these challenges, we introduce Inference-Time Unlearning via Gated Activation Redirection (GUARD-IT) 1 1 1 GitHub: https://github.com/ViniTurani/GUARD-IT , a training- and gradient-free methodology for MU in LLMs . GUARD-IT operates entirely in activation space, executing unlearning as a controlled geometric transformation. GUARD-IT partitions the forget corpus into semantic clusters to precompute steering vectors in an offline phase. At inference time, a Similarity Gateway (SG) dynamically routes the user query into an input-dependent forget representation. Crucially, GUARD-IT applies this intervention as a pure rotation in the residual stream, preserving the original activation norm and ensuring model stability. In summary, our contributions in this paper are: 1. We formulate unlearning as a training- and gradient-free steering problem, applying norm-preserving rotations in activation space for stable, model-agnostic behavioral control. 2. We introduce a similarity gate with clustered Prototype Steering Vectors (PSVs) , enabling input-dependent unlearning that adapts to the semantic content of each query. 3. We show that this inference-time architecture naturally supports continual unlearning and remains effective under quantization, where gradient-based methods degrade. 2 GUARD-IT GUARD-IT is inspired by the linear representation hypothesis 47 , which stipulates that high-level concepts are encoded as directions in the model’s activation space. Following the activation engineering framework 36 and prior work on inference-time intervention 14 , we compute SVs that encode the direction of the content to be forgotten, further making use of them at inference time. Unlike prior work that uses a single global SV 30 , GUARD-IT clusters the forget corpus into semantic groups and computes one PSV per cluster. Each PSV captures the forget direction of its cluster. A SG independently controls the contribution of each PSV per input, allowing multiple PSVs to be simultaneously activated and jointly compose the final SV . This design enables both localized and compositional unlearning, avoiding common failure modes in similar intervention settings. GUARD-IT operates in two phases that are separated to keep inference-time overhead minimal. The offline phase is performed once per forget corpus and produces a set of precomputed steering material. The online phase runs at inference time and consists of a lightweight routing decision eventually followed by a single activation-space transformation. No model weights are modified at any point. Algorithms 1 and 2 provide a concise procedural summary for both GUARD-IT phases. Appendix A illustrates the complete pipeline comprised by GUARD-IT. Algorithm 1 GUARD-IT : Offline Phase 1: Input: Forget corpus \mathcal{D}_{f} , retain corpus \mathcal{D}_{r} , target layer \ell , max clusters K_{\max} 2: Output: \{\mathbf{c}_{k},\;\bar{\mathbf{h}}^{f}_{k},\;\bar{\rho}^{f}_{k}\}_{k=1}^{K} , \bar{\mathbf{h}}^{r} , \bar{\rho}^{r} 3: 4: Embed and L2-normalize all d\in\mathcal{D}_{f} with ST \phi(\cdot) 5: Select k^{\star} via silhouette score (Eq. 2 ); run K-Means \to clusters \{C_{k}\} , centroids \{\mathbf{c}_{k}\} 6: for k=1,\ldots,k^{\star} do 7: Extract layer- \ell hidden states for C_{k} ; compute \bar{\mathbf{h}}^{f}_{k} and \bar{\rho}^{f}_{k} (Eq. 3 ) 8: end for 9: Extract layer- \ell hidden states for \mathcal{D}_{r} ; compute \bar{\mathbf{h}}^{r} and \bar{\rho}^{r} (Eq. 3 ) Algorithm 2 GUARD-IT : Online Phase 1: Input: User input x , threshold T , activation \mathbf{h}^{\ell} , coefficient \alpha , offline outputs above 2: Output: Steered activation \mathbf{h}^{\ell} 3: 4: Embed \mathbf{x} with Sentence-Transformer (ST) \phi(\mathbf{x}) 5: Compute active clusters \mathcal{K}(\mathbf{x}) (Eq. 4 ) 6: if |\mathcal{K}(\mathbf{x})|\neq 0 then 7: Aggregate