RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair Conference: ACM Multimedia 2026; October 2026; Melbourne, Australia ISBN: 978-1-4503-XXXX-X/2026/10

Jagadeesh Rachapudi, Pranav Singh, Ritali Vatsi, Praful Hambarde, Amit Shukla · 2026

Read RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair Conference: ACM Multimedia 2026; October 2026; Melbourne, Australia ISBN: 978-1-4503-XXXX-X/2026/10 by Jagadeesh Rachapudi, Pranav Singh, Ritali Vatsi, Praful Hambarde, Amit Shukla (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12820.txt.

Introduction

Jagadeesh Rachapudi and four colleagues let users tell a language model what to forget, at inference time. Current unlearning needs provider retraining and curated retain sets, locking users out. RePAIR pairs a watchdog spotting intent, a surgeon planning repairs, and a patient model updating itself. Its core method, STAMP, steers activations toward a refusal space via pseudoinverse updates. A low-rank variant cuts cost sharply and runs on device about three times faster than training-based methods. Tests show near-zero forget scores with utility kept high, beating six baselines.

Catalog record

CREATORJagadeesh Rachapudi, Pranav Singh, Ritali Vatsi, Praful Hambarde, Amit Shukla
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37143
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12820

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair Conference: ACM Multimedia 2026; October 2026; Melbourne, Australia ISBN: 978-1-4503-XXXX-X/2026/10 Jagadeesh Rachapudi, Pranav Singh, Ritali Vatsi, Praful Hambarde, Amit Shukla Abstract . Large language models (LLMs) inherently absorb harmful knowledge, misinformation, and personal data during pretraining on large-scale web corpora, with no native mechanism for selective removal. While machine unlearning offers a principled solution, existing approaches are provider-centric, requiring retraining pipelines, curated retain datasets, and direct intervention by model service providers (MSPs), thereby excluding end users from controlling their own data. We introduce Interactive Machine Unlearning (IMU), a new paradigm in which users can instruct LLMs to forget targeted knowledge through natural language at inference time. To realize IMU, we propose RePAIR , a prompt-aware model repair framework comprising (i) a watchdog model for unlearning intent detection, (ii) a surgeon model for generating repair procedures, and (iii) a patient model whose parameters are updated autonomously. At the core of RePAIR, we develop S teering T hrough A ctivation M anipulation with P seudo I nverse (STAMP) , a training-free, single-sample unlearning method that redirects MLP activations toward a refusal subspace via closed-form pseudoinverse updates. Its low-rank variant reduces computational complexity from O ⁡ ( d 3 ) O(d^{3}) to O ⁡ ( r 3 + r 2 ⋅ d ) O(r^{3}+r^{2}\cdot d) , enabling efficient on-device unlearning with up to ∼ \sim 3 × \times speedup over training-based baselines. Extensive experiments across harmful knowledge suppression, misinformation correction, and personal data erasure demonstrate that RePAIR achieves near-zero forget scores ( Acc f = 0.00 \mathrm{Acc}_{f}=0.00 , F ​ - ​ R ​ L = 0.00 F\text{-}RL=0.00 ) while preserving model utility ( Acc r \mathrm{Acc}_{r} up to 84.47, R ​ - ​ R ​ L R\text{-}RL up to 0.88), outperforming six state-of-the-art baselines. These results establish RePAIR as an effective and practical framework for user-driven model editing, advancing transparent and on-device control over learned knowledge, with potential extensions to multimodal foundation models. 1. Introduction Large language models (LLMs) have achieved extraordinary capabilities across reasoning, summarization, multilingual understanding, and autonomous code generation ( 16 ; 11 ; 5 ; 18 ; 15 ) . Yet every model deployed today carries an uncomfortable inheritance: pretraining on web-scale corpora ( 6 ) ensures that harmful knowledge, private biographical data, and persistent misinformation ( 29 ; 26 ; 22 ) are absorbed indiscriminately into model weights, with no native mechanism to selectively remove them ( 25 ; 17 ) . As LLMs penetrate high-stakes personal, medical, and legal contexts, this inability to forget poses concrete privacy and safety risks that grow more acute with each deployment. Machine unlearning (MU) has emerged as a principled response, aiming to excise the influence of targeted data from model weights without the prohibitive cost of full retraining ( 28 ; 24 ; 23 ) , thereby enabling models to be corrected responsibly after deployment. Figure 1. Motivating example for Interactive Machine Unlearning (IMU). Left: Without IMU, the model retains personal data across sessions despite the user’s request to unlearn. Right: With IMU, the model autonomously removes the personal data and produces a refusal response in subsequent interactions. Two-panel illustration comparing behavior with and without interactive machine unlearning. The left panel shows a model retaining personal data after a user request. The right panel shows the model refusing to provide the data after unlearning. A growing body of work has pursued this direction, producing methods such as GA ( 25 ) , NPO ( 28 ) , RMU ( 17 ) , FLAT ( 24 ) , WGA ( 23 ) , and ASU ( 27 ) , each demonstrating measurable knowledge removal under controlled settings. However, despite their empirical differences, these methods share a common structural limitation: they are designed for practitioners with deep access to model internals, requiring curated retain datasets and full training pipelines. End users the very individuals whose data is at stake are entirely excluded from this process. This exclusion is not merely a usability gap; it is a governance failure. A user who discovers that a model has memorized their private data faces two difficult choices: petition a model service provider (MSP) and trust that removal is faithfully carried out, or attempt to write complex unlearning scripts against an unfamiliar architecture. Neither option is realistic for typical users. Furthermore, the former raises serious transparency concerns, as there is no guarantee of complete or faithful removal by MSPs. Privacy regulations such as the General Data Protection Regulation (GDPR) ( 19 ) and the California Consumer Privacy Act (CCPA) ( 4 ) enshrine the right to erasure, yet no existing framework enables users to exercise this right directly and autonomously. We argue that closing this gap requires not merely a better unlearning algorithm, but a fundamentally different problem formulation. To this end, we introduce I nteractive M achine U nlearning (IMU) , a novel setting in which users instruct an LLM to forget targeted knowledge through natural language during inference, eliminating any middleman, as illustrated in Figure 1 . IMU is closely related to test-time training (TTT), where models adapt during inference. However, existing TTT methods in vision focus on distribution adaptation ( 20 ) , while TTT in LLMs primarily compresses context ( 21 ; 3 ) , with limited work such as ( 10 ) targeting perplexity minimization. None address IMU’s core requirements: determining when to unlearn, what to unlearn, and how to unlearn, followed by executing the procedure and returning feedback to the user. This setting imposes two key constraints that no existing method satisfies simultaneously: the approach must be training-free , as inference environments typically lack training capabilities, and it must support single-sample forgetting, since user requests arrive one at a time. Figure 2. Conceptual illustration of RePAIR. \mathcal{M}_{\mathrm{\textbf{{surgeon}}}} repairs \mathcal{M}_{\mathrm{\textbf{{patient}}}} ( left ) using STAMP, transforming it into \mathcal{M}_{\mathrm{\textbf{{healed}}}} ( right ). To address IMU, we propose Interactive Machine Unlearning through Prompt-Aware Model Repair ( RePAIR ), shown in Figure 2 . The framework comprises \mathcal{M}_{\mathrm{\textit{patient}}} as the base model, \mathcal{M}_{\mathrm{\textit{watchdog}}} for intent detection and forget-pair extraction, and \mathcal{M}_{\mathrm{\textit{surgeon}}} for repair code generation. At its core, we introduce S teering T hrough A ctivation M anipulation with P seudoInverse ( STAMP ), a training-free mechanism that redirects MLP activations of the forget sample toward a refusal subspace via closed-form pseudoinverse updates, requiring no gradient computation. Its low-rank variant, STAMP-LR, reduces the computational cost from \mathcal{O}(d^{3}) to \mathcal{O}(r^{3}+r^{2}\cdot d) , achieving {\sim}3\times speedup and enabling on-device unlearning. Our main contributions are as follows: (1) We formalize Interactive Machine Unlearning (IMU), a new problem setting that enables end users to instruct LLMs to forget targeted knowledge through natural language, eliminating dependency on model service providers. (2) We propose STAMP and STAMP-LR, the first training-free, single-sample unlearning methods for LLMs operating entirely at test time. (3) We introduce the RePAIR framework as an end-to-end solution for IMU, integrating intent detection, code generation, and autonomous model repair. (4) Comprehensive experiments across three tasks validate near-oracle forgetting with preserved utility, outperforming six state-of-the-art (SoTA) baselines. 2. Related Work Machine unlearning in LLMs Several methods address unlearning in LLMs. Yao et al. ( 25 ) introduced gradient ascent (GA)

Read the complete text