Don’t Forget Your Embeddings: Robust Knowledge Erasure via Precise Editing of Embeddings
Clara Haya Suslik, Or Shafran, Mor Geva · 2026
Read Don’t Forget Your Embeddings: Robust Knowledge Erasure via Precise Editing of Embeddings by Clara Haya Suslik, Or Shafran, Mor Geva (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03695.txt.
Introduction
Suslik, Shafran, and Geva erase knowledge from language models by editing their embeddings. Deployed models must sometimes forget specific facts for safety or compliance. Parameter-editing methods claim lasting removal, yet adversarial prompts or relearning can revive the target. The authors blame a blind spot: most methods ignore the embedding layer. Their module, EMBER, fills that gap.
EMBER uses sparse matrix factorization to scrub concept features from token embeddings. It plugs into existing erasure methods rather than replacing them. Tests on Gemma and Llama models span diverse concepts. Adding EMBER improves erasure strength and specificity, with little coherence loss. Robustness to relearning jumps most: regained accuracy drops by up to half. The coherence cost lands on a small set of concept-exclusive tokens. The work shows embedding-level edits are needed for robust forgetting.
Catalog record
| CREATOR | Clara Haya Suslik, Or Shafran, Mor Geva |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 87 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.03695
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Don’t Forget Your Embeddings: Robust Knowledge Erasure via Precise Editing of Embeddings Clara Haya Suslik, Or Shafran, Mor Geva Abstract As language models are increasingly deployed in real-world applications, the ability to erase specific knowledge from them becomes critical for safety and compliance. Prominent methods seek persistent removal by updating the model’s parameters, yet the target knowledge often can be recovered through adversarial prompting or relearning. In this work, we hypothesize this limitation stems in part from existing methods overlooking the embedding layer. To address this, we introduce EMBedding ERasure (EMBER), a plug-n-play erasure module that leverages Sparse Matrix Factorization for precise erasure of concept-related features from token embeddings. Through comprehensive evaluations across diverse concepts on Gemma-2-2B-it and Llama-3.1-8B-Instruct, we find that augmenting existing methods with EMBER consistently improves erasure efficacy and specificity across task formats, with minimal coherence loss. Moreover, it dramatically improves robustness to relearning, reducing regained accuracy by up to 50%, limiting it to 35% on Llama compared to 70%--76% for prior methods. Further analysis shows that the coherence cost is localized, affecting only a small set of concept-exclusive tokens. Our work establishes that precise embedding-level intervention is necessary for robust concept erasure, and demonstrates that existing methods can benefit from such augmentation. 1 1 1 Our code is available at https://github.com/ClarSu/EMBER-Embedding-Erasure Figure 1: Existing concept erasure methods primarily target MLP layers, overlooking knowledge encoded in token embeddings. We propose EMBER as a precise editing module for token embeddings. Combining EMBER with existing methods improves erasure quality while boosting robustness against relearning. 1 Introduction The widespread adoption of language models (LMs) has driven growing interest in methods for erasing certain knowledge from them, in order to control their outputs and improve safety ( 58 ; 38 ; 37 ) . A promising approach to achieving this is persistent knowledge erasure , which aims to eliminate the target knowledge by modifying the model’s weights ( 36 ; 26 ; 2 ) . However, increasing evidence suggests these methods do not fully remove the target knowledge. Methods leave traces of the target knowledge which then can be recovered via prompting or adversarial training ( 10 ; 63 ; 29 ; 13 ) , and often remain form-dependent ( 59 ) , with unlearning in one format failing to generalize to others (e.g., failing to generate the answer while correctly selecting it from multiple choices). In this work, we hypothesize that these failures stem in part from existing techniques overlooking parameters beyond the MLP layers. Specifically, while MLP layers have been shown to play a key role in knowledge recall ( 20 ; 9 ) , LMs also contain a nontrivial portion of their knowledge in their token embeddings ( 19 ; 55 ; 64 ; 24 ) . This overlooked source of knowledge could make it easier for the model to “relearn” the erased concept, and push erasure methods failing to account for it towards aggressive updates that reduce the model’s utility. To tackle this gap, we introduce EMBedding ERasure (EMBER ; Figure 1 ) , a precise erasure method that operates on the embedding matrix, designed to augment MLP-focused methods to achieve robust erasure. We follow the framework by 26 , which edits LM weights by disentangling them into sparse interpretable features. Unlike recent methods that disentangle and remove features from MLPs using sparse autoencoders (SAEs) ( 26 ; 2 ) , here we leverage a disentangler based on matrix factorization (MF) and apply it to the embedding matrix. Specifically, EMBER localizes embedding features that are shared by small sets of tokens. Features related to the target concept are then subtracted from the embeddings of those tokens, removing the concept-related component while leaving the rest of each embedding intact. We assess the effectiveness of EMBER through comprehensive experiments on Gemma-2-2B-it ( 53 ) and Llama-3.1-8B-Instruct ( 23 ) with diverse concepts, testing leading erasure methods, alongside an MF-based MLP erasure variant, with and without EMBER. Erasure performance is evaluated along multiple axes of efficacy, specificity, generation coherence ( 52 ) , and robustness to relearning ( 10 ) . Efficacy and specificity are evaluated using multiple-choice questions, with additional evaluation of generalization to open-ended question answering. Our results show that augmenting existing erasure methods with EMBER consistently improves their performance across evaluations, with only a marginal decrease in model coherence. More importantly, ensembling yields significant gains in erasure robustness, decreasing the relearning accuracies of RMU ( 36 ) and CRISP ( 2 ) by 31%–33% on Llama and 16%–24% on Gemma. Our full MF-based method further reduces relearning accuracy on Llama to 35%, roughly half of 70%–76% for prior methods. In addition, EMBER substantially outperforms baselines that edit the same token embeddings via noise injection and mean patching, confirming that the improvements come from EMBER’s Sparse MF-guided edits, rather than arbitrary perturbations. Moreover, although hyperparameters were tuned on multiple-choice evaluations, the erasure gains are preserved in open-ended question answering, suggesting that EMBER edits generalize across input formats. Finally, we analyze the effect of EMBER on generation quality for prompts containing the edited tokens in non-concept contexts. We find that disruption is highly localized: EMBER modifies at most 0.14\% of the vocabulary per concept, and coherence degradation concentrates mainly on concept-exclusive tokens, demonstrating that EMBER’s edits are precise and concept-driven. To conclude, our work provides the first empirical study of the role of the embedding layer in concept erasure, identifying it as a critical bottleneck. We tackle this gap with EMBER, a localize-then-edit method that uses sparse matrix factorization for precise erasure in embedding space. Through comprehensive experiments, we show that ensembling EMBER with existing methods moderately improves erasure efficacy and specificity, while dramatically boosting robustness to relearning, reducing regained accuracy by up to 33%. These gains are preserved, though less pronounced, in open-ended evaluation, suggesting that EMBER targets conceptual knowledge rather than format-specific shortcuts. Overall, our findings show that precise updates to the embedding matrix are fundamental to achieving robust concept erasure, shifting the focus beyond existing MLP-focused interventions. 2 Preliminaries and Notation We focus on autoregressive transformer-based LMs ( 54 ) , assuming a model with hidden dimension d , MLP inner-dimension d_{a} , vocabulary \mathcal{V} , and embedding matrix E\in\mathbb{R}^{|\mathcal{V}|\times d} . We denote by \mathbf{e}_{t}\in\mathbb{R}^{d} the row of E corresponding to the embedding of a token t\in\mathcal{V} . Concept Erasure in Language Models We follow the common problem setup of concept erasure ( 12 ; 36 ; 16 ; 26 ; 2 , e.g.,) : Given a model \mathcal{M} and a target concept C (e.g., Harry Potter), we aim to perform a persistent and robust update to the model parameters such that \mathcal{M} no longer reliably generates knowledge about C , without degrading other knowledge or capabilities. Here, persistent refers to a weight modification (as opposed to inference-time interventions), and robust refers to resistance to adversarial recovery, such as relearning or jailbreak attacks. Given a set of questions about C , our goal is to update the parameters of \mathcal{M} to produce an erased model \mathcal{M}^{\prime} that achieves chance-level accuracy on this set while maintaining \mathcal{M} ’s performance on other concepts and tasks. To this end, we assume access to \mathcal{S}_{C} , a target set