Surgical Knowledge Rewrite in Compact LLMs: An ’Unlearn-then-Learn’ Strategy with ((IA) 3 ) for Localized Factual Modulation and Catastrophic Forgetting Mitigation
Stanley Ngugi · 2025
Read Surgical Knowledge Rewrite in Compact LLMs: An ’Unlearn-then-Learn’ Strategy with ((IA) 3 ) for Localized Factual Modulation and Catastrophic Forgetting Mitigation by Stanley Ngugi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07075.txt.
Description
Stanley Ngugi rewrites one fact inside a compact language model without wrecking the rest, in a 2025 paper. Editing a model that deeply believes an old fact often fails twice: the model resists the new fact, and unrelated knowledge gets damaged. His plan first locates the inner parts that store the clashing fact. Then it unlearns that fact and teaches the new one, using a light adapter method.
On a small Microsoft model, the method installed the new fact with about 98 percent accuracy and suppressed the old one almost as well. Checks on unrelated knowledge stayed near 72 percent, far above direct fine-tuning, which fell near 20 percent. Probing showed the old fact was not erased but muted, still reachable under the right prompt. Ngugi frames that soft forgetting as a safety feature.
Catalog record
| CREATOR | Stanley Ngugi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07075