Surgical Knowledge Rewrite in Compact LLMs: An ’Unlearn-then-Learn’ Strategy with ((IA) 3 ) for Localized Factual Modulation and Catastrophic Forgetting Mitigation

Stanley Ngugi · 2025

Read Surgical Knowledge Rewrite in Compact LLMs: An ’Unlearn-then-Learn’ Strategy with ((IA) 3 ) for Localized Factual Modulation and Catastrophic Forgetting Mitigation by Stanley Ngugi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07075.txt.

Description

Stanley Ngugi rewrites one fact inside a compact language model without wrecking the rest, in a 2025 paper. Editing a model that deeply believes an old fact often fails twice: the model resists the new fact, and unrelated knowledge gets damaged. His plan first locates the inner parts that store the clashing fact. Then it unlearns that fact and teaches the new one, using a light adapter method.

On a small Microsoft model, the method installed the new fact with about 98 percent accuracy and suppressed the old one almost as well. Checks on unrelated knowledge stayed near 72 percent, far above direct fine-tuning, which fell near 20 percent. Probing showed the old fact was not erased but muted, still reachable under the right prompt. Ngugi frames that soft forgetting as a safety feature.

Catalog record

CREATORStanley Ngugi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-07075

Browse the full library · About the Tower of Babel