Learning-Time Encoding Shapes Unlearning in LLMs

Ruihan WuUC, San Diego [email protected], Konstantin GarovUC, San Diego [email protected] Kamalika ChaudhuriUC, San Diego [email protected] · 2025

Read Learning-Time Encoding Shapes Unlearning in LLMs by Ruihan WuUC, San Diego [email protected], Konstantin GarovUC, San Diego [email protected] Kamalika ChaudhuriUC, San Diego [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15076.txt.

Description

Ruihan Wu, Konstantin Garov, and Kamalika Chaudhuri ask how the way a fact is taught to a large language model affects how well that fact can later be removed. Such removal, called unlearning, is needed for privacy, for fixing stale facts, and for wiping harmful content.

Past work took the training path as fixed. Their tests vary it. Facts taught through reworded versions of the same point were easier to unlearn. By contrast, pulling one fact out of a chunk of text proved hard to unlearn cleanly. The authors argue that choices made at teaching time may be central to sound removal.

Catalog record

CREATORRuihan WuUC, San Diego [email protected], Konstantin GarovUC, San Diego [email protected] Kamalika ChaudhuriUC, San Diego [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-15076

Browse the full library · About the Tower of Babel