Gauss-Newton Unlearning for the LLM Era PubID: pubid: This work has been accepted for publication at the IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). The final version will be available on IEEE Xplore.
Lev McKinney 2 4 , Anvith Thudi 2 4 , Juhan Bae 2 , Tara Rezaei 3 , Nicolas Papernot 2 4 , Sheila A. McIlraith 2 4 , and Roger Grosse 2 · 2026
Read Gauss-Newton Unlearning for the LLM Era PubID: pubid: This work has been accepted for publication at the IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). The final version will be available on IEEE Xplore. by Lev McKinney 2 4 , Anvith Thudi 2 4 , Juhan Bae 2 , Tara Rezaei 3 , Nicolas Papernot 2 4 , Sheila A. McIlraith 2 4 , and Roger Grosse 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10568.txt.
Description
Lev McKinney and six co-authors offer a simpler route to making language models forget in a 2026 paper. Standard training can leave models producing outputs their trainers deem unacceptable, and unlearning a forget set often damages conduct on data meant to be retained. The trade-off between forgetting and retention is the central hardness.
The paper shows that a few uphill Gauss-Newton steps computed from the forget set alone give a conceptually simple, state-of-the-art unlearning method. Computing such steps for large models is nontrivial, so the approach leans on Kronecker-factored bend approximations. The combined method, K-FADE, is presented as a useful balance: strong forgetting with limited collateral damage to retained capabilities.
Catalog record
| CREATOR | Lev McKinney 2 4 , Anvith Thudi 2 4 , Juhan Bae 2 , Tara Rezaei 3 , Nicolas Papernot 2 4 , Sheila A. McIlraith 2 4 , and Roger Grosse 2 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-10568