Gauss-Newton Unlearning for the LLM Era PubID: pubid: This work has been accepted for publication at the IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). The final version will be available on IEEE Xplore.

Lev McKinney 2 4 , Anvith Thudi 2 4 , Juhan Bae 2 , Tara Rezaei 3 , Nicolas Papernot 2 4 , Sheila A. McIlraith 2 4 , and Roger Grosse 2 · 2026

Read Gauss-Newton Unlearning for the LLM Era PubID: pubid: This work has been accepted for publication at the IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). The final version will be available on IEEE Xplore. by Lev McKinney 2 4 , Anvith Thudi 2 4 , Juhan Bae 2 , Tara Rezaei 3 , Nicolas Papernot 2 4 , Sheila A. McIlraith 2 4 , and Roger Grosse 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10568.txt.

Description

Lev McKinney and six co-authors offer a simpler route to making language models forget in a 2026 paper. Standard training can leave models producing outputs their trainers deem unacceptable, and unlearning a forget set often damages conduct on data meant to be retained. The trade-off between forgetting and retention is the central hardness.

The paper shows that a few uphill Gauss-Newton steps computed from the forget set alone give a conceptually simple, state-of-the-art unlearning method. Computing such steps for large models is nontrivial, so the approach leans on Kronecker-factored bend approximations. The combined method, K-FADE, is presented as a useful balance: strong forgetting with limited collateral damage to retained capabilities.

Catalog record

CREATORLev McKinney 2 4 , Anvith Thudi 2 4 , Juhan Bae 2 , Tara Rezaei 3 , Nicolas Papernot 2 4 , Sheila A. McIlraith 2 4 , and Roger Grosse 2
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-10568

Browse the full library · About the Tower of Babel