Is Gradient Ascent Really Necessary? Memorize to Forget for Machine Unlearning
Zhuo Huang, Qizhou Wang, Ziming Hong, Shanshan Ye, Bo Han, Tongliang Liu, [1ex] Sydney AI Centre, The University of Sydney; Hong Kong Baptist University · 2026
Read Is Gradient Ascent Really Necessary? Memorize to Forget for Machine Unlearning by Zhuo Huang, Qizhou Wang, Ziming Hong, Shanshan Ye, Bo Han, Tongliang Liu, [1ex] Sydney AI Centre, The University of Sydney; Hong Kong Baptist University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06441.txt.
Description
Forgetting in a trained model is usually forced by gradient ascent, running learning in reverse on the data to remove. Zhuo Huang and 5 co-authors warn that reversal can collapse the model's general skill. They propose extrapolation instead. First, the original model is trained further to memorize the unwanted data while keeping its other answers steady.
That memorization model and the original then define a direction in model space. Stepping from the memorization model through the original and beyond lands on a forget model, reached without any direct ascent. Training stays in the stable descent regime throughout. The authors report cleaner unlearning and better preserved performance, with a method simple to implement.
Catalog record
| CREATOR | Zhuo Huang, Qizhou Wang, Ziming Hong, Shanshan Ye, Bo Han, Tongliang Liu, [1ex] Sydney AI Centre, The University of Sydney; Hong Kong Baptist University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-06441