Learning to Forget using Hypernetworks
Jose Miguel Lara Rangel 1, Stefan Schoepf 1, Jack Foster 1, Usman Anwar 1 · 2024
Read Learning to Forget using Hypernetworks by Jose Miguel Lara Rangel 1, Stefan Schoepf 1, Jack Foster 1, Usman Anwar 1 (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-00761.txt.
Description
Jose Miguel Lara Rangel and co-authors teach models to forget on demand using networks that build other networks. Erasing one dataset's effect from a trained model is now a legal and safety need. Retraining from scratch is often too slow.
Their framework uses a hypernetwork, a network that outputs another network's weights. Guided by diffusion methods, it samples fresh models that never learned the target data. In proof-of-concept tests, the sampled models scored zero on the data meant to be forgotten. They kept strong accuracy on the data meant to stay. Two variants of the design were built and compared. Fast, targeted forgetting could help clean poisoned or private data.
Catalog record
| CREATOR | Jose Miguel Lara Rangel 1, Stefan Schoepf 1, Jack Foster 1, Usman Anwar 1 |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-00761