Learning to Forget using Hypernetworks

Jose Miguel Lara Rangel 1, Stefan Schoepf 1, Jack Foster 1, Usman Anwar 1 · 2024

Read Learning to Forget using Hypernetworks by Jose Miguel Lara Rangel 1, Stefan Schoepf 1, Jack Foster 1, Usman Anwar 1 (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-00761.txt.

Description

Jose Miguel Lara Rangel and co-authors teach models to forget on demand using networks that build other networks. Erasing one dataset's effect from a trained model is now a legal and safety need. Retraining from scratch is often too slow.

Their framework uses a hypernetwork, a network that outputs another network's weights. Guided by diffusion methods, it samples fresh models that never learned the target data. In proof-of-concept tests, the sampled models scored zero on the data meant to be forgotten. They kept strong accuracy on the data meant to stay. Two variants of the design were built and compared. Fast, targeted forgetting could help clean poisoned or private data.

Catalog record

CREATORJose Miguel Lara Rangel 1, Stefan Schoepf 1, Jack Foster 1, Usman Anwar 1
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-00761

Browse the full library · About the Tower of Babel