Optimal Brain Apoptosis

Mingyuan Sun, Zheng Fang, Jiaxu Wang, Junjie Jiang, Delei Kong, Chenming Hu, Yuetong Fang & Renjing Xu · 2025

Read Optimal Brain Apoptosis by Mingyuan Sun, Zheng Fang, Jiaxu Wang, Junjie Jiang, Delei Kong, Chenming Hu, Yuetong Fang & Renjing Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17941.txt.

Description

Mingyuan Sun and co-authors present Optimal Brain Apoptosis, a sharper method for pruning neural networks. Their paper appeared in 2025. Pruning deletes weak parameters so big models run cheaper. Classic methods estimate each parameter's importance with shortcuts. The new method computes a key curvature value, the Hessian-vector product, directly for each parameter. It splits the curvature matrix by layer and finds when cross-layer parts vanish.

That split makes the second-order cost estimate cheap and exact enough to trust. Tests on image data sets with several well-known networks, from VGG to vision transformers, backed the approach. Pruned models kept their accuracy at smaller size. The authors offer the method as a precise tool for shrinking modern networks, and they share the code.

Catalog record

CREATORMingyuan Sun, Zheng Fang, Jiaxu Wang, Junjie Jiang, Delei Kong, Chenming Hu, Yuetong Fang & Renjing Xu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-17941

Browse the full library · About the Tower of Babel