Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries
Chris Kolb, Tobias Weber, Bernd Bischl, & David Rügamer · 2025
Read Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries by Chris Kolb, Tobias Weber, Bernd Bischl, & David Rügamer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-02496.txt.
Description
Making a neural network sparse usually means penalties that gradient descent cannot chew. Chris Kolb and co-authors take a side door in this 2025 paper. They split each weight into several multiplied factors, so smoothness returns. Ordinary penalties on the factors then act like sparsity penalties on the whole. They call the approach deep weight factorization.
The team proves the equivalence with non-convex sparse regularization and studies how training behaves, finding that the starting values and learning rate need special care. Across architectures and datasets, deep factorization beat its two-factor predecessor and popular pruning methods. The authors frame artificial symmetries in the weights as a tool, not a nuisance.
Catalog record
| CREATOR | Chris Kolb, Tobias Weber, Bernd Bischl, & David Rügamer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 120 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-02496