Adaptive Regularization for Sparsity Control in Bregman-Based Optimizers
Ahmad Aloradi, Tim Roith, Emanuël A. P. Habets, Daniel Tenbrinck · 2026
Read Adaptive Regularization for Sparsity Control in Bregman-Based Optimizers by Ahmad Aloradi, Tim Roith, Emanuël A. P. Habets, Daniel Tenbrinck (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07892.txt.
Description
Ahmad Aloradi and three co-authors make sparse neural network training hit a chosen sparsity target without expensive tuning. Sparsity is usually steered indirectly by a regularization weight whose effect on the final sparsity is hard to predict. For Bregman-based optimizers the team found the same target could need weights differing by a hundredfold between variants. Trial-and-error sweeps were the only route.
Their adaptive scheme updates the weight on the fly from the gap between current and target sparsity. Tested on automatic speaker verification with two architectures and two datasets, it reliably landed targets between seventy-five and ninety-nine percent sparsity. It also converged faster early than an oracle-tuned fixed-weight baseline, matched or beat its final error rate, and kept the robustness advantages of the non-adaptive method.
Catalog record
| CREATOR | Ahmad Aloradi, Tim Roith, Emanuël A. P. Habets, Daniel Tenbrinck |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07892