ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Training
Jie Ji 1 Gen Li 1 Kaiyuan Deng 2 Fatemeh Afghah 1 Xiaolong Ma 2 1 Clemson University 2 University of Arizona [email protected] · 2026
Read ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Training by Jie Ji 1 Gen Li 1 Kaiyuan Deng 2 Fatemeh Afghah 1 Xiaolong Ma 2 1 Clemson University 2 University of Arizona [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13115.txt.
Description
Jie Ji and co-authors cut the cost of sharpness-aware training for sparse neural networks. Sparse models save memory and compute, yet their gradient signals turn noisy at high sparsity. That noise hurts convergence and generalization in practice. Their ZO-SAM mixes zero-order estimates into the SAM update, so only one backpropagation step runs per perturbation. Gradient variance drops, and the flat minima that SAM seeks still guide training. Models also held up better under distribution shift.
Catalog record
| CREATOR | Jie Ji 1 Gen Li 1 Kaiyuan Deng 2 Fatemeh Afghah 1 Xiaolong Ma 2 1 Clemson University 2 University of Arizona [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39198 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.13115