ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Training

Jie Ji 1 Gen Li 1 Kaiyuan Deng 2 Fatemeh Afghah 1 Xiaolong Ma 2 1 Clemson University 2 University of Arizona [email protected] · 2026

Read ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Training by Jie Ji 1 Gen Li 1 Kaiyuan Deng 2 Fatemeh Afghah 1 Xiaolong Ma 2 1 Clemson University 2 University of Arizona [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13115.txt.

Description

Jie Ji and co-authors cut the cost of sharpness-aware training for sparse neural networks. Sparse models save memory and compute, yet their gradient signals turn noisy at high sparsity. That noise hurts convergence and generalization in practice. Their ZO-SAM mixes zero-order estimates into the SAM update, so only one backpropagation step runs per perturbation. Gradient variance drops, and the flat minima that SAM seeks still guide training. Models also held up better under distribution shift.

Catalog record

CREATORJie Ji 1 Gen Li 1 Kaiyuan Deng 2 Fatemeh Afghah 1 Xiaolong Ma 2 1 Clemson University 2 University of Arizona [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39198
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13115

Browse the full library · About the Tower of Babel