Learning Provably Improves the Convergence of Gradient Descent

Qingyu Song Xiamen University [email protected] Wei Lin, Hong Xu The Chinese University of Hong Kong [email protected], [email protected] · 2025

Read Learning Provably Improves the Convergence of Gradient Descent by Qingyu Song Xiamen University [email protected] Wei Lin, Hong Xu The Chinese University of Hong Kong [email protected], [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-18092.txt.

Description

Qingyu Song, Wei Lin, and Hong Xu prove that learned optimizers can be trained reliably in this two thousand twenty-five paper. Learn-to-optimize methods train a neural net to solve optimization problems faster than hand-made rules. They shine in practice but lacked proof that their own training settles down. Existing theory leaned on unrealistic assumptions, which the authors show can fail. They prove training convergence for models that learn gradient descent settings on quadratic problems.

Their proof leans on a theory of very wide networks. A fixed starting scheme keeps training stable over long horizons. Tests beat plain gradient descent by a wide margin. Machine learning researchers who deploy learned solvers are the audience.

Catalog record

CREATORQingyu Song Xiamen University [email protected] Wei Lin, Hong Xu The Chinese University of Hong Kong [email protected], [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE203 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-18092

Browse the full library · About the Tower of Babel