Learning Provably Improves the Convergence of Gradient Descent
Qingyu Song Xiamen University [email protected] Wei Lin, Hong Xu The Chinese University of Hong Kong [email protected], [email protected] · 2025
Read Learning Provably Improves the Convergence of Gradient Descent by Qingyu Song Xiamen University [email protected] Wei Lin, Hong Xu The Chinese University of Hong Kong [email protected], [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-18092.txt.
Description
Qingyu Song, Wei Lin, and Hong Xu prove that learned optimizers can be trained reliably in this two thousand twenty-five paper. Learn-to-optimize methods train a neural net to solve optimization problems faster than hand-made rules. They shine in practice but lacked proof that their own training settles down. Existing theory leaned on unrealistic assumptions, which the authors show can fail. They prove training convergence for models that learn gradient descent settings on quadratic problems.
Their proof leans on a theory of very wide networks. A fixed starting scheme keeps training stable over long horizons. Tests beat plain gradient descent by a wide margin. Machine learning researchers who deploy learned solvers are the audience.
Catalog record
| CREATOR | Qingyu Song Xiamen University [email protected] Wei Lin, Hong Xu The Chinese University of Hong Kong [email protected], [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 203 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-18092