Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime
Reza Ghane, Danil Akhtiamov, Babak Hassibi · 2026
Read Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime by Reza Ghane, Danil Akhtiamov, Babak Hassibi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10485.txt.
Description
Reza Ghane, Danil Akhtiamov, and Babak Hassibi study dual space preconditioned gradient descent, a family that includes normalized gradient descent and gradient clipping, in the overparameterized regime. For a linear model with convex loss, they prove full-batch iterates converge exponentially fast to a point that fits the data exactly. That limit point depends on step size for general preconditioners, blurring any neat implicit bias story. They give an approximate bias bound there, and exact convergence to the plain descent limit for isotropic preconditioners, in batch and stochastic forms.
Catalog record
| CREATOR | Reza Ghane, Danil Akhtiamov, Babak Hassibi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50603 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.10485