Scalable Thermodynamic Second-order Optimization
Kaelan Donatella, Samuel Duffield, Denis Melanson, Maxwell Aifer, Phoebe Klett, Rajath Salegame, Zach Belateche, Gavin Crooks, Antonio J. Martinez, Patrick J. Coles · 2025
Read Scalable Thermodynamic Second-order Optimization by Kaelan Donatella, Samuel Duffield, Denis Melanson, Maxwell Aifer, Phoebe Klett, Rajath Salegame, Zach Belateche, Gavin Crooks, Antonio J. Martinez, Patrick J. Coles (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08603.txt.
Description
Kaelan Donatella and co-authors speed up AI training with physics-based chips in this 2025 paper. Training large models costs vast time and power. A class of chips that let physics do part of the math could run training steps that normal chips find too costly.
Their target is a strong training method that uses curve data to set step sizes. That method needs costly matrix flips, which the physics chips can ease. The team's scaled design predicts a growing edge as layers widen. Tests show the gains survive even with heavy rounding noise in the chip math. Forecasts for large vision and graph tasks point to strong speed gains.
Catalog record
| CREATOR | Kaelan Donatella, Samuel Duffield, Denis Melanson, Maxwell Aifer, Phoebe Klett, Rajath Salegame, Zach Belateche, Gavin Crooks, Antonio J. Martinez, Patrick J. Coles |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-08603