Scalable Thermodynamic Second-order Optimization

Kaelan Donatella, Samuel Duffield, Denis Melanson, Maxwell Aifer, Phoebe Klett, Rajath Salegame, Zach Belateche, Gavin Crooks, Antonio J. Martinez, Patrick J. Coles · 2025

Read Scalable Thermodynamic Second-order Optimization by Kaelan Donatella, Samuel Duffield, Denis Melanson, Maxwell Aifer, Phoebe Klett, Rajath Salegame, Zach Belateche, Gavin Crooks, Antonio J. Martinez, Patrick J. Coles (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08603.txt.

Description

Kaelan Donatella and co-authors speed up AI training with physics-based chips in this 2025 paper. Training large models costs vast time and power. A class of chips that let physics do part of the math could run training steps that normal chips find too costly.

Their target is a strong training method that uses curve data to set step sizes. That method needs costly matrix flips, which the physics chips can ease. The team's scaled design predicts a growing edge as layers widen. Tests show the gains survive even with heavy rounding noise in the chip math. Forecasts for large vision and graph tasks point to strong speed gains.

Catalog record

CREATORKaelan Donatella, Samuel Duffield, Denis Melanson, Maxwell Aifer, Phoebe Klett, Rajath Salegame, Zach Belateche, Gavin Crooks, Antonio J. Martinez, Patrick J. Coles
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-08603

Browse the full library · About the Tower of Babel