Temperature is All You Need for Generalization in Langevin Dynamics and other Markov Processes
Itamar Harel, Yonathan Wolanowsky, Gal Vardi, Nathan Srebro, Daniel Soudry · 2025
Read Temperature is All You Need for Generalization in Langevin Dynamics and other Markov Processes by Itamar Harel, Yonathan Wolanowsky, Gal Vardi, Nathan Srebro, Daniel Soudry (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19087.txt.
Description
Itamar Harel and co-authors prove that temperature alone controls how well noisy training generalizes in a 2025 paper. The training method under study wanders downhill on the loss while random noise shakes it, a scheme called Langevin dynamics. Models can be far larger than the data, yet the method still often works.
Their bound on the gap between training and test error depends on the noise level, the starting loss, and the sample count. It needs no assumptions about training time, mixing speed, model size, or slope sizes, which earlier bounds leaned on. The trick is a general fact about Markov processes with a Gibbs steady state, read through a second-law lens: the spread from the start stays bounded. The same proof covers other noisy training rules.
Catalog record
| CREATOR | Itamar Harel, Yonathan Wolanowsky, Gal Vardi, Nathan Srebro, Daniel Soudry |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 148 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-19087