BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems
Konstantin Burlachenko, Peter Richtárik · 2025
Read BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems by Konstantin Burlachenko, Peter Richtárik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13795.txt.
Introduction
Konstantin Burlachenko and Peter Richtárik rebuild neural network training from scratch and find the giants wasteful. Their framework, BurTorch, computes gradients on plain processors with a lean, hand tuned design. Big frameworks carry heavy overhead that swamps small models. On small computation graphs, BurTorch ran up to thousands of times faster and used a sliver of the memory.
Even a shrunken GPT style model trained far faster than in a leading framework. The codebase stays compact and script friendly. The authors argue that classical compiled languages still have much to give deep learning research, especially for quick experiments on one machine.
Catalog record
| CREATOR | Konstantin Burlachenko, Peter Richtárik |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 129 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13795