BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems

Konstantin Burlachenko, Peter Richtárik · 2025

Read BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems by Konstantin Burlachenko, Peter Richtárik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13795.txt.

Introduction

Konstantin Burlachenko and Peter Richtárik rebuild neural network training from scratch and find the giants wasteful. Their framework, BurTorch, computes gradients on plain processors with a lean, hand tuned design. Big frameworks carry heavy overhead that swamps small models. On small computation graphs, BurTorch ran up to thousands of times faster and used a sliver of the memory.

Even a shrunken GPT style model trained far faster than in a leading framework. The codebase stays compact and script friendly. The authors argue that classical compiled languages still have much to give deep learning research, especially for quick experiments on one machine.

Catalog record

CREATORKonstantin Burlachenko, Peter Richtárik
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE129 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-13795

Browse the full library · About the Tower of Babel