Accelerated training through iterative gradient propagation along the residual path
Erwan Fagnou, Paul Caillon, Blaise Delattre & Alexandre Allauzen · 2025
Read Accelerated training through iterative gradient propagation along the residual path by Erwan Fagnou, Paul Caillon, Blaise Delattre & Alexandre Allauzen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17086.txt.
Description
Erwan Fagnou and co-authors speed up deep network training in a 2025 paper. Backpropagation, the core training method, runs layer by layer in strict order, which caps how fast very deep models can learn. Residual links fixed vanishing gradients but left that serial cost in place.
Their method, Highway backpropagation, breaks the gradient into parts that flow along each path in the network. It gathers gradient guesses along the shortcut path, then pushes them through all layers in parallel, round after round. The loop fits ResNets, Transformers, and recurrent nets alike. Broad tests show large speed gains with only slight drops in scores.
Catalog record
| CREATOR | Erwan Fagnou, Paul Caillon, Blaise Delattre & Alexandre Allauzen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-17086