Rethinking the Potential of Layer Freezing for Efficient DNN Training
Chence Yang, Ci Zhang, Lei Lu, Qitao Tan, Sheng Li, Ao Li, Xulong Tang, Shaoyi Huang, Jinzhen Wang, Guoming Li, Jundong Li, Xiaoming Zhai, Jin Lu, Geng Yuan · 2025
Read Rethinking the Potential of Layer Freezing for Efficient DNN Training by Chence Yang, Ci Zhang, Lei Lu, Qitao Tan, Sheng Li, Ao Li, Xulong Tang, Shaoyi Huang, Jinzhen Wang, Guoming Li, Jundong Li, Xiaoming Zhai, Jin Lu, Geng Yuan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15033.txt.
Description
Chence Yang and a large team of co-authors rethought layer freezing in 2025 as a way to train deep networks more cheaply. Freezing early layers stops their updates, but in the classic method those layers still run forward passes to feed the rest. A proposed fix caches their outputs once and trains the upper layers on the stored maps, skipping the forward cost.
That shortcut hides challenges that past work largely skipped, such as how to apply data augmentation to cached maps. The paper revisits the true potential of freezing with those problems in view. Readers get a more honest ledger for a popular cost-cutting trick: what it saves, what it breaks, and what it would take to make the cached version work.
Catalog record
| CREATOR | Chence Yang, Ci Zhang, Lei Lu, Qitao Tan, Sheng Li, Ao Li, Xulong Tang, Shaoyi Huang, Jinzhen Wang, Guoming Li, Jundong Li, Xiaoming Zhai, Jin Lu, Geng Yuan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-15033