TimelyFreeze : Adaptive Parameter Freezing Mechanism for Pipeline Parallelism
Seonghye Cho, Jaemin Han, Hyunjin Kim, Euisoo Jung, Jae-Gil Lee · 2026
Read TimelyFreeze : Adaptive Parameter Freezing Mechanism for Pipeline Parallelism by Seonghye Cho, Jaemin Han, Hyunjin Kim, Euisoo Jung, Jae-Gil Lee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05754.txt.
Description
Training a model too large for one device splits it across a pipeline, and idle gaps called bubbles waste much of the hardware time. Seonghye Cho, Jaemin Han, and 3 co-authors propose TimelyFreeze, which freezes some parameters adaptively to skip backward computation and speed training. Earlier freezing methods often froze too much and cost accuracy.
TimelyFreeze models the pipeline schedule as a directed acyclic graph and solves a linear program for freeze ratios. The ratios minimize batch execution time while respecting accuracy constraints. On a large language model test, training throughput rose substantially, up to forty percent in the best case, with comparable accuracy. The authors report that convergence held up and that the method generalizes across pipeline parallel settings. The work targets teams that train frontier scale models on clusters. It shows scheduling mathematics, not just hardware, can reclaim idle pipeline time.
Catalog record
| CREATOR | Seonghye Cho, Jaemin Han, Hyunjin Kim, Euisoo Jung, Jae-Gil Lee |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-05754