Revisiting Parameter Server in LLM Post-Training
Xinyi Wan, Penghui Qi, Guangxing Huang, Chaoyi Ruan, Min Lin & Jialin Li · 2026
Read Revisiting Parameter Server in LLM Post-Training by Xinyi Wan, Penghui Qi, Guangxing Huang, Chaoyi Ruan, Min Lin & Jialin Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19362.txt.
Description
Xinyi Wan and five co-authors bring back the parameter server for a new era of model training. Modern data parallel training favors collective communication, where all workers sync at once. That works when workloads are balanced, but language model post training has wild variance in sequence lengths.
Under imbalance, synced steps make fast devices wait on slow ones, wasting the fleet. The team revisits the older parameter server design, where workers report to a server on their own clock. For post training's lumpy loads, the old design may beat the new default.
Catalog record
| CREATOR | Xinyi Wan, Penghui Qi, Guangxing Huang, Chaoyi Ruan, Min Lin & Jialin Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-19362