LLMTailor: A Layer-wise Tailoring Tool for Efficient Checkpointing of Large Language Models Conference: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis; November 16–21, 2025; St Louis, MO, USA Workshops of the International Conference for Hi
Minqiu Sun, Xin Huang, Luanzheng Guo, Nathan R. Tallent, Kento Sato, Dong Dai · 2026
Read LLMTailor: A Layer-wise Tailoring Tool for Efficient Checkpointing of Large Language Models Conference: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis; November 16–21, 2025; St Louis, MO, USA Workshops of the International Conference for Hi by Minqiu Sun, Xin Huang, Luanzheng Guo, Nathan R. Tallent, Kento Sato, Dong Dai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-22158.txt.
Description
Training a large language model means checkpointing constantly against crashes, but saving the whole model and optimizer state every time is hugely wasteful. Minqiu Sun and five co-authors observe that layer updates are far from uniform: some layers shift a lot while others barely move. They propose LLMTailor, a tool that saves only layers with significant updates and assembles composite checkpoints from them.
The framework filters and merges layers from different checkpoints, giving fine-grained control over weights and optimizer states that no current tool offers. Evaluations show it works across selection strategies while cutting storage and contention. For training teams, it promises cheaper fault tolerance without giving up the safety net.
Catalog record
| CREATOR | Minqiu Sun, Xin Huang, Luanzheng Guo, Nathan R. Tallent, Kento Sato, Dong Dai |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-22158