Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash
Fucheng Jia, Zewen Wu, Shiqi Jiang, Huiqiang Jiang, Qianxi Zhang, Yuqing Yang, Yunxin Liu, Ju Ren, Deyu Zhang, Ting Cao · 2025
Read Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash by Fucheng Jia, Zewen Wu, Shiqi Jiang, Huiqiang Jiang, Qianxi Zhang, Yuqing Yang, Yunxin Liu, Ju Ren, Deyu Zhang, Ting Cao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-08378.txt.
Description
Phones cannot hold a big language model's full weights in fast memory, which caps how smart on-device AI can be. Fucheng Jia and nine co-authors present ActiveFlow in 2025, a framework that swaps the weights a model actually needs between flash storage and fast memory as it runs.
Only a slice of weights wakes up for each chunk of text. The system predicts which weights later layers will need, using signals from the current layer, so loading overlaps with computing. A self-distillation step keeps the trimmed model faithful to the full one. A scheduling pipeline then divides memory among cached, preloaded, and working weights. The authors report the best speed-cost balance among the methods they compared.
Catalog record
| CREATOR | Fucheng Jia, Zewen Wu, Shiqi Jiang, Huiqiang Jiang, Qianxi Zhang, Yuqing Yang, Yunxin Liu, Ju Ren, Deyu Zhang, Ting Cao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-08378