PreFT : Prefill-only finetuning for inference efficiency
Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu, Dhruv Pai, Ben Keigwin, Dan Jurafsky, Christopher Potts · 2026
Read PreFT : Prefill-only finetuning for inference efficiency by Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu, Dhruv Pai, Ben Keigwin, Dan Jurafsky, Christopher Potts (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14217.txt.
Description
Lanpouthakoun and six co-authors propose PreFT, prefill-only tuning for fast inference. Large language models can now be tailored with parameter tuning, but serving user adapters harms speed, even with special kernels and memory handling. Prefill processes many tokens at once, while decode generates one token at a time, so decode runs far slower with many adapters. PreFT uses the adapter on prefill tokens only and discards it, lifting speed with minimal effect on performance. On 512 adapters for Llama 3.1 70B, PreFT reaches 1.9 times the speed of standard adapters.
Catalog record
| CREATOR | Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu, Dhruv Pai, Ben Keigwin, Dan Jurafsky, Christopher Potts |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 106138 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.14217