Not All Prefills Are Equal: PPD Disaggregation for Multi-turn LLM Serving
Zongze Li, Jingyu Liu, Zach Xu, Yineng Zhang, Tahseen Rabbani, Ce Zhang · 2026
Read Not All Prefills Are Equal: PPD Disaggregation for Multi-turn LLM Serving by Zongze Li, Jingyu Liu, Zach Xu, Yineng Zhang, Tahseen Rabbani, Ce Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13358.txt.
Description
Zongze Li and co-authors rethink prefill-decode disaggregation for multi-turn LLM serving. Each new turn now forces a fresh prefill and repeated KV transfers, which clog bandwidth. Their key insight is that append-prefill, which reuses cached KV states, slows decoding far less than full prefill. That makes local routing to decode nodes a good option for later turns. No fixed routing plan meets all service goals at once, they show. Their PPD system routes by dynamic weights and cut time-to-first-token by about 68 percent.
Catalog record
| CREATOR | Zongze Li, Jingyu Liu, Zach Xu, Yineng Zhang, Tahseen Rabbani, Ce Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44238 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.13358