Not All Prefills Are Equal: PPD Disaggregation for Multi-turn LLM Serving

Zongze Li, Jingyu Liu, Zach Xu, Yineng Zhang, Tahseen Rabbani, Ce Zhang · 2026

Read Not All Prefills Are Equal: PPD Disaggregation for Multi-turn LLM Serving by Zongze Li, Jingyu Liu, Zach Xu, Yineng Zhang, Tahseen Rabbani, Ce Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13358.txt.

Description

Zongze Li and co-authors rethink prefill-decode disaggregation for multi-turn LLM serving. Each new turn now forces a fresh prefill and repeated KV transfers, which clog bandwidth. Their key insight is that append-prefill, which reuses cached KV states, slows decoding far less than full prefill. That makes local routing to decode nodes a good option for later turns. No fixed routing plan meets all service goals at once, they show. Their PPD system routes by dynamic weights and cut time-to-first-token by about 68 percent.

Catalog record

CREATORZongze Li, Jingyu Liu, Zach Xu, Yineng Zhang, Tahseen Rabbani, Ce Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44238
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13358

Browse the full library · About the Tower of Babel