KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
Zaifeng Pan, Ajjkumar Patel, Zhengding Hu, Yipeng Shen, Yue Guan, Wan-Lu Li, Lianhui Qin, Yida Wang, Yufei Ding · 2025
Read KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows by Zaifeng Pan, Ajjkumar Patel, Zhengding Hu, Yipeng Shen, Yue Guan, Wan-Lu Li, Lianhui Qin, Yida Wang, Yufei Ding (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07400.txt.
Description
Pan and co-authors speed up shared saves for teams of chat-bot aides. Such teams chain set aides, each with a fixed lead text. Saved key-value parts let the host skip redo work on each call. Stock hosts drop saves by last-use time, a blunt rule. That rule can toss a save just before its next use. Missed saves then force pricey redo work or swap loads.
KVFlow reads the team plan as a step chart to time saves. Each aide gets a steps-to-next-use score from that chart. Drop rules work at the node grain, tied to those scores. Saves prone to reuse stay, and shared stems stay too. A pre-load step shifts parts from CPU to GPU in the back. That load hides in the shade of work in flight.
Catalog record
| CREATOR | Zaifeng Pan, Ajjkumar Patel, Zhengding Hu, Yipeng Shen, Yue Guan, Wan-Lu Li, Lianhui Qin, Yida Wang, Yufei Ding |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-07400