ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
Yuxing Xiang 1,2 {}^{\text{1,2}} Xue Li 2 {}^{\text{2}} Kun Qian 2 {}^{\text{2}} Wenyuan Yu 2 {}^{\text{2}} Ennan Zhai 2 {}^{\text{2}} Xin Jin 1 {}^{\text{1}} 1 {}^{\text{1}} School of Computer Science, Peking University 2 {}^{\text{2}} Alibaba Group · 2025
Read ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production by Yuxing Xiang 1,2 {}^{\text{1,2}} Xue Li 2 {}^{\text{2}} Kun Qian 2 {}^{\text{2}} Wenyuan Yu 2 {}^{\text{2}} Ennan Zhai 2 {}^{\text{2}} Xin Jin 1 {}^{\text{1}} 1 {}^{\text{1}} School of Computer Science, Peking University 2 {}^{\text{2}} Alibaba Group (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-09999.txt.
Description
Yuxing Xiang and co-authors study the real workload of serving large language models in a live cloud service, in a paper from 2025. Serving means answering user requests with a trained model, at scale and with low delay. Good test workloads are key to building fast, well sized systems.
The team drew data from a world-wide cloud service and mapped patterns for text models as well as newer image-aware and reasoning models. Past studies were smaller and missed many of these traits. Based on their findings, the authors built ServeGen, a tool that creates realistic workloads client by client. In a real use case, the tool avoided a large shortfall in capacity that a naive method would have caused. The tool is shared in a public link.
Catalog record
| CREATOR | Yuxing Xiang 1,2 {}^{\text{1,2}} Xue Li 2 {}^{\text{2}} Kun Qian 2 {}^{\text{2}} Wenyuan Yu 2 {}^{\text{2}} Ennan Zhai 2 {}^{\text{2}} Xin Jin 1 {}^{\text{1}} 1 {}^{\text{1}} School of Computer Science, Peking University 2 {}^{\text{2}} Alibaba Group |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-09999