Archit Patke
1 · 2025
Read Archit Patke by 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08090.txt.
Description
Archit Patke and co-authors present an autoscaler for serving large language models in a 2025 paper. The head title names the system Chiron. Cloud hosts must size their fleets as demand shifts. Requests split into live chats that need answers in seconds and batch jobs that can wait minutes or hours, and mixing the two can break both sets of promises.
Older scalers ignore those time promises, so they scale too much and waste hardware. Chiron uses a layered backpressure signal built from queue size, hardware use, and the promises themselves. It adjusts both the number of serving copies and their batch sizes. In tests, it met promises far more often than rival tools, with gains up to ninety percent in attainment. It also used graphics hardware up to seventy percent more efficiently.
Catalog record
| CREATOR | 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-08090