Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling DOI: XXXXXXX.XXXXXXX CCS: Computing methodologies Artificial intelligence Planning and scheduling Planning for deterministic CCS: Computing methodologies Planning and scheduli
Yihong Jin, Ze Yang · 2025
Read Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling DOI: XXXXXXX.XXXXXXX CCS: Computing methodologies Artificial intelligence Planning and scheduling Planning for deterministic CCS: Computing methodologies Planning and scheduli by Yihong Jin, Ze Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15296.txt.
Description
Yihong Jin and Ze Yang tackle a scaling problem for cloud AI services in this 2025 paper. Demand for AI inference swings up and down through the day. Too few servers slow replies, while too many waste money. The authors propose a framework that balances load in real time and scales capacity on its own.
One part of the system learns how to spread work across servers. Another part forecasts demand with a neural network, so resources can grow before a rush hits. Decisions are spread out rather than made in one place, which guards against single failures. Tests showed better load balance and shorter delays than standard methods, the authors report.
Catalog record
| CREATOR | Yihong Jin, Ze Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-15296