Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling DOI: XXXXXXX.XXXXXXX CCS: Computing methodologies Artificial intelligence Planning and scheduling Planning for deterministic CCS: Computing methodologies Planning and scheduli

Yihong Jin, Ze Yang · 2025

Read Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling DOI: XXXXXXX.XXXXXXX CCS: Computing methodologies Artificial intelligence Planning and scheduling Planning for deterministic CCS: Computing methodologies Planning and scheduli by Yihong Jin, Ze Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15296.txt.

Description

Yihong Jin and Ze Yang tackle a scaling problem for cloud AI services in this 2025 paper. Demand for AI inference swings up and down through the day. Too few servers slow replies, while too many waste money. The authors propose a framework that balances load in real time and scales capacity on its own.

One part of the system learns how to spread work across servers. Another part forecasts demand with a neural network, so resources can grow before a rush hits. Decisions are spread out rather than made in one place, which guards against single failures. Tests showed better load balance and shorter delays than standard methods, the authors report.

Catalog record

CREATORYihong Jin, Ze Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-15296

Browse the full library · About the Tower of Babel