SkyLB: A Locality-Aware Cross-Region Load Balancer for LLM Inference
Tian Xia † Ziming Mao † Jamison Kerney † Ethan J. Jackson †‡ Zhifei Li †§ Jiarong Xing †¶ Scott Shenker †⋄ Ion Stoica † † UC Berkeley ‡ Aviatrix § Renmin University of China ¶ Rice University ⋄ ICSI · 2025
Read SkyLB: A Locality-Aware Cross-Region Load Balancer for LLM Inference by Tian Xia † Ziming Mao † Jamison Kerney † Ethan J. Jackson †‡ Zhifei Li †§ Jiarong Xing †¶ Scott Shenker †⋄ Ion Stoica † † UC Berkeley ‡ Aviatrix § Renmin University of China ¶ Rice University ⋄ ICSI (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24095.txt.
Description
Tian Xia and co-authors present SkyWalker, a load balancer for language model services across regions, in this 2025 paper. Providers run models in several regions on reserved machines. Local-only traffic handling leaves many of those machines idle for hours each day.
Demand rises and falls with local time in each region. SkyWalker shifts work across regions to smooth those daily waves. Providers can then reserve capacity based on world demand, not each region's peak. That change should cut cost while keeping response times steady. The design suits any service whose users sleep on different schedules. Big model hosts face exactly that pattern today. Cloud costs fall when fewer machines sit half empty overnight.
Catalog record
| CREATOR | Tian Xia † Ziming Mao † Jamison Kerney † Ethan J. Jackson †‡ Zhifei Li †§ Jiarong Xing †¶ Scott Shenker †⋄ Ion Stoica † † UC Berkeley ‡ Aviatrix § Renmin University of China ¶ Rice University ⋄ ICSI |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24095