SkyLB: A Locality-Aware Cross-Region Load Balancer for LLM Inference

Tian Xia † Ziming Mao † Jamison Kerney † Ethan J. Jackson †‡ Zhifei Li †§ Jiarong Xing †¶ Scott Shenker †⋄ Ion Stoica † † UC Berkeley ‡ Aviatrix § Renmin University of China ¶ Rice University ⋄ ICSI · 2025

Read SkyLB: A Locality-Aware Cross-Region Load Balancer for LLM Inference by Tian Xia † Ziming Mao † Jamison Kerney † Ethan J. Jackson †‡ Zhifei Li †§ Jiarong Xing †¶ Scott Shenker †⋄ Ion Stoica † † UC Berkeley ‡ Aviatrix § Renmin University of China ¶ Rice University ⋄ ICSI (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24095.txt.

Description

Tian Xia and co-authors present SkyWalker, a load balancer for language model services across regions, in this 2025 paper. Providers run models in several regions on reserved machines. Local-only traffic handling leaves many of those machines idle for hours each day.

Demand rises and falls with local time in each region. SkyWalker shifts work across regions to smooth those daily waves. Providers can then reserve capacity based on world demand, not each region's peak. That change should cut cost while keeping response times steady. The design suits any service whose users sleep on different schedules. Big model hosts face exactly that pattern today. Cloud costs fall when fewer machines sit half empty overnight.

Catalog record

CREATORTian Xia † Ziming Mao † Jamison Kerney † Ethan J. Jackson †‡ Zhifei Li †§ Jiarong Xing †¶ Scott Shenker †⋄ Ion Stoica † † UC Berkeley ‡ Aviatrix § Renmin University of China ¶ Rice University ⋄ ICSI
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-24095

Browse the full library · About the Tower of Babel