SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud PubID: pubid: Accepted to IEEE INFOCOM Workshops 2026 (6G AI-RAN 2026), Tokyo, Japan. This arXiv version is a preprint / author version.

Hariz Yet, Nguyen Thanh Tam, Mao V. Ngo, Lim Yi Shen, Lin Wei, Jihong Park, Binbin Chen, and Tony Q. S. Quek · 2026

Read SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud PubID: pubid: Accepted to IEEE INFOCOM Workshops 2026 (6G AI-RAN 2026), Tokyo, Japan. This arXiv version is a preprint / author version. by Hariz Yet, Nguyen Thanh Tam, Mao V. Ngo, Lim Yi Shen, Lin Wei, Jihong Park, Binbin Chen, and Tony Q. S. Quek (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23722.txt.

Description

Hariz Yet and seven co-authors study SLA-aware distributed LLM inference across tiers. Embodied AI needs sub-second inference near the Radio Access Network, without disrupting real-time baseband work. The team reports reads from a 5G Standalone AI-RAN testbed.

The setup has on-device, RAN-edge, and cloud tiers. On-device runs stay multi-second. At the RAN edge, SLA feasibility is mainly set by model variant choice. Quantized models cluster below zero point five seconds, while unquantized and larger quantized models miss deadlines from stalls and queuing. In the cloud tier, meeting a zero point five second deadline is hard: thirty-two point nine percent of requests finish within zero point five seconds. All variants meet a one point zero second deadline. Under saturated downlink traffic with up to 20 concurrent clients, Multi-Instance GPU isolation preserves baseband timing health.

Catalog record

CREATORHariz Yet, Nguyen Thanh Tam, Mao V. Ngo, Lim Yi Shen, Lin Wei, Jihong Park, Binbin Chen, and Tony Q. S. Quek
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-23722

Browse the full library · About the Tower of Babel