SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud PubID: pubid: Accepted to IEEE INFOCOM Workshops 2026 (6G AI-RAN 2026), Tokyo, Japan. This arXiv version is a preprint / author version.
Hariz Yet, Nguyen Thanh Tam, Mao V. Ngo, Lim Yi Shen, Lin Wei, Jihong Park, Binbin Chen, and Tony Q. S. Quek · 2026
Read SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud PubID: pubid: Accepted to IEEE INFOCOM Workshops 2026 (6G AI-RAN 2026), Tokyo, Japan. This arXiv version is a preprint / author version. by Hariz Yet, Nguyen Thanh Tam, Mao V. Ngo, Lim Yi Shen, Lin Wei, Jihong Park, Binbin Chen, and Tony Q. S. Quek (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23722.txt.
Description
Hariz Yet and seven co-authors study SLA-aware distributed LLM inference across tiers. Embodied AI needs sub-second inference near the Radio Access Network, without disrupting real-time baseband work. The team reports reads from a 5G Standalone AI-RAN testbed.
The setup has on-device, RAN-edge, and cloud tiers. On-device runs stay multi-second. At the RAN edge, SLA feasibility is mainly set by model variant choice. Quantized models cluster below zero point five seconds, while unquantized and larger quantized models miss deadlines from stalls and queuing. In the cloud tier, meeting a zero point five second deadline is hard: thirty-two point nine percent of requests finish within zero point five seconds. All variants meet a one point zero second deadline. Under saturated downlink traffic with up to 20 concurrent clients, Multi-Instance GPU isolation preserves baseband timing health.
Catalog record
| CREATOR | Hariz Yet, Nguyen Thanh Tam, Mao V. Ngo, Lim Yi Shen, Lin Wei, Jihong Park, Binbin Chen, and Tony Q. S. Quek |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-23722