Compact LLM Deployment and World Model Assisted Offloading in Mobile Edge Computing

Ruichen Zhang, Xiaofeng Luo, Jiayi He, Dusit Niyato, Jiawen Kang · 2026

Read Compact LLM Deployment and World Model Assisted Offloading in Mobile Edge Computing by Ruichen Zhang, Xiaofeng Luo, Jiayi He, Dusit Niyato, Jiawen Kang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13628.txt.

Introduction

Running large language models on edge devices means shrinking them without breaking them, then deciding what to run locally versus offload. The ECLD framework jointly applies structured pruning, low-bit quantization, and knowledge distillation to build edge-deployable variants, judged on accessibility, energy use, hallucination rate, and generalization accuracy. On Llama-3.1-8B, Qwen3-8B, and Mistral-12B, compression reached about 70 to 80 percent in storage, with per-query energy down up to 50 percent, largely preserving accuracy and often lowering hallucination.

Offloading is then optimized as a long-term latency problem under per-device energy budgets and quality constraints. It is solved by a world-model proximal policy optimization algorithm that adds a learned recurrent world model to on-policy PPO. That method converged about 50 percent faster, improved final reward 15.8 percent over vanilla PPO, and cut latency 12 to 30 percent. Ruichen Zhang leads the work with five co-authors.

Catalog record

CREATORRuichen Zhang, Xiaofeng Luo, Jiayi He, Dusit Niyato, Jiawen Kang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-13628

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Compact LLM Deployment and World Model Assisted Offloading in Mobile Edge Computing Thanks: R. Zhang and D. Niyato are with the College of Computing and Data Science, Nanyang Technological University, Singapore (e-mail: [email protected], [email protected]). Thanks: X. Luo, J. He, and J. Kang are with the School of Automation, Guangdong University of Technology, Guangzhou 510006, China (e-mail: [email protected], [email protected], [email protected]). Thanks: Z. Xiong is with the School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, BT9 5BN Belfast, U.K. (email: [email protected]). Thanks: Y. Li is with the School of Electrical and Information Engineering, University of Sydney, Sydney, NSW 2006, Australia (e-mail: [email protected]). Ruichen Zhang, Xiaofeng Luo, Jiayi He, Dusit Niyato, Jiawen Kang Abstract This paper investigates compact large language model (LLM) deployment and world-model-assisted inference offloading in mobile edge computing (MEC) networks. We first propose an edge compact LLM deployment (ECLD) framework that jointly applies structured pruning, low-bit quantization, and knowledge distillation to construct edge-deployable LLM variants, and we evaluate these models using four complementary metrics: accessibility, energy consumption, hallucination rate, and generalization accuracy. Building on the resulting compact models, we formulate an MEC offloading optimization problem that minimizes the long-term average inference latency subject to per-device energy budgets and LLM-specific quality-of-service constraints on effective accuracy and hallucination. To solve this problem under unknown and time-varying network dynamics, we develop a world model-proximal policy optimization (PPO) algorithm, which augments an on-policy PPO algorithm with a learned recurrent world model that provides improved value targets and short imagination rollouts. Extensive experiments on Llama-3.1-8B, Qwen3-8B, and Mistral-12B show that ECLD compresses base models by about 70–80% in storage (i.e., from 15.3 GB to 3.3 GB for Llama-3.1-8B) and reduces per-query energy consumption by up to 50%, while largely preserving accuracy and often lowering hallucination compared with quantization-only or pruning-only baselines. Moreover, they also show that world model-PPO speeds up convergence by about 50%, improves the final reward by 15.8% over vanilla PPO, and reduces average inference latency by 12–30% across different user populations, while satisfying the accuracy and hallucination constraints and approaching the generation quality of always-offloading with much of the efficiency of local execution. I Introduction The recent rise of generative artificial intelligence (GAI) has led to the rapid development of large language models (LLMs), such as OpenAI’s ChatGPT and DeepSeek’s R1, which demonstrate remarkable capabilities in text generation, reasoning, and multimodal understanding [ 1 , 2 ] . These frontier models are typically of GPT-3 scale or larger, with on the order of 10^{11} – 10^{12} parameters [ 3 ] . For instance, a 175B-parameter model requires hundreds of gigabytes of memory in FP16 merely to store the weights, while smaller open-source models with 7 – 13 B parameters still demand tens of gigabytes in standard precision. In contrast, mainstream smartphones and edge devices usually provide only 4 – 12 GB of RAM and operate under strict power and thermal limitations, making direct deployment of such LLMs on end devices infeasible [ 4 ] . As a result, most commercial services still rely on cloud-hosted LLMs, with user prompts being transmitted from edge devices to remote data centers [ 5 ] . Although this cloud-centric architecture can deliver strong performance, it exhibits two inherent drawbacks: ( i ) The end-to-end quality of experience is dominated by wide-area network conditions, where round-trip times of hundreds of milliseconds are common even before model computation [ 5 ] . ( ii ) The service remains tightly coupled to backhaul connectivity, which is undesirable for latency-sensitive or intermittently connected edge applications [ 6 ] . To alleviate these limitations, there is a growing interest in compact LLMs that can be deployed locally on mobile phones, tablets, and edge nodes [ 5 ] . By pruning redundant parameters [ 7 ] , distilling knowledge from larger teacher models [ 8 ] , and quantizing weights to low-bit precision [ 9 ] , compact LLMs can reduce the memory footprint from tens of gigabytes to a few gigabytes and substantially lower the per-token computation cost, while still preserving acceptable task performance on many everyday workloads. Nevertheless, purely on-device inference is often insufficient in practice: local compact LLMs may still struggle with complex instructions, domain-specific queries, or long-context reasoning, and their behavior (e.g., accuracy and hallucination rate) is highly sensitive to resource constraints and quantization levels. This motivates a collaborative mobile edge computing (MEC) paradigm, in which each user device runs a lightweight LLM locally and selectively offloads part of the inference workload to an edge server hosting a full-scale LLM [ 10 , 4 ] . In this setting, the key question becomes how to jointly design compact LLM deployment and edge-assisted offloading so that users can enjoy low-latency, energy-efficient, and trustworthy LLM services. However, realizing such an edge-assisted compact LLM system faces several fundamental challenges, as elaborated below. I). Compact LLM Deployment Under Diverse Device Capabilities: Mobile and edge devices span a wide range of CPU configurations, memory capacities, and energy budgets: for example, high-end smartphones may provide several performance cores and 8 – 12 GB of RAM, whereas low-end handsets or IoT nodes often operate with only a few gigabytes of memory and tight battery constraints [ 11 , 12 ] . Designing a single compression configuration for all devices is therefore suboptimal: aggressive pruning and quantization can severely degrade accuracy or exacerbate hallucinations on resource-limited platforms, while conservative compression wastes memory and compute on more capable devices and limits the achievable latency reduction [ 13 , 14 ] . This calls for a systematic deployment framework that jointly coordinates pruning, knowledge distillation, and quantization in a hardware-aware manner. The key challenge is to co-design these steps in a unified pipeline so that the resulting compact LLMs remain lightweight enough for on-device deployment yet accurate enough to support end-to-end LLM services. II). MEC Offloading and QoS-Aware Adaptive Resource Control: Compact-LLM–enabled MEC systems also operate under highly dynamic and diverse conditions in terms of traffic arrivals, wireless channels, and user-level quality-of-service (QoS) requirements. For instance, bursty inference requests from multiple users must be served over interference-limited uplinks, while different applications may impose distinct latency, energy, or content-fidelity targets [ 15 ] . Designing a fixed offloading or power-control rule for all scenarios is therefore suboptimal: aggressive local execution on compact models can violate accuracy or hallucination constraints when tasks are complex or device resources are scarce [ 16 ] , whereas always offloading to the edge server improves QoS but incurs excessive transmission delay and uplink power consumption [ 6 ] . This motivates a QoS-aware adaptive control framework that jointly tunes offloading ratios and transmit powers based on real-time network states while explicitly accounting for application-level QoS and environment uncertainty. The key challenge is to learn such a control policy directly from interaction with the MEC environment in a sample-efficient and stable manner, motivating a world-model–enhanced design that plans over imagined trajectories rather than reacting myopically to instantaneous feedback. To tackle these challenges, two important families of techniques have emerged, i.e., LLM compression and edge deployment and world model-reinforcement learning for adaptive MEC control. I). LLM Compression and Edge Deployment: Recent studies on LLM compression have explored several model–shrinking techniques, including structured pruning,

Read the complete text