Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading
Minrui Xu, Dusit Niyato, Fellow, IEEE, Christopher G. Brinton, Senior Member, IEEE · 2025
Read Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading by Minrui Xu, Dusit Niyato, Fellow, IEEE, Christopher G. Brinton, Senior Member, IEEE (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-14205.txt.
Introduction
Minrui Xu, Dusit Niyato, and Christopher Brinton kept long-context text models responsive at the mobile edge in a 2025 paper. Phones lean on nearby edge servers, but a chatting model keeps learning from its growing context. Which model to cache and where to run each request shifts minute by minute, unlike static offloading problems. Their setup jointly decides caching and offloading with a reinforcement learner that keeps learning at test time. Adapting to live usage patterns rather than freezing after training.
Convergence of the learner is analyzed, and the tuning accounts for how context windows are used. A double Dutch auction mechanism sharpens resource allocation among competing users. Edge serving becomes a moving target the system tracks deliberately.
Catalog record
| CREATOR | Minrui Xu, Dusit Niyato, Fellow, IEEE, Christopher G. Brinton, Senior Member, IEEE |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-14205
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading Minrui Xu, Dusit Niyato, Fellow, IEEE, Christopher G. Brinton, Senior Member, IEEE Abstract Large Language Models (LLMs) can perform zero-shot learning on unseen tasks and few-shot learning on complex reasoning tasks. However, resource-limited mobile edge networks struggle to support long-context LLM serving for LLM agents during multi-round interactions with users. Unlike stateless computation offloading and static service offloading in edge computing, optimizing LLM serving at edge servers is challenging because LLMs continuously learn from context which raises accuracy, latency, and resource consumption dynamics. In this paper, we propose a joint model caching and inference offloading framework that utilizes test-time deep reinforcement learning (T2DRL) to optimize deployment and execution strategies for long-context LLM serving. In this framework, we analyze the performance convergence and design an optimization problem considering the utilization of context windows in LLMs. Furthermore, the T2DRL algorithm can learn in both the training phase and the testing phase to proactively manage cached models and service requests and adapt to context changes and usage patterns during execution. To further enhance resource allocation efficiency, we propose a double Dutch auction (DDA) mechanism, which dynamically matches supply and demand while maximizing social welfare. Finally, experimental results demonstrate that the T2DRL algorithm can reduce system costs by at least 30% compared to baselines while guaranteeing the performance of LLM agents in real-world perception and reasoning tasks. I Introduction Large Language Models (LLMs) [ 1 ] have billions of parameters and demonstrate emerging capability through pre-training on internet-scale datasets. In mobile edge networks, LLMs can empower mobile devices to precept environments and plan for future actions, for running LLM agents on devices and enabling zero-shot learning for unseen perception tasks and few-shot learning for complex reasoning tasks [ 2 , 3 ] . For instance, LLMs can empower mobile devices with environmental understanding and reasoning capabilities, allowing them to adapt flexibly to changing conditions [ 4 ] . Furthermore, LLMs improve the experience of user interaction and achieve automation for mobile devices during the inference process by leveraging the contextual data derived from environmental perceptions and user interactions. However, how to achieve a balance among context, accuracy, and performance for long-context LLM serving in resource-constraint mobile edge networks is still an open question [ 5 ] . Fig. 1: Serving LLMs to handle inputs and tackle complex tasks using CoT prompting in mobile edge networks. As illustrated in Fig. 1 , resource-intensive tasks such as LLM serving, i.e., the deployment and inference of LLMs to perform inference tasks, in mobile edge networks [ 5 ] , often need to be offloaded to edge servers (ESs) or cloud data centers to reduce latency and improve resource utilization efficiency [ 6 ] . Due to the unique context-aware characteristics of LLMs that they can learn in-context examples during few-shot learning [ 2 ] , LLM serving at ESs is significantly different from computation and service offloading [ 7 , 8 ] . Specifically, computation offloading involves transferring computation-intensive tasks from local devices to ESs or a cloud data center [ 9 ] , which is stateless and independent of previous computation outcomes or the historical results of the servers. Similarly, service offloading involves sending user inputs to ESs to invoke pre-cached services [ 10 ] , which is static and remains unchanged across different invocations. As a result, existing optimization frameworks for computation offloading and service offloading are no longer effective for optimizing long-context LLM serving, as they fail to accommodate the dynamic learning capabilities of LLMs and the need for service adaptation based on evolving model contexts. Similar to content caching and service caching in mobile edge networks, model caching for LLMs emerges as a promising solution for provisioning LLM agents with cached LLMs at ESs [ 11 , 12 ] . Specifically, LLMs can learn from input examples and adjust the output by eliciting their reasoning capability via prompting techniques, such as Chain-of-Thought (CoT) [ 13 ] and Self-Consistent CoT [ 14 ] prompting. These CoT prompting can be utilized to improve accuracy but introduce additional resource consumption and latency. Nevertheless, the performance of the LLM agent is not only constrained by the hardware resources available on ESs but also limited by the context windows of LLMs, which are determined by the model architecture. If the model’s inference tokens exceed the context window, the cached model needs to be evicted; otherwise, it will result in a significant decline in model performance. To optimize long-context LLM serving in mobile edge networks, it is essential to actively manage the cached LLMs and model inference requests, adapting to the state transitions of inference requests and cached models. In this paper, we propose a joint model caching and inference offloading framework for long-context LLM serving in mobile edge networks, which allows LLMs to adopt various prompting techniques. In this framework, we formulate the problem of long-context LLM serving as a mixed-integer programming problem to minimize the system cost under the hardware resource constraints and context window constraints. To evaluate the performance of LLMs theoretically, we analyze a convergence rate of language ambiguity with respect to the reasoning paths and steps. To obtain optimal cache and inference management strategies, we propose a test-time deep reinforcement learning (T2DRL) algorithm [ 15 , 16 ] utilizing test-time training (TTT) models as the actor-critic network that can learn not only from historical experience during training but also from real-time interaction during testing. By treating the hidden state as a learnable model, the TTT model-based policy can be updated through self-supervised learning during test time, which overcomes limited expressive power in Long Short-Term Memory (LSTM) networks and the quadratic complexity of attention mechanisms. Based on the optimization results of T2DRL, we formulate the long-context LLM serving market and introduce a double Dutch auction (DDA) mechanism to further enhance the efficiency of resource allocation for maximizing social welfare. This auction dynamically matches the supply of edge server resources with the demand from user devices, ensuring fair and cost-effective distribution of computational resources. In this way, DDA can enhance long-context LLM serving by providing efficient resource allocation, dynamic pricing, and real-time adaptation capabilities that optimize computational resources across edge servers, particularly benefiting complex tasks like multi-path reasoning while maintaining low latency and maximizing social welfare. By leveraging buyer and seller clocks, the auction process adapts to the real-time needs of long-context LLM inference tasks, providing a scalable and transparent solution for dynamic environments. Finally, the experimental results demonstrate that the proposed algorithm can minimize system costs compared with existing baselines. The main contributions can be summarized as follows. • For long-context LLM serving in mobile edge networks, we propose a joint model caching and inference offloading framework to determine resource allocation and deployment strategies for LLM agents in mobile edge networks, which can adapt to dynamic network conditions and device capabilities. • In the framework, we analyze the convergence rate of LLMs handling contexts with CoT prompting. This approach enhances the reasoning and planning capability of LLMs by generating intermediate reasoning thoughts with additional latency and resources. • Furthermore, we model a long-term optimization problem to minimize total system costs