Generalizable Pareto-Optimal Offloading with Reinforcement Learning in Mobile Edge Computing
Ning Yang* Junrui Wen* Meng Zhang Ming Tang · 2025
Read Generalizable Pareto-Optimal Offloading with Reinforcement Learning in Mobile Edge Computing by Ning Yang* Junrui Wen* Meng Zhang Ming Tang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-10474.txt.
Introduction
Phones and wearables send much of their computing to nearby edge servers. Should a task stay local, though? Where should it go? Ning Yang and three co-authors study that offloading problem in this 2025 paper. Delay and energy both matter in mobile edge computing. Nobody knows in advance how a user weighs the two.
Their framework, GMORL, trains one reinforcement learning policy. The policy adapts to shifting preferences. It also adapts to edge systems with different processor speeds and server counts. A histogram-based encoding describes many edge servers compactly. A tailored reward and network design improve generalization. In simulations, the method enlarged the Pareto front's hypervolume. The gain reached up to about double, against benchmarks. That means better trade-offs across the board. It is not a win on one metric bought with the other.
Catalog record
| CREATOR | Ning Yang* Junrui Wen* Meng Zhang Ming Tang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 86 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-10474
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generalizable Pareto-Optimal Offloading with Reinforcement Learning in Mobile Edge Computing Ning Yang* Junrui Wen* Meng Zhang Ming Tang Abstract Mobile edge computing (MEC) is essential for next-generation mobile network applications that prioritize various performance metrics, including delays and energy efficiency. However, conventional single-objective scheduling solutions cannot be directly applied to practical systems in which the preferences (i.e., the weights of different objectives) are often unknown or challenging to specify in advance. In this study, we formulate a multi-objective offloading problem for MEC with multiple edges to minimize the sum of expected long-term energy consumption and delay while considering unknown preferences. To address the challenge of unknown preferences and the potentially diverse MEC systems, we propose a generalizable multi-objective (deep) reinforcement learning (GMORL)-based tasks offloading framework, which employs the Discrete Soft Actor-Critic (Discrete-SAC) method. Our method uses a single policy model to efficiently schedule tasks based on varying preferences and adapt to heterogeneous MEC systems with different CPU frequencies and server quantities. Under the proposed framework, we introduce a histogram-based state encoding method for constructing features for multiple edges in MEC systems, a sophisticated reward function for accurately computing the utilities of delay and energy consumption, and a novel neural network architecture for improving generalization. Simulation results demonstrate that our proposed GMORL scheme enhances the hypervolume of the Pareto front by up to 121.0 % 121.0\% compared to benchmarks. Our code are avavilable at https://github.com/gracefulning/Generalizable-Pareto-Optimal-Offloading-with-Reinforcement-Learning-in-Mobile-Edge-Computing Keywords: Mobile edge computing, multi-objective reinforcement learning, resource scheduling, discrete-soft actor-critic. 1 introduction 1.1 Background and Challenges The rise of next-generation networks and the increasing use of mobile devices have resulted in an exponential growth of data transmission and diverse computing needs. With the emergence of new computing-intensive applications, there is a possibility that device computing capacity may not suffice. To tackle this challenge, mobile edge computing (MEC) has emerged as a promising computing paradigm. MEC enables the offloading of computing workloads to edge or cloud networks, offering the potential for achieving low latency and high efficiency pervez2024energy . In MEC systems, task offloading is crucial in achieving low latency and energy consumption li2018deep . The scheduling of task offloading in MEC systems is challenging due to the dynamic and unpredictable nature of users’ workloads and computing requirements. Some works apply traditional optimization methods to schedule for MEC systems fang2019optimal ; tran2018joint . These methods assume deterministic objective functions that cannot cope well with uncertainty or dynamics in the problem parameters. The application of deep reinforcement learning (DRL) has shown substantial potential in addressing sequential decision-making problems and have demonstrated the effectiveness of applying DRL in MEC systems to address the unknown dynamics. For instance, Cui et al. cui2020latency employed DRL to solve the user association and offloading sub-problems in MEC networks. Lei et al. lei2019multiuser investigated computation offloading and multi-user scheduling algorithms in edge IoT networks and proposed a DRL algorithm to solve the continuous-time problem, supporting implementation based on semi-distributed auctions. Jiang et al. jiang2020stacked proposed an online DRL-based resource scheduling framework to minimize the delay in large-scale MEC systems. However, a challenge that has been overlooked by researchers is the issue of generalization . Challenge 1 DRL policies are typically trained for specific environments, rendering them less adaptable to novel contexts. Nevertheless, it is important to acknowledge that the training and application environments may not always align and that there may be variations in their parameters. Consequently, the scheme must be flexible enough to accommodate a range of diverse and unknown preferences. To achieve the generalization of preferences, we have to seek out new methodologies to address the following questions: Question 1 How should we design a scheduling policy that can apply to various MEC systems with diverse preferences? The challenge of addressing this problem can be summarized in two aspects. First, there may be conflicts between different objectives, such as delay and energy consumption, that cannot be optimized simultaneously. Second, since MEC systems serve diverse applications with varying preferences, it is challenging to design an offloading policy that can generate Pareto optimal solutions under diverse and unknown preferences. It is worth noting that the direct application of single-objective DRL through scalarization, which involves taking a weighted sum, is not a valid approach due to the following issues roijers2013survey : 1. Impossibility : Weights may be unknown when designing or learning an offloading scheme. 2. Infeasibility : Weights may be diverse, which is true when MEC systems have different restrictive constraints on latency or energy. 3. Undesirability : Even if weights are known, nonlinear objective functions may lead to non-stationary optimal policies. To effectively address these challenges, we propose to employ multi-objective reinforcement learning (MORL) to design a task offloading policy. However, this method faces certain limitations. Firstly, when dealing with a large number of preferences, it can become computationally and storage-intensive roijers2013survey . Secondly, since the preference is typically unknown in advance, it becomes infeasible to search for a specific policy that matches a particular preference from a pre-trained set of policies yang2019generalized . Therefore, we propose a novel single-policy MORL method to schedule tasks for MEC systems. To this end, we propose to use a single policy to accommodate diverse preferences . Compared with multi-policy approaches, our single-policy MORL method is more lightweight and more feasible for deployment. Although the MORL approach can deal with diverse preference problems, there are other generalization issues worth considering. Question 2 How should we deploy a well-trained DRL-based policy to new MEC systems with different CPU frequencies and server quantities? Existing DRL methods for task offloading scheduling in MEC networks have, to date, exhibited limited research pertaining to matters of generalization. Yan et al. yan2020offloading introduced a DRL method to optimize offloading scheduling, but they exclusively considered a fixed preference and a set of constant system parameters. Li et al. li2023gasto proposed a meta-reinforcement learning method to lead an DRL-based policy quickly adaptive to new environments. However, this approach lacks the capability to generalize to new environments with varying server quantities. Gao et al. gao2023fast proposed a multi-agent DRL method to schedule tasks for large-scale MEC systems. This method can handle systems with different quantities of servers, but it can only optimize for a single fixed preference. Ren et al. ren2022enhancing exploited learning-experience utility to improve the generalization of a DRL policy. Nonetheless, when the quantity of servers varied, the policy network had to be redesigned and retrained. In contrast, a majority of other studies cui2020latency ; lei2019multiuser ; jiang2020stacked ; li2018deep ; huang2019deep have predominantly disregarded the aspect of generalization in their methodologies. Solving the generalization problem has been the subject of research, and various methods have been proposed. Two widely used technologies to improve the generalization of DRL methods are domain randomization tobin2017domain and adapting online hayes2022practical . These methods utilize context to characterize a system with specific parameters. For a contextual Markov decision process (MDP) kirk2021survey ; ghosh2021generalization , domain randomization approaches train an DRL model in randomized environments to make the model adapt to diverse systems. Therefore, we improve the MORL and propose the generalizable multi-objective reinforcement learning (GMORL). We summarize the differences between our method and other existing works in the study of generalization in Table 1, with comprehensive details