To Offload or Not To Offload: Model-driven Comparison of Edge-native and On-device Processing In the Era of Accelerators
Nathan Ng, David Irwin, Ananthram Swami, Don Towsley, Prashant Shenoy · 2025
Read To Offload or Not To Offload: Model-driven Comparison of Edge-native and On-device Processing In the Era of Accelerators by Nathan Ng, David Irwin, Ananthram Swami, Don Towsley, Prashant Shenoy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15162.txt.
Introduction
Nathan Ng and co-authors weighed computing on a device versus sending work to a nearby edge server in 2025. Phones and other clients are limited, so offloading computation has long been strong. Dedicated accelerator chips now make local processing fast, but edge servers have accelerators too, so the old answer no longer holds. The authors wanted to know when each choice wins.
They built analytic models based on queuing theory that produce plain rules for the expected total delay of each plan. Those rules predict the precise crossover points where offloading starts or stops paying off, across varied accelerators, networks, workloads, and shared multi-tenant conditions. Tests across a range of cases proved the models, giving builders a clear rule for self-tuning offloading choices instead of guesswork.
Catalog record
| CREATOR | Nathan Ng, David Irwin, Ananthram Swami, Don Towsley, Prashant Shenoy |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-15162
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
To Offload or Not To Offload: Model-driven Comparison of Edge-native and On-device Processing In the Era of Accelerators Nathan Ng, David Irwin, Ananthram Swami, Don Towsley, Prashant Shenoy Abstract . Computational offloading is a promising approach for overcoming resource constraints on client devices by moving some or all of an application’s computations to remote servers. With the advent of specialized hardware accelerators, client devices can now perform fast local processing of specific tasks, such as machine learning inference, reducing the need for offloading computations. However, edge servers with accelerators also offer faster processing for offloaded tasks than was previously possible. In this paper, we present an analytic and experimental comparison of on-device processing and edge offloading for a range of accelerator, network, multi-tenant, and application workload scenarios, with the goal of understanding when to use local on-device processing and when to offload computations. We present models that leverage analytical queuing results to derive explainable closed-form equations for the expected end-to-end latencies of both strategies, which yield precise, quantitative performance crossover predictions that guide adaptive offloading. We experimentally validate our models across a range of scenarios and show that they achieve a mean absolute percentage error of 2.2% compared to observed latencies. We further use our models to develop a resource manager for adaptive offloading and show its effectiveness under variable network conditions and dynamic multi-tenant edge settings. DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 1. Introduction Over the past decade, cloud computing has become the predominant approach for running online services in domains ranging from finance to entertainment. In recent years, a new class of online services has emerged that requires low-latency processing to meet end-user requirements. Examples of such applications include autonomous vehicles, interactive augmented and virtual reality (AR/VR), and human-in-the-loop machine learning inference ( Mosqueira-Rey et al., 2022 ) . Edge computing, a complementary approach to cloud computing, is a promising method for meeting the needs of such latency-sensitive applications. Conventional wisdom has held that since edge resources are deployed at the edge of the network and are closer to end-users, edge-native processing can provide lower end-to-end latencies than cloud processing. However, recent research ( Ali-Eldin et al., 2021 ; Wang et al., 2024a ) has shown that this conventional wisdom does not always hold—in certain scenarios, edge processing can yield worse end-to-end latency despite its network latency advantage over the cloud. In particular, since edge clusters are resource-constrained relative to the cloud, research has shown that resource bottlenecks can arise under time-varying workloads and increase the processing latency at the edge, negating its network latency advantage and requiring careful resource management to maintain its benefits ( Wang et al., 2024a ) . A related trend is the emergence of computational offloading from local devices to remote resources such as those at the edge. 1 1 1 Computational offloading to the cloud is also common, but we focus on edge offloading in this paper. Computational offloading ( Satyanarayanan et al., 2009 ; Davis et al., 2004 ) involves using nearby edge resources to take on some, or all, of an application’s processing demands. Such offloading is performed to overcome resource constraints on a device by leveraging the more abundant processing capacity at edge servers. Computational offloading is well-established in mobile computing where mobile devices, which are resource- or battery-constrained, offload local computations to edge servers ( Ren et al., 2019 ) . In this case, local devices pay a so-called mobility penalty ( Satyanarayanan et al., 2019 ) , which is the additional network latency to access edge servers, but then benefit from faster edge processing that outweighs this latency overhead. Conventional wisdom holds that remote edge processing is faster than local processing despite the network overhead of accessing remote resources due to significant resource constraints at local devices. With the advent of hardware accelerators in recent years, the conventional wisdom about the benefits of computational edge offloading for local devices needs to be rethought. For example, the rise of the so-called AI PC with onboard AI accelerators ( Intel Corporation, 2025 ) has enabled local AI processing for many applications without the need for edge offloading. Similarly, mobile phones are equipped with neural accelerators ( Simonite, 2017 ) that enable efficient local inference, while IoT devices such as cameras have specialized accelerators to perform visual tasks (e.g., facial recognition) locally ( Netatmo, 2025 ) . At the same time, edge servers can also be equipped with more powerful accelerators, such as GPUs, enabling offloaded tasks to also run more efficiently or faster than before. Thus, in the era of programmable accelerators, the question of whether to offload computations from local devices to the edge and when it is advantageous to do so needs to be reexamined. In particular, such a reexamination needs to address several questions. (i) Under what scenarios will local on-device processing using accelerators outperform edge offloading? (ii) Are there still scenarios where edge offloading has an advantage over local processing? (iii) How does multi-tenancy at edge servers affect the decision to offload or execute on-device? Motivated by the above questions, this paper takes an analytic model-driven approach to compare local and edge processing in the era of accelerators. Our hypothesis is that there is no a priori clear advantage of edge processing over on-device processing in the presence of accelerators, and that the choice of where to perform the processing depends on hardware, network, and workload characteristics. The novelty of this work lies in deriving closed-form equations for the expected end-to-end latency of each strategy, which are explainable and require no workload-specific training. These equations yield precise, quantitative performance-crossover predictions that can be embedded in OS schedulers and resource managers to dynamically decide whether to run an application component on the device or at the edge. In developing our analytic models and validating our hypothesis, this paper makes the following contributions. • We develop queuing theory–based models to model the behavior of device and edge accelerators when processing requests in on-device and edge offloading scenarios. Our modeling adopts a two-level approach: it first uses estimated service times obtained through profiling or prediction techniques as input, and then embeds them into queuing models to predict end-to-end response times. We develop closed-form analytic bounds using these models to determine when one approach outperforms the other under different hardware, network, and workload scenarios. We further extend the analysis to model multi-tenant edge servers serving multiple clients, and demonstrate how the models can be extended to capture device–edge collaborative (i.e., split) processing. • We experimentally validate our models and bounds across diverse device and edge accelerators, network configurations, and workloads spanning multiple model architectures, including deep neural network (DNN), recurrent neural network (RNN), and large language model (LLM). The results show that our models accurately predict on-device and offloading performance, achieving a 2.2% mean absolute percentage error, with 91.5% of predictions within \pm 5% and all within \pm 10% of the observed latency. • We develop a resource manager that leverages our models to adapt between on-device processing and edge offloading in response to real-world dynamics. We present two case studies demonstrating