Cost-Performance Analysis: A Comparative Study of CPU-Based Serverless and GPU-Based Training Architectures

Amine Barrak, Fabio Petrillo, Fehmi Jaafar · 2025

Read Cost-Performance Analysis: A Comparative Study of CPU-Based Serverless and GPU-Based Training Architectures by Amine Barrak, Fabio Petrillo, Fehmi Jaafar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14920.txt.

Introduction

Amine Barrak, Fabio Petrillo, and Fehmi Jaafar compare serverless architectures for distributed machine learning training in a 2025 cloud computing paper. Training large models demands setups that scale well without wasting money. Serverless computing, where code runs on demand without managed servers, is a candidate design.

The study revisits the authors' SPIRT architecture and measures it against ScatterReduce, AllReduce, and MLLess. Metrics include training time, cost, communication overhead, and fault tolerance. SPIRT cuts training time and communication load through parallel batch processing and in-database operations supported by RedisAI. Traditional designs still show strengths in scalability in some settings, so the comparison maps trade-offs rather than crowning one winner. The work guides teams choosing a training architecture under cost pressure.

Catalog record

CREATORAmine Barrak, Fabio Petrillo, Fehmi Jaafar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-14920

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cost-Performance Analysis: A Comparative Study of CPU-Based Serverless and GPU-Based Training Architectures Amine Barrak, Fabio Petrillo, Fehmi Jaafar Abstract This paper presents a comparative evaluation of four serverless training frameworks: SPIRT, MLLess, LambdaML AllReduce, and ScatterReduce, alongside a GPU-based baseline, using CNN models on CIFAR-10. We assess each architecture across training time, cost, communication overhead, and accuracy under consistent experimental conditions. While GPU-based training achieves the fastest convergence and highest accuracy, serverless frameworks offer cost advantages for lightweight models. Optimizations such as gradient accumulation and in-database computation improve serverless performance. Our findings reveal key trade-offs and highlight the potential of GPU-backed serverless platforms for scalable distributed training. 1 Introduction Machine Learning (ML) continues transforming industries through increasingly sophisticated predictive capabilities. However, these advancements entail substantial computational demands and complex infrastructure management, especially when training moderately-sized models [ 18 ] . For instance, training ResNet-50 on ImageNet takes roughly 14 days using a single NVIDIA GPU [ 18 ] , prompting the adoption of distributed training methods to parallelize workloads, reducing training duration and resource bottlenecks [ 20 ] . Distributed ML architectures typically follow Parameter Server (PS) or Peer-to-Peer (P2P) paradigms [ 16 ] . PS centralizes gradient aggregation through a coordinator node, simplifying synchronization [ 9 ] . Conversely, P2P decentralizes training by distributing parameters and computations across nodes, alleviating central coordination bottlenecks [ 14 ] . Nevertheless, challenges remain, including resource inefficiencies, high operational costs, and system complexity. The increasing computational demands of ML have driven a significant shift toward cloud-based training solutions. Cloud platforms offer scalable infrastructure, cost-effectiveness, and integrated support for parallel training [ 12 ] . However, these benefits often come at the cost of inefficiencies such as over-provisioning and wasted resources. In fact, over-provisioning and always-on resources were estimated to account for $26.6 billion in public cloud waste in 2021 [ 15 ] . A recent survey revealed that 41.1% of data scientists find the use of cloud resources for ML training to be a significant challenge, highlighting the complexity of effectively managing such infrastructure [ 11 ] . Serverless computing, characterized by on-demand resource allocation and seamless scaling, has emerged as a compelling alternative. It offers potential efficiency improvements by dynamically matching resource allocation with workload demands [ 19 , 5 ] . Recent studies validate the effectiveness of serverless computing frameworks in ML, particularly for smaller neural network architectures such as MobileNet and ResNet variants [ 6 , 10 , 13 , 4 , 2 ] . Several solutions utilizing serverless computing for distributed ML training have been proposed, adopting distinct architectural topologies such as Peer-to-Peer (P2P) [ 3 ] , Parameter Server (PS) [ 17 , 5 ] , ScatterReduce, and AllReduce [ 7 ] . A common challenge among these solutions is the stateless nature of serverless computing, requiring external communication channels to maintain state between functions [ 17 , 1 ] . Techniques such as extending function runtime for state checkpointing [ 7 ] or enabling gradient accumulation to optimize parallel processing [ 3 ] have been explored to address these limitations. Despite recent advancements, comprehensive comparisons of existing serverless ML frameworks remain limited, particularly against distributed GPU-based training setups. This paper addresses these gaps through a detailed analysis of end-to-end serverless architectures and an empirical evaluation comparing serverless CPU-based frameworks with distributed GPU-based baselines. To foster reproducibility and further research, we provide a full replication package 1 1 1 https://sites.google.com/view/pdcat-2025 . The key contributions of this paper include: 1. An analysis and comparison of existing serverless ML training architectures, highlighting their respective end-to-end frameworks and operational characteristics. 2. Evaluation of communication overhead reduction techniques in serverless based training architectures, including in-database operations, update filtering, and communication pattern optimization. 3. A detailed cost-performance analysis, identifying conditions under which serverless architectures offer substantial economic and performance advantages over GPU-based training. 2 Comparative Analysis of Training Workflows Figure 1: Abstract training flow shared by all evaluated architectures. Aggregation strategy and synchronization mechanisms vary per framework. Figure 1 illustrates a generalized training workflow in serverless environments. The process includes data fetching, model and dataset loading, gradient computation, saving intermediate state, synchronization, aggregation, and model update. Each framework instantiates this flow differently, reflecting trade-offs in communication overhead, synchronization complexity, and state handling. SPIRT [ 3 ] decentralizes the entire workflow using a peer-to-peer strategy. Each worker independently loads its portion of data and model, computes gradients in parallel over minibatches, and saves these locally in its Redis instance. After averaging gradients internally, workers notify a synchronization queue and wait until all peers report completion. Once synchronized, workers retrieve each other’s averaged gradients, aggregate them again, and store the final result locally before applying the update. SPIRT uniquely performs both gradient averaging and model updates within the database, reducing communication with external storage and enhancing performance under stateless constraints. MLLess [ 5 ] adopts a significance-driven filtering strategy. Each worker loads data and model, computes gradients, and checks whether the change exceeds a predefined threshold. Only significant updates are stored in a shared database, and a message is pushed to other workers and a central supervisor via queues. The supervisor coordinates synchronization by instructing workers when to fetch updates. Workers then aggregate received gradients and update the model. This design reduces communication overhead but introduces a synchronization bottleneck through the central supervisor. ScatterReduce [ 7 ] implements a distributed aggregation strategy by splitting gradients into chunks. Each worker computes gradients and retains one chunk while pushing the rest to shared storage. Workers are assigned specific chunks to aggregate across all peers, then upload the partial aggregates. After all chunks are aggregated, workers download the full set, reconstruct the complete gradient, and perform the model update. This strategy balances computation but incurs significant communication overhead, especially as the number of workers increases. AllReduce [ 7 ] centralizes aggregation. After computing gradients, all workers push their updates to a shared database. A designated master worker performs the full aggregation and uploads the result. All workers then synchronize by fetching this final gradient and applying it to their local models. While simple and efficient for small setups, AllReduce suffers from a scalability bottleneck due to the master’s workload and centralized coordination. GPU-Based Baseline uses multiple GPUs distributed across separate machines. Each GPU processes a different data batch, computes gradients locally, uploads them to a shared S3 bucket, then loads the gradients from other workers and performs local averaging before updating its model. Table 1: Comparative Analysis of ML Training Architectures: Overview of Key Computational Stages in Serverless and Distributed GPU-Based Frameworks Framework Training Stage Content of the Stage SPIRT [ 3 ] Fetch Dataset Each worker fetches its assigned minibatches. Compute Gradients Gradients are computed in parallel for each minibatch, sent to the local Redis database, and averaged within the database. Synchronisation The worker notifies a synchronization queue once gradients are ready, polls the queue until all peers complete, then retrieves averaged gradients from other workers, aggregates them, and stores the result locally. Model Update The final aggregated gradient is used to update the model. MLLess [ 5 ] Fetch Dataset Each worker fetches a single minibatch for processing. Compute Gradients Gradients are computed for the minibatch

Read the complete text