Distributed Learning and Inference Systems: A Networking Perspective

Hesham G. Moussa, Arashmid Akhavain, S. Maryam Hosseini, Bill McCormick · 2025

Read Distributed Learning and Inference Systems: A Networking Perspective by Hesham G. Moussa, Arashmid Akhavain, S. Maryam Hosseini, Bill McCormick (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05323.txt.

Introduction

Hesham Moussa and co-authors view distributed AI through a networking lens in this 2025 paper. Big models usually train and answer from one central cloud. That design raises privacy worries, storage costs, energy demands, and a single point of failure.

Spreading learning across many machines avoids those traps but adds moving parts. The authors propose a framework that treats data and model pieces as a network problem. Their survey maps how training and inference can be split, moved, and coordinated. Edge devices and private data stand to gain most. The paper aims to give system builders a shared map of a scattered field.

Catalog record

CREATORHesham G. Moussa, Arashmid Akhavain, S. Maryam Hosseini, Bill McCormick
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-05323

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Distributed Learning and Inference Systems: A Networking Perspective Hesham G. Moussa, Arashmid Akhavain, S. Maryam Hosseini, Bill McCormick Abstract Machine learning models have achieved, and in some cases surpassed, human-level performance in various tasks, mainly through centralized training of static models and the use of large models stored in centralized clouds for inference. However, this centralized approach has several drawbacks, including privacy concerns, high storage demands, a single point of failure, and significant computing requirements. These challenges have driven interest in developing alternative decentralized and distributed methods for AI training and inference. Distribution introduces additional complexity, as it requires managing multiple moving parts. To address these complexities and fill a gap in the development of distributed AI systems, this work proposes a novel framework, Data and Dynamics-Aware Inference and Training Networks (DA-ITN). The different components of DA-ITN and their functions are explored, and the associated challenges and research areas are highlighted. I Introduction In recent years, human intelligence has been the reference for many researchers working in machine learning. Today we have models that exhibit similar or even surpass human capabilities in many fields including text (e.g., chatGPT), games (e.g., AlphaZero), and object recognition (e.g., YOLOv7) [ 1 ] . While these results are impressive, the underlying model training paradigm relied mostly on centralization, where large volumes of data are collected and processed in a central cloud. However, centralization has become increasingly expensive, especially with the exponential growth in data volumes and model sizes [ 2 ] . This cost is further aggravated in centralized life-long learning, which requires frequent initiation of costly re-training episodes to integrate new data without forgetting past learned tasks [ 3 ] . Additionally, transferring data from distributed nodes poses significant privacy and security risks. Similar limitations also exist in centralized inference setups, especially when hosting large models on central servers which creates single points of failure, server congestion, and elevated computational cost. Furthermore, query and response data may carry sensitive information that, if exposed, may raise security and privacy concerns [ 4 ] . To address these limitations, decentralization has gained considerable attention from researchers in recent years [ 5 ] . Decentralized AI finds its way into many applications such as healthcare, robotics, and mobile networks [ 6 ] . As such, finding ways to enable this paradigm has driven many innovations. Many decentralized training methods like distributed learning, federated learning, gossip learning, split learning, and continual learning [ 7 , 8 ] , along with decentralized inference approaches such as split inference, switched inference, collaborative inference, and multi-modal inference, have been proposed [ 9 , 10 ] . These methods share the common objective of improving scalability, enhancing privacy, and reducing computational and storage demands [ 8 ] . This is achieved by distributing data and models across several networked nodes and enabling extensive mobility of the various components to enable AI functionality [ 11 ] — a concept known as the ”model-follow-data” paradigm. The model-follow-data paradigm views decentralized AI as a network of distributed, yet connected nodes. On the links between these nodes, models, data, and queries are optimally routed to facilitate AI training and inference. For instance, in the case of model-follow-data-based training, the objective is to enable dynamic interactions between data, compute, and models for optimal model training by creating adequate rendezvous points in the network. For example, in vanilla federated learning (FL), distributed data nodes collaborate to train a global model. Each data node receives a copy of the model from a central server, trains it on its local data, and returns the updated weights to the server. The server aggregates these updated weights and creates an updated global model that is ready for the next round of training [ 7 ] . Under vanilla FL, data nodes act as the rendezvous points for model-data interaction. Similarly, in inference, the goal is to provide cheaper and faster query responses by optimizing the placement of models across the network to which queries can be routed efficiently. For instance, in 6G-enabled edge computing, servers near base stations bring computing close to the users; hence, by deploying AI models at these edge servers, inference costs and response times can be significantly reduced [ 12 ] . In this setup, model-hosting facilities are considered the rendezvous points to which queries are routed which necessitates careful deployment designs. Accordingly, under the model-follow-data paradigm, the success of training and inference relies on the careful design and optimization of several components. To address this requirement, we propose a novel network-inspired intelligent decentralized system that we refer to as Data and Dynamics aware Inference and Training Network (DA-ITN) which is the main contribution of this work. DA-ITN aims to fill a gap in the next generation of distributed AI systems. In this paper, we dive into the general framework of DA-ITN and describe its various components. To the best of the authors’ knowledge, this is the first work that introduces such a framework and vision. The article is organized as follows. Section 2 provides an overview of DA-ITN and its various components. Section 3 dives into a high-level example to show DA-ITN in action. Section 4 presents an envisioned forward-looking implementation of DA-ITN as an autonomous system. The paper is wrapped up in section 5 with a list of challenges and potential research areas. II DA-ITN overview DA-ITN aims to address decentralization complexities by: (i) understanding knowledge network topology, resource availability, and authorization; (ii) analyzing data characteristics like type, quality, and variability; (iii) assessing node resources (e.g., computing power, storage, energy) and heterogeneity; (iv) evaluating nodes for reachability, visibility, and trustworthiness; (v) selecting suitable model architectures (e.g., RNNs, CNNs, transformers) for inference or recommending model modification for better training performance; (vi) optimizing training hyperparameters at rendezvous points; (vii) deploying AI models for efficient inference; (viii) managing query-response mobility, and (ix) adapting to network connectivity conditions. To handle these complexities, we envision DA-ITN as a network that contains a fully enabled control plane (CP), a data plane (DP), and an operations and management (OAM) plane as shown in Figure 1 . The network uses its infrastructure to collect information about the participating parties’ underlying data, resources, and reachability statuses, and creates knowledge topologies that aid in making intelligent AI training (Figure 1a ) and inference (Figure 1b ) decisions. In what follows, we discuss each of these DA-ITN setups. For the rest of the document, knowledge and data are used interchangeably unless otherwise stated. (a) (b) Fig. 1: (a) DA-ITN for training (DA-ITN-T). (b) DA-ITN for inference (DA-ITN-I). II-A DA-ITN for Training DA-ITN for Training (DA-ITN-T) is a system designed to offer automated AI training services. It processes multiple users’ training requests, models, and specific training requirements. Leveraging its underlying infrastructure, DA-ITN-T trains the submitted models accordingly. As illustrated in Figure 1a , the training architecture is structured into five layers: (i) the terminal layer, (ii) the tools layer, (iii) the Data, Resource, and Reachability Topology (DRRT) layer, (iv) the DA-ITN control center (DCC) layer, and (v) the OAM layer. These layers interact through control and data planes (CP and DP, respectively), as described below. The tools layer: located at the core of the DA-ITN system, provides

Read the complete text