Towards Decentralized and Sustainable Foundation Model Training with the Edge DOI: XXXXXXX.XXXXXXX Conference: ; July 10-11th, 2025; Cambridge, MA ISBN: 978-1-4503-XXXX-X/18/06 CCS: Computer systems organization Distributed architectures CCS: Social and professional topics Sustainability CCS: Comput

Leyang Xue, Meghana Madhyastha, Randal Burns, Myungjin Lee, Mahesh K. Marina · 2025

Read Towards Decentralized and Sustainable Foundation Model Training with the Edge DOI: XXXXXXX.XXXXXXX Conference: ; July 10-11th, 2025; Cambridge, MA ISBN: 978-1-4503-XXXX-X/18/06 CCS: Computer systems organization Distributed architectures CCS: Social and professional topics Sustainability CCS: Comput by Leyang Xue, Meghana Madhyastha, Randal Burns, Myungjin Lee, Mahesh K. Marina (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01803.txt.

Introduction

Leyang Xue, Meghana Madhyastha, Randal Burns, Myungjin Lee, and Mahesh K Marina put forward a vision for training the largest AI models on spare edge devices, in a 2025 paper. Training such models today takes huge data centers, which burn much energy and place control in few hands. Phones, home hubs, and other edge devices sit idle much of the time.

The authors argue that pooling that spare power could cut the footprint and spread control more widely. They lay out why the idea fits what edge devices can do, with special weight on the gains for sustainability. They also name the main obstacles, from slow home links to keeping data private across many owners. The paper is a call and a road map, not a finished system.

Catalog record

CREATORLeyang Xue, Meghana Madhyastha, Randal Burns, Myungjin Lee, Mahesh K. Marina
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-01803

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Decentralized and Sustainable Foundation Model Training with the Edge DOI: XXXXXXX.XXXXXXX Conference: ; July 10-11th, 2025; Cambridge, MA ISBN: 978-1-4503-XXXX-X/18/06 CCS: Computer systems organization Distributed architectures CCS: Social and professional topics Sustainability CCS: Computing methodologies Artificial intelligence Leyang Xue, Meghana Madhyastha, Randal Burns, Myungjin Lee, Mahesh K. Marina Abstract . Foundation models are at the forefront of AI research, appealing for their ability to learn from vast datasets and cater to diverse tasks. Yet, their significant computational demands raise issues of environmental impact and the risk of centralized control in their development. We put forward a vision towards decentralized and sustainable foundation model training that leverages the collective compute of sparingly used connected edge AI devices. We present the rationale behind our vision, particularly in support of its sustainability benefit. We further outline a set of challenges that need to be addressed to turn this vision into reality. Note: This paper has been accepted to appear at HotCarbon 2025 . 1. Introduction “Foundation models” ( Bommasani et al. 2021 ) , which are models trained at scale on broad data that can then be adapted to a wide range of downstream tasks, are at the heart of the current AI revolution. Such foundation models leverage the power of generative AI and make AI a general-purpose technology, heralding it into the industrial age ( Economist [n. d.] ) . They span domains as diverse as natural language processing (NLP) ( OpenAI 2023 ) , computer vision (CV) ( Rombach et al. 2022 ) , software development ( GitHub [n. d.] ) , networks ( Le et al. 2022 ; Kotaru 2023 ) , biology ( Jumper et al. 2021 ) and more. The immense promise that foundation models offer is in large measure owed to their scale; they exhibit an “emergent” behavior with model size and show a sharp rise in accuracy as they are scaled up beyond a point and trained with large amounts of data ( Kaplan et al. 2020 ) . This has resulted in the compute demand for their training skyrocketing in recent years, needing 1000s of AI accelerators (GPUs, TPUs, etc.) ( Sevilla et al. 2022 ) . Such high compute resource requirements come with a significant economic cost (e.g., ( Smith 2023 ) ) and environmental cost ( e.g., ( Han et al. 2024 ) ), affordable to only a handful of global entities mostly those who run the cloud. This increasing centralization is a big impediment to the collective development of AI to benefit all ( Bartoldson et al. 2023 ; UK Government 2024 ; Borge 2022 ) . Not only that, there is an enormous environmental cost that is rising rapidly, again rooted in the compute-intensive nature of foundation model development ( McQuate 2023 ; Wu et al. 2022 ) . Unsurprisingly then, AI has become one of the big four contributors to global ICT-related carbon footprint ( Knowles 2021 ) . In this paper, we put forward a vision centered on the “edge” as a distributed computing platform to drive future foundation model development in a decentralized and sustainable manner. The key idea is to harness the spare compute across an amorphous collection of connected edge AI devices for foundation model training. There are billions of such mostly idle edge devices across the globe that offer ample opportunity to this end. The approach we advocate builds on the pioneering works in the realm of volunteer computing ( Mengistu and Che 2019 ) , aimed at harnessing the idle computing power from personal devices for distributed computing tasks. SETI@home ( Anderson et al. 2002 ) is a notable example that leveraged a million volunteered computers in the search for extraterrestrial intelligence. Differently from these early works, our focus is on enabling decentralized and sustainable foundation model development via the edge at a lower overall carbon footprint, while maintaining accuracy as with cloud-based training. The decentralized aspect of edge-based foundation model training is obvious. We make a case that it also enhances sustainability overall by presenting our rationale through a three-step argument, as outlined below and elaborated in § 4.2 : \bullet Edge devices are designed with energy efficiency in mind and increasingly feature AI accelerators to support ML tasks, including training. By comparing the energy consumption of cloud- and edge-based training through a series of experiments, we show that it can be several times more energy efficient to train with edge devices. \bullet Edge devices are mostly idle (75% of the time in the case of smartphones) but have a high embodied carbon footprint. The latter is unrelated to their operational use and instead linked to manufacturing, supply chain, and recycling. So, utilizing them better helps amortize their high embodied carbon footprint. \bullet The above enables the opportunity to offload compute from the cloud to the edge, thereby helping reduce the cloud-side carbon footprint. This is because the baseline (embodied + operational) carbon footprint of edge devices would always be incurred simply by individuals owning them. Their better use can yield net gains in carbon footprint due to their relatively better energy efficiency and ample idle time. Our analysis shows that a 4–8× net carbon footprint reduction is possible by replacing the overall carbon footprint of a cloud GPU device with a small addition to the operational carbon footprint of a set of edge devices providing equivalent compute. There have been some efforts in line with our vision aimed at decentralized ML training with volunteered devices ( Ryabinin and Gusev 2020 ; Diskin et al. 2021 ; Ryabinin et al. 2023 ) but they do not account for all the unique characteristics of the edge environment (device heterogeneity and dynamism) and crucially overlook the sustainability dimension altogether. Significant challenges remain to be addressed to bring our vision to fruition, including: (i) Distributed training methods for edge : Scalable and energy-efficient training requires edge-aware workload partitioning and communication minimization, as well as accounting for the carbon footprint; (ii) Training task orchestration at the edge : Coordinating training tasks across dynamic edge environments demands fault tolerance and low-overhead, carbon-aware scheduling; (iii) Holistic, reliable and efficient energy consumption monitoring : Measuring carbon impact at the edge requires accurate, low-overhead energy monitoring across all hardware components. (iv) User incentives, security and privacy : These are key concerns that require maintaining a seamless user experience when their devices are leveraged for training, as well as efficient ways to achieve robustness and ensure training process isolation on user devices. 2. Background 2.1. Foundation Model Development Process Foundation models are characterized by three key traits: training on massive datasets, large model sizes, and broad generalization across tasks ( Bommasani et al. 2021 ) . Large language models (LLMs), such as GPT-4, are a prominent example. Similar models exist for other modalities (e.g., Stable Diffusion ( Rombach et al. 2022 ) for vision) and for multimodal data (e.g., DeepSeek-VL2 ( Wu et al. 2024 ) , QWen2.5-VL ( Bai et al. 2025 ) ). Their development typically involves two phases (Fig. 1 ): pre-training and fine-tuning. Pre-training. In the pre-training phase, foundation models are trained on vast datasets ( e.g., 45TB in GPT-3) ( Brown et al. 2020 ) typically

Read the complete text