Toward Sustainability-Aware LLM Inference on Edge Clusters Conference: 1st International Workshop on Intelligent and Scalable Systems across the Computing Continuum ; November, 2025; Vienna, Austria
Kolichala Rajashekar, Nafiseh Sharghivand, Radu Prodan, Reza Farahani · 2025
Read Toward Sustainability-Aware LLM Inference on Edge Clusters Conference: 1st International Workshop on Intelligent and Scalable Systems across the Computing Continuum ; November, 2025; Vienna, Austria by Kolichala Rajashekar, Nafiseh Sharghivand, Radu Prodan, Reza Farahani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-04088.txt.
Description
Running large language models in distant data centres is flexible but slow and carbon-hungry, while running them on local edge devices is constrained by modest hardware. Kolichala Rajashekar and colleagues explore the middle path in a 2025 distributed computing preprint: inference on clusters of edge devices, routed with sustainability in mind. Their test hardware mixes two kinds of NVIDIA accelerator boards.
The team benchmarked energy use and execution time across prompts and batch configurations, then compared routing strategies. A greedy baseline sends each prompt wherever, while carbon-aware and latency-aware strategies steer prompts to specific hardware using the benchmark data. Batches of four prompts struck the best balance of throughput and energy efficiency. Larger batches risked saturating graphics memory. Where a query runs, the authors argue, is an environmental decision, and measured routing can soften the footprint of everyday model use.
Catalog record
| CREATOR | Kolichala Rajashekar, Nafiseh Sharghivand, Radu Prodan, Reza Farahani |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-04088