Toward Sustainability-Aware LLM Inference on Edge Clusters Conference: 1st International Workshop on Intelligent and Scalable Systems across the Computing Continuum ; November, 2025; Vienna, Austria

Kolichala Rajashekar, Nafiseh Sharghivand, Radu Prodan, Reza Farahani · 2025

Read Toward Sustainability-Aware LLM Inference on Edge Clusters Conference: 1st International Workshop on Intelligent and Scalable Systems across the Computing Continuum ; November, 2025; Vienna, Austria by Kolichala Rajashekar, Nafiseh Sharghivand, Radu Prodan, Reza Farahani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-04088.txt.

Description

Running large language models in distant data centres is flexible but slow and carbon-hungry, while running them on local edge devices is constrained by modest hardware. Kolichala Rajashekar and colleagues explore the middle path in a 2025 distributed computing preprint: inference on clusters of edge devices, routed with sustainability in mind. Their test hardware mixes two kinds of NVIDIA accelerator boards.

The team benchmarked energy use and execution time across prompts and batch configurations, then compared routing strategies. A greedy baseline sends each prompt wherever, while carbon-aware and latency-aware strategies steer prompts to specific hardware using the benchmark data. Batches of four prompts struck the best balance of throughput and energy efficiency. Larger batches risked saturating graphics memory. Where a query runs, the authors argue, is an environmental decision, and measured routing can soften the footprint of everyday model use.

Catalog record

CREATORKolichala Rajashekar, Nafiseh Sharghivand, Radu Prodan, Reza Farahani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-04088

Browse the full library · About the Tower of Babel