Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence

John Violos, Konstantina-Christina Diamanti and Ioannis Kompatsiaris and Symeon Papadopoulos · 2025

Read Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence by John Violos, Konstantina-Christina Diamanti and Ioannis Kompatsiaris and Symeon Papadopoulos (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01869.txt.

Introduction

John Violos, Symeon Papadopoulos and their co-authors survey frugal machine learning, the design of models that do more with less, in 2025. The field aims for good performance while cutting compute, time, energy and data in training and use.

They sort methods into input frugality, learning process frugality and model frugality, by where savings occur. Tools include model compression, efficient hardware and data-light learning, plus tricks such as distillation and dynamic designs that update without full retraining. Case studies span many domains. Smart settings with edge and sensor devices, which face tight limits, are a key driver.

Catalog record

CREATORJohn Violos, Konstantina-Christina Diamanti and Ioannis Kompatsiaris and Symeon Papadopoulos
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01869

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence John Violos, Konstantina-Christina Diamanti and Ioannis Kompatsiaris and Symeon Papadopoulos Abstract Frugal Machine Learning (FML) refers to the practice of developing Machine Learning (ML) models that are efficient, cost-effective, and resource-aware. Approach in this field aim to achieve satisfactory performance while minimizing the computational resources, time, energy and data required for training and inference. This chapter examines recent advancements, applications, and ongoing challenges in FML, highlighting its significance in smart environments that integrate edge computing and IoT devices, often constrained by limited bandwidth or high latency. Key technological enablers are explored, including model compression, energy-efficient hardware, and data-efficient learning techniques, highlighting their role in promoting sustainability and accessibility in Artificial Intelligence (AI). Furthermore, it provides a comprehensive taxonomy of frugal methods, discusses case studies across diverse domains, and identifies future research directions to drive innovation in this evolving field. 1 Introduction FML aims to make AI more accessible and sustainable, particularly in resource-constrained environments such as smart environments, IoT systems, and embedded platforms [ 1 ] . Unlike traditional ML approaches that assume abundant resources, FML focuses on creating models that perform effectively on devices with limited computational power, data, memory, and energy [ 2 ] . This is increasingly important as more applications move towards edge computing, which requires real-time processing on low-power devices [ 3 ] . FML also contributes to sustainability by reducing energy consumption, making AI development more environmentally friendly, while democratizing access to powerful models in settings with limited infrastructure [ 4 ] . It enables scalable and cost-effective deployment across industries such as IoT and Edge computing [ 5 ] , wearable and mobile AI [ 6 ] , autonomous systems [ 7 ] and cybersecurity [ 8 ] , benefiting regions with constrained resources. While closely related to Tiny Machine Learning (TinyML), FML and TinyML differ in focus. FML emphasizes reducing costs associated with data, computation, and time, while maintaining model performance across various ML tasks. TinyML, on the other hand, is specifically designed for ultra-low-power devices like micro-controllers and edge devices, optimizing models for minimal memory, processing power, and energy consumption. Though FML is broader and cost-driven, TinyML is specialized for small, resource-limited hardware platforms. Key technological advancements have fueled the rise of FML, including the shift from centralized cloud computing to edge computing. This transition has driven the demand for lightweight models that can operate on devices with limited processing power. Techniques such as model pruning [ 9 ] , quantization [ 10 ] , and knowledge distillation [ 11 ] have enabled significant reductions in model size with minimal performance loss, making them suitable for embedded systems. Energy-efficient hardware, such as specialized AI accelerators like GPUs [ 4 ] and TPUs [ 12 ] , also play a crucial role in optimizing ML algorithms for low-power devices. These developments support frugal algorithms by reducing the energy consumption required for AI computations. Additionally, techniques such as feature selection [ 13 ] and data selection [ 14 ] help retain the most informative data points, reducing the amount of training data required without compromising the accuracy of ML models. Objectives Description Data Size Minimizing the amount of (labeled) data for the training processes. Computation Reducing the computational load by building lightweight models. Energy Minimizing energy consumption during training and inference. Scalability Designing models that scale across various devices in constrained environments with different levels of resource availability. Monetary Cost Optimizing computational, memory, and data resources to reduce the overall expense of deploying and maintaining ML systems. Model size Decreasing the number of parameters (weights and biases) in ANNs. Compression Reducing the time and resources required to compress, fine-tune, and validate a compressed ANN. Trade-offs FML balances the above objectives with performance, ensuring efficient operation without compromising model accuracy. Table 1: Objectives of FML The chapter relates to the efficiency and sustainability of the AI, Data and Robotics Partnership [ 17 ] . Its main contribution is a comprehensive survey of FML, structured to ensure clarity and accessibility, helping readers easily understand and engage with the topic. Section 2 makes a taxonomy of most prominent FML methods, models and approaches. Section 3 presents various use cases and applications of FML. Section 4 gives the observations about technologies and methodologies. Section 5 casts a critical eye in FML discussing limitations, research gaps and providing suggestions for future work. Finally, we conclude the article in Section 6 . 2 A Taxonomy of FML Methods Frugality in ML models can be achieved through several approaches. The most common approach involves compressing large ANNs into smaller (Subsection 2.1 ). Another approach leverages specialized hardware accelerators such as GPUs, TPUs, FPGAs, and NPUs to enhance computational efficiency while lowering energy consumption (Subsection 2.6 ). Algorithm optimization focuses on selecting lightweight ML techniques or refined versions of existing models to improve efficiency (Subsection 2.2 ). Feature selection streamlines input data by eliminating non-essential features that have minimal impact on model performance (Subsection 2.3 ). Similarly, data sampling reduces training data by discarding entire samples that contribute little to the learning process (Subsection 2.4 ). Furthermore, inverse modeling techniques facilitate the rapid determination of hyperparameter values that influence ML model performance (Subsection 2.5 ). Figure 1: Illustration of the FML Taxonomy 2.1 Model Compression Model compression in FML refers to techniques that reduce the size and computational complexity of ANNs [ 18 ] . This can be achieved through methods such as pruning, quantization, knowledge distillation, and low-rank factorization, or Dynamic Neural Networks that dynamically shrink the size of the ANN based on available computational resources. Pruning Pruning is a widely used FML technique that reduces the size and complexity of ANNs by eliminating redundant or insignificant parameters while maintaining overall performance [ 9 ] . This approach is particularly valuable for deploying deep learning models in environments with limited computational and memory resources, such as mobile devices and embedded systems. Pruning typically follows a two-step process: first, an overparameterized network is trained to achieve high accuracy; then, unnecessary weights, neurons, or filters are identified and pruned [ 19 ] . To mitigate the potential accuracy loss from pruning, a fine-tuning phase is often incorporated to restore the model’s performance. The two main categories of pruning is structured and unstructured. Structured pruning involves removing entire structures such as neurons, filters, channels, or layers, thereby maintaining a regular architecture and facilitating efficient hardware implementation [ 20 ] . This approach contrasts with unstructured pruning, which eliminates individual weights, leading to irregular sparsity patterns that can be challenging for standard hardware to handle [ 21 ] . An example of structured pruning is filter pruning in CNNs, where entire filters are removed based on specific criteria. For instance removing filters with small L1-norm values to compress CNNs [ 22 ] . A third approach is dynamic pruning which refers to methods that adjust the network’s structure during training or inference, allowing the model to adaptively prune and regrow connections based on certain criteria [ 23 ] . An important pruning technique is the magnitude-based pruning, which assesses the importance of parameters based on their magnitude, with smaller weights typically considered less critical [

Read the complete text