Hardware-Aware Federated Learning for Speech Emotion Recognition PubID: pubid: 979-8-3195-1046-4/26/$31.00 ©2026 IEEE
Beyazıt Bestami YÜKSEL, Emrah DİKBIYIK · 2026
Read Hardware-Aware Federated Learning for Speech Emotion Recognition PubID: pubid: 979-8-3195-1046-4/26/$31.00 ©2026 IEEE by Beyazıt Bestami YÜKSEL, Emrah DİKBIYIK (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24712.txt.
Introduction
Beyazit Bestami Yuksel and Emrah Dikbiyik build hardware-aware federated learning for speech emotion recognition. Real deployments mix clients with different compute, memory, and latency, which stretches round times and system cost. Their loop combines hardware profiling, adaptive local epochs, and top-K client picks on session-partitioned IEMOCAP data. Against FedAvg, FedProx, and random client picks under non-IID data, it matches validation accuracy across 50 rounds and 5 trials. It cuts total training time by about 36.5 percent versus FedAvg and lowers cumulative communication cost by 40 percent.
Catalog record
| CREATOR | Beyazıt Bestami YÜKSEL, Emrah DİKBIYIK |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17521 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24712
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hardware-Aware Federated Learning for Speech Emotion Recognition PubID: pubid: 979-8-3195-1046-4/26/$31.00 ©2026 IEEE Beyazıt Bestami YÜKSEL, Emrah DİKBIYIK Abstract Federated learning (FL) enables privacy-preserving collaborative training across distributed edge devices, but real deployments involve heterogeneous clients with different processing power, memory capacity, and communication latency, which often increase round duration and system cost. This paper proposes a hardware-aware federated learning framework for emotion recognition on session-partitioned IEMOCAP that integrates hardware profiling, top- K K client selection, and adaptive local epochs within a unified training loop. We compare the method against FedAvg, FedProx, and random top- K K selection under a non-IID setup and show that, across 50 federated rounds and 5 independent trials, the proposed approach achieves competitive validation accuracy ( 0.352 ± 0.019 0.352{\pm}0.019 ), reduces total training time by about 36.5% compared to FedAvg, and lowers cumulative communication cost by 40%. I Introduction Federated learning is an important paradigm that enables distributed training while keeping data on device and sharing only model updates. It offers a strong alternative for privacy-sensitive domains such as speech analysis and mobile artificial intelligence, as it avoids central collection of raw data. However, most existing FL algorithms assume that client environments are hardware-homogeneous. In real edge deployments, devices differ substantially in processing power, memory, and network latency; this mismatch causes inefficiency, communication overhead, and the straggler effect during training. This study proposes a hardware-aware FL framework for speech emotion recognition (SER) in heterogeneous edge environments. The main contributions are: (i) a joint approach that unifies client selection, workload adaptation, and latency awareness within a single optimization framework, (ii) a dynamic profiling mechanism that embeds CPU, RAM, latency, and training time into the FL loop for online decision making, (iii) a multi-objective client scheduling strategy under the system-level objective defined in Eq. (3), (iv) an adaptive workload scaling mechanism that enables stronger devices to contribute proportionally, and (v) empirical validation on a session-based non-IID setup showing improved accuracy together with approximately 36.5% training-time reduction and 40% communication-cost reduction. II Related Work Federated learning has a broad literature ranging from the foundational FedAvg line [ 1 ] to system design, security [ 18 ] , and scalability discussions [ 8 , 2 ] . On computationally constrained heterogeneous edge devices, FL must cope with uneven compute, memory, and communication capabilities; architectural adaptations, system-level scheduling, synchronous/asynchronous aggregation, distillation, and split learning are surveyed in [ 13 ] . Classical mitigations include FedProx [ 3 ] , resource-aware adaptive FL [ 4 ] , FedCS [ 5 ] , utility-based selection in Oort [ 9 ] , and tiered scheduling in TiFL [ 10 ] . Energy- and hardware-oriented extensions further consider CPU–GPU co-resource management, asynchronous training, and differential-privacy-aware load modeling [ 11 , 12 , 17 ] . Most of these methods do not jointly optimize client selection and local workload adaptation within the same framework; the proposed approach addresses both dimensions together. For efficient training under client heterogeneity, FedEff assigns per-client local epoch budgets from compute and communication profiles via deterministic server-side rules, reducing wall-clock time and waiting delays [ 15 ] . Complementary approaches instead learn heterogeneous local training intensity policies with deep reinforcement learning, adapting per-round workload to client compute capacity and communication delay [ 16 ] . Our HW-FL complements such epoch adaptation with hardware-score-based top- K selection and communication-aware aggregation in a single loop. In affective computing, federated multimodal emotion recognition on edge devices has been explored with lightweight CNN-based audiovisual models, reporting improved classification under privacy-preserving training [ 14 ] . Centralized bimodal audio–text systems on IEMOCAP with data augmentation can also achieve high accuracy [ 7 ] . Nevertheless, hardware-aware FL integration for speech emotion recognition in heterogeneous edge environments remains limited. III System Model The system model considered in this study consists of a federated learning server and multiple connected edge clients. Clients store local data for the speech emotion recognition task without leaving the device and transmit only model parameter updates to the server. Client devices are heterogeneous in terms of CPU core count, memory capacity, and network latency, and these differences directly affect training efficiency. In each federated learning round, the server broadcasts the current global model to selected clients; clients perform local training and send model updates back to the server. The server aggregates these updates to produce a new global model. The proposed hardware-aware mechanisms are integrated into the client selection and local training steps of this basic FL loop. IV Hardware-Aware Federated Learning Framework The hardware-aware federated learning framework proposed in this study addresses hardware profiling, client selection, and adaptive workload assignment within an integrated structure. The overall workflow and interactions among components are presented in Fig. 1 . Figure 1: Overall workflow and components of the hardware-aware federated learning system. IV-A System Architecture The proposed system consists of multiple edge devices connected to a federated learning server. Each client performs local model training using private speech data and periodically sends model updates to the server. The server aggregates received updates to produce a global model. Edge devices include heterogeneous hardware platforms such as laptops, tablets, and smartphones. IV-B Hardware Profiling Each client device performs local hardware profiling to estimate computational capabilities. The hardware profile is represented as H_{i}=(CPU_{i},RAM_{i},T_{i},L_{i}) (1) where CPU_{i} denotes the number of CPU cores, RAM_{i} available memory, T_{i} model training latency, and L_{i} network communication latency. To reduce the effect of scale differences, these quantities are normalized before scoring; for example, \hat{CPU}_{i}=\frac{CPU_{i}}{\max_{j}CPU_{j}},\quad\hat{L}_{i}=\frac{L_{i}}{\max_{j}L_{j}} (2) can be defined. IV-C Hardware-Aware Client Planning and Optimization Based on collected hardware profiles, the federated server computes a hardware score for each client: S_{i}=\alpha\cdot CPU_{i}+\beta\cdot RAM_{i}+\gamma\cdot\frac{1}{T_{i}}-\delta\cdot L_{i} (3) where \alpha , \beta , \gamma , and \delta are weights controlling the importance of processor, memory, training latency, and communication latency, respectively. This score can be interpreted as a proxy that rewards computational capacity and training efficiency while penalizing latency. Clients with higher scores are prioritized for participation in training. Federated round scheduling can be approximated by the following multi-objective problem: \min_{S_{t},\{E_{i}\}}\max_{i\in S_{t}}T_{i}(E_{i})+\lambda\,C(S_{t}) (4) Here, S_{t} denotes the selected client set, E_{i} the number of local epochs for client i , T_{i}(E_{i}) local training time, C(S_{t}) communication load, and \lambda the trade-off coefficient. This formulation approximately expresses a constrained optimization problem that minimizes round completion time under communication and resource constraints. The problem has an NP-hard structure because client selection and workload assignment must be optimized jointly. Therefore, the proposed method uses a hardware-aware score-based heuristic to obtain a practical solution without increasing computational cost excessively. The heuristic showed stable learning behavior in experiments. However, theoretical convergence guarantees are left for future work. IV-D Adaptive Local Training To further improve training efficiency, the number of local training epochs is adjusted dynamically according to each client’s hardware score: E_{i}=E_{\text{base}}\times\frac{S_{i}}{S_{\max}} (5) This mechanism enables stronger devices to contribute more effectively during federated training. The proposed framework is model-agnostic and can be applied to CNN, RNN, and transformer-based architectures with the same scheduling logic. In this study, a lightweight model is preferred primarily to isolate system-level contributions independent of model architecture. To monitor long-term client inclusiveness, participation fairness can also be defined. Let f_{i} be the selection frequency of client