ArabEmoNet: A Lightweight Hybrid 2D CNN-BiLSTM Model with Attention for Robust Arabic Speech Emotion Recognition
Ali Abouzeid, Bilal Elbouardi, Mohamed Maged, Shady Shehata · 2025
Read ArabEmoNet: A Lightweight Hybrid 2D CNN-BiLSTM Model with Attention for Robust Arabic Speech Emotion Recognition by Ali Abouzeid, Bilal Elbouardi, Mohamed Maged, Shady Shehata (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01401.txt.
Introduction
Recognizing emotion in spoken Arabic is hard because data and research for the language are scarce, and large models are costly to run. Ali Abouzeid, Bilal Elbouardi, Mohamed Maged, and Shady Shehata introduce ArabEmoNet, a lightweight model for that task. It reads Mel spectrograms, pictures of sound over time and pitch, with two-dimensional convolutions that keep fine emotional cues.
An attention-equipped bidirectional LSTM then reads those features in both directions to catch how emotion unfolds. With only about one million parameters, the model is many times smaller than large speech systems such as HuBERT base and Whisper, yet the authors report superior results. That small size suits phones and other limited devices. ArabEmoNet aims to bring strong speech emotion tools to a low-resource language in real-world settings.
Catalog record
| CREATOR | Ali Abouzeid, Bilal Elbouardi, Mohamed Maged, Shady Shehata |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-01401
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ArabEmoNet: A Lightweight Hybrid 2D CNN-BiLSTM Model with Attention for Robust Arabic Speech Emotion Recognition Ali Abouzeid, Bilal Elbouardi, Mohamed Maged, Shady Shehata Abstract Speech emotion recognition is vital for human-computer interaction, particularly for low-resource languages like Arabic, which face challenges due to limited data and research. We introduce ArabEmoNet, a lightweight architecture designed to overcome these limitations and deliver state-of-the-art performance. Unlike previous systems relying on discrete MFCC features and 1D convolutions, which miss nuanced spectro-temporal patterns, ArabEmoNet uses Mel spectrograms processed through 2D convolutions, preserving critical emotional cues often lost in traditional methods. While recent models favor large-scale architectures with millions of parameters, ArabEmoNet achieves superior results with just 1 million parameters, which is 90 times smaller than HuBERT base and 74 times smaller than Whisper. This efficiency makes it ideal for resource-constrained environments. ArabEmoNet advances Arabic speech emotion recognition, offering exceptional performance and accessibility for real-world applications. 1 Introduction Speech Emotion Recognition (SER) is essential for improving human-computer interaction, particularly in linguistically diverse contexts like Arabic speech. The complexity of detecting emotions from speech arises from variations in prosody, phonetics, and speaker expression. Over time, SER has evolved from statistical approaches to deep learning, significantly enhancing recognition accuracy. Early SER systems relied on handcrafted acoustic features (e.g., pitch, energy, and MFCCs) processed using classical machine learning models like Support Vector Machines (SVMs) and Gaussian Mixture Models (GMMs) Lieskovska et al. 2021 . While effective, these methods struggled with cross-dataset generalization, particularly in Arabic speech, which exhibits rich phonetic and prosodic diversity. Deep learning mitigated these limitations by enabling automatic feature extraction, with CNNs capturing localized spectro-temporal patterns and LSTMs modeling sequential dependencies Fayek et al. 2017 . However, many Arabic SER systems still rely on MFCCs and 1D convolutions, which fail to capture essential spectral-temporal structures for robust emotion recognition. Transformer-based models Vaswani et al. 2017 introduced attention mechanisms to dynamically focus on emotionally salient speech segments Mirsamadi et al. 2017 . While effective in modeling long-range dependencies and parallelizing computations across emotional speech sequences, their high computational complexity ( O(n^{2}) for self-attention) and substantial memory requirements render them impractical for resource-constrained environments. To address these constraints, we propose ArabEmoNet, a lightweight architecture leveraging Mel spectrograms with 2D convolutions, effectively capturing both fine-grained spectral features and global contextual relationships Kurpukdee et al. 2017 . Our model achieves competitive accuracy with just 0.97M parameters, making it significantly more efficient than HuBERT Hsu et al. 2021 and Whisper Radford et al. 2022 while maintaining state-of-the-art performance. Additionally, we augmented the data by integrating SpecAugment Park et al. 2019 and Additive White Gaussian Noise (AWGN), which enhances the robustness of our model Huh et al. 2024 . Experiments on KSUEmotions Meftah et al. 2021 and KEDAS Belhadj et al. 2022 datasets confirm that ArabEmoNet surpasses prior architectures while maintaining efficiency, marking a significant step forward in Arabic SER. The main contributions of this paper can be summarized as follows: • We propose ArabEmoNet: a novel lightweight hybrid architecture combining 2D Convolutional Neural Networks (CNN) with Bidirectional Long Short-Term Memory (BiLSTM) and an attention mechanism • ArabEmoNet (1M parameters) achieves superior results with just 1 million parameters—90 times smaller than HuBERT base (95M parameters) and 74 times smaller than Whisper (74M parameters). • We demonstrate ArabEmoNet’s superior performance by achieving state-of-the-art results on the KSUEmotion and KEDAS datasets, surpassing previous benchmark models. Figure 1: ArabEmoNet:2D CNN-Attention and BiLSTM Model Architecture. 2 Related Work Speech Emotion Recognition (SER) has been an active area of research for decades. Traditional approaches often relied on statistical evaluations of speech features such as pitch, energy, and spectral coefficients, combined with classifiers such as SVMs or Hidden Markov Models (HMMs) Iliou and Anagnostopoulos 2009 and Nwe et al. 2003 . Although these methods provided foundational information, they struggled to generalize across different datasets and languages. The advent of deep learning significantly advanced SER by enabling the extraction of hierarchical feature representations. Convolutional Neural Networks (CNNs) demonstrated their ability to capture local spectral features, while Recurrent Neural Networks (RNNs), including Long Short-Term Memory (LSTM) networks, excelled at modeling temporal dependencies Sainath et al. 2015 and Trigeorgis et al. 2016 . End-to-end frameworks combining CNNs and RNNs have further improved performance by learning from raw or minimally processed data Fayek et al. 2017 . Attention mechanisms, introduced in Bahdanau et al. 2015 and further refined in Vaswani et al. 2017 , have recently shown promise in SER. They allow models to focus on the most relevant parts of the input, which is particularly useful for tasks such as emotion recognition, where only a subset of characteristics may contribute to emotional cues Mirsamadi et al. 2017 and Neumann and Vu 2017 . Previous work in Hifny and Ali 2019 proposed an efficient Arabic Speech Emotion Recognition (SER) system, combining Convolutional Neural Networks (CNNs), Bidirectional LSTMs (BiLSTMs), and attention mechanisms, which achieved state-of-the-art performance on the KSUEmotions dataset. However, their approach relied on 13-feature Mel Frequency Cepstral Coefficients (MFCCs) and 1D convolutions, which may restrict the richness of captured acoustic features. Building on the insights from their work and the demonstrated importance of leveraging richer representations by Meng et al. 2019 and Lieskovska et al. 2021 , we diverge from Hifny and Ali 2019 by utilizing Log-Mel spectrograms as input, which offer a more comprehensive acoustic feature representation, and employs 2D convolutions to effectively capture spatial dependencies within these spectrograms Furthermore, advances in data augmentation techniques, such as SpecAugment Park et al. 2019 and Gaussian noise, have addressed data scarcity challenges, enhancing the generalization capabilities of SER systems. Studies have also investigated the impact of kernel sizes in CNN layers and regularization techniques to optimize performance Kumar and Raj 2024 . 3 Proposed Approach In this work, we introduce ArabEmoNet, a dedicated 2D NN-Attention and BiLSTM framework optimized for Arabic Speech Emotion Recognition. Our model processes Log-Mel spectrograms to effectively capture the multifaceted nature of emotional speech through three complementary components: 2D convolutional layers that identify emotion-specific spectral patterns, bidirectional LSTMs that model the temporal evolution of emotional cues, and an attention mechanism that highlights emotionally salient segments within utterances. This integrated approach addresses the unique challenges of recognizing Arabic emotional expressions while maintaining a lightweight, efficient architecture. Figure 1 illustrates our complete model design. 3.1 Input Prepossessing For our classification model, raw audio signals are transformed into Log-Mel spectrograms. This process involves computing the Mel spectrogram using a Fast Fourier Transform (FFT) window length of 2048 samples and a hop length of 256 samples. We generate 128 Mel bands across a frequency range from 80 Hz to 7600 Hz . A Hann window is applied to each frame to minimize spectral leakage. Subsequently, the resulting Mel spectrogram is converted to a logarithmic scale (decibels), referenced to the maximum power, to optimize the dynamic range for neural network processing. 3.2 Data Augmentation To improve the generalization ability of the model and mitigate overfitting, we incorporate Gaussian noise augmentation during training. This technique simulates variations in the input data and leads to a more robust model. Optimization is performed using the Adam optimizer, which adapts learning rates for each parameter based on the first and second moments of the gradients. Additionally, we