Towards Cross-Lingual Audio Abuse Detection in Low-Resource Settings with Few-Shot Learning
Aditya Narayan Sankaran, Reza Farahbakhsh, Noel Crespi · 2024
Read Towards Cross-Lingual Audio Abuse Detection in Low-Resource Settings with Few-Shot Learning by Aditya Narayan Sankaran, Reza Farahbakhsh, Noel Crespi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01408.txt.
Introduction
Aditya Narayan Sankaran, Reza Farahbakhsh, and Noel Crespi study abuse detection in speech across languages in a 2024 paper. Most work on online abuse deals with text in rich-resource languages, leaving audio in languages with little data largely untouched. The team asked whether strong speech models trained elsewhere could spot abuse in Indian languages from just a few examples.
They took traits from two large pre-trained speech models and used a meta-learning method that learns how to learn from small samples. On a data set covering ten languages, they varied the number of examples given for each new language. The setup carried over across languages well, and a look at the learned traits helped explain model behavior. The authors offer the work as a path for low-resource audio safety tools.
Catalog record
| CREATOR | Aditya Narayan Sankaran, Reza Farahbakhsh, Noel Crespi |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01408
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Cross-Lingual Audio Abuse Detection in Low-Resource Settings with Few-Shot Learning Aditya Narayan Sankaran, Reza Farahbakhsh, Noel Crespi Abstract Online abusive content detection, particularly in low-resource settings and within the audio modality, remains underexplored. We investigate the potential of pre-trained audio representations for detecting abusive language in low-resource languages, in this case, in Indian languages using Few Shot Learning (FSL). Leveraging powerful representations from models such as Wav2Vec and Whisper, we explore cross-lingual abuse detection using the ADIMA dataset with FSL. Our approach integrates these representations within the Model-Agnostic Meta-Learning (MAML) framework to classify abusive language in 10 languages. We experiment with various shot sizes (50-200) evaluating the impact of limited data on performance. Additionally, a feature visualization study was conducted to better understand model behaviour. This study highlights the generalization ability of pre-trained models in low-resource scenarios and offers valuable insights into detecting abusive language in multilingual contexts. 1 Introduction The widespread adoption of social media for everyday communication requires safeguards and moderation to create a safe space for the user and the social community. With audio-based social media platforms like Twitter (now X) Spaces, Clubhouse, Discord, ShareChat etc, moderating offensive language and hate speech has become essential in maintaining a safe space for people online. These platforms host users from diverse linguistic backgrounds, especially in multilingual countries like India. India is home to several languages with more than 30 Million speakers of languages and has experienced a phenomenal increase in the use of online social media services, including Facebook, Twitter (now X), Instagram, LinkedIn, and YouTube, with over 250 million users (and growing) helping them in Social interactions and conversations Ganguly and Kumaraguru 2019 ; Palakodety and KhudaBukhsh 2020 . 76% of Indians spend an estimated 1 hour and 29 minutes daily using social media through smartphones, with adolescents between the ages of 13 and 19 making up 31% of the overall number of people who use social media Dar and Nagrath 2022 ; Srivastava et al. 2019 . Given that a large share of users of social media are teenagers and young adults, there is an absolute need to create a safe online space for them to express their views freely without being exposed to hate-filled and offensive content. In an era where social media and entertainment companies are being scrutinized more carefully than ever before, laxity related to user privacy or community rule violations could prove harmful and costly for social audio platforms. There have been reports of incidents where Clubhouse rooms engaged in racist and anti-Semitist talks 1 1 1 https://tinyurl.com/fslanti . This issue raises concerns about how it can be more difficult to safeguard user privacy and security on Audio Social Media platforms as traditional content moderation techniques utilized for text-based media do not necessarily work well with audio-based platforms. Abusive language detection and Hate Speech detection in Low Resource Language settings has been a popularly researched topic, especially in the text modality. Transfer Learning-based approaches have shown incredible performance in abuse detection using existing popular models like BERT, RoBERTa, XLM-RoBERTa etc Mozafari et al. 2019 ; Ranasinghe and Zampieri 2021 . Works by Mozafari et al. 2022 and Awal et al. 2024 also show the ability of meta-learning-based models to outperform transfer learning-based models in a cross-lingual abuse detection task in Low resource settings. Even abusive content detection on images and videos has been accelerated with the contribution of multimedia datasets and progress in Deep Learning techniques Gao et al. 2020 ; Alcântara et al. 2020 . Abuse detection in audio is relatively unexplored given the limited availability of such datasets to work with and the complex task of collecting and annotating a large set of audio samples, that too in a variety of languages and accents, especially in highly multilingual countries like India where there are 1369 rationalised languages and more than 500 dialects, with 22 scheduled languages having over 1.17 billion speakers Khanuja et al. 2021 ; Sengupta 2018 . A simple approach for abuse detection is to employ an Automatic Speech Recognition (ASR) model to transcribe the audio to text and then work with them using existing NLP techniques with models like Whisper and Wav2Vec boasting low Word-Error Rates for the transcription task. This is what Ghosh et al. 2021 did by crawling through popular voice datasets for abusive data called DeToxy and performed a Two-Step approach by transcribing audio to text and using BERT for downstream classification. A major problem with these approaches is that they tend to miss out on abusive words since they are usually not spoken clearly and completely, thereby limiting the ability to spot abusive keywords Gupta et al. 2022 . Recent advancements in Pre-Trained Audio Representations, such as those demonstrated by Shor et al. 2022 and Saeed et al. 2021 , show promising results in various audio abuse detection-related tasks, particularly embeddings pre-trained on large datasets like Audioset have been successfully transferred to multiple classification tasks, including various audio pattern recognition tasks Kong et al. 2020 . Building on the success of Meta-Learning techniques, such as Model-Agnostic Meta-Learning (MAML) Finn et al. 2017 , Prototypical Networks Snell et al. 2017 , and Contrastive Learning Yang et al. 2022 , our work applies MAML in a low-resource setting, leveraging its demonstrated efficacy in tasks such as document classification van der Heijden et al. 2021 , speech recognition Singh et al. 2022 , and abstract summarization Huh and Ko 2022 . Thus, we present the contributions 2 2 2 Codebase: https://github.com/callmesanfornow/fsl-audio-abuse.git of our work enumerated below: 1. We propose a MAML-based few-shot cross-lingual audio abuse classification methodology, leveraging pre-trained audio representations from Whisper Radford et al. 2022 and Wav2Vec Baevski et al. 2020 . We also assess the effectiveness of these pre-trained features under two feature normalization strategies: L2 normalization and Temporal Mean. 2. Our method is evaluated on the ADIMA dataset Gupta et al. 2022 , with Whisper achieving top accuracy scores ranging from 78.98% to 85.22% in the 100-shot setting, using L2-Norm feature normalization. 3. We provide a visual analysis of pre-trained audio features from the best-performing normalization setting, examining how language similarity can enhance cross-lingual abuse detection, especially in Low-Resource Languages. This study offers valuable insights into optimal strategies for audio abuse detection and identifies potential directions for future research. 2 Related Works 2.1 Audio Abuse Detection The introduction of DeToxy Ghosh et al. 2021 , a large-scale multi-modal dataset was a significant advancement in the field of audio toxicity detection, particularly in the context of spoken utterances. DeToxy improved text-based methods’ performance and lessened keyword bias, paving the way for more robust audio abuse detection. Building on this work, Facebook AI extended on DeToxy by creating MuTox Costa-jussà et al. 2024 , one of the first large-scale multilingual audio datasets for toxicity detection. MuTox includes 20,000 audio utterances in English and Spanish and 4,000 utterances across 19 other languages. However, it is worth noting that this dataset only includes three Indian languages: Bengali, Hindi, and Urdu. The first Bengali Audio Abuse dataset was introduced by Rahut et al. 2020 , where Transfer Learning was applied to extract features from 960 voice recordings of native Bengali speakers. Following this, ADIMA Gupta et al.