Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language.
E Shoukat, R Irfan, I Basharat, MA Tahir, S Shaukat · 2025
Read Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language. by E Shoukat, R Irfan, I Basharat, MA Tahir, S Shaukat (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-09722.txt.
Introduction
Ezzah Shoukat and four co-authors build a deep learning model that flags toxic content in Urdu text, in a 2025 study. Online abuse research covers many languages, but South Asian languages lag behind. Urdu adds hard traits: one word can carry many spellings, and writers often mix in English. The team's model reads text both ways with a gated recurrent design and adds an attention layer that weighs the most telling words.
Four rival designs were trained for comparison, with and without pre-trained word vectors. The new model beat all rivals, reaching eighty-four percent accuracy in its best run. The attention layer clearly helped. The pre-trained vectors, though, did not help on this abuse data set. The authors offer the model as a base for safer Urdu online spaces.
Catalog record
| CREATOR | E Shoukat, R Irfan, I Basharat, MA Tahir, S Shaukat |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-09722
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language. E Shoukat, R Irfan, I Basharat, MA Tahir, S Shaukat Abstract With the increased use of the internet and social networks for online discussions, the spread of toxic and inappropriate content on social networking sites has also increased. Several studies have been conducted in different languages. However, there is less work done for South Asian languages for inappropriate content identification using deep learning techniques. In Urdu language, the spellings are not unique, and people write different common spellings for the same word, while mixing it other languages, like English in the text makes it more challenging, and limited research work is available to process such language with the finest algorithms. The use of attention layer with a deep learning model can help handling the long-term dependencies and increase its efficiency . To explore the effects of the attention layer, this study proposes attention-based Bidirectional GRU hybrid model for identifying inappropriate content in Urdu Unicode text language. Four different baseline deep learning models; LSTM, Bi-LSTM, GRU, and TCN, are used to compare the performance of the proposed model. The results of these models were compared based on evaluation metrics, dataset size, and impact of the word embedding layer. The pre-trained Urdu word2Vec embeddings were utilized for our case. Our proposed model BiGRU-A outperformed all other baseline models by yielding 84% accuracy without using pre-trained word2Vec layer. From our experiments, we have established that the attention layer improves the model’s efficiency, and pre-trained word2Vec embedding does not work well with an inappropriate content dataset. [type=editor, ] [orcid=0000-0002-2719-9852 ] [orcid=0000-0002-2335-2776 ] [] keywords Attention,deep learning,inappropriate language detection,natural language processing,social media,text processing. † † titlenote: This work is not sponsored/supported by any organization. † † credit: Methodology, Data curation, Writing - Original draft preparation, Visualization † † credit: Formal analysis, Supervision, Writing – review & editing, Project administration, Validation † † credit: Review & editing, Validation † † credit: Conceptualization, Formal analysis, Supervision, Writing – review & editing, Project administration, Validation † † credit: Formal analysis, Visualization, Validation, Project administration 1 Introduction Social media networks like Twitter, Facebook, and YouTube allow a wide diversity of people from all around the world. Users find it more convenient to write and express their thoughts about products, movies, or articles in their local language than in English Akhter et al. 2020 . At the same time, billions of people using social media platforms are prone to cyber-crimes such as bullying, threats, and scams. The Government of Pakistan (GoP) has taken precautionary steps to avoid uncontrollable consequences after a series of events in recent years Rizwan et al. 2020 . These included slandering of political parties, and their leaders, hurting the sentiments of religious minorities, and targeted harassment of women sharing their point of view. Such actions give away the notions of nation’s struggles with online hate speech dialogues and necessitates the immediate need for automated filtering system. Social media sites need to be able to detect, categorize, and clean the inappropriate content before it is posted online. Inappropriate content in this study is used broadly to refer to any type of the content that could be hurtful. Abusive language and profanities are major part of the inappropriate or violating content. Urdu is a resource-scarce language with a complex morphological structure, unique characters, and low linguistic resources Mandl et al. 2019 . Several native Urdu speakers communicate their sentiments, views, and thoughts on social media using the Urdu script. Hence, automatic filtering and identification of inappropriate content in the Urdu language are equally important to help non-Urdu speakers understand the sentiments and to improve the quality of conversation for native Urdu speakers. Many studies been conducted using Machine learning (ML) and Deep learning (DL) algorithms to detect inappropriate content in different languages Chakraborty and Seddiqui 2019 , Sigurbergsson and Derczynski 2020 , Subramani et al. 2019 . From basic ML models to simple recurrent neural networks to the most advanced models with transformers are the approaches followed by researchers. In this domain, much work been done in English and many other languages. But current literature lacks inappropriate content detection in Urdu script using DL techniques. In recent years many researchers have publicly shared their annotated Urdu corpus for future research workers. However, finding domain-specific data is still a challenging task. Most of the annotated datasets available are either in Roman Urdu or of small size to apply advanced deep neural networks and analyze their results. Furthermore, only classical ML models are explored extensively on the available datasets. The proposed strategy aims to address the gaps discussed till yet. The following are the key contributions of this research: • A large inappropriate content corpus, developed by combining three publicly available annotated datasets. The first two were in Urdu Script, but the third was converted from Roman Urdu to Urdu script. • A Bidirectional Gated Recurrent Unit with attention layer (BiGRU-A) hybrid model for identification of inappropriate content is proposed. • Result comparison of state-of-art DL models, i.e. Long short-term Memory (LSTM), Bidirectional Long short-term Memory (Bi-LSTM), Gated Recurrent Unit (GRU), Temporal Convolutional Network (TCN). • Comparative analysis of baseline models and proposed model with and without using pre-trained word embedding model word2vec for the Urdu language. • The impact of dataset size on the performance of DL models is studied. The paper is comprised of following sections; introduction followed by the study of existing literature, research gap, problem statement, methodology, experiments and discussion on their results, conclusions, future work and references. 2 Literature Review This section provides in-depth information on studies related to the identification of inappropriate language. It explores conventional and neural network approaches. Finally, we will provide a detailed comparative analysis of the work on offensive content detection in the Urdu language. Previous studies have explored classical ML models and a few Neural Networks(NN) for the problem of offensive language detection in Urdu script. On large data sets, ML models struggle to perform well and usually learn just directed text features. When compared to standard ML models, the efficiency of DL models has significantly improved as the size of dataset increases Mathew et al. 2020 . The NLP community has been concentrating on internet platforms including Twitter,YouTube, Instagram, Facebook, and online blogs to identify harmful language Balakrishnan et al. 2019 , Alakrot et al. 2018 , Rani and Ojha 2019 , Lee et al. 2018a . 2.1 Machine Learning approaches In Chakraborty and Seddiqui 2019 , Support Vector Machine(SVM) with the linear kernel is proposed to achieve the best results in detecting inappropriate language in Bengali dataset. In their suggested approach, both Unicode the Bengali characters, as well as Unicode emoticons, were taken into consideration as acceptable input in our suggested approach. Authors in Hajibabaee et al. 2022 employed ML classifiers SVM, Logistic Regression (LR), Decision Trees (DT) and Random forest (RF), etc. along with modular cleaning of Twitter dataset, and built a tokenizer. Threatening comments in Twitter tweets were categorized using NB in Lee et al. 2018b . Multi-class classification in Roman Urdu content extracted from YouTube comments is presented by Sajid et al. 2020 . For feature selection, n-gram and TF-IDF techniques