Who Wrote This? Identifying Machine vs Human-Generated Text in Hausa

Babangida Sani, Aakansha Soy, Sukairaj Hafiz Imam, Ahmad Mustapha, Lukman Jibril Aliyu, Idris Abdulmumin, Ibrahim Said Ahmad, Shamsuddeen Hassan Muhammad · 2025

Read Who Wrote This? Identifying Machine vs Human-Generated Text in Hausa by Babangida Sani, Aakansha Soy, Sukairaj Hafiz Imam, Ahmad Mustapha, Lukman Jibril Aliyu, Idris Abdulmumin, Ibrahim Said Ahmad, Shamsuddeen Hassan Muhammad (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13101.txt.

Introduction

Babangida Sani and co-authors present a 2025 paper on telling machine text from human text in Hausa. Language models now write fluently, which invites plagiarism and fake news. Detectors exist for big languages, but Hausa, a major West African tongue, had none at scale. The team scraped seven Hausa news outlets for human articles. A commercial model then wrote matched fake articles from the same headlines.

Four Africa-tuned pre-trained models were fine-tuned on the paired data. The best, AfroXLMR, reached about 99 percent accuracy and a matching F1 score. The data set is public for further work. The authors frame the tool as a guard for low-resource language media. It is the first large detector of its kind for Hausa.

Catalog record

CREATORBabangida Sani, Aakansha Soy, Sukairaj Hafiz Imam, Ahmad Mustapha, Lukman Jibril Aliyu, Idris Abdulmumin, Ibrahim Said Ahmad, Shamsuddeen Hassan Muhammad
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE16 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-13101

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Who Wrote This? Identifying Machine vs Human-Generated Text in Hausa Babangida Sani, Aakansha Soy, Sukairaj Hafiz Imam, Ahmad Mustapha, Lukman Jibril Aliyu, Idris Abdulmumin, Ibrahim Said Ahmad, Shamsuddeen Hassan Muhammad Abstract The advancement of large language models (LLMs) has allowed them to be proficient in various tasks, including content generation. However, their unregulated usage can lead to malicious activities such as plagiarism and generating and spreading fake news, especially for low-resource languages. Most existing machine-generated text detectors are trained on high-resource languages like English, French, etc. In this study, we developed the first large-scale detector that can distinguish between human- and machine-generated content in Hausa. We scrapped seven Hausa-language media outlets for the human-generated text and the Gemini-2.0 flash model to automatically generate the corresponding Hausa-language articles based on the human-generated article headlines. We fine-tuned four pre-trained Afri-centric models (AfriTeVa, AfriBERTa, AfroXLMR, and AfroXLMR-76L) on the resulting dataset and assessed their performance using accuracy and F1-score metrics. AfroXLMR achieved the highest performance with an accuracy of 99.23% and an F1 score of 99.21%, demonstrating its effectiveness for Hausa text detection. Our dataset is made publicly available 1 1 1 https://github.com/TheBangis/hausa_corpus to enable further research. 1 Introduction Hausa is among the most spoken Chadic languages, belonging to the Afroasiatic phylum. Over 100 million people are estimated to speak the language, with the majority of speakers living in Northern Nigeria and the Republic of Niger, respectively Inuwa-Dutse 2021 . However, from computational linguistics, it is regarded as a low-resource language, having insufficient resources to support tasks involving Natural Language Processing (NLP; Adam et al. 2023 ; Muhammad et al. 2023 ). Hausa language is written in either the Latin (or Boko ) and Arabic (or Ajami ) script Jaggar 2006 . The Boko script, existing since the 1930s, was introduced by the British colonial administration, and is used in education, government, and digital communication. The Ajami script, an order writing system of the Hausa language that existed in pre-colonial times, is used mostly in religious, cultural, and informal writing. For the purpose of our work, and as Hausa is widely written nowadays, we scraped and generated data based on the Latin-based script. Large language models (LLMs) are becoming mainstream and easily accessible, ushering in an explosion of machine-generated content over various channels, such as news, social media, question-answering (QA) forums, educational, and even academic contexts Wang et al. 2023 . The human-like quality of texts generated by LLMs models for different languages including Hausa language is always advancing, allowing them to generate diverse content. LLMs, intentionally or unintentionally, have the potential to be used to create and propagate harmful or misleading content, such as fake news or hate speech Xie et al. 2024 , or even fake or artificial scholarship. To ensure the authenticity, accuracy, and trustworthiness of content, there is a need for machine-generated text detectors. Extensive research has been undertaken to differentiate between machine-generated texts (MGTs) and human-generated texts (HGTs), primarily employing model-based approaches Wang et al. 2023 ; Alshammari 2024 ; Ji et al. 2024 . In existing studies, (i) focus has mainly been on high-resource languages like English; (ii) there are no reliable detectors for detecting human vs. AI-generated text in the Hausa language; (iii) ensuring content authenticity is difficult, especially for low resource languages like Hausa Ji et al. 2024 . We aim, therefore, to develop an automatic detector to classify human-generated and machine-generated text in Hausa, focused on the news domain, hence filling this gap. The following are our contributions: • We are the first to develop a Hausa detector that is capable of differentiating HGT and MGT in Hausa. We believe it would help in ensuring content authenticity in digital communication, academia, and mitigating fake news. • We curated a dataset that consists of human-generated data by scraping seven Hausa media outlets and machine-generated data using Gemini, addressing the lack of high-quality data in the area. • By focusing on the Hausa language, we contribute to the expanding NLP capabilities for low-resource languages. • All our resources will be open-source to encourage future academic research in the Hausa language. 2 Related Work 2.1 Detection of MGTs before ChatGPT Radford et al. 2019 raised concerns regarding using machine-generated text for malicious purposes such as spam, fake news, plagiarism, and disinformation. The GLTR (Giant Language Model Test Room) tool Gehrmann et al. 2019 , released in June 2019, is an open-source system for detecting GPT-2-generated text using baseline statistical methods. Later that year, OpenAI enhanced the Roberta model Liu et al. 2019 by introducing a dedicated GPT-2 detector Radford et al. 2019 . Another major advancement was the GROVER model Zellers et al. 2019 , which can both generate and detect fake news. With 5,000 self-generated articles and extensive real news content, GROVER achieved a 92% detection accuracy, surpassing models like the Plug and Play Language Model (PPLM) Dathathri et al. 2019 and BERT Devlin et al. 2019 . Another study by Ippolito et al. 2019 examines the detection of machine-generated texts (MGTs) from GPT-2 with decoding strategies such as top-k, untruncated random sampling, and nucleus sampling in English. They discovered that optimized BERT was best but had poor cross-strategy generalization, whereas automatic classifiers performed better than humans, who misclassified AI text more than 30% of the time. AraGPT-2 Antoun et al. 2020 introduced the first advanced Arabic language model that aides in distinguishing human-written and machine-generated Arabic text. 2.2 Detection of MGTs after ChatGPT The launch of ChatGPT in late 2022 and later sequential models like GPT-4, Gemini, Claude, Llama, DeepSeek, etc., have posed new challenges as machine-generated texts (MGTs) mimic human writing styles more effectively than ever before. This raises concerns and the need for detection models to discern between HGTs and MGTs in different fields, such as academia, to mitigate plagiarism. In 2024, a study by Jawaid et al. 2024 presents a systemic approach for discerning between HWTs and MGTs using a combination of deep learning models, textual feature-based models, and machine learning models. Similarly, Xie et al. 2024 used eight traditional machine learning models and integrate statistical analysis, linguistic patterns, sentiment analysis and fact-checking as factors to differentiate between human-generated and machine-generated content across the three different datasets. In another study, Mitrović et al. 2023 examines ChatGPT-generated short text detection using DistilBERT and a perplexity-based classifier on online reviews, creating three datasets: human-written, ChatGPT-generated, and ChatGPT-rephrased. DistilBERT achieved 98% accuracy on original AI-generated text but only 79% on rephrased text, indicating the challenge of detecting AI-rewritten text. 3 Methodology 3.1 Datasets We used both human-generated text (HGTs) and machine-generated text (MTGTs) for the Hausa language in the news domain. We collected 2,586 HGTs from seven different local and international news outlets and generated equal amounts of texts for the MGTs by leveraging the Gemini-2.0-flash closed-source model. We merged the datasets into a single file and created a source column to label whether a text is a HGT or MGT and then shuffled the data for effective training and evaluation. Table 3 provides information on the composition of our dataset in terms of the number of sentences, words, and unique words. Statistic Count Total Sentences 6,737 Total Words 3,376,976 Unique Words 49,883 Table 1:

Read the complete text