GATech at AbjadGenEval Shared Task: Multilingual Embeddings for Arabic Machine-Generated Text Classification
Ahmed Khaled Khamis · 2026
Read GATech at AbjadGenEval Shared Task: Multilingual Embeddings for Arabic Machine-Generated Text Classification by Ahmed Khaled Khamis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10007.txt.
Introduction
Khamis reports a GATech entry to a shared task on spotting machine generated Arabic text. He fine tunes the multilingual E5 large encoder to sort text as human or machine written, and tries several ways to pool token vectors. Weighted layer pooling, attention pooling and gated fusion all lost to plain mean pooling, he finds. A side pattern helps explain the task: human written texts run much longer than machine ones.
Catalog record
| CREATOR | Ahmed Khaled Khamis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 11526 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.10007
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GATech at AbjadGenEval Shared Task: Multilingual Embeddings for Arabic Machine-Generated Text Classification Ahmed Khaled Khamis Abstract We present our approach to the AbjadGenEval shared task on detecting AI-generated Arabic text. We fine-tuned the multilingual E5-large encoder for binary classification, and we explored several pooling strategies to pool token representations, including weighted layer pooling, multi-head attention pooling, and gated fusion. Interestingly, none of these outperformed simple mean pooling, which achieved an F1 of 0.75 on the test set. We believe this is because complex pooling methods introduce additional parameters that need more data to train properly, whereas mean pooling offers a stable baseline that generalizes well even with limited examples. We also observe a clear pattern in the data: human-written texts tend to be significantly longer than machine-generated ones. 1 Introduction When ChatGPT 11 and similar models started producing fluent Arabic text, it became clear that detection tools would need to catch up. Unlike English, where several detectors already exist, Arabic has received far less attention, mostly due to its morphological complexity and the diversity of written styles across regions. AbjadGenEval 1 3 8 addresses this gap with a shared task specifically for Arabic machine-generated text detection. We approached this as a classification problem: take a pre-trained multilingual encoder 12 (E5-large) 13 , add a classification head, and fine-tune on the provided data. The interesting part was figuring out how to pool the token representations. We implemented weighted layer pooling (learning which transformer layers matter most), attention-based pooling (learning which tokens to focus on), and gated fusion (learning how to combine multiple pooling outputs). After all that engineering, plain mean pooling gave us the best results. This paper makes three contributions: • A systematic comparison of pooling strategies for Arabic text classification, demonstrating that simple mean pooling outperforms complex learned aggregation methods on limited training data. • Observations about the dataset: human-written texts average 632 words versus 303 for machine-generated • A training recipe with layer-wise learning rate decay and multi-sample dropout regularization Our final system scores 0.75 F1 on the shared task test set. 1 1 1 Code: https://github.com/KickItLikeShika/abjadgeneval 2 Background 2.1 Task Setup The AbjadEval task frames Arabic AI-generated text detection as a binary classification problem. Given an input text x , the system must predict a label y\in\{\text{human},\text{machine}\} indicating whether the text was written by a human or generated by an AI system. 2.2 Dataset The competition dataset consists of 5,298 Arabic text samples with a balanced class distribution (50% human, 50% machine-generated). Table 1 summarizes the dataset statistics. Statistic Human Machine Samples 2,649 2,649 Avg. Words 632.0 303.0 Avg. Characters 3,806.7 1,865.5 Max Words 3,068 1,969 Table 1: Dataset statistics by class. Human-written texts are significantly longer than machine-generated texts. A notable characteristic of the dataset is the substantial length difference between classes: human-written texts average 632 words compared to only 303 words for machine-generated texts. This suggests that text length could be a discriminative feature, though our model learns to capture more nuanced patterns. 2.3 Related Work Previous approaches to AI-generated text detection have employed statistical methods 4 , fine-tuned language models, and watermarking techniques 7 . For Arabic specifically, transformer-based models like AraBERT 2 and CAMeL-BERT 5 have shown strong performance on various NLP tasks. Recent work on multilingual text embeddings, particularly the E5 family 13 , has demonstrated excellent cross-lingual transfer capabilities. More recent work has shifted toward fine-tuning language models directly for detection. The intuition is simple: if a model like BERT 12 can learn what "natural" text looks like during pre-training, it should also be able to learn what generated text looks like with supervised fine-tuning. DetectGPT 10 took a different approach, using perturbation-based methods that don’t require any training data at all—though these zero-shot methods typically lag behind supervised ones when labeled data is available. 3 System Overview 3.1 Model Architecture Our system is built on the multilingual E5-large encoder 13 , which consists of 24 transformer layers with a hidden size of 1,024. We add a classification head on top of the pooled representations. The architecture follows a standard encoder-classifier setup. Input text is first tokenized and passed through the E5-large encoder, which produces a contextualized representation for each token. These token-level representations are then aggregated into a single fixed-size vector using a pooling operation. Finally, this pooled vector passes through a classification head that outputs probabilities for each class (human or machine). 3.2 Pooling Strategies We experimented with several pooling strategies before settling on mean pooling: Mean Pooling: The simplest approach, where we average the hidden states across all non-padded tokens. Each token contributes equally to the final representation. Weighted Layer Pooling: Instead of using only the final transformer layer, this method learns to combine outputs from multiple layers. The intuition is that different layers capture different types of information: lower layers tend to encode surface-level features while higher layers capture more semantic content. We assign a learnable weight to each layer, and take a weighted average, with weights normalized using softmax. Multi-Head Attention Pooling: Rather than treating all tokens equally, this approach learns which tokens to focus on. We use 8 learnable query vectors, each attending to the token sequence independently. The resulting 8 context vectors are concatenated and projected back to the hidden dimension. Gated Fusion: When combining multiple pooling methods, we use learned sigmoid gates to control how much each pooling output contributes. The gates are computed from the concatenation of all pooling outputs, allowing the model to dynamically weight different representations based on the input. 3.3 Classification Head The pooled representation passes through a feed-forward layer with layer normalization, GELU activation, and dropout before the final classifier. We also use multi-sample dropout 6 : during training, we apply 5 different dropout masks (with rates 0.1, 0.15, 0.2, 0.25, and 0.3) and average the resulting logits. This acts like a small ensemble within a single forward pass, improving regularization without additional inference cost. 3.4 Loss Function We use Focal Loss 9 instead of standard cross-entropy. Focal loss down-weights easy examples and focuses training on harder cases by scaling the loss based on prediction confidence. 4 Experimental Setup 4.1 Data Split We trained on the full competition training set containing 5,298 samples. Due to the blind test evaluation setup. 4.2 Hyperparameters Table 2 details our training configuration. Parameter Value Model multilingual-e5-large Max Sequence Length 512 tokens Batch Size 16 Gradient Accumulation 4 steps Effective Batch Size 64 Learning Rate 2\times 10^{-5} Weight Decay 0.01 LLRD Decay Factor 0.95 Epochs 2 Warmup Ratio 10% Scheduler Cosine with warmup Table 2: Training hyperparameters. 4.3 Layer-wise Learning Rate Decay We apply layer-wise learning rate decay (LLRD) to prevent catastrophic forgetting of pretrained knowledge. Lower transformer layers receive smaller learning rates. 4.4 Other Implementation Details We utilized Dynamic Padding to ensure that sequences are padded to max length within each batch for efficiency. 5 Results Our system achieved an F1 score of 0.75 on the official test set using mean pooling with the E5-large encoder. 5.1 Pooling Strategy Results Table 3 presents our comparison of pooling strategies during development. Pooling Strategy Test F1 Mean Pooling 0.75 Weighted Layer Pooling + Attention + Gated Fusion 0.70 Weighted Layer Pooling + Attention 0.71