Aaron at SemEval-2026 Task 9: Multilingual Polarization Detection Using Transformer-Based Models with Class Weighting and Threshold Tuning

Aaron Bundi Anampiu · 2026

Read Aaron at SemEval-2026 Task 9: Multilingual Polarization Detection Using Transformer-Based Models with Class Weighting and Threshold Tuning by Aaron Bundi Anampiu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30857.txt.

Introduction

Aaron Bundi Anampiu describes a competition system for spotting polarising content online in many languages. The SemEval-2026 shared task covers three subtasks: detecting polarisation, classifying its type, and identifying how it shows itself, across 22 languages. Anampiu entered for English and Swahili, using transformer models matched to each language, RoBERTa-base for English and AfroXLMR-base for Swahili.

Class-weighted losses countered the severe label imbalance, and tuning a threshold per label sharpened the multi-label decisions. Test F1 macro scores reached 0.7901 for English and 0.7910 for Swahili on detection, with type and manifestation scores between 0.4615 and 0.5830 across the two languages. Error analysis showed the models struggle most with dehumanisation and lack of empathy.

Catalog record

CREATORAaron Bundi Anampiu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30857

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Aaron at SemEval-2026 Task 9: Multilingual Polarization Detection Using Transformer-Based Models with Class Weighting and Threshold Tuning Aaron Bundi Anampiu Abstract This paper describes our submission to SemEval-2026 Task 9 on detecting multilingual, multicultural, and multievent online polarization. We address all three subtasks: binary polarization detection, polarization type classification, and manifestation identification for English and Swahili. Our approach leverages transformer-based models (RoBERTa-base for English, AfroXLMR-base for Swahili) with class-weighted loss functions to address severe label imbalance and per-label threshold tuning to optimize multi-label classification. On the test set, we achieve F1 macro scores of 0.7901 (English) and 0.7910 (Swahili) for Subtask 1, 0.4615 (English) and 0.4808 (Swahili) for Subtask 2 and 0.4791 (English) and 0.5830 (Swahili) for Subtask 3, which give competitive performance on the leaderboard, demonstrating the effectiveness of our methods for handling imbalanced multi-label polarization detection. Our error analysis reveals that models struggle with dehumanization detection and lack of empathy. 1 Introduction Social media platforms have become central spaces for public discourse, enabling millions of users to share opinions, engage in debates, and form communities. However, these platforms have also witnessed a concerning rise in polarized content and messages that divide audiences along ideological, political, racial, religious, or other lines Grover et al. 2019 . Polarization poses significant societal challenges. It amplifies echo chambers, reduces constructive dialogue, can incite real-world conflicts, and disproportionately affects marginalized communities Conover et al. 2011 . Automatic detection of polarizing content is therefore crucial for content moderation, understanding radicalization mechanisms, and promoting healthier online discourse. SemEval-2026 Task 9 Naseem et al. 2026a addresses this challenge through three subtasks: (1) binary detection of polarization, (2) multi-label classification of polarization types (political, racial/ethnic, religious, gender/sexual, other), and (3) multi-label identification of polarization manifestations (stereotype, vilification, dehumanization, extreme language, lack of empathy, invalidation). The task covers 22 languages from diverse platforms including news websites, Reddit, blogs, and regional forums. We focus on English and Swahili, employing a transformer-based approach with three key contributions: (1) language-specific model selection (RoBERTa Liu et al. 2019 for English and AfroXLMR base Alabi et al. 2022 for Swahili), (2) class-weighted loss functions to address severe label imbalance in multi-label tasks, and (3) per-label threshold tuning to optimize classification boundaries. Our system achieves strong performance on all the Subtasks. It was ranked 2nd for Swahili on Subtask 3 among the 16 participating teams. 2 Background and Related Work 2.1 Task Description SemEval-2026 Task 9 provides 3,000 to 5,000 annotated instances per language Naseem et al. 2026b spanning elections, conflicts, gender rights, and migration. Subtask 1 (Binary Detection) classifies text as polarized or non-polarized based on context and overall meaning. Subtask 2 (Type Classification) identifies five polarization types: political/ideological, racial/ethnic, religious, gender/sexual, and other (multi-label). Subtask 3 (Manifestation Identification) classifies six manifestations: stereotype, vilification, dehumanization, extreme language, lack of empathy, and invalidation (multi-label). 2.2 Related Work Prior work on detecting harmful online content has primarily focused on hate speech Fortuna and Nunes 2018 , toxic comments Wulczyn et al. 2017 , and offensive language Zampieri et al. 2019 . These tasks share similarities with polarization detection but differ in important ways. Early approaches used traditional machine learning with hand-crafted features Davidson et al. 2017 , including n-grams, sentiment lexicons, and linguistic patterns. More recent work has adopted deep learning, with Badjatiya et al. 2017 demonstrating that LSTMs with word embeddings significantly outperform traditional methods. Devlin et al. 2019 showed that BERT-based models achieve better results on hate speech benchmarks. The development of multilingual pre-trained models has enabled effective transfer across languages. mBERT Devlin et al. 2019 and XLM-RoBERTa Conneau et al. 2020 demonstrated strong zero-shot cross-lingual transfer capabilities. However, these models are trained primarily on high-resource languages and may underperform on African languages. Recent work has developed language models specifically for African languages. AfriBERTa Ogueji et al. 2021 and AfroXLMR Alabi et al. 2022 were pre-trained on diverse African language corpora, including Swahili, and have shown superior performance on African NLP tasks compared to general multilingual models. We adopt AfroXLMR for Swahili based on these findings. 3 System Overview Our pipeline has four components: preprocessing, model architecture, training with class-weighted loss, and post-training threshold tuning. Figure 1 illustrates the full pipeline. Figure 1: Complete system architecture pipeline showing the four main components: preprocessing, model architecture (language-specific transformers), training with class-weighted loss, and post-training threshold tuning. The pipeline is applied to all three subtasks. 3.1 Data Preprocessing We perform minimal preprocessing: retain original casing, preserve emojis and special characters that may signal polarization, and remove only control characters and null bytes. Data are split 80/20 into train/validation using stratified sampling to maintain class distribution. 3.2 Model Architecture English Models: RoBERTa-base Liu et al. 2019 (125M parameters) was selected over BERT and XLM-RoBERTa based on ablation results (Table 6 ): its dynamic masking and removal of next-sentence prediction yield the best English-specific performance, while XLM-RoBERTa’s multilingual pre-training dilutes English-specific representations. Swahili Models: AfroXLMR-base Alabi et al. 2022 (270M parameters) is specifically adapted for African languages through continued pre-training on diverse African language corpora. It significantly outperforms multilingual alternatives like mBERT and XLM-RoBERTa on Swahili tasks Adelani et al. 2022 . Tokenization: RoBERTa’s byte-level BPE tokenizer (50K vocab) for English; AfroXLMR’s SentencePiece tokenizer (250K vocab) for Swahili. Maximum sequence length is 128 tokens for Subtask 1 and 256 for Subtasks 2–3. Classification head: A dropout layer (rate 0.1) followed by a linear layer maps 768-dimensional embeddings to 2, 5, or 6 output logits for Subtasks 1, 2, and 3 respectively. 3.3 Class-Weighted Loss Function To address severe label imbalance, we implement class-weighted binary cross-entropy loss with positive class weight for label computed as: w_{j}=\frac{n_{\text{neg}}^{j}}{n_{\text{pos}}^{j}} (1) where n_{\text{neg}}^{j} and n_{\text{pos}}^{j} are negative and positive sample counts for label j . This increases the penalty for misclassifying minority classes. For Subtask 2 English, weights range from 1.73 (political) to 41.95 (gender/sexual), reflecting the extreme rarity of gender-related polarization. Weights for Swahili Subtask 2 reach 48.05 (gender/sexual). Training loss curves showed no instability; gradient clipping (max-norm 1.0) was applied throughout as a precaution. 3.4 Threshold Tuning After training we perform per-label threshold search on the validation set, scanning [0.1,0.9] in steps of 0.05 to maximise macro F1 per label. This improves macro F1 by 0.87–5.98 points over the standard 0.5 threshold. 3.5 Training Configuration We use AdamW Loshchilov and Hutter 2019 (learning rate 2e-5, weight decay 0.01) with a linear warmup schedule, chosen as the standard optimizer for fine-tuning transformers. Batch size is 16 for Subtask 1 and 8 with gradient accumulation ( \times 2) for Subtasks 2–3 to fit 256-token sequences in 15 GB GPU memory. We train for 5–8 epochs with early stopping (patience 2) based on validation macro F1, using FP16 mixed precision on a single GPU via Google Colab. Random seed is fixed to 42 for all experiments. 4 Experimental Setup 4.1 Data Splits We use provided train/dev splits. Training data are further split 80/20 using stratified sampling (binary labels for Subtask 1; label-count strata for Subtasks 2–3). The test set is used exclusively for final evaluation. 4.2 Evaluation Metrics All subtasks use macro F1 as the primary metric. We compare our system against two baselines: (i) a POLAR Baseline , and (ii) an mBERT

Read the complete text