PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation

Srikar Kashyap Pulipaka · 2026

Read PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation by Srikar Kashyap Pulipaka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05159.txt.

Introduction

Srikar Kashyap Pulipaka describes a competition system for detecting political polarisation in text across twenty-two languages, entered in SemEval-2026 Task 9. The system fine-tunes separate Gemma models for each language using low-rank adaptation, in two sizes. Synthetic training data from a large language model supplements the real data through direct generation, paraphrasing, and contrastive pair creation, followed by multi-stage quality filtering.

Tuning decision thresholds per language on development data added several points without retraining, Pulipaka reports. A weighted ensemble of the two model sizes placed second overall among participating teams, with first place in three languages. Rival architectures that looked strong in development dropped sharply on the test set.

Catalog record

CREATORSrikar Kashyap Pulipaka
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.05159

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation Srikar Kashyap Pulipaka Abstract We present our system for SemEval-2026 Task 9: Multilingual Polarization Detection, a binary classification task spanning 22 languages. Our approach fine-tunes separate Gemma 3 models (12B and 27B parameters) per language using Low-Rank Adaptation (LoRA), augmented with synthetic data generated by a large language model (LLM). We employ three synthetic data strategies (direct generation, paraphrasing, and contrastive pair creation) using GPT-4o-mini, with a multi-stage quality filtering pipeline including embedding-based deduplication. We find that per-language threshold tuning on the development set yields 2 to 4% F1 improvements without retraining. We also use weighted ensembles of 12B and 27B model predictions with per-language strategy selection. Our final system achieves a mean macro-F1 of 0.811 across all 22 languages, ranking 2nd overall of the participating teams, with 1st place finishes in 3 languages and top-3 in 8 languages. We also find that alternative architectures (XLM-RoBERTa, Qwen3) that showed strong development set performance suffered 30 to 50% F1 drops on the test set, highlighting the importance of generalization. 1 Introduction Team PSK participated in SemEval-2026 Task 9 ( 8 ) on detecting attitude polarization in social media text across 22 typologically diverse languages. The task requires binary classification of social media text as polarized or non-polarized, where polarization encompasses stereotyping, vilification, dehumanization, and intolerance. The task presents several challenges: the dataset sizes vary across languages, polarization manifests differently across cultures, and some languages exhibit class imbalance. In our work, we explore per-language fine-tuned Gemma models with synthetic data augmentation, ensemble methods, and post-hoc threshold tuning. We find that robust generalization, rather than maximizing development set performance, is the critical factor for success. The remainder of this paper is structured as follows: Section 2 gives a brief overview of related work, Section 3 describes the data, Section 4 describes our methodology, Section 5 discusses our results, and we conclude in Section 6 . 2 Related Work Recent work has explored LLM-generated data for text classification augmentation. 1 found that LLM augmentation primarily benefits low-data scenarios. 14 demonstrated 5.6 to 8.9 point improvements for extremely low-resource languages using lexicon-conditioned generation. Our augmentation pipeline draws on backtranslation ( 12 ) . 3 demonstrated that noisy backtranslations provide stronger training signals than clean outputs. Our contrastive pair generation is inspired by counterfactual data augmentation for hate speech detection ( 7 ) , where minimal-edit counterfactuals improve classifier robustness. We use LoRA ( 4 ) for parameter-efficient fine-tuning, and our per-language threshold tuning follows work by 5 on F1 maximization and 10 on per-class threshold optimization for macro-averaged metrics. 3 Data 3.1 Task Data SemEval-2026 Task 9 Subtask 1 uses the POLAR dataset ( 9 ) , a binary classification task: given a social media text in one of 22 languages, the task is to classify it as polarized (1) or non-polarized (0). The primary evaluation metric is macro-averaged F1-score. The 22 languages are: Amharic (amh), Arabic (arb), Bengali (ben), Burmese (mya), Chinese (zho), English (eng), German (deu), Hausa (hau), Hindi (hin), Italian (ita), Khmer (khm), Nepali (nep), Odia (ori), Persian (fas), Polish (pol), Punjabi (pan), Russian (rus), Spanish (spa), Swahili (swa), Telugu (tel), Turkish (tur), and Urdu (urd). Training set sizes range from approximately 1,700 (Punjabi) to 7,000 (Swahili) samples. Several languages exhibit class imbalance, most notably Khmer (10:1 polarized), Hausa (8:1 non-polarized), Hindi (6:1 polarized), and Amharic (3:1 polarized). We split the original data into 80% train and 20% validation before adding any synthetic data, ensuring the validation set contains only real data. 3.2 Data Augmentation To provide more data to our models, we generated synthetic training data using GPT-4o-mini with three complementary strategies: Direct Generation (50%). We generated new samples natively in the target language, covering five culturally relevant topic categories: political, ethnic/racial, religious, social class, and international relations. Label-Preserving Paraphrasing (30%). We created paraphrases of real training samples with temperature 0.7, filtered to ensure cosine similarity below 0.90 with the original. Contrastive Pairs (20%). We generated minimal pairs on the same topic, one polarized and one non-polarized, to sharpen class boundaries. In preliminary experiments for lower-resource languages, we also explored backtranslation through pivot languages and cross-lingual transfer from related languages (e.g., Hindi to Nepali, Bengali to Odia). Quality Filtering. We applied a multi-stage pipeline to ensure data quality: (1) basic cleaning and length filtering, (2) label leakage detection via regex patterns, (3) embedding-based deduplication using paraphrase-multilingual-MiniLM-L12-v2 ( 11 ) with a 0.90 cosine similarity threshold (both intra-synthetic and against original training data), and (4) round-trip translation consistency checks (threshold 0.70) for translated samples. Synthetic samples were added only to the training portion at a configurable ratio. We generated approximately 1,000 synthetic samples per language. 4 Methodology Figure 1 provides an overview of our end-to-end system pipeline. Figure 1: Overview of our multilingual polarization detection pipeline, from real-data splitting and synthetic augmentation to Gemma model training, threshold tuning, and per-language strategy selection. 4.1 Model Architecture We chose Google’s Gemma 3 family ( 13 ) as our primary model because of its broad multilingual pretraining mix, which provides meaningful coverage across all 22 POLAR languages rather than concentrating on a narrow set of high resource ones. Our model comparison in Section 5 confirms this choice: Gemma transferred from development to test substantially better than XLM-RoBERTa or Qwen3, both of which have more skewed pretraining distributions. We fine-tuned the Gemma 3 family using LoRA with a sequence classification head. We used two model sizes: • Gemma 3 12B : Our primary model. We used 8-bit quantization on a single A100 GPU, with LoRA rank 16, a learning rate of 5\times 10^{-5} , and trained for 3 epochs. • Gemma 3 27B : Our ensemble model. We used 4-bit NF4 quantization ( 2 ) with gradient checkpointing across 2 A100 GPUs via device_map="auto" . Both models use 2 output labels, eager attention, and bfloat16 precision. We set the pad token to the end-of-sequence (EOS) token. All models were trained with batch size 1 to 2, gradient accumulation of 2 to 4 steps (effective batch size 4), warmup ratio 0.1, max sequence length 128 to 256 tokens, and random seed 42. We used the AdamW optimizer with cosine learning rate decay. 4.2 Ensemble and Threshold Tuning Threshold Tuning. For each language, we searched over thresholds t\in\{0.3,0.35,0.4,\ldots,0.7\} to maximize development set macro-F1, rather than using the default t=0.5 . Ensemble Strategies. For each language, we evaluated four strategies on the development set: 1. 12B predictions with tuned threshold 2. 27B predictions with tuned threshold 3. Average ensemble: p=\frac{p_{\text{12B}}+p_{\text{27B}}}{2} 4. Weighted ensemble: p=w\cdot p_{\text{12B}}+(1{-}w)\cdot p_{\text{27B}} , with w\in\{0.3,0.4,0.6,0.7\} We selected the strategy achieving the highest development F1 per language. 4.3 Infrastructure Experiments were run on a computing cluster with NVIDIA A100 40GB GPUs. The 12B model requires 1 GPU while the 27B model requires 2 GPUs. 5 Results We first provide an overview of our official results, then discuss the results of our internal experiments on the development data. 5.1 Official Results We were allowed to submit 5 systems. Table 1 summarizes our test submissions. Sub Strategy Mean F1 1 Best Gemma-12B per lang 0.797 2 Hybrid (XLM-R, Qwen3) 0.665 4 Ensemble

Read the complete text