SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakraborty, Dheeraj Kodati, Sahar Moradizeyveh, Firoj Alam, Ye Kyaw Thu, Shantipriya Parida, Ihsan Ayyub Qazi, Lilian Wanzare, Nelson Odhiambo Onyango, Clemencia Siro, Ibrahim Said Ahmad, Adem Chanie Ali, Martin Semmann, Chris Biemann, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam, [2mm] Macquarie University, University of Hamburg, Zayed University, HKBK College of Engineering, University of Turin · 2026
Read SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization by Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakraborty, Dheeraj Kodati, Sahar Moradizeyveh, Firoj Alam, Ye Kyaw Thu, Shantipriya Parida, Ihsan Ayyub Qazi, Lilian Wanzare, Nelson Odhiambo Onyango, Clemencia Siro, Ibrahim Said Ahmad, Adem Chanie Ali, Martin Semmann, Chris Biemann, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam, [2mm] Macquarie University, University of Hamburg, Zayed University, HKBK College of Engineering, University of Turin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06817.txt.
Introduction
Naseem and thirty-three colleagues run SemEval-2026 Task 9 on multilingual online polarization. The shared task spans 22 languages with over 110,000 annotated instances, each tagged for polarization presence, type, and manifestation. Teams tackled three jobs: detecting presence, identifying type, and recognizing manifestation. Over 1,000 participants joined with more than 10,000 submissions, yielding final entries from 67 teams and 73 system papers. The report gives baselines and reviews top systems, noting the most common and most effective methods across jobs and languages, with the dataset public.
Catalog record
| CREATOR | Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakraborty, Dheeraj Kodati, Sahar Moradizeyveh, Firoj Alam, Ye Kyaw Thu, Shantipriya Parida, Ihsan Ayyub Qazi, Lilian Wanzare, Nelson Odhiambo Onyango, Clemencia Siro, Ibrahim Said Ahmad, Adem Chanie Ali, Martin Semmann, Chris Biemann, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam, [2mm] Macquarie University, University of Hamburg, Zayed University, HKBK College of Engineering, University of Turin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35771 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.06817
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakraborty, Dheeraj Kodati, Sahar Moradizeyveh, Firoj Alam, Ye Kyaw Thu, Shantipriya Parida, Ihsan Ayyub Qazi, Lilian Wanzare, Nelson Odhiambo Onyango, Clemencia Siro, Ibrahim Said Ahmad, Adem Chanie Ali, Martin Semmann, Chris Biemann, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam, [2mm] Macquarie University, University of Hamburg, Zayed University, HKBK College of Engineering, University of Turin Abstract We present SemEval-2026 Task 9, a shared task on online polarization detection, covering 22 languages and comprising over 110K annotated instances. Each data instance is multi-labeled with the presence of polarization, polarization type, and polarization manifestation. Participants were asked to predict labels in three subtasks: (1) detecting the presence of polarization, (2) identifying the type of polarization, and (3) recognizing the polarization manifestation. The three tasks attracted over 1,000 participants worldwide and more than 10k submission on Codabench. We received final submissions from 67 teams and 73 system description papers. We report the baseline results and analyze the performance of the best-performing systems, highlighting the most common approaches and the most effective methods across different subtasks and languages. The dataset of this task is publicly available. 1 1 1 https://github.com/Polar-SemEval/data-public 1 Introduction Online polarization, defined as sharp division and antagonism between social, political, or identity groups, has become a pervasive threat to democratic institutions, civil discourse, and social cohesion worldwide ( 87 ) . It is often fueled by biased or inflammatory content in digital media, strengthening echo chambers and undermining mutual understanding ( 27 ) . Polarized discourse amplifies ideological divides and can escalate into hate speech, harassment, and real-world violence 66 ; 53 . Therefore, early detection of polarization is essential for designing interventions that promote healthier online ecosystems. In this shared task, we provide participants with POLAR , a large-scale, multilingual, multicultural, and multi-event dataset for fine-grained polarization detection 57 . The task challenges participants to develop systems that can automatically detect and classify polarized content across multiple languages, cultural contexts, and event types. POLAR covers 22 languages spanning seven language families and comprises over 110,000 annotated instances (see Figure 1 for the geographic and linguistic diversity represented). Table 1 presents the data distribution across the train, development, and test splits. This shared task supports three complementary subtasks: • Binary Polarization Detection: Determine whether a given text expresses polarization. We refer to this task as PolarDetect . • Polarization Type Classification: Identify the social dimension underlying polarization (e.g., political, religious, racial). We refer to this task as PolarType . • Manifestation Identification: Detect how polarization is rhetorically manifested, including strategies such as stereotyping, deindividuation, vilification, dehumanization, extreme language, and other rhetorical devices. We refer to this task as PolarManifest . Each team could submit results for subtask 1, 2, 3, or all the three subtasks in one or more languages. Our official evaluation metrics were the average Macro F 1. Our tasks attracted over 1000 participants, with 546 final submissions in test phase and 73 system description papers. Subtask 1 received the most submissions (267), followed by subtask 2 with 161, and subtask 3 with 120. Figure 1: Languages represented in a world map covered by POLAR , covering diverse linguistic and regional contexts. The language and societal context can present itself across varied areas. Language assignments to countries and regions are approximate. Lang. Train Dev Test Total Inner Agr. ( \kappa ) amh 3,332 166 1,501 4,999 0.59 arb 3,380 169 1,521 5,070 0.25 ben 3,333 166 1,501 5,000 0.59 deu 3,180 159 1,432 4,771 0.10* eng 3,222 160 1,452 4,834 0.39 fas 3,295 164 1,484 4,943 0.78 hau 3,651 182 1,644 5,477 0.48 hin 2,744 137 1,236 4,117 0.49 ita 3,334 166 1,538 5,038 0.39 khm 6,640 332 2,988 9,960 0.83 mya 2,889 144 1,301 4,334 0.13 nep 2,005 100 903 3,008 0.79 ori 2,368 118 1,066 3,552 0.46 pan 1,700 100 809 2,609 0.55* pol 2,391 119 1,077 3,587 0.46 rus 3,348 167 1,508 5,023 0.39 spa 3,305 165 1,488 4,958 0.26 swa 6,991 349 3,147 10,487 0.56 tel 2,366 118 1,066 3,550 0.7 tur 2,364 115 1,093 3,572 0.46 urd 3,563 177 1,606 5,346 0.29 / 0.70* zho 4,280 214 1,927 6,421 0.64 Total 73,681 3,687 33,288 110,656 Table 1: Data distribution across the train, development, and test splits, along with inner agreement. Inner Agre. denotes inter-annotator agreement per language (Fleiss’s \kappa unless otherwise noted). ∗ denotes exceptions: German uses Krippendorff’s \alpha ; Punjabi reports identical Krippendorff’s \alpha and Cohen’s \kappa ; Urdu reports Fleiss’s \kappa / Cohen’s \kappa . 2 Related Work Online polarization poses a threat to social cohesion, exacerbated by social media echo chambers and biased content ( 87 ; 34 ; 27 ) . As social media and other online platforms become key arenas for political and cultural discourse, the need for early detection and nuanced understanding of polarization has grown significantly. Polarization detection is important for content moderation, peace building, responsible digital governance, and healthy democracy. Foundational research has defined polarization as both intergroup hostility and blind ingroup cohesion ( 6 ) , and has highlighted its relationship with hate speech, fragmentation, and incivility ( 54 ) . A growing body of research has documented the role of online spaces in intensifying polarization across different regions ( 42 ; 9 ; 28 ; 74 ) . However, most computational work focuses on high-resource languages and event- or region-specific datasets, limiting generalizability ( 42 ) . This leaves a significant gap in our ability to generalize findings across cultures, languages, and events, especially for low-resource languages or multilingual regions. The lack of standardized datasets across languages has hindered progress in developing and evaluating polarization detection models with cross-lingual or cross-cultural capabilities. Recent shared tasks on hate speech and toxicity ( 11 ; 62 ) have expanded the language and domain coverage, yet remain less fine-grained regarding polarization’s diverse types and rhetorical manifestations. This shared task addresses this gap by presenting the comprehensive, fine-grained dataset benchmark for multilingual, multicultural, and multievent online polarization, enabling robust cross-lingual and context-aware modeling. 3 POLAR Dataset Construction 3.1 Operational Definitions Our work 57 defines polarization as the increasing extremity of opinions, beliefs, or behaviors, resulting in heightened inter-group divisions and conflict. Besides, we defined polarization types including political, racial or ethnic, religious, gender or sexual, ant other. We further distinguish polarization by its rhetorical manifestations, containing stereotype, vilification, dehumanization, extreme language, lack of Empathy, and Invalidation 3.2 Data Collection We collected data from a range of online platforms, including major social media sites, local news or commentary forums. For several languages, including Burmese, polish, and Chinese, we sampled and re-annotated instances from existing toxic or hate speech datasets. The curated the dataset to cover diverse real-world events, grounding event selection in the sociopolitical and socioeconomic contexts specific to each language and cultural setting. The data span a broad range of events and issues, including armed conflicts, elections and party politics, public health crises, large-scale migration, climate change, and