Notebook for the EXIST2025 Lab at CLEF 2025
Roberto Labadie-Tamayo, Adrian Jaques Böck, Djordje Slijepčević, Xihui Chen, Andreas Babic, Matthias Zeppelzauer · 2025
Read Notebook for the EXIST2025 Lab at CLEF 2025 by Roberto Labadie-Tamayo, Adrian Jaques Böck, Djordje Slijepčević, Xihui Chen, Andreas Babic, Matthias Zeppelzauer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20924.txt.
Introduction
Roberto Labadie-Tamayo and five co-authors report their entry to the EXIST shared task on sexism in social posts. Their notebook paper is from two thousand twenty-five. The task has three parts: spot sexism in tweets, judge the intent of the source, and sort sexism into types.
They build three models. One uses plain descriptive words as a clear middle step, so its choices can be read by a human. A second blends those words with transformer context, and a third is a tuned multilingual transformer. Scores place them near the top for English and Spanish. The clear models also give reasons at the level of one post or a whole class.
Catalog record
| CREATOR | Roberto Labadie-Tamayo, Adrian Jaques Böck, Djordje Slijepčević, Xihui Chen, Andreas Babic, Matthias Zeppelzauer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-20924
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Notebook for the EXIST2025 Lab at CLEF 2025 Roberto Labadie-Tamayo, Adrian Jaques Böck, Djordje Slijepčević, Xihui Chen, Andreas Babic, Matthias Zeppelzauer Abstract Sexism has become widespread on social media and in online conversation. To help address this issue, the fifth Sexism Identification in Social Networks (EXIST) challenge is initiated at CLEF 2025. Among this year’s international benchmarks, we concentrate on solving the first task aiming to identify and classify sexism in social media textual posts. In this paper, we describe our solutions and report the results for all the three subtasks: Subtask 1.1 – Sexism Identification in Tweets , Subtask 1.2 – Source Intention in Tweets , and Subtask 1.3 – Sexism Categorization in Tweets . We implement three models to address each subtask which constitute three individual runs, i.e., Speech Concept Bottleneck Model (SCBM), Speech Concept Bottleneck Model with Transformer (SCBMT) and a fine-tuned XLM-RoBERTa transformer model. SCBM uses descriptive adjectives as human-interpretable bottleneck concepts. SCBM leverages large language models (LLMs) to encode input texts to a human-interpretable representation of descriptive adjectives, which is then used to train a light-weight classifier for downstream tasks. SCBMT extends SCBM by fusing the adjective-based representation with the contextual embeddings computed by transformers to balance interpretability and classification performance. Beyond the competitive results, these two models offer fine-grained explanations at both the instance level (local) and class level (global). We also investigate how additional metadata available in the benchmarks, e.g., annotators’ demographic profiles, can be leveraged in our models. For the binary classification task (Subtask 1.1), XLM-RoBERTa, fine-tuned on the provided data augmented with datasets from previous years, ranks 6 th 6^{\text{th}} for English and Spanish content and 4 th 4^{\text{th}} for English content in the Soft-Soft evaluation. Our SCBMT model achieves 7 th 7^{\text{th}} place for English and Spanish content and 6 th 6^{\text{th}} for Spanish content in the Soft-Soft evaluation. [orcid= 0000-0003-4928-8706, [email protected], ] 1 *1 [orcid= 0000-0003-1972-0473, [email protected] ] 1 *1 [orcid= 0000-0002-2295-7466, [email protected] ] [orcid=0000-0002-8131-5092, [email protected] ] [orcid=0009-0008-8927-2593, [email protected] ] [orcid= 0000-0003-0413-4746, [email protected] ] keywords Sexism Detection,Sexism Identification ,Social Media Retrieval ,Concept Bottleneck Models ,Transformer Models ,Natural Language Processing † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: CLEF 2025 Working Notes, 9 – 12 September 2025, Madrid, Spain † † address: St. Pölten University of Applied Sciences, 3100 St. Pölten, Austria † † corresp: Corresponding authors. † † These authors contributed equally. † † footnotetext: Preprint July 2025. This version is a preprint uploaded to arXiv. This work has been accepted for presentation in the sEXism Identification in Social neTworks (EXIST) task at the 16 th Conference and Labs of the Evaluation Forum (CLEF 2025). 1 Introduction Discriminatory views and statements have increased on social media platforms in recent years, particularly targeting women. This phenomenon, with other issues stemming from social media, e.g., hate speech Demus et al. 2022 and disinformation Schütz et al. 2021 , significantly impacts psychological well-being of those affected and may even lead to physical violence. The EXIST (sEXism Identification in Social neTworks) task at CLEF 2025 Plaza et al. 2025a ; Plaza et al. 2025b aims to advance automatic tools for identifying sexism in the multimedia content of social media. We focus on the task of sexism identification in textual posts, which forms Task 1 of the EXIST benchmark. In this paper, we describe our approaches to the three subtasks, i.e., Subtask 1.1 – Sexism Identification in Tweets , Subtask 1.2 – Source Intention in Tweets , and Subtask 1.3 – Sexism Categorization in Tweets and report the results on the provided data. We employ three distinct machine learning (ML) models to address each subtask and provide three different experimental runs for each. Our first approach utilizes the Speech Concept Bottleneck Model (SCBM), a human-interpretable model that leverages large language models (LLMs) to map input text into an adjective-based representation. Originally developed for recognizing online hate and counter speech, SCBM offers both global and local explainability. Each dimension of the intermediate representation reflects the degree to which a specific adjective captures an emotion or subjective opinion expressed in the input text. A lightweight classifier is then trained on these representations for downstream prediction tasks. Compared to concept representations based on topics or linguistic patterns, using adjectives as bottleneck concepts significantly improves the interpretability of the model’s outputs. Our second model, the Speech Concept Bottleneck Model with Transformer (SCBMT), extends SCBM by combining the adjective-based representations with transformer-generated embeddings. This fusion enables the model to capture a more comprehensive range of textual features by integrating interpretable and contextual information. Finally, we fine-tune a multilingual RoBERTa transformer model as a baseline to benchmark the performance of our proposed methods. The structure of the paper. We describe in detail our methodology used to address the challenge, including the developed models and evaluation settings (e.g., datasets, concept list), in Section 2 . The results are presented in Section 3.1 , followed by the discussion and conclusions in Section 4 . 2 Methodology and Evaluation With the three models employed, we conducted three independent runs for each subtask of the tweet classification challenge of the benchmark. In this section, we describe the datasets used, the developed ML models, and the approaches applied for each submitted run of the subtasks. 2.1 Datasets In addition to the dataset (EXIST2025) of this year’s benchmark, we utilized data provided in previous years’ benchmarks (EXIST2021-EXIST2024). The combination increases the scale of training data, and thereby improves model robustness and generalizability. EXIST2025 Dataset. The EXIST2025 dataset Plaza et al. 2025a contains over 10,000 text posts from X/Twitter, written in both English and Spanish, with a balanced distribution between the two languages. The dataset is split into three subsets: training (6,920 posts), development (1,038 posts), and testing (2,076 posts). Each post is annotated by six individuals, and demographic information about the annotators (e.g., gender, age group, ethnicity, education level, and country) is also provided. Each post is assigned three labels corresponding to the dataset’s three subtasks: Sexism Identification , Source Intention , and Sexism Categorization . Subtask 1.1 – Sexism Identification is a binary classification task in which each post is labeled as either SEXIST or NON-SEXIST. Subtask 1.2 – Source Intention is a multiclass classification task that aims to determine the intention behind sexist content, with possible labels being DIRECT, REPORTED, or JUDGEMENTAL. Subtask 1.3 – Sexism Categorization is a multi-label classification task that identifies specific types of sexist content. The possible categories include: IDEOLOGICAL-INEQUALITY, STEREOTYPING-DOMINANCE, OBJECTIFICATION, SEXUAL-VIOLENCE, and MISOGYNY-NON-SEXUAL-VIOLENCE. EXIST2024 Dataset. The dataset from the EXIST2024 Plaza et al. 2024 benchmark includes posts from X/Twitter and Gab. We used thetraining set, which consists of 6,920 instances in total, i.e., 3,260 in English and 3,660 in Spanish, and the development set, which includes 1,038 samples. The test set was not used due to missing annotations. The class definitions for the tasks of Sexism Identification, Source Intention, and Sexism Categorization are the same as those in the EXIST2025 dataset. EXIST2022 Dataset. We used the training dataset from the EXIST2022 benchmark Rodríguez-Sánchez et al. 2022 . This dataset includes posts from X/Twitter and Gab, with a