Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?

Yifan Wang, Mayank Jobanputra, Ji-Ung Lee, Soyoung Oh, Isabel Valera, Vera Demberg · 2025

Read Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection? by Yifan Wang, Mayank Jobanputra, Ji-Ung Lee, Soyoung Oh, Isabel Valera, Vera Demberg (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22291.txt.

Introduction

Hate speech detectors can inherit bias from their training data, and their black-box nature hides when that bias strikes. Yifan Wang and five co-authors study whether explanations of a model's inputs can help, in this 2025 paper accepted at ICLR 2026. Input-based explanations highlight which words drove a prediction. Earlier work debated their value for fairness, mostly in small qualitative studies. This paper gives the first systematic, large-scale quantitative test, across both encoder-only and decoder-only models.

The authors examine three uses. Explanations proved effective at flagging biased predictions. They also served as useful supervision during training, reducing bias in the resulting model. But they were unreliable for the third use: choosing the fairest model among candidates. Explanations are a working tool for finding and unlearning bias, not a trustworthy scoreboard for comparing models.

Catalog record

CREATORYifan Wang, Mayank Jobanputra, Ji-Ung Lee, Soyoung Oh, Isabel Valera, Vera Demberg
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-22291

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection? Yifan Wang, Mayank Jobanputra, Ji-Ung Lee, Soyoung Oh, Isabel Valera, Vera Demberg Abstract Natural language processing (NLP) models often replicate or amplify social bias from training data, raising concerns about fairness. At the same time, their black-box nature makes it difficult for users to recognize biased predictions and for developers to effectively mitigate them. While some studies suggest that input-based explanations can help detect and mitigate bias, others question their reliability in ensuring fairness. Existing research on explainability in fair NLP has been predominantly qualitative, with limited large-scale quantitative analysis. In this work, we conduct the first systematic study of the relationship between explainability and fairness in hate speech detection, focusing on both encoder- and decoder-only models. We examine three key dimensions: (1) identifying biased predictions, (2) selecting fair models, and (3) mitigating bias during model training. Our findings show that input-based explanations can effectively detect biased predictions and serve as useful supervision for reducing bias during training, but they are unreliable for selecting fair models among candidates. 1 Introduction Language models (LMs) pre-trained on large-scale natural language datasets have shown great capacities in various NLP tasks ( Wang et al. 2018 ; Gao et al. 2023 ) . However, previous studies have shown that they can replicate and amplify stereotypes and social bias present in their training data and demonstrate biased behaviors ( Sheng et al. 2021 ; Gupta et al. 2024 ; Gallegos et al. 2024 ) . Such behaviors risk the underrepresentation of marginalized groups and the unfair allocation of resources, raising serious concerns in critical applications ( Blodgett et al. 2020 ) . Meanwhile, current NLP models are mostly based on black-box neural networks. Despite their strong capacities, the complex architecture and large number of parameters of these models make it hard for humans to understand their behaviors ( Bommasani et al. 2021 ) . To understand neural NLP models, different types of explanations have been devised, such as input-based explanations ( Yin and Neubig 2022 ; Deiseroth et al. 2023 ; Madsen et al. 2024 ; Wang et al. 2025a ) , natural language explanations ( Ramnath et al. 2024 ; Wang et al. 2025b ) , and concept-based explanations ( Yu et al. 2024 ; Raman et al. 2024 ) . Among these, input-based explanations, often referred to as rationales, indicate the contribution of each token to models’ predictions, and thus provide the most direct insights into models’ behaviors ( Arras et al. 2019 ; Atanasova et al. 2022 ; Lyu et al. 2024 ) . Explainability has long been deemed critical to improving fairness. Researchers believe that if the use of sensitive features is evidenced by model explanations, then they can easily detect biased predictions and impose fairness constraints by guiding models to avoid such faulty reasoning ( Meng et al. 2022 ; Sogancioglu et al. 2023 ) . However, recent studies have challenged this assumption, suggesting that the relationship between explainability and fairness is complex and that explanations may not always reliably detect or mitigate bias ( Dimanov et al. 2020 ; Slack et al. 2020 ; Pruthi et al. 2020 ) . Unfortunately, to the best of our knowledge, current studies are mostly limited to qualitative analysis on a small set of explanation methods ( Balkir et al. 2022 ; Deck et al. 2024 ) . Our work takes a step toward bridging explainability and fairness by providing the first comprehensive quantitative analysis in the context of hate speech detection, a task where both fairness and explainability are particularly critical. Specifically, we address the following three research questions to investigate the role of explainability in promoting fairness within the task of hate speech detection: • RQ1: Can input-based explanations be used to identify biased predictions? • RQ2: Can input-based explanations be used to automatically select fair models? • RQ3: Can input-based explanations be used to mitigate bias during model training? Our results show that input-based explanations can effectively detect biased predictions and help reduce bias during model training, but they are less reliable for automatic fair model selection. Furthermore, our analyses indicate that explanation-based bias detection remains robust even when models are trained to reduce reliance on sensitive features, and that these explanations outperform LLM judgments in identifying bias. 2 Related Work Bias in NLP The presence of social bias and stereotypes has significantly shaped human language and LMs trained on it ( Blodgett et al. 2020 ; Sheng et al. 2021 ) . As a result, these models often exhibit biased behaviors ( Gallegos et al. 2024 ) , such as stereotypical geographical relations in the embedding space ( Bolukbasi et al. 2016 ; May et al. 2019 ) and stereotypical associations between social groups and certain concepts in the model outputs ( Fang et al. 2024 ; Wan and Chang 2025 ) . More critically, disparities in model predictions and performance across social groups ( Zhao et al. 2018 ; Sheng et al. 2019 ) can significantly compromise user experiences of marginalized groups and risk amplifying bias against them, therefore drawing great concerns in critical use cases. Input-based Model Explanations Input-based explanations in NLP models aim to attribute model predictions to each input token ( Lyu et al. 2024 ) . They can be broadly categorized based on how they generate explanations: gradient-based ( Simonyan et al. 2014 ; Kindermans et al. 2016 ; Sundararajan et al. 2017 ; Enguehard 2023 ) , propagation-based ( Bach et al. 2015 ; Shrikumar et al. 2017 ; Ferrando et al. 2022 ; Modarressi et al. 2022 ; Modarressi et al. 2023 ) , perturbation-based ( Li et al. 2016 ; Ribeiro et al. 2016 ; Lundberg and Lee 2017 ; Deiseroth et al. 2023 ) , and attention-based methods ( Bahdanau et al. 2015 ; Abnar and Zuidema 2020 ) . While most prior work has focused on encoder-only models, recent studies have also explored explaining the behaviors of generative models ( Yin and Neubig 2022 ; Ferrando et al. 2022 ; Enouen et al. 2024 ; Cohen-Wang et al. 2024 ) . Bridging Explainability and Fairness Explainability is often considered essential for achieving fairness in machine learning systems ( Balkir et al. 2022 ; Deck et al. 2024 ) . One line of research investigates model bias by analyzing explanations ( Prabhakaran et al. 2019 ; Jeyaraj and Delany 2024 ; Sogancioglu et al. 2023 ) . For instance, Muntasir and Noor 2025 shows that a biased model relied on gendered words as key features in its predictions, as revealed by LIME explanations. Similarly, Stevens et al. 2020 demonstrates that biased models often place high importance on gender and race features when examined with SHAP explanations. Extending this line of evidence, Meng et al. 2022 finds that features with higher importance scores are associated with larger disparities in model performance on a synthetic medical dataset using deep learning models. Another line of research focuses on mitigating bias with explanations ( Dimanov et al. 2020 ; Kennedy et al. 2020 ; Rao et al. 2023 ) . For example, Hickey et al. 2020 improves fairness by reducing reliance

Read the complete text