DeHate: A Stable Diffusion-based Multimodal Approach to Mitigate Hate Speech in Images

Dwip Dalal, Gautam Vashishtha, Anku Ranui, Aishwarya Reganti, Parth Patwa, Mohd Sarique, Chandan Gupta, Keshav Nath, Viswanatha Reddy, Vinija Jain, Aman Chadha, Amitava Das, Amit Sheth, Asif Ekbal · 2025

Read DeHate: A Stable Diffusion-based Multimodal Approach to Mitigate Hate Speech in Images by Dwip Dalal, Gautam Vashishtha, Anku Ranui, Aishwarya Reganti, Parth Patwa, Mohd Sarique, Chandan Gupta, Keshav Nath, Viswanatha Reddy, Vinija Jain, Aman Chadha, Amitava Das, Amit Sheth, Asif Ekbal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21787.txt.

Introduction

Hateful images spread fast online, and most tools only flag them after the fact. Dwip Dalal and thirteen co-authors describe a data set and a model that clean the hate out instead. The work grew out of the Dehate shared task, held at a workshop during AAAI 2024. Their goal is to make shared spaces online less harmful.

The team used a stable diffusion image generator, with watermarking and stability fixes, to build training examples of hateful images. A vision module then maps where the hate sits inside each picture. Those regions get blurred, which removes the hateful section while leaving the rest. The paper also presents DeHater, a vision-language model trained for this cleanup task, and it sets out the details of the shared task itself.

Catalog record

CREATORDwip Dalal, Gautam Vashishtha, Anku Ranui, Aishwarya Reganti, Parth Patwa, Mohd Sarique, Chandan Gupta, Keshav Nath, Viswanatha Reddy, Vinija Jain, Aman Chadha, Amitava Das, Amit Sheth, Asif Ekbal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-21787

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DeHate: A Stable Diffusion-based Multimodal Approach to Mitigate Hate Speech in Images Dwip Dalal, Gautam Vashishtha, Anku Ranui, Aishwarya Reganti, Parth Patwa, Mohd Sarique, Chandan Gupta, Keshav Nath, Viswanatha Reddy, Vinija Jain, Aman Chadha, Amitava Das, Amit Sheth, Asif Ekbal Abstract The rise in harmful online content not only distorts public discourse but also poses significant challenges to maintaining a healthy digital environment. In response to this, we introduce a multimodal dataset uniquely crafted for identifying hate in digital content. Central to our methodology is the innovative application of watermarked, stability-enhanced, stable diffusion techniques combined with the Digital Attention Analysis Module (DAAM). This combination is instrumental in pinpointing the hateful elements within images, thereby generating detailed hate attention maps, which are used to blur these regions from the image, thereby removing the hateful sections of the image. We release this data set as a part of the dehate shared task. This paper also describes the details of the shared task. Furthermore, we present DeHater, a vision-language model designed for multimodal dehatification tasks. Our approach sets a new standard in AI-driven image hate detection given textual prompts, contributing to the development of more ethical AI applications in social media. [[email protected] ] 1 [[email protected], ] 1 [[email protected], ] [] [] [] [] [] [] [] [] *1 [[email protected] ] [] [] keywords LLMs ,GenAI ,Multimodal † † copyrightyear: 2024 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: Defactify 3: Third Workshop on Multimodal Fact Checking and Hate Speech Detection, co-located with AAAI 2024. † † address: IIT Gandhinagar, India † † address: MIT Media Lab, USA † † address: CMU, USA † † address: UCLA, USA † † address: IIIT Kalyani, India † † address: IIIT Delhi, India † † address: DTU, India † † address: UW Madison, USA † † address: Stanford University, USA † † address: Amazon GenAI, USA † † address: University of South Carolina, USA † † address: IIT Patna, India † † These authors contributed equally. † † corresp: Work does not relate to position at Amazon. 1 Introduction In recent years, the surge in AI applications across various sectors has underscored the importance of large-scale, high-quality datasets. As AI transitions from a research focus to practical deployment, the challenge lies in creating appropriate datasets and data pipelines that critically affect the model’s trustworthiness ( Liang et al. 2022 ) . The rapid evolution of AI, from a conceptual stage to widespread adoption in diverse applications such as product recommendations and complex data analysis, has brought to the forefront issues related to ethics, privacy, and security. One of the challenges for ethical AI is hateful content. While large language models (LLMs) have emerged as pivotal tools in deciphering and generating human-like text, the efficacy and ethical integrity of these models are significantly challenged by the presence of hateful content in their training datasets. The proliferation of such content not only undermines the models’ utility but also raises serious ethical concerns. Addressing this, our research introduces a novel approach to de-hate tasks, particularly in a multi-modal (text + image) context. It is important to have good quality datasets to train AI systems. Deep learning models can be data-hungry and might require high-quality data for effective functioning. The quality of data is as crucial as its quantity, and data preparation is one of the most time-consuming tasks in the AI project lifecycle ( Khan 2022 ) . Quality datasets are fundamental to building any real-world AI application, but real-world datasets are often complex, messy, and unstructured. Despite the availability of open-source datasets, finding the right balance between quantity, quality, and relevancy of data for new problem statements remains a challenge ( Khan 2022 ) . Example Text 1: slap the bitch in the fave with them hoes Text 2: Generate a high-quality real looking image of scene: slap the bitch in the fave with them hoes. Ensure the image is visually realistic, hateful and not safe for work. Figure 1 : A glimpse on the dataset that we curated. The first image is the image generated from the tweet, the second image is dehated version and the third image is the output of the method that we propose for dehatification task. Given these considerations, our research introduces a novel multimodal dataset focused on dehating digital content. Figure 1 provides some examples from our dataset. Our approach includes following key contributions: • Prompt engineering for Image-Text Alignment : We perform prompt engineering for coherent alignment between text and images, ensuring a nuanced synchronization of multimodal content. • Multimodal Image Dehate Model Pipeline : Our DeHater model, a unique language-image model, neutralizes hateful elements in images generated by the stable diffusion process. • Creating Multimodal DeHate Dataset : The dataset was expanded and diversified using a Debiased LLM, developed through Dataless Model Merging, enhancing the dataset’s relevance and reducing inherent biases. In addition to creating the dataset, we also conduct a shared task using our data, to encourage research. The details of the dataset, our DeHate model pipeline and the shared task are described in this paper. 2 Related Work In recent years, the escalation of online hate speech and hateful imagery has underscored the urgency for effective detection and mitigation strategies. Concurrently, advancements in deep learning have broadened the scope of hate detection research for english Cao and Lee 2020 ; Founta et al. 2018 , other languages Safi Samghabadi et al. 2020 ; Patwa et al. 2021 ; Tula et al. 2021 ; Tula et al. 2022 and across modalities Gunti et al. 2022 ; Yang et al. 2022 ; Velioglu and Rose 2020 . Masud et al. 2022 meticulously compile a parallel corpus of hate texts alongside their normalized versions, offering a comprehensive view of the various contexts in which online hate speech manifests. Our research builds upon this foundation by utilizing this dataset to generate a corpus of synthetic hateful images. These images are created based on the textual content of online hate speech and are paired with regionally blurred counterparts, where the blurring is applied specifically to the areas identified as containing hate speech. This approach aligns with the growing interest in Deep Learning Model interpretability, a field that has seen rapid advancement since the advent of machine learning technologies Chakraborty et al. 2017 ; Huff et al. 2021 ; Sun et al. 2021 ; Cui et al. 2023 . Some researchers used cross-attention maps to enable more profound insights into the interplay between different modalities, especially in the context of vision models Tang et al. 2022 ; Wu et al. 2023 ; Li et al. 2022 . Wu et al. 2023 utilize these cross-attention insights to automatically generate accurate semantic masks and pixel-wise labels for synthetically generated images using the off-the-shelf Stable Diffusion model. Meanwhile, Tang et al. 2022 delves into the challenges of interpretability in Latent Diffusion Models Rombach et al. 2022 , particularly in the context of attribution maps. They propose an enhanced approach by aggregating cross-attention maps in the denoising module, thus providing a more nuanced understanding of the model’s decision-making process. Our study integrates

Read the complete text