Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models

Pushkar Shukla Aditya Chinchure Emily Diana Alexander Tolbert, Kartik Hosanagar Vineeth N. Balasubramanian Leonid Sigal Matthew A. Turk · 2025

Read Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models by Pushkar Shukla Aditya Chinchure Emily Diana Alexander Tolbert, Kartik Hosanagar Vineeth N. Balasubramanian Leonid Sigal Matthew A. Turk (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17280.txt.

Introduction

Text-to-image models carry biases — about age, ethnicity, gender, and more — and fixing one can quietly bend another. Pushkar Shukla and co-authors build a tool, BiasConnect, that maps these interactions. It intervenes on one bias axis at a time and estimates how mitigation there will ripple to the others. Those estimates correlated strongly with the outcomes actually observed after mitigation.

On top of the map they build InterMit, a mitigation method steered by the user's target distributions and priorities across axes. It reached lower bias than traditional techniques while taking fewer mitigation steps and producing better images. The implementation needs no retraining and can wrap around many existing debiasing methods. The work argues that fairness in image generation must be handled as one connected system, not as a row of separate dials.

Catalog record

CREATORPushkar Shukla Aditya Chinchure Emily Diana Alexander Tolbert, Kartik Hosanagar Vineeth N. Balasubramanian Leonid Sigal Matthew A. Turk
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17280

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models Pushkar Shukla Aditya Chinchure Emily Diana Alexander Tolbert, Kartik Hosanagar Vineeth N. Balasubramanian Leonid Sigal Matthew A. Turk Abstract The biases exhibited by text-to-image (TTI) models are often treated as independent, though in reality, they may be deeply interrelated. Addressing bias along one dimension—such as ethnicity or age—can inadvertently affect another, like gender, either mitigating or exacerbating existing disparities. Understanding these interdependencies is crucial for designing fairer generative models, yet measuring such effects quantitatively remains a challenge. To address this, we introduce BiasConnect , a novel tool for analyzing and quantifying bias interactions in TTI models. BiasConnect uses counterfactual interventions along different bias axes to reveal the underlying structure of these interactions and estimates the effect of mitigating one bias axis on another. These estimates show strong correlation (+0.65) with observed post-mitigation outcomes. Building on BiasConnect , we propose InterMit , an intersectional bias mitigation algorithm guided by user-defined target distributions and priority weights. InterMit achieves lower bias (0.33 vs. 0.52) with fewer mitigation steps (2.38 vs. 3.15 average steps), and yields superior image quality compared to traditional techniques. Although our implementation is training-free, InterMit is modular and can be integrated with many existing debiasing approaches for TTI models, making it a flexible and extensible solution. Figure 1: An example for which BiasConnect estimates a negative impact of bias mitigation along one axis on another axis. For this query, increasing the gender diversity (Gen) skews age distribution (Age) for images of musicians generated by Flux-dev. 1 Introduction Text-to-Image (TTI) models such as DALL-E ( Ramesh et al., 2021 ) , Imagen ( Saharia et al., 2022 ) , and Stable Diffusion Rombach et al. (2022) have become widely used for generating visual content from textual prompts. Despite their impressive capabilities, these models often inherit and amplify biases present in their training data Wang et al. (2022b) ; Chinchure et al. (2024) ; Cho et al. (2023) . These biases manifest across multiple social and non-social dimensions – including gender, race, clothing, and age – leading to skewed or inaccurate representations. As a result, TTI models may reinforce harmful stereotypes and societal norms ( Bender et al., 2021 ; Birhane and Prabhu, 2021 ) . While significant efforts have been made to evaluate and mitigate societal biases in TTI models ( Wang et al., 2023a ; Cho et al., 2023 ; Ghosh and Caliskan, 2023 ; Esposito et al., 2023 ; Bianchi et al., 2023 ; Chinchure et al., 2024 ) , these approaches often assume that biases along different dimensions (e.g., gender and race) are independent of each other. Consequently, they do not account for relationships between these dimensions. For instance, as illustrated in Figure 1 , mitigating gender (male, female) may effectively diversify the gender distribution in a set of generated images, but this mitigation step may negatively impact the diversity of another bias dimension, such as age. This relationship between two bias dimensions highlights the intersectional nature of these biases. The concept of intersectionality , first introduced by Crenshaw ( Crenshaw, 1989 ) , motivates the need to understand how overlapping social identities such as race, gender, and class contribute to systemic inequalities. In TTI models, these intersections can have a significant impact. As a motivating study, we independently mitigated eight bias dimensions over 26 occupational prompts on Stable Diffusion 1.4, using a popular bias mitigation strategy, ITI-GEN Zhang et al. (2023) (see A.5 ). We found that while the targeted biases were reduced in most cases, biases along other axes were negatively affected in over 29\% of the cases. This suggests that for an effective bias mitigation strategy, it is crucial to understand which biases are intersectional. Additionally, it is important to strive towards building a more holistic bias mitigation algorithm that can either mitigate multiple biases simultaneously or predict what biases cannot be mitigated together. To understand how biases in TTI models influence one another, we propose BiasConnect , the first analysis tool that evaluates biases while explicitly modeling their intersectional relationships. Unlike prior methods that treat biases in isolation, BiasConnect identifies how mitigating one bias can positively or negatively affect others. Specifically, BiasConnect uses a novel metric, the Intersectional Sensitivity ( IS ), to quantify how mitigation along one axis affects others. These IS scores show a strong correlation (+0.65) with observed intersectional outcomes post-mitigation. We validate our approach through robustness studies and qualitative analyses, demonstrating its utility for auditing open-source TTI models. Furthermore, we extend BiasConnect with a holistic intersectional bias mitigation algorithm, InterMit . While we propose an effective and straightforward implementation in this paper, InterMit is modular and can be integrated with any existing sequential bias mitigation method. Unlike prior approaches that assume fixed ideal distributions and treat all biases equally, InterMit allows users to define arbitrary target distributions, select specific bias axes, and assign custom priority weights to each bias—enabling flexible joint mitigation and informed reasoning about conflicting biases. In our evaluation, InterMit outperforms existing methods by mitigating biases more effectively, producing higher-quality images, and requiring fewer mitigation steps. Moreover, unlike other methods, it can handle a larger number ( >3 ) of bias axes and alerts users when mitigation along one axis adversely affects others. Figure 2: An overview of BiasConnect . We use a counterfactual-based approach to measure how interventions along a single bias axes impact other bias axes. Our metric Intersectional Sensitivity estimates how bias mitigation on one axis impacts another. Our results are visualized as a a matrix called the Bias Intersectionality Matrix. 2 Related Work 2.1 Intersectionality and Bias in AI Intersectionality, introduced by Crenshaw Crenshaw (1989) , describes how multiple forms of oppression—such as racism, sexism, and classism—intersect to shape unique experiences of discrimination. Two key models define this concept: the additive model, where oppression accumulates across marginalized identities, and the interactive model, where these identities interact synergistically, creating effects beyond simple accumulation Curry (2018) . In the context of AI, most existing work Diana and Tolbert (2023) ; Kavouras et al. (2023) ; Kearns et al. (2018) aligns more closely with the additive model, focusing on quantifying and mitigating biases in intersectional subgroups. This perspective has influenced fairness metrics Diana et al. (2021) ; Foulds et al. (2020) ; Ghosh et al. (2021) designed to assess subgroup-level performance, extending across various domains, including natural language processing (NLP) Lalor et al. (2022) ; Lassen et al. (2023) ; Guo and Caliskan (2021) ; Tan and Celis (2019) and recent large language models Kirk et al. (2021) ; Ma et al. (2023) ; Devinney et al. (2024) ; Bai et al. (2025) , multimodal research Howard et al. (2024) ; Hoepfinger (2023) , and computer vision Wang et al. (2020) ; Steed and Caliskan (2021) . These approaches typically measure disparities across predefined demographic intersections and propose mitigation strategies accordingly. Our work aligns with the interactive model of intersectionality, using counterfactual analysis with TTI models, where we intervene on a single bias axis to assess its ripple effects on others. 2.2 Bias in Text-to-Image Models Extensive research has been conducted on evaluating and mitigating social biases in both image-only models Buolamwini and Gebru (2018) ; Seyyed-Kalantari

Read the complete text