FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models
Zihao Fu, Ryan Brown, Shun Shao, Kai Rawal, Eoin Delaney, Chris Russell · 2025
Read FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models by Zihao Fu, Ryan Brown, Shun Shao, Kai Rawal, Eoin Delaney, Chris Russell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21363.txt.
Introduction
Zihao Fu and five co-authors tackle bias in text-to-image models, which often copy and even strengthen social stereotypes about groups such as gender and race groups. Their tool, FairImagen, debiases a model after training, working only on the numerical form of the prompt, so the image model itself stays untouched.
A fair version of principal component analysis projects prompt codes into a space that keeps the prompt's meaning while stripping out group-specific signals. Added noise strengthens the effect, and a single unified projection can debias several traits at once, including combined settings. Across gender, race, and combined tests, fairness improves clearly while picture quality and faithfulness to the prompt dip only moderately, beating other after-the-fact methods.
Catalog record
| CREATOR | Zihao Fu, Ryan Brown, Shun Shao, Kai Rawal, Eoin Delaney, Chris Russell |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21363
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models Zihao Fu, Ryan Brown, Shun Shao, Kai Rawal, Eoin Delaney, Chris Russell Abstract Text-to-image diffusion models, such as Stable Diffusion, have demonstrated remarkable capabilities in generating high-quality and diverse images from natural language prompts. However, recent studies reveal that these models often replicate and amplify societal biases, particularly along demographic attributes like gender and race. In this paper, we introduce FairImagen 1 1 1 https://github.com/fuzihaofzh/FairImagen , a post-hoc debiasing framework that operates on prompt embeddings to mitigate such biases without retraining or modifying the underlying diffusion model. Our method integrates Fair Principal Component Analysis to project CLIP-based input embeddings into a subspace that minimizes group-specific information while preserving semantic content. We further enhance debiasing effectiveness through empirical noise injection and propose a unified cross-demographic projection method that enables simultaneous debiasing across multiple demographic attributes. Extensive experiments across gender, race, and intersectional settings demonstrate that FairImagen significantly improves fairness with a moderate trade-off in image quality and prompt fidelity. Our framework outperforms existing post-hoc methods and offers a simple, scalable, and model-agnostic solution for equitable text-to-image generation. Firefighter (Base) Gender Debias (FairImagen) Yoga Instructor (Base) Race Debias (FairImagen) Figure 1: FairImagen mitigates demographic biases in text-to-image generation. Compared to baseline Stable Diffusion (Base), our method (FairImagen) produces more balanced representations across demographic attributes such as gender and race, while preserving visual quality and semantic fidelity. 1 Introduction Recent advances in text-to-image generation have led to the widespread adoption of models such as Stable Diffusion [ 1 , 2 ] , DALL·E [ 3 , 4 ] , Imagen [ 5 ] , and Parti [ 6 ] . These can produce photorealistic and diverse images from natural language prompts. These models leverage powerful vision-language encoders such as CLIP [ 7 ] to align textual and visual modalities, enabling open-ended image generation from arbitrary input. Due to their flexibility, scalability, and accessibility, these systems are increasingly integrated into applications across design, content creation, and interactive media [ 8 , 9 ] . However, studies have shown that these generative models often replicate and even amplify social biases present in the training data [ 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 ] . For example, prompts such as: “a photo of a CEO” or “a nurse” typically yield images depicting white males and females, respectively, reflecting gender and racial stereotypes. These biases pose serious concerns regarding fairness, representation, and downstream harms, particularly as generative models are integrated into public-facing systems. To mitigate such biases, researchers have proposed a variety of debiasing techniques. Methods fall into three main categories: Prompt-based , Fine-tuning , and Post-hoc editing ( Table 1 ). Prompt-based approaches [ 10 , 19 , 20 ] modify the input to influence the model’s output, but often require per image heuristic rewriting and manually-curated prompts. Fine-tuning methods [ 21 , 13 , 22 ] retrain or adapt parts of the model to encode fairness objectives, but they are computationally intensive and require access to model internals. Post-hoc editing methods [ 23 , 24 , 13 ] modify prompt embeddings at inference without updating model weights, offering a lightweight and deployment-friendly alternative. Each category exhibits differing trade-offs in terms of fidelity, interpretability, and generalizability. We focus on post-hoc editing methods due to their simplicity and compatibility with a wide range of off-the-shelf diffusion models. Approaches such as SDID [ 23 ] and TBIE [ 24 ] demonstrated the feasibility of manipulating prompt embeddings to mitigate demographic bias. SDID identified a gender direction by subtracting CLIP embeddings of pairs of hand-crafted prompts; adding or subtracting this vector in generation. This heavily relies on the group bias being linearly separable and correctable via modifying embeddings in a single direction. This is inappropriate for debiasing involving more than two demographic groups, e.g., ethnicity. TBIE improves over SDID by applying PCA on CLIP embeddings of gender-related words to identify bias directions in a data-driven way. However, PCA is performed on simple gendered words without an explicit optimization criterion for content alignment. As a result, the debiasing process is often overly aggressive, removing not just demographic cues but also key semantic information. This results in semantic drift, loss of prompt fidelity, and unnatural image generation. Both methods offer limited control over the trade-off between fairness and fidelity, and neither generalizes well to intersectional prompts or unseen groups. To overcome these limitations, we propose a novel post-hoc debiasing framework we call FairImagen (Fair Image Generation). It explicitly integrates Fair Principal Component Analysis (FairPCA) [ 25 ] into the Stable Diffusion pipeline and optimizes for semantic preservation while minimizing group-dependent variance. FairImagen operates in three stages: first, it extracts CLIP-based prompt embeddings; second, it applies a fairness-aware projection using FairPCA to remove group-dependent directions from both pooled and token-level embeddings; finally, it synthesizes images from the transformed embeddings using a modified Stable Diffusion decoder. To further enhance performance, we incorporate an empirical noise injection scheme to avoid overly neutralized outputs, and propose a unified cross-demographic debiasing formulation to jointly mitigate intersectional bias. Unlike existing post-hoc approaches, FairImagen offers precise control over the trade-off between fairness and content alignment. It is fully compatible with off-the-shelf diffusion models, supports multiple demographic attributes simultaneously, and preserves visual quality while effectively reducing bias. Our contributions are as follows: (1) We introduce a post-hoc fairness framework that integrates FairPCA with diffusion-based text-to-image generation, enabling bias mitigation without model retraining. (2) We propose empirical noise injection to obscure residual demographic signals and improve fairness-performance trade-offs. (3) We develop a cross-demographic debiasing formulation that handles multiple protected attributes in a unified manor, avoiding over-pruning from sequential projections. (4) We conduct extensive quantitative and qualitative evaluations across gender, race, and joint debiasing tasks, demonstrating that our method outperforms existing post-hoc baselines. Criteria Prompt-based Fine-tuning Post-hoc editing Training-free ✓ ✗ ✓ Black-box compatible ✓ ✗ ✓ Low human effort ✗ ✓ ✓ Low computational cost ✓ ✗ ✓ Generalizable to new prompts ✗ ✓ ✓ Strong bias mitigation ✗ ✓ ✓ Preserves prompt fidelity ✓ ✓ ✗ Easy deployment ✗ ✗ ✓ Table 1: Comparison of prompt-based, fine-tuning-based, and post-hoc editing methods for debiasing text-to-image generation. 2 Related Works Existing debiasing methods for text-to-image generation can be categorized into three types: Prompt-based, Fine-tuning-based, and Training-free methods. As summarized in Table 1 , no single category is universally superior; the choice of method often depends on the specific application scenario and deployment constraints. Prompt-based methods mitigate bias by modifying the input prompts. Friedrich et al. 2023 proposed Fair Diffusion using fairness-guided prompts constructed from demographic opposites. Sakurai and Sato 2025 utilized LLMs to automatically detect and revise biased prompts. Bansal et al. 2022 and Chuang et al. 2023 examined ethical interventions and latent direction projection. Kim et al. 2023 and Al Sahili et al. 2024 developed learned fairness prompts, and Bianchi et al. 2023 assessed the impact of biased prompts at scale. These methods are flexible but often rely on heuristic or external guidance for every single image. This can be somewhat opaque and laborious [ 20 , 13 ] . Fine-tuning based methods update model parameters to enforce fairness. Li et