Prompting Away Stereotypes? Evaluating Bias in Text-to-Image Models for Occupations

Shaina Raza, Maximus Powers, Partha Pratim Saha, Mahveen Raza, Rizwan Qureshi · 2025

Read Prompting Away Stereotypes? Evaluating Bias in Text-to-Image Models for Occupations by Shaina Raza, Maximus Powers, Partha Pratim Saha, Mahveen Raza, Rizwan Qureshi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-00849.txt.

Introduction

Shaina Raza leads five authors in total, with Maximus Powers, Partha Pratim Saha, Mahveen Raza, and Rizwan Qureshi, in testing whether prompting can reduce stereotypes in text-to-image models for occupations. Such models are powerful creative tools but risk amplifying harmful social biases in who they depict.

They build a pilot benchmark covering five roles: chief executive, nurse, software engineer, teacher, and athlete. Five leading models, closed and open, receive neutral prompts and fairness-aware prompts designed to encourage demographic diversity. Outputs are annotated for gender and race. Prompting shifts representation substantially, but effects vary by model: some diversify well, others overcorrect into unrealistic uniformity, and some barely respond.

Catalog record

CREATORShaina Raza, Maximus Powers, Partha Pratim Saha, Mahveen Raza, Rizwan Qureshi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-00849

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Prompting Away Stereotypes? Evaluating Bias in Text-to-Image Models for Occupations Shaina Raza, Maximus Powers, Partha Pratim Saha, Mahveen Raza, Rizwan Qureshi Abstract Text-to-Image (TTI) models are powerful creative tools but risk amplifying harmful social biases. We frame representational societal bias assessment as an image curation and evaluation task and introduce a pilot benchmark of occupational portrayals spanning five socially salient roles (CEO, Nurse, Software Engineer, Teacher, Athlete). Using five state-of-the-art models: closed-source (DALL·E 3, Gemini Imagen 4.0) and open-source (FLUX.1-dev, Stable Diffusion XL Turbo, Grok-2 Image), we compare neutral baseline prompts against fairness-aware controlled prompts designed to encourage demographic diversity. All outputs are annotated for gender (male, female) and race (Asian, Black, White), enabling structured distributional analysis. Results show that prompting can substantially shift demographic representations, but with highly model-specific effects: some systems diversify effectively, others overcorrect into unrealistic uniformity, and some show little responsiveness. These findings highlight both the promise and the limitations of prompting as a fairness intervention, underscoring the need for complementary model-level strategies. We release all code and data for transparency and reproducibility https://github.com/maximus-powers/img-gen-bias-analysis . 1 Introduction T ext- T o- I mage (TTI) models are now widely used in creative and professional workflows lee2023aligning . However, alongside their utility, prior work cho2023generative ; vice2023quantifyingbiastexttoimagegenerative ; sakurai2025fairt2i ; liu2023sport shows that these systems often reproduce and even amplify harmful social biases, particularly along race and gender. Occupational portrayals are of particular concern where the skewed depictions of professions (e.g., male CEOs, female nurses) reinforce long-standing stereotypes and can shape public perceptions in ways that exacerbate inequity hirota2022gender ; amodio2006stereotyping . Addressing these issues is essential to ensure that TTI models are not only powerful, but also socially responsible. Two broad approaches have emerged to mitigate representational bias tan2019assessing in LLMs and TTI. Upstream interventions, such as dataset curation or retraining, can improve fairness but are often costly and opaque in commercial systems shahbazi2023representation ; saeed2025beyond . Downstream interventions, most notably prompt engineering, are more accessible, yet their effectiveness is mixed: some studies report reduced bias hutchinson_social_2020 , while others find unstable or limited effects luccioni2023stable . While recent work has begun to examine multiple TTI models, evaluations are often limited to a single demographic axis (typically gender) or race but lack occupational framing with these. This highlights the need for cross-model studies that examine race and gender together in occupational portrayals and provide empirical evidence on their effects. In this paper, we frame the problem as an image curation and evaluation task . We construct a benchmark of occupational portrayals by systematically generating images from widely used TTI models in five socially significant occupations: CEO, Nurse, Software Engineer, Teacher, and Athlete. For each occupation, we compare the outputs from neutral baseline prompts with those from fairness-aware controlled prompts explicitly designed to encourage diversity. All generated images are then annotated for race/ethnicity (Asian, Black, White) and gender (female, male), enabling structured evaluation of both baseline distributions and prompt-induced shifts. While our analysis is necessarily limited to a small set of occupations and simplified demographic categories, it provides a transparent and reproducible step toward understanding the broader effectiveness and limitations of prompting as a fairness intervention. Our main contributions can be encapsulated as follows: • We construct a pilot benchmark of \sim 500 occupational images for systematically evaluating demographic bias in two families of TT1 generation models: commercial closed-source (OpenAI DALL-E 3 betker2023improving and Gemini Imagen 4.0 google2025imagen4 ) and open-source (Flux flux2024 , SD stability2023sdxlturbo and Grok 2 xai2024grok2 ). • We provide a cross-model comparison of prompt interventions on both racial and gender representation across multiple occupations. • We analyze not only whether prompts shift distributions, but also whether these shifts risk overcorrection, producing new imbalances. We open-source data and code for the reproducibility of experiments. Our results show that prompt-based interventions can substantially shift model outputs, but their effects vary widely across systems. Some models respond with meaningful diversification, others overcorrect into unrealistic uniformity, and some show minimal change. These patterns illustrate both the potential and the limitations of prompting as a fairness tool, suggesting that complementary model-level interventions remain necessary. Our analysis presents a case study with a pilot dataset (not a comprehensive benchmark), and it offers a transparent, reproducible step toward understanding how prompt-based interventions influence demographic portrayals. 2 Related Work Table 1 : Related work at a glance (✓/✗). Legend: MM = multi-model; O+C = open+closed models; Occ = occupational framing; R+G = joint race+gender; Prompt = prompt mitigation; Overcorr = overcorrection analysis. Study MM O+C Occ R+G Prompt Overcorr hutchinson_social_2020 ✗ ✗ ✗ ✗ ✓ ✗ luccioni2023stable ✓ ✓ ✓ ✓ ✗ ✗ vice2023quantifyingbiastexttoimagegenerative ✓ ✓ ✗ ✓ ✗ ✗ sun2024smiling ✗ ✗ ✓ ✗ ✗ ✗ seshadri2023bias ✗ ✗ ✗ ✓ ✗ ✗ chinchure2024tibet ✓ ✗ ✗ ✓ ✗ ✗ wang2024biaspainter ✓ ✓ ✓ ✓ ✗ ✗ friedrich2024fairdiffusion ✓ ✗ ✓ ✓ ✗ ✗ li2023fairmapping ✓ ✗ ✓ ✓ ✗ ✗ This work (pilot) ✓ ✓ ✓ ✓ ✓ ✓ Text-to-image (TTI) systems often produce harmful social biases. Early analyses documented representational harms in language and vision systems, motivating demographic audits and fairness guidelines for multimodal models. Subsequent studies measured bias directly in TTI outputs, finding systematic under-representation along gender and race axes and correlations with labor-market stereotypes luccioni2023stable ; vice2023quantifyingbiastexttoimagegenerative . Occupational prompts particularly reveal slice, where models often default to gendered portrayals of professions. Meanwhile, related work has highlighted upstream drivers: large web-scale corpora such as LAION-5B carry known demographic skews and safety concerns, requiring caution for downstream use schuhmann2022laion5b . Beyond single-attribute audits, intersectional evaluation is crucial. The facial-analysis literature established that performance and representation disparities compound across gender and skin tone buolamwini2018gendershades , and resources like FairFace enable balanced labeling across multiple race groups karkkainen2021fairface . In TTI, recent audits increasingly adopt joint lenses, reporting amplification gaps when demographic targets in generations depart from observed training distributions chakraborty2025biasmap ; luccioni2023stable . Methodologically, several frameworks generalize bias discovery and measurement. TIBET proposes counterfactual perturbations and explanations that apply across models and prompts chinchure2024tibet , while BiasPainter automates bias triggering via neutral edits across occupations, activities, and traits, reducing annotation load wang2024biaspainter . Broader evaluation proposals argue for holistic auditing that spans alignment, quality, and ethics dimensions, not only representation lee2023holisticeval . Mitigation techniques range from downstream to model-level. Prompt-based approaches (e.g., Fair Diffusion) allow post-deployment control toward target demographic ratios without retraining friedrich2024fairdiffusion , while learned prompt or reference-guided inclusivity methods trade off sample quality and controllability clemmer2024precisedebias . Model-level strategies include distributional alignment via finetuning with soft tokens or adapted sampling friedrich2024fairdiffusion and lightweight mapping of conditioning embeddings for fairer generations li2023fairmapping . Several works note practical risks, e.g., quality degradation, mode collapse, or overshooting toward target groups, when controls are mis-specified, underscoring the need to monitor overcorrection as a distinct failure mode. Building on these strands (as shown in Table 1 ), we jointly evaluate race and gender under an occupational framing across five open/closed TTI systems, then study prompt-level interventions and explicitly quantify unintended shifts (e.g., overcorrection). This complements prior single-model or single-attribute audits by offering a compact, reproducible protocol spanning multiple systems, professions, and intersectional outcomes. Figure 1 : The end-to-end framework of

Read the complete text