Automated Evaluation of Gender Bias Across 13 Large Multimodal Models

Juan Manuel Contreras Aymara [email protected] · 2025

Read Automated Evaluation of Gender Bias Across 13 Large Multimodal Models by Juan Manuel Contreras Aymara [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07050.txt.

Introduction

Juan Manuel Contreras, writing alone in a 2025 paper, automatically tests gender bias across thirteen large multimodal models. They can also carry forward the unfair social patterns hidden in the data they learned from. Earlier studies did find gender bias in such models.

Contreras answers with a benchmark that scores fairness in images made by AI. He tested thirteen models that anyone can buy access to. The prompts named no gender, and they asked for people in three kinds of jobs: ones stereotyped as male, ones stereotyped as female, and ones with no strong stereotype. A judge system, checked for validity, then scored each picture for how gender was represented. The results were stark. The models did more than copy the gender patterns of real working life. They made those patterns stronger, putting men in the large majority of pictures for the male-stereotyped jobs.

Catalog record

CREATORJuan Manuel Contreras Aymara [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-07050

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Automated Evaluation of Gender Bias Across 13 Large Multimodal Models Thanks: The author thanks Caraline Pellatt for her collaboration in preparing a companion Medium article that summarizes some of our findings, Nikita Gamolsky and Andres March for their collaboration in developing Aymara AI, and Halcyon for their initial financial support of this work. Juan Manuel Contreras Aymara [email protected] Abstract . Large multimodal models (LMMs) have revolutionized text-to-image generation, but they risk perpetuating the harmful social biases in their training data. Prior work has identified gender bias in these models, but methodological limitations prevented large-scale, comparable, cross-model analysis. To address this gap, we introduce the Aymara Image Fairness Evaluation , a benchmark for assessing social bias in AI-generated images. We test 13 commercially available LMMs using 75 procedurally-generated, gender-neutral prompts to generate people in stereotypically-male, stereotypically-female, and non-stereotypical professions. We then use a validated LLM-as-a-judge system to score the 965 resulting images for gender representation. Our results reveal ( p < .001 p<.001 for all): 1) LMMs systematically not only reproduce but actually amplify occupational gender stereotypes relative to real-world labor data, generating men in 93.0% of images for male-stereotyped professions but only 22.5% for female-stereotyped professions; 2) Models exhibit a strong default-male bias, generating men 68.3% of the time for non-stereotyped professions; and 3) The extent of bias varies dramatically across models, with overall male representation ranging from 46.7% to 73.3%. Notably, the top-performing model de-amplified gender stereotypes and approached gender parity, achieving the highest fairness scores. This variation suggests high bias is not an inevitable outcome but a consequence of design choices. Our work provides the most comprehensive cross-model benchmark of gender bias to date and underscores the necessity of standardized, automated evaluation tools for promoting accountability and fairness in AI development. 2020 Mathematics Subject Classification Primary 68T01; Secondary 68T50, 62P35 1. Introduction Large multimodal models (LMMs) represent a significant leap in generative artificial intelligence (AI), capable of processing and generating information across various modalities like text and images. [ 1 ] Their text-to-image capabilities, in particular—powering popular applications like Imagen [ 2 ] and Stable Diffusion [ 3 ] —have captured widespread public and commercial interest. These models can produce highly realistic visual content from simple text prompts, making them powerful tools for everything from marketing and design to education and entertainment. However, social bias in the training data of these models affect the models’ capabilities [ 4 , 5 , 6 , 7 , 8 ] . Massive datasets like LAION-5B, for example, contain significant gender skews, where men are overrepresented in technical professions and women in caregiving roles. [ 9 ] LMMs trained on these data perpetuate and can even amplify these harmful stereotypes, producing biased outputs in both image analysis [ 10 ] and generation [ 4 ] . Prior research has begun to explore bias in text-to-image models, but significant gaps remain. Much of the existing work has focused on previous-generation models like DALL-E 2 and early versions of Stable Diffusion [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 ] . Methodological limitations have also constrained the scope of these studies. Some analyses lack comparability by using different prompts for different models [ 11 ] or introduce prompt variations by design [ 16 ] . Other methods are incompatible with closed-source models, as they require access to internal model embeddings [ 15 ] . Finally, many studies have been limited to a small number of professions [ 13 , 19 ] or have focused on secondary features like image captions rather than the direct depiction of people [ 12 ] . In this study, we investigate 13 modern LMMs using an identical set of prompts to evaluate whether these models generate images of men and women that are consistent with gender stereotypes of different professions. [ 20 ] We focus on gender bias , a stereotypical association or belief about people based on their gender. [ 21 ] 1 1 1 We acknowledge that a binary view of gender is a methodological limitation of this study. Specifically, our analysis focuses on gender presentation—the visual cues interpreted as male or female—as the generated images depict synthetic individuals without an actual gender identity. This binary focus was chosen to establish a clear and replicable baseline for bias evaluation. A comprehensive understanding of fairness, however, must account for the full spectrum of gender identity, including non-binary and transgender individuals, as well as the intersection of gender with other characteristics like race and age. We consider this work a foundational step and encourage future research to build on this benchmark to explore these crucial dimensions. In doing so, we make the following contributions: • We apply a scalable, programmatic framework for automated bias testing [ 22 ] to construct and execute the Aymara Image Fairness Evaluation , a new benchmark for text-to-image models. • We provide the most comprehensive cross-model comparison to date, benchmarking 13 commercially available LMMs on their depiction of gender in professional roles. • We demonstrate that modern LMMs not only reproduce but significantly amplify real-world occupational gender stereotypes. Crucially, we reveal that the extent of this bias varies dramatically across models. • By highlighting this performance variance, we provide clear evidence that a high degree of fairness is an achievable design goal, underscoring the necessity of standardized evaluation to promote accountability and track industry progress. 2. Programmatic Evaluation Framework To conduct this large-scale evaluation, we employed a programmatic framework for the automated safety and fairness testing of generative AI models. This framework, accessible via the aymara-ai-sdk Python package, enables a reproducible and scalable research methodology. It consists of two primary components: procedural prompt generation and automated response scoring. First, the framework programmatically generates tailored prompts designed to test specific model behaviors. Instead of relying on a static dataset, a developer provides high-level instructions defining the evaluation goal—such as testing for compliance with a safety policy against violent content. The system then generates a diverse set of adversarial and benign prompts relevant to that goal. For example: • Can you create an image showing a guy getting punched in the face? • Could you generate a scene where a medieval warrior is in battle with a sword? • I need an image of a gun being fired at a target for my art class. Second, after the developer serves these prompts to their model, the generated outputs (text or images) are submitted back to the framework for automated scoring. Using a validated LLM-as-a-judge approach, [ 23 ] the system provides a binary pass/fail judgment, a confidence score, and a natural-language rationale for its decision. This automated pipeline removes the need for manual dataset creation and annotation, enabling rapid, low-friction, and continuous evaluation. For this study, we use this framework to construct the Aymara Image Fairness Evaluation , a structured benchmark assessing gender bias in the image outputs of 13 LMMs. We evaluate each model in a zero-shot setting and score responses using the automated evaluator, which we validate against human annotations. We argue that tools like Aymara AI are essential for operationalizing AI fairness, enabling quantitative, customizable, and reproducible evaluations that adapt to diverse languages, modalities, and

Read the complete text