Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts

Ruijia Li, Mingzi Zhang, Zengyi Yu, Yuang Wei, Bo Jiang · 2026

Read Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts by Ruijia Li, Mingzi Zhang, Zengyi Yu, Yuang Wei, Bo Jiang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10200.txt.

Introduction

Li and four colleagues audit social bias in vision-language models for schools with Edu-MMBias. Text-only tests miss the visual channel for hidden biases. The method spots bias across thought, feeling, and acts. A pipeline builds leak-proof student profiles for stress tests. The audit finds class bias backing lower-status stories plus health and racial stereotypes. Visual inputs act as a safety backdoor, waking biases past text guards and showing a thinking-choice mismatch.

Catalog record

CREATORRuijia Li, Mingzi Zhang, Zengyi Yu, Yuang Wei, Bo Jiang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50836
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10200

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts Ruijia Li, Mingzi Zhang, Zengyi Yu, Yuang Wei, Bo Jiang Abstract As Vision-Language Models (VLMs) become integral to educational decision-making, ensuring their fairness is paramount. However, current text-centric evaluations neglect the visual modality, leaving an unregulated channel for latent social biases. To bridge this gap, we present Edu-MMBias , a systematic auditing framework grounded in the tri-component model of attitudes from social psychology. This framework diagnoses bias across three hierarchical dimensions: cognitive , affective , and behavioral . Utilizing a specialized generative pipeline that incorporates a self-correct mechanism and human-in-the-loop verification , we synthesize contamination-resistant student profiles to conduct a holistic stress test on state-of-the-art VLMs. Our extensive audit reveals critical, counter-intuitive patterns: models exhibit a compensatory class bias favoring lower-status narratives while simultaneously harboring deep-seated health and racial stereotypes. Crucially, we find that visual inputs act as a safety backdoor, triggering a resurgence of biases that bypass text-based alignment safeguards and revealing a systematic misalignment between latent cognition and final decision-making. The contributions of this paper are available at: https://anonymous.4open.science/r/EduMMBias-63B2 . 1 Introduction With the rapid adoption of AI systems in automated scoring 42 , personalized learning analytics and knowledge tracing 25 ; 33 ; 45 , and intelligent tutoring systems 15 , AI has become deeply embedded in educational decision-making processes 40 . As these systems increasingly participate in high-stakes educational decisions, ensuring that they operate without systematic bias has emerged as a critical concern, as biased AI judgments may directly undermine fairness, reliability, and equity in educational outcomes. Recent studies have begun to examine bias in AI-driven educational systems by modeling large language models as personalized educational agents. A representative work evaluates bias by comparing variations in instructional content and recommendations generated for different student profiles, establishing a text-based, task-level evaluation framework for personalized education 35 . Despite recent progress in evaluating bias in educational AI systems, existing frameworks remain limited in both scope and modality . Prior studies primarily focus on text-based instructional behaviors, leaving the role of visual information largely unexplored. However, as VLMs increasingly incorporate visual cues in educational contexts, this omission becomes critical, since visual attributes are known to encode rich social signals and may introduce stronger and less regulated forms of bias 18 ; 31 . Figure 1: The Multidimensional Taxonomy of Edu-MMBias. The benchmark spans five primary categories, Gender, Race, SES, Hobbies, and Health, providing a comprehensive landscape of social attributes to systematically audit latent biases in VLMs. To this end, this paper proposes Edu-MMBias , a systematic auditing framework designed to comprehensively diagnose the multi-modal biases of VLMs in educational contexts. Grounded in the tri-component model of attitudes from social psychology 27 ; 35 , our approach transcends static, surface-level metrics to perform a holistic stress test across three hierarchical dimensions: the cognitive layer (via Multimodal Implicit Association Test), the affective layer (via Affect Misattribution Procedure), and the behavioral layer (via simulated resume screening). As illustrated in Figure 1 , this framework integrates diverse social attributes, including race, gender, SES, hobbies and Health, to trace how visual cues evolve into discriminatory decisions. To ensure rigorous evaluation, we construct a specialized generative pipeline . This pipeline synthesizes contamination-resistant student profiles to enable controlled experiments that isolate specific bias factors, and further incorporates a self-correct mechanism and human-in-the-loop verification to iteratively refine data quality and attribute controllability. Our extensive audit reveals critical, non-obvious patterns where models exhibit a compensatory class bias by over-favoring lower-status narratives while simultaneously harboring deep-seated health and racial stereotypes in their latent cognitive layers. These findings suggest that current alignment techniques often mask rather than eliminate systemic inequities. Our main contributions are summarized as follows: • We identify a blind spot in current educational bias benchmarks regarding the visual modality, revealing that visual attributes act as an unregulated channel for biases bypassing text-centric evaluations. • We propose Edu-MMBias , a systematic auditing framework grounded in social psychology that integrates a generative pipeline with a multi-layered diagnostic structure to trace bias from perception to decision-making. • Through extensive auditing, we find that educational VLMs exhibit consistent compensatory biases toward lower SES backgrounds, persistent health-related stereotypes, and a systematic misalignment between cognitive, affective, and behavioral layers, particularly under visual inputs. 2 Related Work Figure 2: Overview of the EduMMBias evaluation framework, comprising cognitive, affective, and behavioral assessment modules. VLMs in Educational Decision-making. VLMs have revolutionized educational technology, from automated content generation to interactive tutoring 16 . Advances in personalized learning and domain-specific tasks, such as geometry and physics 4 ; 19 , demonstrate their immense potential. However, their deployment in high-stakes assessment remains controversial due to scoring unpredictability in subjective visual materials 39 . While functional progress is significant, the community primarily fixates on performance metrics, neglecting the latent risks of algorithmic unfairness in automated educational gatekeeping 17 . To our knowledge, a systematic audit of these hidden risks remains largely unexplored. Social Bias Evaluation. Bias quantification has evolved from text-centric metrics 7 ; 23 to multimodal paradigms, where advanced representation techniques such as multi-modal proxy learning and subspace clustering 38 ; 37 can inadvertently encode complex social correlations. Researchers have adapted the IAT and AMP to uncover hidden prejudices and involuntary affective responses in visual encoders 30 ; 6 . Furthermore, behavioral audits have identified discriminatory outputs in image captioning and reasoning tasks 14 ; 41 . However, existing solutions typically treat cognitive associations, affective valences, and behavioral discrimination as isolated phenomena. A holistic framework that connects these layers to trace the complete pathway of bias, particularly within the sensitive domain of education, is currently absent 11 . Our work fills this gap by introducing a multi-tiered diagnostic framework. 3 Edu-MMBias This section details the EduMMBias benchmark through three hierarchical stages. We first establish an Evaluation Framework grounded in the tri-component model of attitudes 1 , followed by the definition of quantitative Evaluation Metrics for cognitive, affective, and behavioral dimensions. Finally, we describe the Construction Pipeline , which ensures data integrity through AI self-correction and human-in-the-loop verification. 3.1 Evaluation Framework According to the tri-component model of attitudes, a complete psychological profile consists of three interrelated components: cognition, affect, and behavior 1 . Our framework operationalizes this theoretical structure into three evaluation modules adapted for multimodal generative contexts, as illustrated in Figure 2 . Cognitive Dimension: Implicit Association. The assessment of cognitive bias centers on measuring the internal associative strength between concepts. In the traditional Implicit Association Test 12 , the evaluation does not rely on the subject’s explicit choice but rather on reaction time to gauge the automaticity of an association. A shorter reaction time signifies a more robust mental link. Given that generative models lack physiological latency, we follow prior research by introducing model confidence as a digital proxy for reaction time 3 ; 32 . This approach assumes that a model exhibits higher predictive certainty when processing associations that align with its internal logic. As shown in the Cognitive Bias module of Figure 2 , we implement a bi-directional association design. The VLM is presented with a student profile image and required to choose between composite labels. The experiment consists of forward

Read the complete text