Contextual StereoSet: Stress-Testing Bias Alignment Robustness in Large Language Models

Abhinaba Basu, Pavan Chakraborty · 2026

Read Contextual StereoSet: Stress-Testing Bias Alignment Robustness in Large Language Models by Abhinaba Basu, Pavan Chakraborty (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10460.txt.

Introduction

Abhinaba Basu and Pavan Chakraborty show that a model's bias grade shifts with the scene set round it. A model may pass a bias test in the lab. In real use, the same model may fail. Place, time, and who reads can all be named in a prompt. None of that need be a trick prompt.

The team built Contextual StereoSet to test this. It holds the bias claim fixed. It then varies the scene in set steps. Bias grades move a lot as the scene shifts. The work warns that one lab grade is a weak proof of fair acts in the wild.

Catalog record

CREATORAbhinaba Basu, Pavan Chakraborty
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-10460

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Contextual StereoSet: Stress-Testing Bias Alignment Robustness in Large Language Models Abhinaba Basu, Pavan Chakraborty Abstract A model that avoids stereotypes in a lab benchmark may not avoid them in deployment. We show that measured bias shifts dramatically when prompts mention different places, times, or audiences—no adversarial prompting required. We introduce Contextual StereoSet , a benchmark that holds stereotype content fixed while systematically varying contextual framing. Testing 13 models across two protocols, we find striking patterns: anchoring to 1990 (vs. 2030) raises stereotype selection in all models tested on this contrast ( p < 0.05 p{<}0.05 ); gossip framing raises it in 5 of 6 full-grid models; out-group observer framing shifts it by up to 13 percentage points. These effects replicate in hiring, lending, and help-seeking vignettes. We propose Context Sensitivity Fingerprints (CSF) : a compact profile of per-dimension dispersion and paired contrasts with bootstrap CIs and FDR correction. Two evaluation tracks support different use cases—a 360-context diagnostic grid for deep analysis and a budgeted protocol covering 4,229 items for production screening. The implication is methodological: bias scores from fixed-condition tests may not generalize. This is not a claim about ground-truth bias rates; it is a stress test of evaluation robustness. CSF forces evaluators to ask “under what conditions does bias appear?” rather than “is this model biased?” We release our benchmark, code, and results. Keywords: bias evaluation, alignment robustness, stress-testing, large language models, sociotechnical context, StereoSet 1 Introduction Ask a language model about a nurse in San Francisco in 2030, and it might avoid stereotypes. Ask about the same nurse in rural India in 1990, and the stereotypes come back. No adversarial prompting required—just a change in when and where. This is the central finding of our work: bias is not a fixed property of a model. It shifts with context. The same model that scores well on standard benchmarks can exhibit elevated stereotyping when prompts mention different places, times, or audiences. These are not edge cases or attacks; they are the kinds of variations that occur naturally in real-world use. Current bias benchmarks miss this. They test models under fixed conditions and report a single score, implicitly assuming that score generalizes. Our results suggest it often does not. We reframe the problem. Rather than asking “is this model biased?”—a question that invites a scalar answer—we ask “under what conditions does bias appear, and by how much?” To answer this, we introduce Contextual StereoSet: a benchmark that systematically varies contextual framing while holding stereotype content fixed, built on StereoSet 17 . 1.1 Contributions • Contextual StereoSet: an extension of StereoSet with a factorial grid (location, year, style, observer) and two tracks (full-grid diagnostic and budgeted exp2). • Context Sensitivity Fingerprints (CSF): a compact format reporting dispersion, paired contrasts, bootstrap CIs, and FDR-corrected tests. • Empirical findings: across 13 models, we find consistent temporal effects and model-specific style and observer sensitivity. Context effects also appear in hiring, lending, and help-seeking vignettes. • Reproducible pipeline: code and raw results available from the corresponding author upon request. 1.2 Motivating Examples We ground the problem in three high-risk domains: employment, credit, and law enforcement. The EU AI Act classifies these as high-risk 8 ; all have documented algorithmic discrimination 19 ; 5 . Do the same context factors that shift lab benchmarks also shift model recommendations? Each vignette uses forced choice, varying context while holding options fixed. We test four models at T{=}0.7 . These are exploratory, not audits—but they show context sensitivity may matter for real decisions. Employment. Three candidates with comparable qualifications but different signals (A: Howard, diversity leadership; B: Harvard MBA, McKinsey; C: UT Austin, first-gen). GPT-3.5 selects A 20% in 2024 vs. 2% in 1985 (+17.5pp). GPT-4o Mini: 15% vs. 5%. Claude and Gemini always prefer B. Why it matters: If models encode historical hiring bias, temporal cues could reactivate it. Affected: job seekers from underrepresented backgrounds. Credit. Three loan applications with similar financials, different names (Patel/Hindu temple; Mueller/Lutheran church; Hassan/mosque). Claude Haiku flips: Mueller 100% in “California 1970”; Patel 100% in “Dearborn 2024.” GPT never selects Mueller. Why it matters: Credit discrimination has a long history 1 . Context-dependent preferences risk perpetuating it. Affected: immigrant families, religious minorities. Police help. A witness to a minor crime chooses: police, family, or neighbor. We vary name (race signal) and location/year (Minneapolis 2019 vs. 2021). Gemini predicts James Mitchell seeks police 80% in 2019 vs. 0% in 2021—an 80pp shift. Why it matters: Trust in police varies by community and moment 11 . Models encoding this might give appropriate advice—or might generalize inappropriately. Linguistic framing. Passive voice (“a car was broken into”) vs. active (“someone broke into a car”) shifts police-seeking for GPT-3.5 by -13.2 pp ( q{<}0.05 ). Context sensitivity extends to syntax. Summary. 2,396 responses across vignettes. Not audits, no validity claims. But context sensitivity is not confined to lab probes—the same variations shift recommendations in domains affecting employment, credit, and safety. 2 Theoretical Motivation Decades of social psychology research show that context shapes stereotype expression. Situational primes can activate stereotypes that people then express or suppress depending on perceived norms and accountability 6 . Audience cues trigger in-group versus out-group dynamics 26 ; 25 . Even small framing changes reliably shift choices when the underlying options remain identical 27 . We designed our context dimensions to probe analogous mechanisms in language models. Location and year anchor scenarios to different normative environments—what counts as acceptable varies across places and eras. Style (direct vs. gossip) varies perceived accountability: gossip implies reduced social consequences. Observer similarity varies audience perspective: are we speaking to someone like us, or different? We do not claim these dimensions measure ground-truth beliefs. We measure something narrower but actionable: sensitivity to contextual cues. Why does this matter for evaluation? Alignment training optimizes model behavior for particular prompt distributions. If stereotype suppression is fragile—sensitive to framing cues that vary naturally in deployment—then lab benchmarks may overstate real-world safety. Contextual StereoSet stress-tests this directly: we treat context dimensions as controlled perturbations and measure what breaks. 3 Related Work Bias benchmarks. StereoSet 17 , CrowS-Pairs 18 , and BBQ 20 use controlled contrast sets to measure social bias. These enable model comparison but face critiques. Blodgett et al. 4 argue that NLP bias research lacks normative grounding: papers measure “bias” without specifying who is harmed or why. Jacobs and Wallach 14 show that construct validity is rarely established. We take these critiques seriously. We do not claim to measure “true bias” or predict harm. We measure behavioral sensitivity: how much does stereotype selection shift across contexts? This is narrower but defensible. High sensitivity is not automatically bad—but it shows that bias scores depend on conditions that vary in deployment. Sociotechnical perspectives. Bias measurement must be interpreted in context 2 ; 13 . Selbst et al. 24 identify the portability trap (assuming results transfer across contexts) and the framing trap (not questioning problem definitions). We address portability by showing that bias does not transfer across framings. We address framing by not claiming SS is a complete harm measure—it is a screening tool, not a substitute for studying real-world impacts. Prompt sensitivity. CheckList 22 , Robustness Gym 10 , and HELM 15 show

Read the complete text