AI Alignment Amplifies the Role of Race, Gender, and Disability in Hiring Decisions
Ze Wang, Guobin Shen, Michael Thaler · 2026
Read AI Alignment Amplifies the Role of Race, Gender, and Disability in Hiring Decisions by Ze Wang, Guobin Shen, Michael Thaler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13866.txt.
Introduction
Wang, Shen and Thaler test hiring bias in 29 language models across 177 jobs that cover near half of jobs in the United States. They find the models favor female and Black job seekers, yet penalize disabled ones. The gaps match the effect of six months to a year of added schooling. Post-training alignment, which tunes models to human norms, makes the gaps far worse. It lifts the female and Black edge about fourfold, and deepens the penalty for disabled job seekers.
Catalog record
| CREATOR | Ze Wang, Guobin Shen, Michael Thaler |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52247 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13866
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AI Alignment Amplifies the Role of Race, Gender, and Disability in Hiring Decisions Ze Wang, Guobin Shen, Michael Thaler Abstract keywords AI fairness, hiring discrimination, language models, post-training alignment, disability Humans increasingly delegate decisions to language models, yet whether these systems reproduce or reshape human patterns of discrimination remains unclear 36 , 30 , 12 , 31 , 44 , 5 , 21 , 22 , 26 . Here we run a large-scale study to analyse whether language models use demographic information in hiring decisions. We show, across 27 models and 177 occupations 33 , 41 , 42 , 43 , that language models give female and Black candidates hiring advantages relative to otherwise-comparable male and white candidates, while giving disabled candidates disadvantages. The differences are meaningful in magnitude: the role of race, gender, and disability status is comparable to six months to one year of additional education. Post-training alignment is the primary driver: relative to matched pre-trained models, alignment amplifies advantages for female and Black candidates by 325% and 330%, and disadvantages for disabled candidates by 171%. Compared with previous human correspondence studies, language models reverse the direction of racial discrimination, attenuate the disability penalty, and amplify the female advantage by 190%. Alignment changes how models use qualification signals: alignment increases returns to skills and work experience overall, but relatively more so for female and Black candidates. Meanwhile, the absence of qualification signals harms marginalised groups more, particularly for disabled candidates, differences that may explain the asymmetry of alignment effects across groups we observe. 1 Introduction Artificial intelligence systems increasingly participate in decisions that were once exclusively human, from advising on medical diagnoses 27 to shaping judicial outcomes 28 to screening job candidates 32 . As these systems shape more consequential outcomes, people have begun to examine whether they inherit the cognitive and social biases documented in decades of psychology and economics research 17 , 12 , 26 . Among the most consequential of human biases is discrimination based on demographic identity—gender, race, and disability 7 , 37 , 6 , 2 —which has been extensively documented in labour markets through correspondence experiments 8 , 29 , 38 , 4 , 31 . We use hiring as a testing ground to investigate whether language models reproduce these human patterns of discrimination, or create new ones. Figure 1: Per-model demographic effects. Log odds ratios ( \beta_{d} ) for each of 27 models; positive values indicate the model gives the historically-marginalised group higher hiring odds, negative values indicate a penalty. Blue: instruction-tuned models; grey: pre-trained models. Dark blue and dark grey bars indicate pooled estimates across all instruction-tuned and all pre-trained models, respectively (Extended Data Table 1). Error bars indicate 95% confidence intervals. a , Female vs. male. b , Black vs. White. c , Disabled vs. non-disabled. Here, we show, across 27 models and 177 occupations covering nearly half of U.S. employment 33 , 41 , 42 , 43 (Extended Data Fig. 1), that language models incorporate demographics when asked to make hiring decisions: holding qualifications fixed, female and Black candidates are more likely to be hired, while disabled candidates are less likely to be hired (Fig. 1). Each model chooses which of two candidates would be better to hire: the two candidates are paired with independently sampled qualifications and explicitly disclosed demographic profiles, isolating the causal effect of demographic identity on hiring decisions (Methods). We find alignment is the primary driver of demographic effects: comparing nine sets of models that share the same architecture but differ in whether they are pre-trained and post-trained, we find that post-training alignment amplifies advantages for female and Black candidates by 325% and 330%, and amplifies the disability disadvantage by 171%, while also increasing qualification responsiveness by 218–444% (Methods III; Extended Data Table 2). These effects vary with occupation characteristics: female and Black advantages are largest in higher-wage occupations, and the disability disadvantage is largest when disabled workers are most prevalent (Fig. 2). Compared with previous human correspondence studies 8 , 29 , 38 , 4 , 31 , language models reverse the direction of racial discrimination, substantially attenuate the disability penalty, but amplify the female advantage (Fig. 3). Investigating the mechanisms, we find that language models’ underlying behaviour is consistent with statistical discrimination documented among human employers 2 , 37 , 6 , 10 : models reward the same qualifications more generously when held by historically-marginalised-group candidates (differential returns), and the absence of qualification signals disproportionately harms marginalised-group candidates (information asymmetry). We find that alignment amplifies the returns to general skills for all three marginalised groups relative to their counterparts and amplifies the returns to work experience for female and Black candidates but not disabled candidates, while introducing information-asymmetry effects that impose higher penalties on marginalised-group candidates, with the largest penalty on disability. 2 Asymmetric effects across marginalised groups We measure the hiring advantage or penalty associated with each demographic group by regressing pairwise decisions on candidate attribute differences (Methods III; Supplementary Information, ‘Derivations’). Every instruction-tuned model favours female candidates over otherwise identical male candidates, and all but two favour Black candidates (Fig. 1). The effects of disability show no comparable pattern: the coefficients are heterogeneous in sign, and the average effect is significantly negative. Pre-trained models show the same directional asymmetry but with smaller magnitudes. Pooling across all instruction-tuned models, the aggregate specification reveals significant advantages for female ( +0.115 , P<0.001 ) and Black ( +0.077 , P<0.001 ) candidates and a significant penalty for disabled candidates ( -0.058 , P<0.001 ; Extended Data Table 1). In odds terms, the female advantage corresponds to roughly 12% higher hiring odds and the Black advantage to 8%—effects equivalent to approximately one year of additional education for gender and seven months for race (one additional year of education increases hiring odds by 13%)—while the disability penalty implies roughly 6% lower odds. Beyond demographic effects, instruction-tuned models are far more responsive to qualifications: pre-trained models are near-random (pseudo R^{2}\approx 0 ), whereas instruction-tuned models achieve markedly better fit (pseudo R^{2}=0.28 ; Extended Data Table 1), with all qualification coefficients increasing three- to fivefold—patterns robust to nonparametric specification (Methods III). The asymmetric advantages thus emerge alongside a dramatic increase in qualification responsiveness, suggesting alignment drives both. We test this with nine matched instruct–base pairs that share the same architecture (Extended Data Table 2). Alignment amplifies the female advantage by 325% ( +0.094 , P<0.001 ) and the Black advantage by 330% ( +0.073 , P<0.001 ), while worsening the disability penalty by 171% ( -0.049 , P<0.001 ). Simultaneously, alignment increases all qualification weights by 218–444%, transforming pre-trained models from near-random decision-makers into systems highly responsive to candidate qualifications. Another important question is whether these demographic effects interact; for instance, whether candidates belonging to multiple historically-marginalised groups face compounding advantages or penalties beyond the sum of individual effects. We test this by adding pairwise interaction terms (female \times Black, female \times disabled, Black \times disabled) and find no significant interactions in either instruction-tuned or pre-trained models (Extended Data Table 3), suggesting that effects are close to additive. 3 Cross-occupation variation and human comparison Figure 2: Cross-occupation variation in demographic effects. Each point represents one of 177