Small Changes, Big Impact: Demographic Bias in LLM-Based Hiring Through Subtle Sociocultural Markers in Anonymised Resumes

Bryan Chen Zhengyu Tan 1,4 Shaun Khoo 2 Bich Ngoc Doan 3 Zhengyuan Liu 4 Nancy F. Chen 4 Roy Ka-Wei Lee 1,2 1 Singapore University of Technology and Design (SUTD) 2 GovTech Singapore 3 EPFL 4 Agency for Science, Technology and Research (A*STAR), Singapore · 2026

Read Small Changes, Big Impact: Demographic Bias in LLM-Based Hiring Through Subtle Sociocultural Markers in Anonymised Resumes by Bryan Chen Zhengyu Tan 1,4 Shaun Khoo 2 Bich Ngoc Doan 3 Zhengyuan Liu 4 Nancy F. Chen 4 Roy Ka-Wei Lee 1,2 1 Singapore University of Technology and Design (SUTD) 2 GovTech Singapore 3 EPFL 4 Agency for Science, Technology and Research (A*STAR), Singapore (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05189.txt.

Introduction

Bryan Chen Zhengyu Tan and team stress-test hiring fairness in language models using anonymized resumes carrying only subtle sociocultural markers. They expand 100 neutral resumes into 4100 variants across four ethnicities and two genders, differing solely in job-irrelevant markers like languages, activities, volunteering, and hobbies. Across 18 models, direct contrast and score-and-shortlist tests show demographics inferred with high accuracy and systematic favor for markers linked to Chinese and Caucasian males. Language markers alone suffice for ethnicity, hobbies and activities drive gender inference, and account prompts can paradoxically amplify bias.

Catalog record

CREATORBryan Chen Zhengyu Tan 1,4 Shaun Khoo 2 Bich Ngoc Doan 3 Zhengyuan Liu 4 Nancy F. Chen 4 Roy Ka-Wei Lee 1,2 1 Singapore University of Technology and Design (SUTD) 2 GovTech Singapore 3 EPFL 4 Agency for Science, Technology and Research (A*STAR), Singapore
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63443
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05189

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Small Changes, Big Impact: Demographic Bias in LLM-Based Hiring Through Subtle Sociocultural Markers in Anonymised Resumes Bryan Chen Zhengyu Tan 1,4 Shaun Khoo 2 Bich Ngoc Doan 3 Zhengyuan Liu 4 Nancy F. Chen 4 Roy Ka-Wei Lee 1,2 1 Singapore University of Technology and Design (SUTD) 2 GovTech Singapore 3 EPFL 4 Agency for Science, Technology and Research (A*STAR), Singapore Abstract Large Language Models (LLMs) are increasingly deployed in resume screening pipelines. Although explicit PII (e.g., names) is commonly redacted, resumes typically retain subtle sociocultural markers (languages, co-curricular activities, volunteering, hobbies) that can act as demographic proxies. We introduce a generalisable stress-test framework for hiring fairness, instantiated in the Singapore context: 100 100 neutral job-aligned resumes are augmented into 4100 4100 variants spanning four ethnicities and two genders, differing only in job-irrelevant markers. We evaluate 18 18 LLMs in two realistic settings: (i) Direct Comparison (1v1) and (ii) Score & Shortlist (top-scoring rate), each with and without rationale prompting. Even without explicit identifiers, models recover demographic attributes with high F1 and exhibit systematic disparities, with models favouring markers associated with Chinese and Caucasian males. Ablations show language markers suffice for ethnicity inference, whereas gender relies on hobbies and activities. Furthermore, prompting for explanations tends to amplify bias. Our findings suggest that seemingly innocuous markers surviving anonymisation can materially skew automated hiring outcomes. 1 Introduction Figure 1: Overview of the experimental framework. (A) We generate 100 neutral resumes and inject them with sociocultural markers to create 4100 demographic variants. (B) We test 18 LLMs using two evaluation settings: Direct Comparison (1v1) and Score & Shortlist . (C) Results reveal systematic disparities where ethnicity drives the macro-ranking (top vs. bottom tiers) and gender drives the micro-ranking (male advantage within tiers). The integration of Large Language Models (LLMs) into recruitment pipelines promises unprecedented efficiency and scalability, yet simultaneously raises critical fairness concerns that demand rigorous investigation ( 8 ; 15 ) . As organisations increasingly deploy AI-powered tools for resume screening, candidate ranking, and preliminary assessments, the potential for these systems to encode and amplify demographic biases has become a pressing societal issue ( 36 ; 19 ) . The consequences of biased hiring algorithms extend beyond individual candidates to perpetuate systemic inequalities in employment, reinforcing historical patterns of discrimination ( 64 ) . While extensive prior research has documented LLM biases against explicit demographic indicators, including names ( 42 ; 4 ; 27 ) , gender pronouns ( 29 ) , and racial identifiers ( 4 ) , real-world hiring systems increasingly employ anonymisation techniques to redact Personally Identifiable Information (PII) ( 5 ; 34 ; 32 ) . This practice, while well-intentioned, may provide a false sense of security: candidates’ resumes remain rich with subtle, ostensibly job-irrelevant sociocultural markers that can implicitly signal race, gender, and socioeconomic status ( 46 ; 33 ) . These markers (spoken languages, co-curricular activities, volunteering experiences, hobbies) constitute sociocultural proxies that survive conventional anonymisation. Our study investigates whether these subtle markers alone are sufficient to trigger demographic bias in state-of-the-art LLMs deployed in hiring contexts. We develop a generalisable experimental framework applicable to any cultural setting, which we instantiate within the Singapore context for several methodologically important reasons. Singapore’s multiethnic, multilingual society, comprising Chinese (74.3%), Malay (13.5%), Indian (9.0%), and other ethnic groups, including Caucasians ( 13 ) , provides a natural laboratory with clearly distinguishable sociocultural markers across demographic groups ( 38 ) . Furthermore, Singapore’s recent AI safety evaluations have highlighted the inadequacy of Western-centric bias testing ( 9 ) , underscoring the need for region-specific bias audits ( 25 ) . The framework we present is designed for adaptation: researchers can replicate our methodology by substituting appropriate demographic variations and cultural markers. Research Questions. We organise our investigation around three core research questions: RQ1 (Demographic Recoverability): Can LLMs infer gender and ethnicity from anonymised resumes containing only sociocultural markers, and which marker categories drive this inference? RQ2 (Outcome Bias): How do demographic markers alter hiring outcomes under two complementary evaluation paradigms: (a) Direct Comparison (a 1v1 forced-choice between two candidates) and (b) Score & Shortlist (independent scoring of a pool to find top-scoring candidates), and how sensitive are these outcomes to prompt variations? RQ3 (Demographic Disparities): Which demographic groups receive the most and least favourable treatment, and how do these disparities manifest across models and settings? Contributions. Our work makes three key contributions: (1) We introduce a generalisable evaluation framework (Figure 1 ) comprising 4100 resumes with controlled demographic markers that diagnoses hiring biases in LLMs via Direct Comparison (1v1) and Score & Shortlist , with prompt sensitivity analysis that surface distinct bias pathways. (2) We provide empirical evidence that LLMs can infer demographic stereotypes from subtle markers (language suffices for ethnicity; hobbies/activities drive gender inference) and that these inferences can propagate to hiring outcomes. (3) We demonstrate that prompting for decision rationale, as an interpretability and debiasing strategy, neither guarantees nor systematically improves fairness and can paradoxically amplify bias in certain models. 2 Related Work Bias in Large Language Models. The study of bias in NLP systems has evolved from foundational work on word embeddings ( 35 ; 21 ) to comprehensive audits of modern LLMs. Researchers have documented pervasive biases across gender ( 29 ; 1 ) , race ( 41 ; 40 ) , and intersectional dimensions ( 30 ; 58 ) . Recent surveys provide taxonomies of bias sources, evaluation metrics, and mitigation strategies ( 19 ; 17 ) . A particularly relevant strand of research examines implicit versus explicit biases in LLMs. 6 demonstrate that models passing explicit fairness tests can still harbour implicit biases detectable through indirect evaluation. 31 show that LLM agents reveal biases through their decisions rather than explicit statements, with more advanced models sometimes exhibiting greater implicit bias despite reduced explicit bias. This distinction is critical, as it suggests that surface-level safeguards may be insufficient, motivating our investigation into biases triggered by subtle, non-explicit signals. Bias in AI-Assisted Hiring. Algorithmic hiring tools have attracted significant scrutiny for their potential to perpetuate employment discrimination ( 8 ; 15 ; 59 ; 49 ) . 4 conducted a comprehensive audit of GPT-3.5, finding systematic race and gender biases in resume assessment and generation tasks. 27 evaluated LLMs on job-resume matching, discovering that while recent models have reduced explicit gender and race biases, implicit biases concerning educational background remain significant. 42 and 61 both found significant preferences for White-associated profiles over Black-associated counterparts. However, these studies largely rely on demographic perturbation via explicit PII (names), which overlooks the fact that such identifiers can be easily redacted. We instead examine a more resilient source of bias by asking whether LLMs are influenced by sociocultural proxies (e.g., hobbies, languages); implicit signals that persist even after anonymisation. Cultural and Sociocultural Bias. Beyond individual-level demographics, researchers have documented cultural biases in LLMs that favour Western, English-speaking perspectives ( 54 ; 50 ) . Within the hiring domain, 51 found systematic biases against Indian interview transcripts compared to UK ones, with disparities linked to linguistic features rather than explicit demographic signals. These findings (that bias can be triggered by subtle, culturally-linked patterns) directly motivate our decision to instantiate a generalisable

Read the complete text