Unmasking Implicit Bias: Evaluating Persona-Prompted LLM Responses in Power-Disparate Social Scenarios
Bryan Chen Zhengyu Tan, Roy Ka-Wei Lee, [email protected] · 2025
Read Unmasking Implicit Bias: Evaluating Persona-Prompted LLM Responses in Power-Disparate Social Scenarios by Bryan Chen Zhengyu Tan, Roy Ka-Wei Lee, [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01532.txt.
Description
Bryan Tan and Roy Lee probe implicit bias in persona prompted language models in this 2025 paper. When a model is told to answer as a person of a given age, race, or politics, its replies can shift in quality. The authors measure those shifts two ways: semantic distance between replies, and a judged win rate for reply quality. They ran five models through one hundred social scenarios across nine demographic axes.
A default persona emerged: replies drifted toward the voice of a middle aged, able bodied, native born, Caucasian, atheist man with centrist views. Some demographic prompts drew lower quality replies. Power gaps widened the spread of both meaning and quality across groups. The authors argue that bias grows exactly where stakes are uneven. They offer the framework as a way to expose such drift before deployment.
Catalog record
| CREATOR | Bryan Chen Zhengyu Tan, Roy Ka-Wei Lee, [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01532