On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena
Tarek Naous, Wei Xu · 2025
Read On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena by Tarek Naous, Wei Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-04662.txt.
Description
Tarek Naous and Wei Xu trace where cultural bias in language models comes from in a 2025 paper. Models often favor Western-linked entities, even when they work in a non-Western tongue. The pair built a test set to hunt for the roots of that tilt.
Their bench pairs Arabic and English across more than 58,000 entities from Arab and Western cultures, set in natural masked sentences. Models did better in English than in Arabic. Trouble grew for Arabic words with many senses and for words shared with other tongues that use the same script. The way text is chopped into tokens made the flaw worse.
Catalog record
| CREATOR | Tarek Naous, Wei Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-04662