On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena

Tarek Naous, Wei Xu · 2025

Read On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena by Tarek Naous, Wei Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-04662.txt.

Description

Tarek Naous and Wei Xu trace where cultural bias in language models comes from in a 2025 paper. Models often favor Western-linked entities, even when they work in a non-Western tongue. The pair built a test set to hunt for the roots of that tilt.

Their bench pairs Arabic and English across more than 58,000 entities from Arab and Western cultures, set in natural masked sentences. Models did better in English than in Arabic. Trouble grew for Arabic words with many senses and for words shared with other tongues that use the same script. The way text is chopped into tokens made the flaw worse.

Catalog record

CREATORTarek Naous, Wei Xu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-04662

Browse the full library · About the Tower of Babel