A Japanese Benchmark for Evaluating Social Bias in Reasoning Based on Attribution Theory
Taihei Shiotani, Masahiro Kaneko, Naoaki Okazaki · 2026
Read A Japanese Benchmark for Evaluating Social Bias in Reasoning Based on Attribution Theory by Taihei Shiotani, Masahiro Kaneko, Naoaki Okazaki (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-00568.txt.
Description
Shiotani and colleagues build a benchmark for social bias in Japanese reasoning, built on attribution theory. Most benchmarks for Japanese translate English data, which may miss local contexts. They also judge bias only in answers, missing bias hidden in the reasoning itself. The new dataset, JUBAKU-v2, fixes the ending and tests how acts get assigned to in-groups and out-groups during reasoning. With 216 cases showing local biases, tests verify it can detect score gaps across models.
Catalog record
| CREATOR | Taihei Shiotani, Masahiro Kaneko, Naoaki Okazaki |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 15291 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.00568