A Japanese Benchmark for Evaluating Social Bias in Reasoning Based on Attribution Theory

Taihei Shiotani, Masahiro Kaneko, Naoaki Okazaki · 2026

Read A Japanese Benchmark for Evaluating Social Bias in Reasoning Based on Attribution Theory by Taihei Shiotani, Masahiro Kaneko, Naoaki Okazaki (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-00568.txt.

Description

Shiotani and colleagues build a benchmark for social bias in Japanese reasoning, built on attribution theory. Most benchmarks for Japanese translate English data, which may miss local contexts. They also judge bias only in answers, missing bias hidden in the reasoning itself. The new dataset, JUBAKU-v2, fixes the ending and tests how acts get assigned to in-groups and out-groups during reasoning. With 216 cases showing local biases, tests verify it can detect score gaps across models.

Catalog record

CREATORTaihei Shiotani, Masahiro Kaneko, Naoaki Okazaki
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15291
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.00568

Browse the full library · About the Tower of Babel