AmharicStoryQA : A Multicultural Story Question Answering Benchmark in Amharic

Israel Abebe Azime, Abenezer Kebede Angamo, Hana Mekonen Tamiru, Dagnachew Mekonnen Marilign, Philipp Slusallek, Seid Muhie Yimam, Dietrich Klakow · 2026

Read AmharicStoryQA : A Multicultural Story Question Answering Benchmark in Amharic by Israel Abebe Azime, Abenezer Kebede Angamo, Hana Mekonen Tamiru, Dagnachew Mekonnen Marilign, Philipp Slusallek, Seid Muhie Yimam, Dietrich Klakow (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02774.txt.

Description

Israel Abebe Azime and 6 co-authors build a story question-answering benchmark in Amharic that treats culture as more than language, in a 2026 paper. Multilingual evaluations often equate a language with one culture and read model scores as language understanding. Narratives from different Ethiopian regions share a language yet differ in content, and those differences move evaluation results.

AmharicStoryQA grounds long-sequence questions in culturally diverse stories from Amharic-speaking regions. Testing existing models reveals a significant narrative understanding gap. Regional differences in scores are pronounced, and supervised fine-tuning improves performance unevenly across regions and settings. The authors argue benchmarks must be culturally grounded within a language, not just translated into it. Low-resource language evaluation, they show, hides its hardest variation at the regional level.

Catalog record

CREATORIsrael Abebe Azime, Abenezer Kebede Angamo, Hana Mekonen Tamiru, Dagnachew Mekonnen Marilign, Philipp Slusallek, Seid Muhie Yimam, Dietrich Klakow
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02774

Browse the full library · About the Tower of Babel