AmharicStoryQA : A Multicultural Story Question Answering Benchmark in Amharic
Israel Abebe Azime, Abenezer Kebede Angamo, Hana Mekonen Tamiru, Dagnachew Mekonnen Marilign, Philipp Slusallek, Seid Muhie Yimam, Dietrich Klakow · 2026
Read AmharicStoryQA : A Multicultural Story Question Answering Benchmark in Amharic by Israel Abebe Azime, Abenezer Kebede Angamo, Hana Mekonen Tamiru, Dagnachew Mekonnen Marilign, Philipp Slusallek, Seid Muhie Yimam, Dietrich Klakow (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02774.txt.
Description
Israel Abebe Azime and 6 co-authors build a story question-answering benchmark in Amharic that treats culture as more than language, in a 2026 paper. Multilingual evaluations often equate a language with one culture and read model scores as language understanding. Narratives from different Ethiopian regions share a language yet differ in content, and those differences move evaluation results.
AmharicStoryQA grounds long-sequence questions in culturally diverse stories from Amharic-speaking regions. Testing existing models reveals a significant narrative understanding gap. Regional differences in scores are pronounced, and supervised fine-tuning improves performance unevenly across regions and settings. The authors argue benchmarks must be culturally grounded within a language, not just translated into it. Low-resource language evaluation, they show, hides its hardest variation at the regional level.
Catalog record
| CREATOR | Israel Abebe Azime, Abenezer Kebede Angamo, Hana Mekonen Tamiru, Dagnachew Mekonnen Marilign, Philipp Slusallek, Seid Muhie Yimam, Dietrich Klakow |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02774