Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding

Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Grigori Sidorov, Dietrich Klakow, Philipp Slusallek, Olga Kolesnikova, Seid Muhie Yimam · 2024

Read Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding by Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Grigori Sidorov, Dietrich Klakow, Philipp Slusallek, Olga Kolesnikova, Seid Muhie Yimam (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-17837.txt.

Description

Tadesse Destaw Belay and a large team tested language models in two thousand twenty-four on reading emotions in four Ethiopian languages. Emotion labels often overlap, since one text can hold joy and anger at once. This multi-label task is little studied in models, above all for low-resource languages.

They built EthioEmo, a data set for Amharic, Afan Oromo, Somali, and Tigrinya, and also tested an English set. They compared large models used cold or with a few examples against smaller tuned models. Even English results fell short, and the gap to low-resource languages was large. Scores varied by language and model type. The data set is shared in public.

Catalog record

CREATORTadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Grigori Sidorov, Dietrich Klakow, Philipp Slusallek, Olga Kolesnikova, Seid Muhie Yimam
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-17837

Browse the full library · About the Tower of Babel