Sentiment Analysis of German Sign Language Fairy Tales
Fabrizio Nunnari, Siddhant Jain, Patrick Gebhard · 2026
Read Sentiment Analysis of German Sign Language Fairy Tales by Fabrizio Nunnari, Siddhant Jain, Patrick Gebhard (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16138.txt.
Introduction
Nunnari, Jain, and Gebhard present a dataset and model for sentiment analysis of German sign language fairy tales. First, they analyze negative, neutral, and positive valence in fairy tale text segments using four models and majority voting, reaching inter-annotator agreement. Second, they extract face and body features from sign segments using MediaPipe. Finally, they train an XGBoost model on video features. Results show about 63 percent balanced accuracy. Analysis finds eyebrow, mouth, hip, elbow, and shoulder motion contribute, showing face and body matter equally for sentiment.
Catalog record
| CREATOR | Fabrizio Nunnari, Siddhant Jain, Patrick Gebhard |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30160 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16138
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sentiment Analysis of German Sign Language Fairy Tales Fabrizio Nunnari, Siddhant Jain, Patrick Gebhard Abstract We present a dataset and a model for sentiment analysis of German sign language (DGS) fairy tales. First, we perform sentiment analysis for three levels of valence (negative, neutral, positive) on German fairy tales text segments using four large language models (LLMs) and majority voting, reaching an inter-annotator agreement of 0.781 Krippendorff’s alpha. Second, we extract face and body motion features from each corresponding DGS video segment using MediaPipe. Finally, we train an explainable model (based on XGBoost) to predict negative, neutral or positive sentiment from video features. Results show an average balanced accuracy of 0.631. A thorough analysis of the most important features reveal that, in addition to eyebrows and mouth motion on the face, also the motion of hips, elbows, and shoulders considerably contribute in the discrimination of the conveyed sentiment, indicating an equal importance of face and body for sentiment communication in sign language. Keywords: German sign language, DGS, sentiment analysis, fairy tales, explainable AI, machine learning. 1 Introduction Sign languages (SLs) are the native language of 70 million people in the world [ 6 ] and they are the main communication languages among Deaf communities. SLs rely on hands, fingers, torso, shoulders, gaze, head motion and facial expressions to convey meaning. Although performance of SL translation pipelines is still rudimentary, some companies start offering online automated translation tools (e.g.: Signapse - https://www.signapse.ai , Migam - https://migam.ai , and Nagish - https://sign.mt ). The output of SL synthesis systems is however criticized as being too robotic and unexpressive, to the extent that the semantics of the message is not correctly communicated. One reason for that, is the lack of explicit support for the communication of “sentiments”, and “emotions” in SL translation systems. Expressing emotions in spoken communication (e.g., through facial expressions, hand gestures, and body postures) can be seen as an addition to the basic message conveyed via words. In contrast, in SL the whole body and face are already involved in the communication of the semantics of the message, and such a dichotomy between grammatical communication and emotion expression is not possible. Research on emotion expression in SL is still emerging (e.g., see the ExEmSiLa workshop [ 8 ] ), but it is carried on mainly by SL linguists through time-consuming manual annotation and human observations of video material, without the support of modern automated machine learning tools. In contrast, for written languages, sentiment analysis is a well established branch of natural language processing (NLP) that automatizes the inference of an emotion (e.g., happiness, sadness) or the valence (positive, neutral, negative). This paper investigates the automated sentiment analysis of SL. The goal being to deploy a computational model that can correctly infer valence from the analysis of SL video segments 1 1 1 We refer to a SL video clip conveying a semantic information as a segment , instead of sentence, because there is no exact correspondence between the two. . Valence (also known as Pleasure) is one of the three dimensions used to quantify emotions in the PAD space (Pleasure/Valence, Arousal, Dominance) [ 13 ] . In this first investigation, we target only the valence dimension. For this work, we rely on the DGS-Fabeln-1 dataset [ 15 , 16 ] , a parallel corpus of 7 German sign language fairy tales, consisting of 574 German text segments each associated with a video clip. Figure 1 shows an example. Figure 1: A frame from the first sentence of the tale The Hare and the Hedgehog from the DGS-Fabeln-1 corpus. The corresponding German sentence is “Der Hase und der Igel. Es war einmal: So fangen Märchen an. Ein Märchen ist eine sehr alte Geschichte. Dieses Märchen heißt: Der Hase und der Igel. Das Märchen geht so:” [ENG: “The Hare and the Hedgehog. Once upon a time: that’s how fairy tales begin. A fairy tale is a very old story. This fairy tale is called: The Hare and the Hedgehog. The fairy tale goes like this:”]. The corresponding video lasts about 15 seconds. We focus on fairy tales because they naturally cover the whole range of positive to negative valence, thus displaying a high degree of expressivity in sign language performance. This seems to be a better candidate with respect to the most popular dataset for DGS used in natural language processing, i.e., the RWTH-PHOENIX-2014-T corpus [ 2 ] , which is limited to the domain of weather forecast. To perform our tests, we first take advantage of the state-of-the-art performance of LLMs in sentiment classification [ 10 ] . Four publicly available large language models (LLMs) are used to extract sentiment labels from the text segments, and a final label is assigned through majority voting. Second, the MediaPipe library [ 12 ] is used to analyse SL videos and extract blend-shapes and landmarks, which are further elaborated to extract information about velocities, accelerations, mutual distances, accumulated distances, bone rotations, and peak frequency. Finally, XGBoost is used to build a model predicting sentiment from video features. The details are reported in section 3 . This paper brings the following three contributions. First , we release DGS-Fabeln-1-SE (Sentiment Estimation), which extends DGS-Fabeln-1 with: i) the sentimental labels predicted by the 4 LLMs and their majority voting, and ii) the MediaPipe motion data of all segments, including an extra set of features precomputed for the classification task. Second , we deploy an explainable prediction model, based on XGBoost, capable of predicting sentiment in DGS-Fabeln-1 video segments. Third , we report and comment on the list of the most important features for the classification of valence from SL videos. The DGS-Fabeln-1-SE dataset is publicly available at https://doi.org/10.5281/zenodo.18879038 . The code to process the dataset is available at https://github.com/DFKI-SignLanguage/LREC2026-DGS-Fabeln-1-SE 2 Related work 2.1 Sentiment analysis through LLMs Manual sentiment annotation is a time- and resource-consuming task. Given the increased reliability of automated sentiment analysis, we opted for a procedural annotation of the DGS-Fabeln-1 dataset. The system introduced by [ 7 ] is considered among the best for sentiment analysis on the German language. However, an early test on fairy tales showed a peculiar behavior, with more than 90% of the text chunks classified as “neutral”. A manual check on a few samples led us to consider such library inappropriate for this task. Indeed, Klähn et al. [ 10 ] report that while fine-tuned German BERT models achieve 94-96% F1 scores on contemporary German text, these models experience catastrophic performance degradation in other contexts, like historical German. In contrast, LLMs can generalize much better in unseen domains (zero-shot learning). For example, also Suter and Mecker [ 20 ] report that GPT 4 demonstrated a “substantial to near perfect agreement” with human annotators on German text classification. In our work, we relied on a combination of the latest available versions of four different LLMs (GPT5, Sonic, Mistral, GPT OSS 20B) and ran a majority voting to compensate for disagreements. 2.2 Sentiment Analysis on SLs [ 17 ] performed sentiment analysis on Indian sign language. They trained a mixed model using both video and text to predict labels in 4193 frames, singularly annotated for negative, neutral, and positive valence. They