Evaluation of a Sign Language Avatar on Comprehensibility, User Experience & Acceptability Conference: ACM International Conference on Intelligent Virtual Agents; September 16–19, 2025; Berlin, Germany ACM International Conference on Intelligent Virtual Agents (IVA Adjunct ’25), September 16–19, 202
Fenya Wasserroth, Eleftherios Avramidis, Vera Czehmann, Tanja Kojic, Fabrizio Nunnari, Sebastian Möller · 2025
Read Evaluation of a Sign Language Avatar on Comprehensibility, User Experience & Acceptability Conference: ACM International Conference on Intelligent Virtual Agents; September 16–19, 2025; Berlin, Germany ACM International Conference on Intelligent Virtual Agents (IVA Adjunct ’25), September 16–19, 202 by Fenya Wasserroth, Eleftherios Avramidis, Vera Czehmann, Tanja Kojic, Fabrizio Nunnari, Sebastian Möller (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05358.txt.
Introduction
Fenya Wasserroth and co-authors tested an adjustable sign language avatar in a 2025 study with expert users of German Sign Language. The avatar ran on a head-worn display, and users could adjust some settings. The team asked whether those controls would aid understanding, ease of use, and acceptance.
Users liked having controls, yet scores for understanding and ease stayed low in both modes. Missing mouth shapes and face cues, plus blurred hand shapes, held the system back. Stress ran higher with the adjustable form, and users found the hand gestures for settings hard to learn. The lesson is that avatars must be clear by default before personal controls can help.
Catalog record
| CREATOR | Fenya Wasserroth, Eleftherios Avramidis, Vera Czehmann, Tanja Kojic, Fabrizio Nunnari, Sebastian Möller |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05358
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Evaluation of a Sign Language Avatar on Comprehensibility, User Experience & Acceptability Conference: ACM International Conference on Intelligent Virtual Agents; September 16–19, 2025; Berlin, Germany ACM International Conference on Intelligent Virtual Agents (IVA Adjunct ’25), September 16–19, 2025, Berlin, Germany DOI: 10.1145/3742886.3756719 ISBN: 979-8-4007-1996-7/2025/09 35 CCS: Computing methodologies Natural language processing CCS: Applied computing Language translation CCS: Human-centered computing Accessibility systems and tools CCS: Human-centered computing Mixed / augmented reality CCS: Human-centered computing Natural language interfaces CCS: Human-centered computing User studies CCS: Human-centered computing Usability testing Fenya Wasserroth, Eleftherios Avramidis, Vera Czehmann, Tanja Kojic, Fabrizio Nunnari, Sebastian Möller Abstract . This paper presents an investigation into the impact of adding adjustment features to an existing sign language (SL) avatar on a Microsoft Hololens 2 device. Through a detailed analysis of interactions of expert German Sign Language (DGS) users with both adjustable and non-adjustable avatars in a specific use case, this study identifies the key factors influencing the comprehensibility, the user experience (UX), and the acceptability of such a system. Despite user preference for adjustable settings, no significant improvements in UX or comprehensibility were observed, which remained at low levels, amid missing SL elements (mouthings and facial expressions) and implementation issues (indistinct hand shapes, lack of feedback and menu positioning). Hedonic quality was rated higher than pragmatic quality, indicating that users found the system more emotionally or aesthetically pleasing than functionally useful. Stress levels were higher for the adjustable avatar, reflecting lower performance, greater effort and more frustration. Additionally, concerns were raised about whether the Hololens adjustment gestures are intuitive and easy to familiarise oneself with. While acceptability of the concept of adjustability was generally positive, it was strongly dependent on usability and animation quality. This study highlights that personalisation alone is insufficient, and that SL avatars must be comprehensible by default. Key recommendations include enhancing mouthing and facial animation, improving interaction interfaces, and applying participatory design. Figure 1. Avatar modification example Avatar modification example. † † cc-license: by 1. Introduction Over 70 million people worldwide are deaf and use more than 200 national sign languages (SLs) ( World Federation of the Deaf 2021 ) . For many, SL is their first and preferred language, and written text remains a less accessible second language. SL interpreters thus support communication between hearing individuals and deaf people. According to the Federal Association of Sign Language Interpreters in Germany, around 850 interpreters serve approximately 80,000 hearing-impaired individuals in Germany ( bgsd 2020 ) . As a result, there are everyday situations where no interpreter is available, complicating communication and limiting access—for example, to train announcements. SL avatars could help improve accessibility, making research in this area crucial. While one might argue displaying text subtitles on AR glasses could be sufficient, this overlooks the cultural and linguistic significance of SLs. Prioritising text over SL in assistive technologies risks further marginalising SL users and perpetuating a hearing-centric view of communication. If successful, communication barriers between deaf and hearing communities could be reduced long-term. These avatars are not intended to replace human interpreters, but to support the autonomy of deaf individuals in situations where interpreters are unavailable ( Krausneker and Schügerl 2022 ) . At the same time, augmented reality (AR) technology is evolving, finding applications in fields like surgery ( Kim et al. 2017 ) and automotive displays. AR can embed artificial information into the real environment—an advantage that can benefit SL avatars ( Yang et al. 2022a ) . In research led by hearing individuals, expertise from the deaf and hard-of-hearing community is particularly important ( Fox et al. 2023 ) . This study examines how avatar adjustments—such as scalable size and adjustable spatial positioning—affect comprehensibility, user experience, and acceptability in an AR SL assistant. A user study with a mixed-method approach was conducted that combined quantitative data from standardised questionnaires with qualitative responses from an interview with the participant. The following research questions guide the investigation: • Comprehensibility : Can adjustment options of a SL avatar improve the comprehensibility of the signed sentences? • User experience : Do adjustment options contribute to a better user experience of a SL avatar? • Acceptability : To what extent do the adjustable setting options contribute to the acceptability of a SL avatar? Are there differences in acceptability and, if so, can factors be identified that explain these differences? 2. Related work 2.1. Sign language avatars Research in this area focuses on three core themes: acceptance, qualitative user requirements, and technical development. Qualitative requirements include both the technical necessities of transmitting SL and the needs of the deaf and hard-of-hearing community. SL involves hand shapes, movement, location, facial expressions, and lip movements. These components must be animated and synchronised, and signs combined into fluid sentences ( Bernhard et al. 2022 ; Krausneker and Schügerl 2022 ) . Facial expressions and lip movements are crucial for understanding and emotional expression ( Kipp et al. 2011 ; Lacerda et al. 2023 ) . Presentation and control aspects are also being studied. Avatars may have comic-like or human-like appearances ( Aziz and Othman 2023 ) , though comic styles can be situationally appropriate ( Kipp et al. 2011 ) . More human-like avatars risk entering the "uncanny valley", reducing acceptance ( Mori et al. 2019 ; Bragg et al. 2019 ) . Personalisation is valued, but customisations that improve comprehension—like size or speed—are considered more important than appearance ( Nolte et al. 2023 ; Kipp et al. 2011 ) . Technically, avatars must convert spoken language into signed output. This involves automated translation and/or avatar synthesis—challenging, because SL is neither linear nor written. Methods involving motion capture (MoCap) or glosses are common but limited ( Wolfe et al. 2023 ) . Precise synchronisation of visual elements is required ( Wolfe et al. 2023 ; Aziz and Othman 2023 ) . The use of Augmented Reality for SL avatars has been suggested in order to allow deaf or hard of hearing people to see an interpreter next to the speaking person or other visual information simultaneously, without having to switch their gaze ( Nguyen et al. 2021b ; Nguyen et al. 2021a ; Luo et al. 2022 ) . Many efforts have focused on educational use cases ( Kožuh et al. 2015 ; Soogund and Joseph 2019 ; Quandt 2020 ; Aditama et al. 2021 ; Cabanillas-Carbonell et al. 2022 ; Yang et al. 2022b ; Liang et al. 2024 ) , most of them in prototypical stage with either no or only limited user evaluation. Among these, ( Yang et al. 2022b ) contains a user study similar to ours, showing that users prefer a full-body vs. a half-body signing AR avatar, with the possibility to adjust its position. The DFKI avatar for DGS ( Nunnari et al. 2023 ) , on which this work is based (Figure 1 ), is an animated AR SL interpreter currently running on Microsoft HoloLens 2. The version used for this experiment can sign six pre-recorded sentences related to public transport, developed within the AVASAG research project ( Nunnari et al. 2021 ) . Users can start animations via a menu after a brief loading time. The avatar’s size, 3D position, and orientation are adjustable.