See-Through Face Display for DHH People: Enhancing Gaze Awareness in Remote Sign Language Conversations with Camera-Behind Displays DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY Woodstock ’18: ACM
Kazuya Izumi, Akihisa Shitara, Yoichi Ochiai · 2025
Read See-Through Face Display for DHH People: Enhancing Gaze Awareness in Remote Sign Language Conversations with Camera-Behind Displays DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY Woodstock ’18: ACM by Kazuya Izumi, Akihisa Shitara, Yoichi Ochiai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03141.txt.
Introduction
Kazuya Izumi, Akihisa Shitara, and Yoichi Ochiai present a display system in a 2025 paper for remote sign language talks. Eye contact matters in signed conversation, but a normal video call breaks it: each person looks at the screen, not at the camera. Past fixes used software to shift the gaze or bulky mirror rigs.
Their design puts a camera behind a clear display. Each user looks at the partner's face on the screen while the camera films through it, so gaze lines up on both sides. A working video system was built and compared with standard video calls. The design cuts image warping and simplifies setup at home.
Catalog record
| CREATOR | Kazuya Izumi, Akihisa Shitara, Yoichi Ochiai |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-03141
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
See-Through Face Display for DHH People: Enhancing Gaze Awareness in Remote Sign Language Conversations with Camera-Behind Displays DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY Woodstock ’18: ACM Symposium on Neural Gaze Detection, June 03–05, 2018, Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Conference: 2025 Symposium on Eye Tracking Research and Applications; May 26–29, 2025; Tokyo, Japan 2025 Symposium on Eye Tracking Research and Applications (ETRA ’25), May 26–29, 2025, Tokyo, Japan DOI: 10.1145/3715669.3726848 ISBN: 979-8-4007-1487-0/2025/05 CCS: Human-centered computing Displays and imagers Kazuya Izumi, Akihisa Shitara, Yoichi Ochiai Abstract . This paper presents a sign language conversation system based on the See-Through Face Display to address the challenge of maintaining eye contact in remote sign language interactions. A camera positioned behind a transparent display allows users to look at the face of their conversation partner while appearing to maintain direct eye contact. Unlike conventional methods that rely on software-based gaze correction or large-scale half-mirror setups, this design reduces visual distortions and simplifies installation. We implemented and evaluated a videoconferencing system that integrates See-Through Face Display, comparing it to traditional videoconferencing methods. We explore its potential applications for Deaf and Hard of Hearing (DHH), including multi-party sign language conversations, corpus collection, remote interpretation, and AI-driven sign language avatars. Collaboration with DHH communities will be key to refining the system for real-world use and ensuring its practical deployment. Figure 1 . (a, b) Deaf and hard of hearing users communicate via sign language with a conversation partner displayed on the screen. (c) The See-Through Face Display consists of a transparent display and a camera placed directly behind it, so that the user’s face appears superimposed on the camera lens (reproduced from ( Izumi et al. 2024 ) ). † † cc-license: by 1. Introduction This position paper presents a discussion between the first author, who possesses expertise in gaze analysis and gaze interaction, and the second author, who is a Deaf person and has been conducting accessibility research for Deaf and hard of hearing (DHH). The paper focuses on accessibility applications that utilize gaze analysis and gaze interaction technologies for DHH. The paper discusses applications that utilize See-Through Face Display technology by Izumi et al ( Izumi et al. 2024 ) , because we aim to the application incorporated a gaze natural interactions in sign language conversation as future work’s goal. The application examples include applications for online sign language conversations among multiple Deaf people, AI-powered avatar assistance, and remote sign language interpretation services for Deaf in settings such as reception areas. Many studies on gaze analysis targeting DHH have conducted research, as reported in the survey paper by Agrawal and Peiris ( Agrawal and Peiris 2021 ) . Specifically, in sign language conversation, gaze shifts can mean indicators of starting to speak after a speaker changes ( Watanabe et al. 2011 ; Coates and Sutton-Spence 2001 ) . Furthermore, gaze interaction should be considered in UI design approaches such as “Sign Language User Interface (SUI)” ( Bragg et al. 2020 ) . However, current UIs and systems often fail to consider gaze interaction needs of the Deaf. This is likewise the case in display-based sign language conversation contexts, such as remote sign language interactions and sign language avatars, where the lack of direct eye contact remains a persistent challenge. On the other hand, early attempts to support eye contact in remote conversations date back to systems such as ClearBoard ( Ishii and Kobayashi 1992 ) , and this functionality has since been integrated into commercial products such as NVIDIA Maxine Eye Contact 1 1 1 https://developer.nvidia.com/blog/improve-human-connection-in-video-conferences-with-nvidia-maxine-eye-contact/ . Unlike these approaches, the display setup proposed by Izumi et al. features a smaller form factor, eliminates visual artifacts, and avoids image-processing techniques that can distort users’ gaze. This preservation of natural gaze relationships is particularly important in sign language conversation, where gaze cues play a critical role in turn-taking and engagement. Given these advantages, we propose that this system is well-suited for remote video-based sign language conversation and have accordingly developed a multi-party videoconferencing system based on the See-Through Face Display. In this paper, we introduce the developed system and present a comparison of user eye contact differences between our system and conventional videoconferencing systems. Furthermore, we discuss evaluations focusing on the gaze behavior and user experience of DHH users, as well as future applications such as dataset collection of gaze and sign language interactions and the potential utilization of AI-driven sign language avatars. 2. Related Work 2.1. Eye Contact in Remote Video Conversations Mutual gaze, or eye contact, is a key element of face-to-face communication and serves as an important channel for nonverbal signaling, shaping various aspects of social interaction ( Mazur et al. [n. d.] ) . In the context of remote video communication, however, the issue of lack of eye contact caused by the misalignment of the camera with a user’s on-screen face has been a subject of discussion in the Human-Computer Interaction community for some time ( Jaklič et al. 2017 ) . To address this issue, two major approaches have been proposed: hardware-based and software-based solutions. Notable hardware approaches employ half-mirrors or carefully positioned cameras to minimize parallax and achieve near-direct eye contact ( Ishii and Kobayashi 1992 ; Okada et al. 1994 ; Otsuka 2016 ) . These methods preserve natural gaze cues with minimal visual distortion. However, these methods require specialized equipment, involve complex setups, and often lack scalability for everyday use. Software-based approaches include switching the video viewpoint based on head pose estimation ( Yang and Zhang 2002 ; Vertegaal et al. 2003 ) and manipulating eye regions in the video through image processing ( Wood et al. 2018 ; Wang et al. 2021 ; He et al. 2021 ) . While these methods do not require the complex setups characteristic of hardware-based approaches, these methods often introduce visual artifacts, such as misaligned eye regions or jittery gaze movements, which can reduce perceived authenticity and engagement. The See-Through Face Display bridges the gap between hardware and software solutions by integrating a transparent display and an embedded camera into a compact, user-friendly design. Unlike traditional hardware-based approaches, it eliminates complex installations, and unlike software-based methods, it avoids noticeable visual distortions. The display determines each user’s position relative to others. This setup ensures that when two participants make eye contact, it appears to outside viewers that only these two individuals are exchanging glances. 2.2. Target to Deaf and Hard of Hearing Regarding gaze analysis for DHH, multiple studies have reported and there is a summarized survey paper ( Agrawal and Peiris 2021 ) that compiles these findings. In general, gaze related elements are sometimes used as grammatical components in sign language ( Garcia and Sallandre 2020 ) . In addition, gaze plays an important role during speaker transitions and the initiation of utterances ( Watanabe et al. 2011 ; Coates and Sutton-Spence 2001 ) . Thompson et al. also reported relationships between eye gaze and verb agreement in sign language. ( Thompson et al. 2006 ) . However, issues with gaze interaction for DHH have been noted, including the mismatch between gaze direction and spatial position in