Behavioral Engagement in VR-Based Sign Language Learning: Visual Attention as a Predictor of Performance and Temporal Dynamics

Davide Traini, José Manuel Alcalde-Llergo, Mariana Buenestado-Fernández, Domenico Ursino, Enrique Yeguas-Bolívar * · 2026

Read Behavioral Engagement in VR-Based Sign Language Learning: Visual Attention as a Predictor of Performance and Temporal Dynamics by Davide Traini, José Manuel Alcalde-Llergo, Mariana Buenestado-Fernández, Domenico Ursino, Enrique Yeguas-Bolívar * (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19535.txt.

Introduction

Traini and co-authors study engagement in SONAR, a virtual reality app for learning sign language. Three measures were tracked: visual attention, video replay frequency, and post-playback viewing time. Learners trained at their own pace, then took a validation quiz. Visual attention correlated strongly with quiz scores, and viewing time also helped. Replay frequency showed no real link to success. Attention traces peaked at dense segments of the videos. The results stress steady, well-placed attention in immersive learning.

Catalog record

CREATORDavide Traini, José Manuel Alcalde-Llergo, Mariana Buenestado-Fernández, Domenico Ursino, Enrique Yeguas-Bolívar *
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70112
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19535

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Behavioral Engagement in VR-Based Sign Language Learning: Visual Attention as a Predictor of Performance and Temporal Dynamics Davide Traini, José Manuel Alcalde-Llergo, Mariana Buenestado-Fernández, Domenico Ursino, Enrique Yeguas-Bolívar * Abstract Understanding how learners engage with immersive sign language training environments is essential for advancing virtual reality-based education and inclusion. This study analyzes behavioral engagement in SONAR, a virtual reality application designed for sign language training and validation. We focus on three automatically derived engagement indicators (Visual Attention (VA), Video Replay Frequency (VRF), and Post-Playback Viewing Time (PPVT)) and examine their relationship with learning performance in a sample of 117 university students. Participants completed a self-paced Training phase with 12 sign language instructional videos, followed by a Validation quiz assessing retention. We employed Pearson correlation analysis to examine the relationships between engagement indicators and quiz performance, followed by binomial Generalized Linear Model (GLM) regression to assess their joint predictive contributions. Additionally, we conducted temporal analysis by aggregating moment-to-moment VA traces across all learners to characterize engagement dynamics during the learning session. Results show that VA exhibits a strong positive correlation with quiz performance (r = 0.76),followed by PPVT (r = 0.66), whereas VRF shows no meaningful association. A binomial GLM confirms that VA and PPVT are significant predictors of learning success, jointly explaining a substantial proportion of performance variance ( p ​ s ​ e ​ u ​ d ​ o − R 2 pseudo-R^{2} = 0.83). Going beyond outcome-oriented analysis, we characterize temporal engagement patterns by aggregating moment-to-moment VA traces across all learners. The temporal profile reveals distinct attention peaks aligned with informationally dense segments of both training and validation videos, as well as phase-specific engagement dynamics, including initial acclimatization, oscillatory attention cycles during learning, and pronounced attentional peaks during assessment. Together, these findings highlight the central role of sustained and strategically allocated visual attention in VR-based sign language learning and demonstrate the value of behavioral trace data for understanding and predicting learner engagement in immersive environments. keywords virtual reality; sign language education; behavioral engagement; visual attention; learning analytics; head-gaze-tracking ; VR learning environments; immersive learning † † firstpage: 1 † † volume: 1 † † issue: 1 † † articlenumber: 0 † † year: 2026 † † copyright-year: 2026 † † external-editor: Firstname Lastname † † received: 22 January 2026 † † revised: 19 February 2026 † † address: 1 Department of Information Engineering, Università delle Marche, Ancona, 60121, Italy; [email protected] (D.T.); [email protected] (D.U.) 2 Department of Computer Science and AI, Universidad de Córdoba, Córdoba, 14071, Spain; [email protected] 3 Department of Education, Universidad de Córdoba, Córdoba, 14071, Spain; [email protected] † † corresponding: Correspondence: [email protected] † † reftitle: References \DeclareCaptionLabelFormat subcaptionlabel( #2 ) 1 Introduction Virtual reality (VR) has demonstrated significant potential for enhancing learning outcomes across diverse educational domains Yu and Xu 2022 . The immersive nature of VR enables embodied, multimodal learning experiences particularly suited for skill-based domains such as sign language instruction Wang et al. 2024 , where learners must simultaneously encode handshape, orientation, movement, location, and facial expressions. Recent systematic reviews confirm that VR-based sign language learning systems produce measurable improvements in learning outcomes and engagement compared to conventional instruction Berrezueta-Guzman et al. 2025 . Despite these promising developments, a critical gap persists in the literature: while studies document the general effectiveness of VR sign language systems, they have not systematically examined how objective behavioral engagement indicators extracted from learner interactions relate to user engagement. Contemporary VR learning research emphasizes behavioral metrics derived from built-in tracking: head and hand movements, movement entropy, and exploration patterns. These metrics can be used as synchronous indicators of engagement and intrinsic motivation, providing real-time insight into cognitive states Mallek et al. 2024 . Indeed, these behavioral indicators are scalable, continuously available from the learning platform, and can support real-time adaptive feedback without additional instrumentation. The present study addresses this gap by proposing the following research questions: (i) Which specific behavioral indicators, automatically captured during VR interaction, predict learning success in sign language training? and (ii) Do these indicators evolve dynamically throughout the learning process? Addressing these questions is essential for developing adaptive VR learning systems that can provide real-time support based on learner engagement patterns. Specifically, we examined how three indicators of behavioral engagement, namely Visual Attention (VA), Post-Playback Viewing Time (PPVT), and Video Replay Frequency (VRF), correlate with learning performance, and we employed a Generalized Linear Model (GLM) regression to evaluate their combined predictive power. Finally, we analyzed how VA changes over the course of the interaction with the VR sign language application. Our framework is grounded in established learning science theories: eye-tracking research establishing gaze duration as a proxy for cognitive engagement Abeysinghe et al. 2025 ; Adhanom et al. 2023 , cognitive load theory identifying fixation patterns as indicators of information processing Ekin et al. 2025 , and self-regulated learning theory showing that voluntary content revisits reflect metacognitive strategies Wang et al. 2022 ; Galbraith 2004 . Specifically, this study examines learner engagement in SONAR (the interested reader can find the application at the following link: https://sidequestvr.com/app/37864/ (accessed on 1 March 2026) ), a VR application developed within the ISENSE project for sign language training. This application places users in the role of a hearing student interacting with a deaf avatar on a simulated university campus during the first day of class. The participants, 117 university students, completed a self-paced Training phase that allowed video replays, followed by a Validation quiz aimed at assessing retention. We then evaluate whether the engagement indicators are associated with learning success by relating them to quiz performance through correlational analyses and regression-based modeling. In addition, we move beyond aggregate measures by examining how visual attention changes over time throughout the interaction, highlighting temporal engagement patterns across the learning and assessment phases. Our principal findings are threefold: (i) VA and PPVT are strong, statistically significant predictors of learning performance, with VA emerging as the dominant indicator; (ii) VRF shows negligible correlation with performance, suggesting that engagement quality supersedes repetition quantity; (iii) temporal analysis reveals structured engagement patterns with characteristic peaks during training and assessment phases, confirming the task-driven nature of the observed attention patterns. These findings extend recent work on learning analytics into sign language VR contexts Tao et al. 2025 . Practically, these metrics can be automatically computed to enable real-time attention-aware learning analytics, supporting adaptive instructional interventions such as refocusing prompts or targeted content review. The remainder of this paper is organized as follows. Section 2 reviews literature on VR for education, VR-based sign language instruction, and engagement measurement in immersive environments. Section 3 describes the SONAR VR application. Section 4 formally introduces the engagement framework and research questions. Section 5 presents the study design and data collection procedures. Section 6 reports quantitative analyses addressing the research questions. Section 7 interprets findings in the context of prior work and discusses limitations. Section 8 summarizes contributions and implications for adaptive VR learning systems. 2 Related Work This section positions our work in relation to existing research on VR-based education, inclusion, and engagement measurement, with a specific focus on sign language learning. Specifically, in Section 2.1 , we review research on VR for education, highlighting evidence on learning gains, motivational benefits, and

Read the complete text