A Real-Time Error Prevention System for Gaze-Based Interaction in Virtual Reality Based on Anomaly Detection DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Huma
Björn R. Severitt, Yannick Sauer, Nora Castner, Siegfried Wahl · 2026
Read A Real-Time Error Prevention System for Gaze-Based Interaction in Virtual Reality Based on Anomaly Detection DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Huma by Björn R. Severitt, Yannick Sauer, Nora Castner, Siegfried Wahl (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-15146.txt.
Introduction
Björn R. Severitt, Yannick Sauer, Nora Castner, and Siegfried Wahl guard against wrong clicks made by the eyes alone. Their 2026 arXiv paper deals with gaze-based interaction in virtual reality. Gaze control is hands-free and feels natural, but stray looks can trigger inputs no one meant.
The team builds a real-time error prevention system. A temporal convolutional network autoencoder watches the gaze signal as it unfolds. It flags odd patterns in gaze dynamics during selection tasks. Flagged selections can then be stopped in time before they cause an error in the scene.
Catalog record
| CREATOR | Björn R. Severitt, Yannick Sauer, Nora Castner, Siegfried Wahl |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-15146
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Real-Time Error Prevention System for Gaze-Based Interaction in Virtual Reality Based on Anomaly Detection DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Human-centered computing Interaction techniques CCS: Human-centered computing Human computer interaction (HCI) CCS: Human-centered computing User interface management systems CCS: Human-centered computing User interface programming CCS: Human-centered computing Systems and tools for interaction design CCS: Human-centered computing Empirical studies in interaction design Björn R. Severitt, Yannick Sauer, Nora Castner, Siegfried Wahl Abstract . Gaze-based interaction enables intuitive, hands-free control in immersive environments, but remains susceptible to unintended inputs. We present a real-time error prevention system (EPS) that uses a temporal convolutional network autoencoder (TCNAE) to detect anomalies in gaze dynamics during selection tasks. In a visual search task in VR, 41 participants used three gaze-based methods—dwell time, gaze and head direction alignment, and nod—with and without EPS. The system reduced erroneous selections by up to 95% for dwell time and gaze and head, and was positively received by most users. Performance varied for nodding and between individuals, suggesting the need for adaptive systems. Objective metrics and subjective evaluations show that anomaly-based error prevention can improve gaze interfaces without disrupting interaction. These findings demonstrate the potential of anomaly-based error prevention for gaze interfaces and suggest applications in VR, AR, and assistive technologies. 1. Introduction Gaze-based interaction offers an intuitive and hands-free way to communicate with digital systems, especially in immersive environments and assistive technologies where traditional input devices are impractical ( Severitt et al. 2024a ; Sidenmark et al. 2021 ) . Despite its potential, gaze interaction still faces challenges in accurately interpreting the user’s intentions. A well-known issue is the Midas touch problem ( Jacob 1990 ) , in which unintended gaze behavior is mistakenly registered as intentional input. To mitigate such errors, various confirmation mechanisms have been proposed, including head gestures, dwell times, and combined gaze-head alignment ( Sidenmark and Gellersen 2019 ; Špakov and Majaranta 2012 ; Esteves et al. 2015 ) . However, even with these mechanisms, false selections persist and can undermine user trust—especially in collaborative or safety-critical scenarios ( Padovani et al. 2019 ; Flook et al. 2019 ) . Given previous findings that gaze behavior reflects signs of confusion or correction even before a selection is confirmed ( Severitt et al. 2024b ) , we focus our anomaly detection approach on the immediate temporal vicinity of the selection event to enable early error detection. In our previous work ( Severitt et al. 2025a ) , we presented a theoretical proof-of-concept for using anomaly detection to identify erroneous gaze-based selections, defining errors as statistical outliers in gaze dynamics. This concept was further substantiated by empirical results in ( Severitt et al. 2025b ) , where we showed that even with confirmation mechanisms, selection errors frequently occurred, negatively impacting the user experience. At the same time, the success of gaze-based interaction depends not only on technical accuracy, but also on the quality of the user experience. Error prevention mechanisms may be most effective when they work in the background, reducing errors without disrupting interaction. Ideally, users can continue to focus on their task while the system silently supports them, contributing to a more natural and trustworthy experience. From this perspective, both reliable anomaly detection and unobtrusive support are important. The goal of this work is therefore not only to investigate the accuracy of error prevention, but also to explore how such systems can contribute to smoother eye-based interaction, with potential implications for consumer and assistive technologies. On this basis, the present study evaluates a real-time eps ( eps ) based on an ae ( ae ) with a tcn ( tcn ), a model architecture that has proven effective for anomaly detection in time series data ( He and Zhao 2019 ) . The system is trained to reconstruct typical gaze patterns for correct selections, with deviations from this learned representation interpreted as potential errors. We implement and test the eps in a vr ( vr ) game using a visual search task and compare three gaze-based interaction methods—Dwell Time, Gaze and Head and Nod—with and without the system. Specifically, we address two central questions: (1) How do participants experience and evaluate the system in terms of usefulness, trust, and overall preference, and (2) can the eps reliably prevent errors in a real-time environment across different interaction methods? 2. Related Work Gaze-based interaction has become increasingly important in various fields, including assistive technologies ( Parisay et al. 2021 ; Istance et al. 2008 ; Rajanna and Hammond 2016 ) , ar ( ar )/ vr ( vr ) ( Mohan et al. 2018 ; Velloso et al. 2016 ; Gardony et al. 2024 ; Schweigert et al. 2019 ) , and hci ( hci ) in general ( Gemicioglu et al. 2023 ) . An important goal of these approaches is to reduce unintended selections, known as the Midas touch problem , in order to improve both the user experience and trust in the system. Recent studies also show that features derived from eye-tracking data can be used to infer user intentions ( David-John et al. 2021 ; Bovo et al. 2020 ) and detect interaction errors ( Severitt et al. 2023 ; Bovo et al. 2020 ; Bruder et al. 2016 ) , highlighting the potential of gaze as a powerful input modality. Recent studies have demonstrated the potential of machine learning to improve gaze-based input by contextualizing gaze behavior within the interaction environment. For example, Shevtsova et al. 2025 shows that machine learning techniques were applied to gaze data with free behavior in a visually complex game environment, resulting in a significant reduction in false alarms and an improvement in user satisfaction—all without explicit intention confirmation. Such results underscore the increasing suitability of data-driven methods for improving gaze-based interaction. In addition, sequence modeling techniques such as tcn were used to classify user errors and predict intentions directly from raw gaze histories, achieving performance well above chance levels ( Sendhilnathan et al. 2022 ) . Continuing this work, Sendhilnathan et al. 2025 demonstrated that incorporating multimodal inputs into the tcn framework further improves error classification. Although these approaches are effective, they are generally based on a fixed taxonomy of error classes, which makes it difficult to generalize them to previously unknown or unexpected interaction errors. In contrast, our approach reframes selection errors not as predefined classes, but as anomalies within the continuous temporal gaze signal. By treating errors as rare, statistically distinct events, we align our methodology with established techniques in time series anomaly detection, as surveyed by Schmidl et al. 2022 . In our previous work, we demonstrated a theoretical proof-of-concept for applying anomaly detection algorithms to gaze-based interaction data ( Severitt et al. 2025a ) . In this study, we extend that foundation by implementing and evaluating anomaly detection models on real gaze recordings from a target selection task in a vr environment. 3. Methods In this section, we explain the anomaly detection method we use for the eps and the ve ( ve ) to test this method, including the various gaze-based interaction methods.