Investigating the Effect of Encumbrance on Gaze- and Touch-based Target Acquisition on Handheld Mobile Devices DOI: XXXXXXX.XXXXXXX Conference: ; ; CCS: Human-centered computing Human computer interaction (HCI) CCS: Human-centered computing Interaction techniques CCS: Human-centered computing Empiri

Omar Namnakani, Yasmeen Abdrabou, John H. Williamson, Mohamed Khamis · 2025

Read Investigating the Effect of Encumbrance on Gaze- and Touch-based Target Acquisition on Handheld Mobile Devices DOI: XXXXXXX.XXXXXXX Conference: ; ; CCS: Human-centered computing Human computer interaction (HCI) CCS: Human-centered computing Interaction techniques CCS: Human-centered computing Empiri by Omar Namnakani, Yasmeen Abdrabou, John H. Williamson, Mohamed Khamis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-16472.txt.

Introduction

Omar Namnakani and three co-authors study hands-full phone use in a 2025 arXiv paper. People often operate handheld devices while carrying things, walking, or otherwise encumbered. Gaze input could help, but nobody had measured how encumbrance affects it.

In a study with 24 participants, the team compared gaze with dwell time, gaze plus touch, and one- or two-handed touch, each with and without visual feedback where relevant. Touch was generally faster, but gaze, especially with feedback, performed consistently whether participants were encumbered or not. Preferences flipped with circumstance: people favored gaze while encumbered and touch when free-handed. The findings guide designers choosing input methods for mobile interfaces that must handle situational impairments gracefully.

Catalog record

CREATOROmar Namnakani, Yasmeen Abdrabou, John H. Williamson, Mohamed Khamis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-16472

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Investigating the Effect of Encumbrance on Gaze- and Touch-based Target Acquisition on Handheld Mobile Devices DOI: XXXXXXX.XXXXXXX Conference: ; ; CCS: Human-centered computing Human computer interaction (HCI) CCS: Human-centered computing Interaction techniques CCS: Human-centered computing Empirical studies in HCI Omar Namnakani, Yasmeen Abdrabou, John H. Williamson, Mohamed Khamis Abstract . The potential of using gaze as an input modality in the mobile context is growing. While users often encumber themselves by carrying objects and using mobile devices while walking, the impact of encumbrance on gaze input performance remains unexplored. To investigate this, we conducted a user study ( N = 24 N=24 ) to evaluate the effect of encumbrance on the performance of 1) Gaze using Dwell time (with/without visual feedback), 2) GazeTouch (with/without visual feedback), and 3) One- or two-hand touch input. While Touch generally performed better, Gaze, especially with feedback, showed a consistent performance regardless of whether participants were encumbered or unencumbered. Participants’ preferences for input modalities varied with encumbrance: they preferred Gaze when encumbered, and touch when unencumbered. Our findings enhance understanding of the effect of encumbrance on gaze input and contribute towards selecting appropriate input modalities in future mobile user interfaces to account for situational impairments. Figure 1 . We evaluate gaze- and touch-based target selection on handheld mobile devices while walking encumbered (carrying bags) and unencumbered. Besides using gaze alone, we examine multimodal interaction combining gaze for pointing and touch for selection, with touch input included as a baseline. Left: a participant walking while carrying two bags and interacting using gaze (Encumbered). Right: a participant walking and interacting using gaze without carrying bags (Unencumbered). 1. Introduction Gaze interaction on mobile devices is an emerging field of research that investigates the use of eye movements to operate smartphones and tablets ( 32 ) . It holds promise for becoming a natural part of everyday interaction ( 42 ; 11 ) Given the current advances in front-facing cameras and processing power of the said devices, eye tracking on handheld mobile devices is advancing ( 32 ) , with recent work exploring the use of gaze as an input modality on handheld mobile devices by evaluating gaze interaction techniques in the mobile context ( 45 ; 44 ; 38 ) . Using gaze as an input modality can be advantageous, particularly in situations where touch interaction is limited, such as while walking, holding objects, while wearing gloves, or when the screen is wet. ( 63 ; 29 ; 21 ) . Prior work showed that users move through the environment and carry objects daily, which encumbers them while walking and interacting with their mobile devices ( 47 ) . Although touch is the mainstream input modality for mobile devices ( 51 ) , walking while encumbered has been shown to negatively affect its performance, as accuracy decreases ( 46 ; 49 ) . Given that prior work suggested using alternative modalities depending on the mobile context ( 64 ) , advancing gaze as an alternative input when users are encumbered requires exploring the effect of encumbrance on its performance, as prior work suggested that new input modalities should be tested for encumbrance ( 48 ) . In this paper, we investigate the effects of encumbrance on Gaze input using dwell time, which is the most common gaze interaction technique evaluated in the literature ( 27 ; 42 ; 10 ; 26 ) . Dwell time was introduced to mitigate the Midas touch problem, and showed promising results in the mobile context, where users preferred it ( 38 ; 44 ) compared to other techniques, such as Pursuits ( 75 ; 14 ) and gestures ( 43 ; 30 ) . Besides using Dwell time, we also explore the impact of encumbrance on multimodal interaction, when using gaze in combination with touch (GazeTouch), as prior work suggested that gaze has the potential to complement other modalities ( 53 ) , where gaze is used for pointing and touch for selection ( 53 ; 54 ; 55 ) . Additionally, we incorporated touch input using two common postures as baselines: 1H-thumb (one-handed use, where the user holds the phone using their preferred hand and interacts with the thumb), and 2H-index (two-handed use, where one hand holds the phone and the other interacts with the index finger), enabling comparisons of users’ perceptions and preferences under different levels of encumbrance. In a user study with 24 participants, for a discrete selection task on a handheld mobile device, we evaluated the impact of encumbrance on the performance of three input modalities when participants performed selection while walking encumbered and walking unencumbered, in terms of selection time, task completion rate, error counts, perceived workload (NASA-TLX), perceived usability (Likert scale), and user preference. Participants used three input modalities 1) Gaze input using dwell time with visual feedback (Gaze-F) and without (Gaze), 2) touch when combined with gaze, where gaze is used for pointing and touch for manipulating, with visual feedback (GazeTouch-F) and without (GazeTouch), and 3) Touch input, one handed (1H-thumb) and two handed (2H-index). Based on statistical tests using ANOVA and Bayesian analysis, we found that the performance of both 1H-thumb and 2H-index was affected by encumbrance, where participants took a longer time to select, completed fewer tasks, and committed more errors when walking encumbered compared to when unencumbered. On the other hand, Gaze, especially with feedback (Gaze-F), showed comparable performance between walking encumbered and unencumbered. GazeTouch with and without feedback showed comparable selection time and error counts, but participants completed less tasks when encumbered compared to when unencumbered. Participants perceived GazeTouch-F, 1H-thumb, and 2H-index to cause significantly higher workload when encumbered than when unencumbered. Participants’ preferences for input modalities varied with encumbrance: they preferred Gaze when encumbered, and touch when unencumbered. Our findings enhance our understanding of the effect of encumbrance on gaze input and contribute towards selecting appropriate interaction techniques to counteract the problem of such encumbrance when interacting with handheld mobile devices. Contribution Statement This work provides two contributions. We quantify, for the first time, the effects of encumbrance and walking on gaze-based target selection for handheld mobile devices. Second, we present the first empirical results, showing that gaze may have the potential to be used for target selection on handheld mobile devices, especially when touch-based interaction is limited, highlighting the importance of not excluding gaze solely based on objective performance. 2. Background and Related Work Our work builds on prior research on gaze interaction on handheld mobile devices, touch-based interaction, multimodal gaze-based interaction, and on situational impairments induced by the mobile context. 2.1. Encumbrance, Mobility, and Situationally Induced Impairments to Touch Input on Handheld Mobile Devices Prior work has explored the impact of encumbrance, such as when users carry objects, a frequent situation in everyday life, on the performance of interaction techniques, for example, wrist rotation gestures ( 48 ) . The researchers suggested that designers should test their interaction techniques with a range of different encumbrances to ensure that they are useful in real-world contexts. Following that and given the popularity of touch screen mobile devices, a plethora of other work focused on the performance of touch input on smartwatches and mobile devices when users are encumbered ( 46

Read the complete text