GazeSwipe: Enhancing Mobile Touchscreen Reachability through Seamless Gaze and Finger-Swipe Integration Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokoha

Zhuojiang Cai, Jingkai Hong, Zhimin Wang, Feng Lu · 2025

Read GazeSwipe: Enhancing Mobile Touchscreen Reachability through Seamless Gaze and Finger-Swipe Integration Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokoha by Zhuojiang Cai, Jingkai Hong, Zhimin Wang, Feng Lu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21094.txt.

Introduction

Zhuojiang Cai, Jingkai Hong, Zhimin Wang, and Feng Lu present a phone interaction method, in a paper published in 2025. Big phone screens are hard to use with one thumb, since far corners sit out of reach. Their method, GazeSwipe, pairs eye gaze with a finger swipe, so a look aims the touch and the swipe confirms it.

The system reads gaze from the phone's front camera, with no extra gear and no setup routine where users stare at dots. Swipes correct for gaze errors, and a quiet self-tuning step sharpens gaze accuracy as people use the phone. In tests on phones and tablets, the method hit high success rates and users preferred it over rivals. The design could make large phones friendlier for one-handed use, in transit or with a hand full.

Catalog record

CREATORZhuojiang Cai, Jingkai Hong, Zhimin Wang, Feng Lu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-21094

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GazeSwipe: Enhancing Mobile Touchscreen Reachability through Seamless Gaze and Finger-Swipe Integration Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/3706598.3713739 ISBN: 979-8-4007-1394-1/25/04 CCS: Human-centered computing HCI design and evaluation methods CCS: Human-centered computing Interaction techniques Zhuojiang Cai, Jingkai Hong, Zhimin Wang, Feng Lu Abstract . Smartphones with large screens provide users with increased display and interaction space but pose challenges in reaching certain areas with the thumb when using the device with one hand. To address this, we introduce GazeSwipe, a multimodal interaction technique that combines eye gaze with finger-swipe gestures, enabling intuitive and low-friction reach on mobile touchscreens. Specifically, we design a gaze estimation method that eliminates the need for explicit gaze calibration. Our approach also avoids the use of additional eye-tracking hardware by leveraging the smartphone’s built-in front-facing camera. Considering the potential decrease in gaze accuracy without dedicated eye trackers, we use finger-swipe gestures to compensate for any inaccuracies in gaze estimation. Additionally, we introduce a user-unaware auto-calibration method that improves gaze accuracy during interaction. Through extensive experiments on smartphones and tablets, we compare our technique with various methods for touchscreen reachability and evaluate the performance of our auto-calibration strategy. The results demonstrate that our method achieves high success rates and is preferred by users. The findings also validate the effectiveness of the auto-calibration strategy. Figure 1. To address the issue of thumb reachability on smartphones and tablets when held with one hand, we introduce the gaze-based pointing, enabling users to intuitively reach any position by simply gazing at the target. By combining this with the finger-touch input, users can precisely interact with targets through swipe gestures. Additionally, we introduce a user-unaware auto-calibration method that eliminates the need for explicit gaze calibration, enhancing gaze accuracy during use and making interactions more seamless and efficient. The basic concept of the proposed method, starting from the issue of one-handed reachability on smartphones. Initially, gaze estimation is introduced to achieve full-screen reachability, followed by precise target selection through touch compensation. Finally, the accuracy of gaze guidance is enhanced by incorporating our auto-calibration strategy. 1. Introduction In recent years, the rapid advancement of smartphone and tablet technology has been characterized by a significant increase in screen sizes. While these larger displays have undoubtedly enriched user experiences by providing more space for content presentation and interaction, they have also introduced a series of ergonomic challenges, particularly concerning one-handed usage ( Eardley et al. 2017 ; Kim et al. 2012 ; Ikematsu et al. 2020 ) . As screen sizes approach or exceed the 5.5-inch threshold, users find it increasingly difficult to reach certain areas of the screen with one hand ( Voelker et al. 2020a ) . This often requires frequent adjustments to the grip, making operations more cumbersome and time-consuming, and increasing the risk of accidental drops. Imagine you are holding your smartphone with one hand while the other is occupied by carrying a bag, holding onto a bus or subway handrail, or steering a shopping cart, and you need to tap a button at the top of the screen. Alternatively, picture yourself walking through the office, one hand holding a coffee cup while the other operates a tablet, or even writing on a whiteboard with one hand while trying to use a tablet with the other. In these situations, reaching distant parts of the screen becomes difficult and unstable. These scenarios highlight the reachability issues of large touchscreens when used with one hand, emphasizing the need for a more convenient and efficient solution. To address the reachability issue, several solutions have been proposed. Screen Transform Techniques, such as Sliding Screen ( Kim et al. 2012 ) , TiltReduction ( Chang et al. 2015 ) , and One-handed Mode features on various smartphones ( Huawei 2019 ) , adjust the user interface by scaling or shifting it to the lower areas of the screen, bringing content within the thumb’s comfortable reach. While these methods can enhance reachability and improve grip stability, they come at the cost of reducing the usable screen area. Scaling the interface often results in smaller elements, leading to imprecise operations, while shifting the interface to display only a part of the content makes it inconvenient for continuous operations across different regions of the screen. Moreover, since the thumb’s reachable area remains fixed regardless of screen size, the required degree of scaling or shifting increases with larger displays, making these techniques impractical for tablets. Another category of solutions, Cursor-based Techniques, such as BezelCursor ( Li and Fu 2013 ) , ForceRay ( Corsten et al. 2019 ) , and HeadReach ( Voelker et al. 2020a ) , enable users to control a virtual cursor to reach targets anywhere on the screen via touch. While these strategies avoid the limitations of Screen Transform Techniques, their interaction experiences are often less intuitive and require long thumb movements, making the interaction more effortful and time-consuming. Given that gaze information naturally and intuitively reflects user intent, increasing research is integrating it into mobile device interactions ( Nagamatsu et al. 2010 ; Dybdal et al. 2012 ; Khamis et al. 2018 ) . For example, gaze has been used to enhance mobile voice assistants, improve text editing efficiency, and facilitate cross-device collaboration ( Mayer et al. 2020 ; Sindhwani et al. 2019 ; Voelker et al. 2020b ) . CursorShift ( Pfeuffer and Gellersen 2016 ) achieved gaze and touch interaction on tablets in a controlled lab environment. However, this method requires expensive external eye-tracking devices and additional explicit calibration processes, making it impractical for everyday use on mobile devices and insufficient for effectively solving reachability problems. Recently, Apple has also introduced gaze tracking on iPhones and iPads as an accessibility feature for people with disabilities, allowing users to gaze at a target and dwell to confirm interactions ( Butler 2024 ) . However, this feature requires calibration before use, and after calibration, changes in head position may lead to inaccurate performance. Moreover, the reliance on gaze alone for unimodal interaction makes it susceptible to the Midas touch problem. Therefore, it cannot serve as a viable solution for the everyday reachability issue. To this end, we propose GazeSwipe, a multimodal interaction technique that combines eye gaze with finger-swipe gestures to enhance reachability on smartphones and tablets in an intuitive and low-effort manner. With GazeSwipe, users simply gaze at the target and use a thumb swipe to adjust the gaze cursor and complete the interaction. Our approach offers three key advantages: 1) it requires no additional hardware, utilizing the front-facing camera for gaze estimation; 2) it employs a user-unaware auto-calibration strategy, eliminating the need for explicit gaze calibration and improving gaze accuracy during use; and 3) Since gaze is fast and intuitive, while finger-swipe is subtle and effortless, the combination delivers a seamless and low-friction interaction experience for everyday use on smartphones and tablets. We conduct two user studies to evaluate our interaction technique. In the first study, we examine the impact of different calibration strategies on GazeSwipe. The results show that our auto-calibration strategy significantly improves gaze estimation accuracy while

Read the complete text