Implicit Search Intent Recognition using EEG and Eye Tracking: Novel Dataset and Cross-User Prediction Conference: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION; October 9–13, 2023; Paris, France INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (ICMI ’23), October 9–13, 2023, Paris, France DO
Mansi Sharma, Shuang Chen, Philipp Müller, Maurice Rekrut, Antonio Krüger · 2025
Read Implicit Search Intent Recognition using EEG and Eye Tracking: Novel Dataset and Cross-User Prediction Conference: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION; October 9–13, 2023; Paris, France INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (ICMI ’23), October 9–13, 2023, Paris, France DO by Mansi Sharma, Shuang Chen, Philipp Müller, Maurice Rekrut, Antonio Krüger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01860.txt.
Introduction
Mansi Sharma, Shuang Chen, and three colleagues read a person's search intent from brain waves and eye movements, in a paper published in two thousand twenty-five. Helpers, human or machine, need to know whether someone is merely glancing around a scene or hunting for a specific target.
Earlier studies combined EEG and eye tracking for this, but with two flaws. They forced fixed search times, unlike real searches that end when the target pops out. And their predictors needed training data from the very user being tested, which rarely exists in practice. The authors release a new dataset designed around natural search behavior. They then tackle cross-user prediction: judging a new person's intent from other people's data. Solving that makes intent-aware assistance deployable. The work was presented at a conference on multimodal interaction.
Catalog record
| CREATOR | Mansi Sharma, Shuang Chen, Philipp Müller, Maurice Rekrut, Antonio Krüger |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-01860
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Implicit Search Intent Recognition using EEG and Eye Tracking: Novel Dataset and Cross-User Prediction Conference: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION; October 9–13, 2023; Paris, France INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (ICMI ’23), October 9–13, 2023, Paris, France DOI: 10.1145/3577190.3614166 ISBN: 979-8-4007-0055-2/23/10 CCS: Human-centered computing Human computer interaction (HCI) Mansi Sharma, Shuang Chen, Philipp Müller, Maurice Rekrut, Antonio Krüger Abstract . For machines to effectively assist humans in challenging visual search tasks, they must differentiate whether a human is simply glancing into a scene ( navigational intent ) or searching for a target object ( informational intent ). Previous research proposed combining electroencephalography (EEG) and eye-tracking measurements to recognize such search intents implicitly, i.e., without explicit user input. However, the applicability of these approaches to real-world scenarios suffers from two key limitations. First, previous work used fixed search times in the informational intent condition - a stark contrast to visual search, which naturally terminates when the target is found. Second, methods incorporating EEG measurements addressed prediction scenarios that require ground truth training data from the target user, which is impractical in many use cases. We address these limitations by making the first publicly available EEG and eye-tracking dataset for navigational vs. informational intent recognition, where the user determines search times. We present the first method for cross-user prediction of search intents from EEG and eye-tracking recordings and reach 84.5 % 84.5\% accuracy in leave-one-user-out evaluations - comparable to within-user prediction accuracy ( 85.5 % 85.5\% ) but offering much greater flexibility. Figure 1 . Examples of the visual scenes we created for our study. In total, we created 120 unique scenes. 1. Introduction Visual search is ubiquitous in daily life. For example, searching for a desired chocolate bar on a supermarket shelf or the wrench in a cluttered workshop. Approaches that can automatically - and without explicit user input - infer humans’ search targets have the potential to assist humans and avoid unnecessary frustration or delays ( Barz et al. 2020 ; Sattar et al. 2015 ) . For an assistive system to effectively help humans in visual search tasks, it has to solve one key problem before even considering which target the user might be searching for: the system needs to be able to recognize that the user is searching at all (i.e., having an informational intent ) and not simply looking at the scene with no such purpose in mind (called navigational intent in the literature) ( Jang et al. 2014b ; Jang et al. 2014a ; Sharma et al. 2023 ) . Previous studies on the recognition of navigational versus informational intent from EEG and eye-tracking data proved the general feasibility of this task ( Jang et al. 2014b ; Jang et al. 2014a ; Park et al. 2014 ; Kang et al. 2015 ) . However, they suffer from limitations concerning the study design and the prediction scenarios that reduce their applicability to the real world. Using fixed time limits (usually 5 s) in visual search tasks is a common practice in previous studies ( Kang et al. 2015 ; Park et al. 2014 ) , as it allows for better control over the experiment and ensures that all users complete the task in a consistent amount of time. However, this approach does not allow for the evaluation of unrestricted search duration, where user has no time constraints. This type of design allows for a more naturalistic approach to studying visual search behavior. It can provide insight into how participants prioritize search strategies and allocate their attention over time. Concerning the prediction scenario, previous studies on multi-modal navigational vs. informational intent recognition from EEG and eye-tracking are limited to cross-user scenarios where data from a given user can appear both in the test- and training sets ( Park et al. 2014 ) . In contrast, practical scenarios often require a model to apply to unseen target users without needing to collect training data. A final but crucial limitation to achieve progress in search intent recognition based on EEG and eye-tracking is the need for a publicly available dataset that will provide us valuable insights into real-world search behavior and help us generalize our intent recognition models across a larger group of users. Our work addresses these shortcomings by proposing a novel EEG- and eye-tracking dataset for navigational vs. informational intent classification where the search duration depends entirely on the time it takes to find the target. We intentionally select the industrial workplace scenarios where visual search often helps workers quickly and accurately identify equipment, components, and tools, which can improve efficiency, safety, and productivity. Furthermore, we propose the first multi-modal approach to the best of our knowledge for navigational vs. informational intent prediction in a strict leave-one-user-out evaluation scenario. Our experiments reveal that feature selection is crucial in achieving high performance across users, unlike within-user prediction. Our specific contributions are threefold: (1) We present MindGaze 1 1 1 https://doi.org/10.5281/zenodo.8239061 , the first publicly available dataset for informational vs. navigational intent prediction with a large variety of different workplaces in Unity ( Haas 2014 ) (see Figure 1 for examples). The dataset consists of EEG and eye-tracking recordings of 15 participants, performing 3600 trials. (2) We present the first approach for cross-user (leave-one-user-out) prediction of navigational vs. informational intent from EEG and eye-tracking, revealing that appropriate choice of features plays a crucial role in cross-user prediction. (3) We conduct extensive evaluations for within-user and cross-user scenarios and compare different multi-modal fusion strategies (early, late, and hybrid fusion). Subsequently, we perform experiments with the smaller windows of 0.5 s, 1 s, 1.5 s, and 2 s to show the potential for near real-time intent prediction applications. Figure 2 . (Left) Navigational scan path, (Right) Informational scan path, search target: Screwdriver 2. Related Work The implicit (i.e., without explicit user input) prediction of user intents are of great interest in human-machine interaction (HMI), as it can help to adapt the behavior of machines without the overhead and discomfort associated with explicit input. Implicit intent recognition systems often rely on challenging-to-interpret measurements like eye-tracking, pupil dilation, or EEG ( Majaranta and Bulling 2014 ; Zander et al. 2014 ) . Previous research worked on implicitly predicting web users’ click intents ( Slanzi et al. 2017 ) , search targets ( Sattar et al. 2015 ; Barz et al. 2020 ) , improving the performance of Motor Imagery task ( Cheng et al. 2020 ) their next focus of attention ( Müller et al. 2020 ; Ward et al. 2016 ; Steil et al. 2018 ) , or choice of ingredients ( Huang et al. 2015 ) when preparing a meal. In this paper, we focus on the task of distinguishing between navigational intent and informational intent, a pre-requisite to assist humans when searching for a target object in cluttered environments ( Kang et al. 2015 ) . In the following, we discuss previous work on this task. Subsequently, we compare datasets for navigational vs. informational intent recognition recorded in previous work. Table 1 . Datasets for search intent recognition. Availability shows whether the dataset is publicly available or not. Size is a product of the total number of users, scenes shown