Interpreting Radiologist’s Intention from Eye Movements in Chest X-ray Diagnosis DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artifici

Trong Thang Pham, Anh Nguyen, Zhigang Deng, Carol C. Wu, Hien Nguyen, Ngan Le · 2025

Read Interpreting Radiologist’s Intention from Eye Movements in Chest X-ray Diagnosis DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artifici by Trong Thang Pham, Anh Nguyen, Zhigang Deng, Carol C. Wu, Hien Nguyen, Ngan Le (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-12461.txt.

Introduction

Trong Thang Pham and co-authors build, in a 2025 paper, a model that reads a radiologist's intent from eye movements during chest X-ray reading. Expert readers scan images with a mental checklist, hunting for specific diseases. Earlier gaze models tracked where eyes landed but not what the reader was looking for. The authors treat each stretch of gaze as goal-driven search and set out to name the goal.

Their transformer model, RadGazeIntent, turns fine-grained fixation data into clear intent labels over time and space. To train and test it, they reworked existing eye-tracking sets into three intent styles: orderly search, uncertain exploration, and a hybrid of both. The model beat baselines at predicting which finding a radiologist was examining at each moment, across all three sets. The work could feed training tools and smarter reading aids.

Catalog record

CREATORTrong Thang Pham, Anh Nguyen, Zhigang Deng, Carol C. Wu, Hien Nguyen, Ngan Le
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-12461

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Interpreting Radiologist’s Intention from Eye Movements in Chest X-ray Diagnosis DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence CCS: Applied computing Health care information systems Trong Thang Pham, Anh Nguyen, Zhigang Deng, Carol C. Wu, Hien Nguyen, Ngan Le Abstract . Radiologists rely on eye movements to navigate and interpret medical images. A trained radiologist possesses knowledge about the potential diseases that may be present in the images and, when searching, follows a mental checklist to locate them using their gaze. This is a key observation, yet existing models fail to capture the underlying intent behind each fixation. In this paper, we introduce a deep learning-based approach, RadGazeIntent , designed to model this behavior: having an intention to find something and actively searching for it. Our transformer-based architecture processes both the temporal and spatial dimensions of gaze data, transforming fine-grained fixation features into coarse, meaningful representations of diagnostic intent to interpret radiologists’ goals. To capture the nuances of radiologists’ varied intention-driven behaviors, we process existing medical eye-tracking datasets to create three intention-labeled subsets: RadSeq (Systematic Sequential Search), RadExplore (Uncertainty-driven Exploration), and RadHybrid (Hybrid Pattern). Experimental results demonstrate RadGazeIntent’s ability to predict which findings radiologists are examining at specific moments, outperforming baseline methods across all intention-labeled datasets. Figure 1 . From existing gaze datasets (e.g., EGD and REFLACX) that are illustrated in the left image, existing research on gaze-assisted medical AI primarily focus on developing systems to assist radiologists by performing tasks such as disease classification, report generation, or mimicking visual search patterns through scanpath prediction tasks. Our work extends apart from these tasks and focuses on understanding radiologists by interpreting the radiologist’s intention behind each of their captured gaze points. 1. Introduction Radiologists rely on eye movements to navigate and interpret medical images, leading to a natural question: "what are they trying to find at this moment?" Most current works attempt to mimic radiologists to create digital twins rather than using data from them to understand them, which could be crucial in interactive systems. To the best of our knowledge, no existing work deciphers the underlying intentions behind each fixation in medical imaging analysis. As shown in Table 1 , existing Aritificial Intelligence approaches (AI) fall short in intention prediction. I-AI ( Pham et al. 2024a ) , despite designing models to mimic radiologist decision-making processes, only focuses on predicting heatmaps (spatial information) without explaining the purpose of each fixation. Similarly, EyeXNet ( Hsieh et al. 2024 ) , while using fixation data, takes full heatmaps as input without considering temporal characteristics, and is used for localization rather than explaining fixation meaning. The work most similar to ours is ChestSearch ( Pham et al. 2024b ) , which predicts how a radiologist would view an chest X-ray (CXR) image given a specific finding to search for. However, it is limited to a single finding, whereas in reality, radiologists look for multiple findings, and each gaze point does not always serve only one class, leading to complex relationships. Furthermore, the GazeSearch data from ChestSearch significantly reduce fixation points from the original data, which would be difficult to implement in real scenarios for an interactive system. In contrast, our RadGazeIntent model directly takes the fixation sequence and chest X-ray image to predict which finding each fixation is used to identify, along with confidence scores. Because we strive to understand radiologists’ intentions, we designe our model based on the premise that eye movements ( Neves et al. 2024 ) provide a window into cognitive processes. We conceptualize intention through three complementary perspectives as illustrated in Fig. 1 : First, intention may manifest as a systematic sequential search where radiologists follow a mental checklist, targeting specific findings sequentially and perform medical visual search tasks sequentially ( Pham et al. 2024b ) . Second, intention could reflect uncertainty-driven exploration, where radiologists respond opportunistically to visual cues without predetermined targets, in practice, this means that for a predefined set of findings, radiologists identify and report whatever they happen to observe ( Chen et al. 2022 ) . Third, intention might follow a hybrid pattern where radiologists initially conduct a brief reference scan of the entire image before focusing intently on a single pathology, effectively combining the systematic and opportunistic approaches ( Chen et al. 2020 ) . In this paper, we cannot completely rule out any of these possibilities, so we process the original data and evaluate all methods across multiple definitions to provide the most comprehensive perspective. To support these different interpretations of radiologist behavior, we introduce three corresponding datasets: RadSeq (Systematic Sequential Search), RadExplore (Uncertainty-driven Exploration), and RadHybrid (Hybrid Pattern). Each dataset represents a different conceptualization of how radiologists allocate their visual attention during diagnosis. To model the intention behind each fixation, we use a transformer-based architecture called RadGazeIntent to model gaze sequences with three major characteristics: incorporating both peripheral and foveal information to mimic the visual information humans perceive ( Yang et al. 2020 ) , ensuring fixation information adheres to causality (earlier fixations cannot access information from later ones), and recognizing that fixations are not independent but complementary to each other, meaning adjacent fixations can be combined into more complex features. This approach allows our model to learn patterns from "fine-grained and noisy" fixation data and transform them into "coarse and abstract representations that cluster only the most relevant information." Our RadGazeIntent model employs a transformer architecture with a pooling mechanism to achieve this transformation. Our main contributions include: • Benchmark: Three new benchmark datasets, RadSeq, RadExplore, and RadHybrid, representing different conceptualizations of radiologist’s intention. • RadGazeIntent: A novel framework, RadGazeIntent, for classifying radiologists’ fixations according to their underlying intentions, bridging the gap between visual search patterns and diagnostic reasoning. • Evaluation: A comprehensive evaluation across multiple settings for predicting radiologist intentions, demonstrating our model’s ability to generalize across varied intention definitions and consistently outperform baseline approaches. Table 1 . Comparison of previous works on eye tracking assistance methods. Most existing works using eye tracking datasets primarily utilize information in heatmap form (spatial modeling) to solve problems within their corresponding settings. Recently, ChestSearch ( Pham et al. 2024b ) has focused on the visual search problem and proposed models to incorporate temporal information. It is also notable that a temporal classifier proposed by Karargyris et al. ( Karargyris et al. 2021 ) can model temporal aspects by using RNNs on heatmaps of gaze sequences. However, despite having both temporal and spatial modeling capabilities, no existing work has tackled the problem of understanding the intention behind each gaze point. Methods Temporal Modeling Spatial Modeling Intention Interpretation Tasks I-AI ( Pham et al. 2024a ) ✗ ✓ ✗ Disease Classification GazeRadar ( Bhattacharya et al. 2022 ) ✗ ✓ ✗ Disease Localization EGGCA-Net ( Peng et al. 2024 ) ✗ ✓ ✗ Report Generation EyeXNet ( Hsieh et al. 2024 ) ✗ ✓ ✗ Disease Localization Karargyris et al. ( Karargyris et al. 2021 ) ✓ ✓ ✗ Disease Classification ChestSearch ( Pham et al. 2024b ) ✓ ✓ ✗ Scanpath Prediction RadGazeIntent (Ours) ✓ ✓ ✓ Intention Interpretation 2. Related Work 2.1. Gaze-Assisted

Read the complete text