Deep Learning in Mild Cognitive Impairment Diagnosis using Eye Movements and Image Content in Visual Memory Tasks
Tomás Silva Santos Rocha a, Anastasiia Mikhailova b, Moreno I. Coco c, José Santos-Victor d · 2025
Read Deep Learning in Mild Cognitive Impairment Diagnosis using Eye Movements and Image Content in Visual Memory Tasks by Tomás Silva Santos Rocha a, Anastasiia Mikhailova b, Moreno I. Coco c, José Santos-Victor d (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-23016.txt.
Introduction
Tomás Rocha and three co-authors test in 2025 whether eye movements can flag early memory loss. Mild thinking decline often comes before dementia, and world cases are set to double by 2050. Cheap, scalable checks are in demand.
Forty-four people, twenty-four with mild decline and twenty healthy peers, did a visual memory task while their eyes were tracked. A deep model read the gaze paths and heat maps, plus the test images themselves. The best setup caught about two-thirds of cases and cleared about three-quarters of healthy peers. Those scores rival a like study on Alzheimer's disease, despite a smaller set and a shorter task here.
Catalog record
| CREATOR | Tomás Silva Santos Rocha a, Anastasiia Mikhailova b, Moreno I. Coco c, José Santos-Victor d |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-23016
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep Learning in Mild Cognitive Impairment Diagnosis using Eye Movements and Image Content in Visual Memory Tasks Tomás Silva Santos Rocha a, Anastasiia Mikhailova b, Moreno I. Coco c, José Santos-Victor d Abstract The global prevalence of dementia is projected to double by 2050, highlighting the urgent need for scalable diagnostic tools. This study utilizes digital cognitive tasks with eye-tracking data correlated with memory processes to distinguish between Healthy Controls (HC) and Mild Cognitive Impairment (MCI), a precursor to dementia. A deep learning model based on VTNet was trained using eye-tracking data from 44 participants (24 MCI, 20 HCs) who performed a visual memory task. The model utilizes both time series and spatial data derived from eye-tracking. It was modified to incorporate scan paths, heat maps, and image content. These modifications also enabled testing parameters such as image resolution and task performance, analysing their impact on model performance. The best model, utilizing 700 × 700 p x 700\times 700px resolution heatmaps, achieved 68% sensitivity and 76% specificity. Despite operating under more challenging conditions (e.g., smaller dataset size, shorter task duration, or a less standardized task), the model’s performance is comparable to an Alzheimer’s study using similar methods (70% sensitivity and 73% specificity). These findings contribute to the development of automated diagnostic tools for MCI. Future work should focus on refining the model and using a standardized long-term visual memory task. 1 Introduction The WHO’s “Global Status Report on the Public Health Response to dementia” [ 21 ] highlights dementia as a growing global health issue, affecting over 55 million people, projected to rise to 139 million by 2050, with a disproportionate impact expected in middle-income countries. This emphasizes the urgent need for diagnostic tools and interventions, particularly as populations age. Dementia is characterized by cognitive decline that interferes with daily life, including memory loss and impaired thinking. Alzheimer’s disease is the most common form, though other types include vascular, frontotemporal, and Lewy body dementia [ 4 ] . Before developing dementia, patients go through a stage called Mild Cognitive Impairment (MCI), experiencing mild cognitive changes that do not significantly interfere with daily life activities. Early detection of MCI provides a critical window for interventions that may delay progression to dementia [ 22 ] . There are two types: amnestic, which impacts memory and has a higher risk of Alzheimer’s, and non-amnestic, which affects attention, language, or visuospatial skills and may indicate risk for other dementias, such as frontotemporal or vascular dementia [ 25 ] . MCI may also involve single-domain or multiple-domain impairments, with the latter suggesting more significant progression to dementia [ 25 , 10 ] . 1.1 Diagnosing MCI MCI is influenced by age, genetics (e.g., APOE \varepsilon 4 allele), lifestyle, and cardiovascular health and is associated with biomarkers such as hippocampal atrophy and amyloid-beta accumulation. However, diagnosing it promptly remains challenging, as neuroimaging and genetic tests are primarily used for the confirmation of the diagnosis, and lifestyle factors lack diagnostic precision [ 15 , 12 ] . The limited availability of specialists restricts the widespread use of standard cognitive tests like MMSE and MoCA. When combined with delayed symptom recognition, this often results in MCI being detected only when cognitive decline is noticeable, limiting early intervention opportunities [ 2 , 5 ] . To tackle these challenges, digital assessments on computers, tablets, and smartphones are emerging as practical and cost-effective alternatives [ 27 ] . When integrated with wearable sensors, cameras, or eye-tracking devices, these tools capture many relevant data without adding time costs, enhancing their diagnostic potential [ 14 ] . Eye-tracking integration in MCI diagnosis has gained attention, with studies suggesting that eye movement patterns could serve as early indicators of cognitive decline [ 36 ] . 1.2 Eye-movements: A window to the memories Eye movements, essential for visual perception, offer insight into cognitive processing and memory. Key movements include saccades, rapid gaze shifts between points, and fixations, where the eyes pause to process information. Saccades allow quick repositioning but temporarily suppress visual intake, while fixations enable detailed analysis, with durations varying by task [ 24 ] . Fixations are particularly useful, as their longer periods allow for easier detection. Eye movements significantly impact memory mechanisms, aiding encoding and retrieval. Increased fixations and shifts in gaze while encoding a scene improve memory recall, as eye movements facilitate scene exploration [ 8 ] . During encoding, eye movement patterns differ notably between MCIs and healthy individuals. MCI patients often have delayed saccadic responses and less accurate saccade targeting, leading to inefficient visual exploration. Their fixations are longer but fewer, possibly indicating attention and visual processing difficulties. This reduced fixation frequency and longer saccadic latency contribute to limited scene exploration, impacting their ability to recognize and remember visual details [ 36 ] . In the literature, we also see that image content influences eye movements by engaging in a process where our eyes selectively focus on elements perceived as necessary, consequently influencing the memory processes [ 32 ] . 1.3 Deep Learning and MCI To enhance the digital tools deep learning algorithms are being employed, improving diagnostic accuracy and scalability, thus making the timely detection of MCI a possibility [ 36 ] . Algorithms based on various methods have been employed, such as Recursive Neural Networks (RNN), Convolutional Neural Networks (CNN), autoencoders, transformers, and others [ 30 ] . The transformer based architectures usually perform better than the others, however they require large amounts of data to be properly trained [ 30 , 16 ] . One problem with recent studies is the fact that, typically there is no differentiation between MCI patients and patients with more advanced dementias, such as Alzheimer’s, which can bring biases to the results obtained [ 30 ] . Building upon recent advancements, this study explores deep learning models that use eye-tracking data collected from MCI patients and Healthy Controls (HCs). The participants perform a visual long-term active memory task to predict MCI, which have been shown to have better diagnostic accuracy [ 34 , 36 ] . Additionally, this paper investigates incorporating image content, a topic that has not been extensively researched, and memory performance to try to enhance the diagnostic process further. 2 Method Initially, we planned to use our collected data; however, the process is ongoing, and the sample size is insufficient. Instead, we utilized data from [ 6 ] , whose experiment closely aligned with our study. This study [ 6 ] examined how semantically related images impact memory performance in MCI and HC groups. 2.1 Participants and Procedure Selection criteria: (a) ages between 50 and 90; (b) minimum of three schooling years; (c) normal or corrected-to-normal vision with no history of eye surgery; (d) no history of neurological (except memory disturbances) or psychiatric disorders; (e) no history of alcohol/substance abuse or medications affecting cognition. Data collection resulted in 44 participants (24 MCI patients: 71.92\pm 9.06 years old, 9.83\pm 4.50 schooling years; 20 HCs: 68.50\pm 8.79 years old, 11.05\pm 5.10 schooling years), and a total of 79 sessions after filtering for (a) follow-up participation; (b) performance above chance; (c) successful eye-tracker calibration. The visual memory task was conducted on a 17-inch monitor ( 1600\times 900 resolution,