Graph Representations for Reading Comprehension Analysis using Large Language Model and Eye-Tracking biomarker
Yuhong Zhang, Jialu Li, Shilai Yang, Yuchen Xu, Gert Cauwenberghs, Tzyy-Ping Jung · 2025
Read Graph Representations for Reading Comprehension Analysis using Large Language Model and Eye-Tracking biomarker by Yuhong Zhang, Jialu Li, Shilai Yang, Yuchen Xu, Gert Cauwenberghs, Tzyy-Ping Jung (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-11972.txt.
Introduction
Yuhong Zhang and co-authors build a 2025 method to compare how people and language models understand what they read. Reading comprehension grows through childhood and stays central to learning. As large language models take on reading tasks, researchers want to know where machine reading matches human reading and where it parts ways.
Earlier work by the team used eye-tracking, which records where a reader's eyes linger, to test how word relevance lines up with attention. Looking at single words alone gave a shallow view. Here an AI agent builds graph views of the text, linking ideas rather than lone words. The eye data then tests those richer maps. The approach aims to show, in a form people can inspect, how a model weighs the parts of a passage.
Catalog record
| CREATOR | Yuhong Zhang, Jialu Li, Shilai Yang, Yuchen Xu, Gert Cauwenberghs, Tzyy-Ping Jung |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-11972
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Graph Representations for Reading Comprehension Analysis using Large Language Model and Eye-Tracking biomarker Yuhong Zhang, Jialu Li, Shilai Yang, Yuchen Xu, Gert Cauwenberghs, Tzyy-Ping Jung Abstract Reading comprehension is a fundamental skill in human cognitive development. With the advancement of Large Language Models (LLMs), there is a growing need to compare how humans and LLMs understand language across different contexts and apply this understanding to functional tasks such as inference, emotion interpretation, and information retrieval. Our previous work used LLMs and human biomarkers to study the reading comprehension process. The results showed that the biomarkers corresponding to words with high and low relevance to the inference target, as labeled by the LLMs, exhibited distinct patterns, particularly when validated using eye-tracking data. However, focusing solely on individual words limited the depth of understanding, which made the conclusions somewhat simplistic despite their potential significance. This study used an LLM-based AI agent to group words from a reading passage into nodes and edges, forming a graph-based text representation based on semantic meaning and question-oriented prompts. We then compare the distribution of eye fixations on important nodes and edges. Our findings indicate that LLMs exhibit high consistency in language understanding at the level of graph topological structure. These results build on our previous findings and offer insights into effective human-AI co-learning strategies. Keywords — Large Language Models, Graph Theory, Reading Comprehension, Eye-tracking I INTRODUCTION Reading comprehension is a fundamental cognitive skill that remains a critical challenge in education and healthcare [ 1 ] . Globally, approximately 61% of adolescents fail to achieve minimum reading comprehension proficiency by the end of lower secondary education [ 2 , 3 ] . Studies have shown that attention deficits and low reading efficiency significantly impact students’ ability to process and retain information effectively [ 4 , 5 , 6 ] . From a biomedical perspective, these challenges often stem from underlying neurological conditions, learning disabilities, or developmental disorders, making accurate assessment and intervention crucial for both educational and clinical applications. In the field of biomedical engineering, developing precise, quantitative methods for assessing cognitive function represents a significant challenge and opportunity for advancing healthcare delivery. Recent LLMs advances have provided new insights into cognitive and human intelligence studies, particularly in reading comprehension tasks. This underscores the growing need to compare how humans and LLMs understand language across different contexts. Our previous work [ 7 , 8 ] leveraged LLMs, particularly GPT-4, to label individual words in reading sentences, identifying whether they belong to groups of high relevance for question inference. Biomarkers from EEG and eye-tracking validated this approach, showing strong alignment with markers labeled with LLM. Although promising, this method has a limitation: sentence meaning often depends on phrases or word groups rather than individual words alone. This constraint makes it difficult to capture the broader semantic meaning of text and its relevance to comprehension tasks. Furthermore, word-level labeling fails to effectively characterize which parts of a sentence contain critical information that readers should focus on. Readers often spend more time fixating on key phrases or sentence segments essential for understanding and answering questions rather than isolated words. Researchers have investigated graph-based text representations to address the limitations of single-word representations. Graph-based representations depict words and phrases as nodes, with edges connecting the nodes representing their semantic relationships. Graph structures have long been used to represent sentence meanings [ 9 ] , revealing the underlying logical flow of sentences and aligning closely with human cognitive processes [ 10 , 11 , 12 ] . Unlike traditional bag-of-words (BoW) models, which assume term independence, graph-based text representations provide a structured approach to capturing linguistic relationships by modeling words or phrases as nodes and their syntactic or semantic connections as edges [ 13 , 14 ] . Beyond simple word co-occurrence, graph-theoretic techniques offer deeper insights into text analysis. Graph centrality measures—including degree centrality, PageRank, and betweenness centrality—identify key terms within a sentence, mirroring how humans prioritize information while reading. Additionally, k-core decomposition extracts dense subgraphs that highlight the most coherent and informative parts of a text. Similarly, Graph-of-Words (GoW) models capture concept interconnections more effectively, reflecting the relationships readers form during text comprehension [ 14 , 15 ] . We propose leveraging LLM-generated graph-based representations to advance reading comprehension analysis, integrating eye-tracking biomarkers, which have long been used in cognitive research. Prior studies have shown that eye-tracking data effectively predict reading comprehension performance [ 16 ] . This study examines various prompt heuristics and their corresponding knowledge graphs generated using LLMs. We analyze their topological structural features and compare them with LLM-generated labels to assess node importance, validating these findings with biomarkers. Our approach represents a novel contribution to biomedical engineering by establishing a framework that combines physiological measurements, artificial intelligence, and graph theory for cognitive assessment. The remainder of this paper is organized as follows: Section 2 introduces the Zurich Cognitive Language Processing (ZuCo) 1.0 datasets used for textual content and eye-tracking analysis. Section 3 presents our methodology for converting text sequences into graph structures and describes graph theory-based analysis methods. Section 4 discusses our results, focusing on graph measurement statistics and eye-gaze biomarkers. Finally, Section 5 summarizes our findings and outlines future research directions, particularly emphasizing potential clinical applications and biomedical implications for advancing healthcare technology and patient care. II Dataset The ZuCo 1.0 dataset is a comprehensive multimodal resources that integrate high-density 128-channel EEG and eye-tracking data from 12 native English speakers engaged in various language comprehension and reading tasks [ 17 ] . It comprises 21,629 words, 1,107 sentences, and 154,173 fixations recorded over 4-6 hours of natural text reading. An EyeLink 1000 Plus eye tracker, operating at 500 Hz, tracked eye movements and pupil size. This study fully leverages all tasks available in the ZuCo 1.0 datasets to analyze a range of eye-fixation features. High-resolution eye-tracking enables the examination of cognitive processes such as attention, language processing, and cognitive load, supporting research in psycholinguistics, cognitive neuroscience, and human-computer interaction [ 17 ] . The dataset includes three reading tasks: • Task 1 (Reading Task - Natural Sentences): Participants read grammatically correct and semantically meaningful English sentences in a natural reading setting. • Task 2 (Reading Task - Random Words): Subjects read sequences of unconnected, randomly ordered English words, used to isolate low-level lexical processing from higher-order sentence integration. • Task 3 (Reading Task - Baseline/Resting): Participants fixate on a central cross in the absence of any textual stimulus, providing a resting-state baseline for both eye and EEG signals. III METHOD III-A Workflow Overview Fig. 1 illustrates the workflow. We begin by extracting sentences from three different tasks within the ZuCo 1.0 dataset. Based on semantic and logical transitions, we develop corresponding prompts to convert sentences into nodes and edges. Next, we use task-oriented reading questions to generate additional prompts that direct LLMs to label key nodes and edges, reflecting their internal reasoning process. Graph metrics, such as node centrality (degree, closeness, betweenness), are calculated to characterize graph properties and compared with the results provided by the AI agent. More importantly, we align human eye-tracking biomarkers to validate the computational results, specifically the number of fixations on nodes and edges. Fig. 1 : Workflow for analyzing