From Scores to Strategies: Towards Gaze-Informed Diagnostic Assessment for Visualization Literacy
Kathrin Schnizer · 2026
Read From Scores to Strategies: Towards Gaze-Informed Diagnostic Assessment for Visualization Literacy by Kathrin Schnizer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21898.txt.
Introduction
Schnizer argues that chart literacy tests should read eye movements, not just right and wrong answers. Scores alone say little about how a reader worked through a chart. Gaze data, from visualization research, reveal mental load that timing and accuracy miss. They also expose strategy gaps between fluent and struggling readers. The proposal merges test scores with gaze markers like attention per chart part, cross-part linking, and path spread. That mix could split fluent success from labored success. A roadmap lists open needs in evidence and design.
Catalog record
| CREATOR | Kathrin Schnizer |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18015 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21898
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Scores to Strategies: Towards Gaze-Informed Diagnostic Assessment for Visualization Literacy DOI: https://doi.org/10.48550/arXiv.2603.21898 Conference: Workshop on Data Literacy for the 21st Century at CHI 2026; April 26; Barcelona, Spain CCS: Human-centered computing Information visualization CCS: Human-centered computing Empirical studies in HCI Kathrin Schnizer Abstract . Visualization literacy assessments typically rely on correctness to classify performance, providing little evidence about how readers arrive at their answers. We argue that gaze can address this gap as an implicit process signal that complements standardized tests without sacrificing their scalability. Synthesizing findings from visualization and related research, we show that gaze metrics capture cognitive load invisible to accuracy and response time, and reflect strategy differences in attention allocation that track proficiency. We propose assessments that integrate literacy scores with gaze-derived process indicators — component-level attention profiles, integration frequency, and viewing path dispersion — to distinguish fluent comprehension from labored success. This would shift literacy assessment from binary classification toward nuanced characterization of how readers navigate, integrate, and coordinate information across chart components. A roadmap identifies open challenges in empirical grounding, generalizability, assessment design, and practical feasibility. 1. Introduction Visualizations are an effective means to communicate trends and patterns in data. As we generate and collect increasing amounts of data, it becomes crucial that people can correctly read and interpret data visualizations to extract meaningful insights, a skill often referred to as visualization literacy ( 16 ; 4 ; 3 ) . To promote visualization literacy effectively, we need reliable ways to assess whether and how individuals understand a given visualization. Most visualization literacy assessments rely on explicit, outcome-based responses such as whether an answer is correct, rather than capturing how comprehension unfolds ( 18 ) . As a result, they primarily support classification of performance, offering limited evidence about attention allocation, reading strategies, and where processing difficulties occur; information necessary for diagnosis beyond binary success or failure. In practice, visualization literacy is often operationalized through performance metrics on standardized single-choice tasks ( 21 ; 16 ; 9 ) . These approaches are easy to administer and compare across studies, but they mainly reflect what users conclude, not how they navigated, prioritized, or integrated information to reach a conclusion. At the other end of the spectrum, free exploration and open-response approaches can reveal salient features, misconceptions, and reasoning ( 2 ; 14 ; 24 ; 23 ) , yet they frequently require lengthy responses and intensive manual coding, increasing effort for both participants and evaluators. Together, these complementary approaches highlight the need for process-sensitive measures that can add diagnostic evidence to standardized assessments while remaining scalable. In this position paper, we argue for augmenting outcome-based visualization literacy assessment with gaze as an implicit process signal. Our central claim is that gaze can bridge the two assessment needs identified above: the scalability of standardized tests and the diagnostic richness of qualitative approaches. We synthesize evidence from visualization research showing that gaze metrics capture processing demands masked by correct responses and systematic strategy differences linked to proficiency. Building on this, we outline a vision of assessments that report literacy scores alongside gaze-derived process indicators — such as attention distribution across chart components, integration between complementary elements, and viewing path dispersion — distinguishing fluent comprehension from labored success. We conclude with a roadmap addressing empirical grounding and diagnostic interpretation, generalizability across items and readers, and the design of actionable assessment reports. 2. Gaze for Process-Informed Visualization Literacy Assessment In the following, we motivate the use of gaze as process evidence for visualization literacy assessment. We first clarify what gaze can and cannot provide as a behavioral signal, then synthesize evidence from visualization research showing that gaze reflects processing demands that outcome measures miss and differentiates reading strategies linked to proficiency. Building on this evidence, we outline our vision for gaze-informed diagnostic assessment and a roadmap of open challenges. 2.1. Gaze as a Process Trace of Attention Allocation Eye-tracking metrics such as fixation durations, refixations, and transitions between regions provide time-resolved traces of where and when readers allocate overt visual attention while engaging with a data visualization. While gaze does not unambiguously reveal cognitive processing or intent, it provides direct behavioral evidence of attention allocation as it is shaped by task goals ( 19 ; 1 ) , bottom-up salience ( 12 ) , and cognitive demands such as working memory load ( 5 ; 31 ) . Because gaze traces combine temporal and spatial information, they allow us to follow how attention is distributed across chart components over time (e.g., axes, marks, legend, or text), offering a window into reading and interaction processes that outcome measures alone cannot capture. 2.2. Gaze Reflects Processing Demands Standardized visualization literacy tests such as the VLAT ( 16 ) , Mini-VLAT ( 21 ) , and CALVI ( 9 ) assess literacy through accuracy on task items. While this reveals which items a reader failed, it provides no signal about how demanding successfully completed items were for a given reader. Response time partially addresses this by capturing overall time on trials, and has been used to demonstrate that equivalent tasks vary in processing demands depending on visualization design ( 25 ; 6 ) . However, it does not reveal where in the chart effort was concentrated or which components impacted difficulty. As a result, a strong literacy score can mask processing struggles, limiting diagnosis to readers who fail rather than also identifying those whose correct responses mask effortful processing. Gaze metrics can address this gap by providing spatiotemporal evidence of how processing demands are distributed across chart components, even when the outcome is correct. The diagnostic value of such component-level process information has been demonstrated by 18 , who showed that capturing how readers engaged with chart elements during incorrect trials — through explicit input on reading strategies — enabled categorization of distinct literacy barriers beyond what accuracy alone could reveal. Gaze offers a pathway to obtain process evidence implicitly, extending the diagnostic potential to correct trials where struggles currently remain invisible. At the level of individual gaze events, fixation frequency and duration, as well as saccade amplitude, have been shown to systematically vary with task complexity, with saccade size emerging as a particularly discriminatory parameter ( 8 ) . Importantly, these metrics can be mapped to specific chart components when combined with area-of-interest definitions. 6 demonstrated that readers produced more refixations on task-relevant elements in visualization layouts associated with longer completion times, indicating that spatial gaze patterns capture where processing effort concentrates at a granularity that response time alone cannot provide. Complementing fixation-based metrics, pupil dilation provides a physiological index of cognitive load that is partially decoupled from overt viewing behavior. Task difficulty has been shown to systematically modulate pupil size ( 22 ) , a signal that has been leveraged to adapt system behavior in real time ( 15 ; 10 ) . In the context of visualization, 29 found that a visualization intervention which failed to improve task performance was associated with larger pupil diameters, suggesting elevated cognitive load, demonstrating the kind of masked difficulty that accuracy-based assessment cannot detect. More broadly, 30 showed that eye-tracking signals explained approximately 50% of variance in subjectively rated text comprehensibility, establishing that