Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention
\authororcid Minsuk Chang0009-0007-5088-8991, \authororcid Yao Wang0000-0002-3633-8623, \authororcid Huichen Will Wang0009-0007-5941-4047, Yuanhong Zhou, \authororcid Andreas Bulling0000-0001-6317-7303, and \authororcid Cindy Xiong Bearfield0000-0002-1451-4083 · 2025
Read Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention by \authororcid Minsuk Chang0009-0007-5088-8991, \authororcid Yao Wang0000-0002-3633-8623, \authororcid Huichen Will Wang0009-0007-5941-4047, Yuanhong Zhou, \authororcid Andreas Bulling0000-0001-6317-7303, and \authororcid Cindy Xiong Bearfield0000-0002-1451-4083 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03713.txt.
Introduction
Minsuk Chang and co-authors link chart-reading skill to eye movement in a 2025 study. People differ in how well they read charts, and those gaps shape what a chart tells them. The team ran a study with 235 people who took three chart skill tests. Eye tracking showed clear gaps between strong and weak readers.
Strong readers kept a tight focus on the key parts of a chart. Weaker readers roamed more and locked on less. From these patterns the team built two models. One predicts where a viewer will look, given their skill level. The other works in reverse, guessing skill from a single gaze map with 86 percent accuracy in under a minute. Both point toward charts tuned to each reader.
Catalog record
| CREATOR | \authororcid Minsuk Chang0009-0007-5088-8991, \authororcid Yao Wang0000-0002-3633-8623, \authororcid Huichen Will Wang0009-0007-5941-4047, Yuanhong Zhou, \authororcid Andreas Bulling0000-0001-6317-7303, and \authororcid Cindy Xiong Bearfield0000-0002-1451-4083 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-03713
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention \authororcid Minsuk Chang0009-0007-5088-8991, \authororcid Yao Wang0000-0002-3633-8623, \authororcid Huichen Will Wang0009-0007-5941-4047, Yuanhong Zhou, \authororcid Andreas Bulling0000-0001-6317-7303, and \authororcid Cindy Xiong Bearfield0000-0002-1451-4083 Abstract Accounting for individual differences can improve the effectiveness of visualization design. While the role of visual attention in visualization interpretation is well recognized, existing work often overlooks how this behavior varies based on visual literacy levels. Based on data from a 235-participant user study covering three visualization tests (mini-VLAT, CALVI, and SGL), we show that distinct attention patterns in visual data exploration can correlate with participants’ literacy levels: While experts (high-scorers) generally show a strong attentional focus, novices (low-scorers) focus less and explore more. We then propose two computational models leveraging these insights: Lit2Sal – a novel visual saliency model that predicts observer attention given their visualization literacy level, and Sal2Lit – a model to predict visual literacy from human visual attention data. Our quantitative and qualitative evaluation demonstrates that Lit2Sal outperforms state-of-the-art saliency models with literacy-aware considerations. Sal2Lit predicts literacy with 86% accuracy using a single attention map, providing a time-efficient supplement to literacy assessment that only takes less than a minute. Taken together, our unique approach to consider individual differences in salience models and visual attention in literacy assessments paves the way for new directions in personalized visual data communication to enhance understanding. \onlineid 1490 \vgtccategory Research \vgtcpapertype theory/model \authorfooter Minsuk Chang, Yuanhong Zhou, and Cindy Xiong Bearfield is with Georgia Tech. E-mail: {minsuk, yzhou842, cxiong}@gatech.edu Yao Wang and Andreas Bulling is with University of Stuttgart. E-mail: {yao.wang, andreas.bulling}@vis.uni-stuttgart.de Huichen Will Wang is with University of Washington. E-mail: [email protected] \teaser Predicted visual saliency map from Lit2Sal model on three literacy assessment test (mini-VLAT, CALVI, SGL) scores for treemap and histogram. High-literacy group represents the top 20% on each test, and Low-literacy group represents the bottom 20%. The saliency map for high scores on mini-VLAT and CALVI focuses more on the title and axis labels than low scores, also showing more focused attention. Refer to Section 4 for further details. Introduction keywords Visualization Literacy Assessment, Visual Attention and Saliency, Visual Saliency Models. Visualization can guide users to notice key patterns in data. Yet what counts as the ’right’ design often depends on who is looking. People interpret data on a deeply personal level [ 78 , 64 ] , thus visualization design effectiveness can depend on their analytic tasks [ 48 ] , their goals [ 67 ] , and most importantly, their visualization literacy [ 58 , 62 ] . Existing research has demonstrated that we can improve visualization design by understanding where people look in a visualization [ 16 ] . Researchers have built saliency models to predict which parts of a visualization are most likely to attract a viewer’s attention [ 53 , 18 , 15 , 85 , 76 ] . These models have proven valuable in supporting visualization and tool design, such as attention-aware UI [ 55 ] , chart compression [ 16 ] , and image quality evaluations [ 80 ] . However, existing models still largely assume universal viewing patterns among people and overlook differences driven by cognitive abilities such as visualization literacy [ 79 , 56 ] . We posit that individuals with varying levels of visualization literacy interpret visualizations through distinct viewing patterns and can be effectively captured by models of their visual attention. Therefore, we argue that we can improve existing saliency models by accounting for the unique patterns in viewers’ visual attention depending on their literacy levels. In this paper, we conduct user study (N = 235) using three established literacy tests: the mini Visualization Literacy Assessment Test (mini-VLAT) [ 62 ] , the Critical thinking Assessment for Literacy in VIsualizations (CALVI) [ 35 ] , and the Subjective Graph Literacy assessment (SGL) [ 34 ] . We recorded participants’ responses and attention maps generated using the established BubbleView [ 40 ] technique, which approximates gaze tracking with mouse clicks. Our analyses reveal striking attention differences between high- and low-literacy groups in each assessment : Experts focus on specific regions of visualizations, creating concentrated ‘hot spots’ that reflect their targeted attention. In contrast, the viewing patterns of novices tended to be more distributed, with less intense focal points. This finding provides strong evidence that performance on visualization literacy tests does correlate with distinct visual attention patterns when interpreting visualizations. Informed by these insights, we then introduce two novel saliency models for literacy-aware attention prediction ( Lit2Sal ) and visual attention-based visualization literacy prediction ( Sal2Lit ). Lit2Sal extends the state-of-the-art saliency model VisSalFormer [ 86 ] to predict where people look in a visualization while accounting for their literacy levels, across three literacy assessment categories: mini-VLAT [ 62 ] for visualization comprehension, CALVI [ 35 ] for critical thinking abilities, and SGL for self-perceived literacy, both individually and holistically. By integrating these measures, our model generates distinct saliency predictions for novices and experts. Sal2Lit can predict visualization literacy levels for all three tests with an average accuracy of 86% with a human attention map from only one visualization. If we expand the input data to include attention maps from three visualizations, our model accuracy increases by over 87% (mini-VLAT) to 90% (CALVI, SGL). This time-efficient proxy supplements existing literacy assessments and reveals visual processes that differentiate novices and experts in visual data exploration. Contributions: We conducted a crowdsourced study and a series of analyses, discovering how different aspects of visualization literacy (e.g., basic comprehension, critical thinking, and self-assessed ability) correlate with viewers’ gaze patterns. Our literacy-to-saliency model, Lit2Sal, enhances existing saliency models by incorporating individual differences and aligning attention predictions with a viewer’s visualization literacy level. Conversely, our saliency-to-literacy model, Sal2Lit, takes a perception-driven approach to reframe how visualization literacy can be assessed through observed visual attention patterns. Together, they advance our fundamental understanding of visual patterns in visual data exploration. By embracing personalization, these models empower researchers and practitioners to design personalized visualizations and efficiently estimate literacy levels. Figure 1 : Overview of our study from data collection to model experiments. We first collected attention data from 235 participants using BubbleView [ 40 ] while they solved 27 visualization literacy test questions consisting of mini-VLAT [ 62 ] and CALVI [ 35 ] . The data was used to test hypotheses in Section 3.4 - 3.6 and train two models: Lit2Sal ( Section 4 ), and Sal2Lit ( Section 5 ). 1 Related Work 1.1 Visual Attention in Visualizations Where do people look in a visualization? Cognitive and vision scientists have long studied human attention to understand how visual features such as color, contrast, and orientation can guide attention patterns [ 81 , 39 ] . With the rise of large-scale eye-tracking datasets and advances in machine learning, researchers have developed more sophisticated models to predict human attention patterns across images, text, and data visualizations. This research identifies salient regions that most likely attract human attention [ 74 , 6 , 23 ] . Saliency models serve as critical tools for evaluating the visual hierarchy in attention allocation in scene interpretation [ 39 ] and object identification [ 7 ] . While conventional