Accessible Text Descriptions for UpSet Plots

A. McNutt 1 , M. K. McCracken 1 , I. J. Eliza 1 , D. Hajas 2 , J. Wagoner 1 , N. Lanza 1 , J. Wilburn 1 , S. Creem-Regehr 1 , and A. Lex 1 1 University of Utah, Salt Lake City, Utah 2 Global Disability Innovation Hub, University College London · 2025

Read Accessible Text Descriptions for UpSet Plots by A. McNutt 1 , M. K. McCracken 1 , I. J. Eliza 1 , D. Hajas 2 , J. Wagoner 1 , N. Lanza 1 , J. Wilburn 1 , S. Creem-Regehr 1 , and A. Lex 1 1 University of Utah, Salt Lake City, Utah 2 Global Disability Innovation Hub, University College London (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17517.txt.

Introduction

Andrew McNutt and co-authors build a tool in 2025 that writes plain text accounts of a complex chart type for blind and low-vision readers. Charts are a main way science shares facts, yet their sense is locked in shape and color. The chart type shows how sets overlap, and it had drawn little past access work.

The team mined real charts for shared patterns, then built a system that can caption any chart of this kind. They tested the captions in long talks with eleven blind and low-vision users. The talks showed what the text got right and where it fell short. Chart makers gain a path to share set data with readers who cannot see the plot.

Catalog record

CREATORA. McNutt 1 , M. K. McCracken 1 , I. J. Eliza 1 , D. Hajas 2 , J. Wagoner 1 , N. Lanza 1 , J. Wilburn 1 , S. Creem-Regehr 1 , and A. Lex 1 1 University of Utah, Salt Lake City, Utah 2 Global Disability Innovation Hub, University College London
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE98 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17517

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Accessible Text Descriptions for UpSet Plots A. McNutt 1 , M. K. McCracken 1 , I. J. Eliza 1 , D. Hajas 2 , J. Wagoner 1 , N. Lanza 1 , J. Wilburn 1 , S. Creem-Regehr 1 , and A. Lex 1 1 University of Utah, Salt Lake City, Utah 2 Global Disability Innovation Hub, University College London Abstract Data visualizations are typically not accessible to blind and low-vision (BLV) users. Automatically generating text descriptions offers an enticing mechanism for democratizing access to the information held in complex scientific charts, yet appropriate procedures for generating those texts remain elusive. Pursuing this issue, we study a single complex chart form: UpSet plots. UpSet Plots are a common way to analyze set data, an area largely unexplored by prior accessibility literature. By analyzing the patterns present in real-world examples, we develop a system for automatically captioning any UpSet plot. We evaluated the utility of our captions via semi-structured interviews with (N=11) BLV users and found that BLV users find them informative. In extensions, we find that sighted users can use our texts similarly to UpSet plots and that they are better than naive LLM usage. \SpecialIssuePaper \CGFccby Volume: 44 Issue: 3 ccs Human-centered computing Accessibility systems and tools ccs Human-centered computing Visualization † † year: 2025 † † year: 2025 † † editors: W. Aigner, N. Andrienko, and B. Wang † † subject: Eurographics Conference on Visualization CGF Vol No † † teaser: UpSet plots are a prominent technique for visualizing set data. Despite their popularity, they remain inaccessible to blind or low vision (BLV) users. We seek to make this chart type accessible to all by developing a descriptive text-generation system. Here we show a subset of the descriptive data patterns that appears in their usage that enable our text descriptions. 1 Introduction Being able to read and understand the content of information presented in data visualizations is crucial to fully grasp the content of scientific articles. Yet, most visualizations are not accessible to blind or low-vision users (BLV) [ BHMG17 , SL21 ] . A common way to make images accessible is to provide text descriptions [ JMK∗22 ] that communicate the content of a chart to screen readers [ ZLL∗22 ] . However, most scientific journals do not provide useful text descriptions for visualizations. This lack of accessibility likely contributes to an under-representation of disabled individuals in STEM [ Nat23 ] . The reasons for the lack of text descriptions in scientific publications are manifold—ranging from a lack of awareness, to technical challenges when providing text descriptions in scientific papers, to adding yet another burden to the onerous publishing process, and lack of knowledge on how to write useful text descriptions. Automatic construction of text descriptions would seem then to offer a partial solution: reducing the burden of writing text descriptions and potentially forming a foundation for a human to refine. Whereas pixel-based images are commonly analyzed and captioned using computer vision techniques (e.g., on Facebook [ WWFS17 ] ), many chart forms are computationally represented and thus amenable to automated analysis and thus description. Tang et al. [ TBS23 ] highlight that effective text descriptions should be specific to the chart and dataset type, yet, automatic approaches for data and chart aware charts are currently limited to simple charts, like one-dimensional line and bar charts. The construction of text for more bespoke or uncommon chart types has been identified as important next steps, although efforts so far have been limited [ KKK23 , LPS∗24 ] . Complicating these explorations is that usage and data patterns of complex charts are not as well as understood as those present in quotidian charts forms. For instance, widely used plots, such as scatterplots, exhibit well-understood patterns, such as clusters or outliers, that algorithms can identify to extract information and potentially generate meaningful text descriptions. Yet such patterns do not exist for exotic charts (such as necklace maps [ SV10 ] ), boutique charts (as in My Life with Long Covid [ Lui23 ] ), or even relatively well-known charts like tree maps [ KKK23 ] . We seek to improve the process of automatic descriptive text creation, by considering the specific case of UpSet plots [ LGS∗14 ] . UpSet plots offer an intriguing test case for text description-generation, as there has been substantial adoption (particularly in the biomedical domain with over 4000 citations to the two papers that introduce them [ LGS∗14 , CLG17 ] ) making them a graphical form with potentially high impact if made more broadly accessible. At the same time they consider a relatively uncommon data type; compared to the ubiquity of tabular data, set data is less frequently used—making them novel compared to prior designs. They are hence an ideal exemplar of complex or scientific charts; offering fertile ground on which to explore alternative text design. To explore text descriptions for UpSet plots, we developed a system that automatically generates descriptive texts. To do so, we first identified common patterns in UpSet plots via a survey and qualitative analysis of published UpSet figures ( Sec. 3.1 ). Based on the results of these patterns, we conducted a design process for effective text descriptions on UpSet, consistently collecting feedback from our blind coauthor, and drawing on insights from prior work [ LS22 ] . We then implemented text generation that automatically detects the patterns we describe and generates the description. We leverage a JSON-based description of the UpSet plot and the original dataset in the process. We implement our tool as a web-API, so that all kinds of implementations of UpSet plots can leverage our text generation, and demonstrate the process with two instances: an interactive, web-based version of UpSet and a Python library. We design our texts generically, both to be generally useful and so that they can be manually adapted to specific situations. To evaluate the quality of our results, we conducted a semi-structured interview study with (N=11) BLV participants ( Sec. 4 ). We find that the text descriptions gave participants a “decent sense as to like what [UpSet Plots] might look like” and that they were informative and appropriate in length. We also received suggestions to improve the descriptions, for example, by adding a glossary and by structuring the text with bullet points. More essentially, we find that participants were generally able to conduct at least simple analytic tasks using text descriptions that would be normally carried out using the visual representations. We complement these findings with two contextualizing experiments, Sec. 5 . First, we explored whether our descriptive texts contained the same level of data analytic functionality as visual UpSets. To do so, we conducted a (N=83) crowd work study with sighted users. Although this does not show that our texts are equivalent to their visual representations, we do find that sighted users perform similarly well with text and visual versions in answering a series of factual questions. We also find that preference and correctness are highest for the combined text and visual condition, indicating that providing text descriptions for complex charts could have a “curb-cut” effect, benefiting both BLV and sighted users. This also constitutes a minor contribution: this

Read the complete text