Playing telephone with generative models: “verification disability,” “compelled reliance,” and accessibility in data visualization

Frank Elavsky Carnegie Mellon University, Cindy Xiong Bearfield Georgia Tech · 2025

Read Playing telephone with generative models: “verification disability,” “compelled reliance,” and accessibility in data visualization by Frank Elavsky Carnegie Mellon University, Cindy Xiong Bearfield Georgia Tech (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-12192.txt.

Introduction

Frank Elavsky and Cindy Xiong Bearfield warn about generative models as accessibility aids in data visualization. Their twenty twenty-five paper joins two fields of expertise: access work and bias study. Models now write charts and also describe charts for blind and low-vision readers.

The catch is checking. A blind reader cannot verify a model chart description or probe its bias. The authors name this a verification disability, where reliance is compelled rather than earned by trust. Human bias in chart talk was already a known risk. Adding a model middleman can hide that bias. The paper urges designs that let users question outputs, not just receive them.

Catalog record

CREATORFrank Elavsky Carnegie Mellon University, Cindy Xiong Bearfield Georgia Tech
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-12192

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Playing telephone with generative models: “verification disability,” “compelled reliance,” and accessibility in data visualization Frank Elavsky Carnegie Mellon University, Cindy Xiong Bearfield Georgia Tech Abstract This paper is a collaborative piece between two worlds of expertise in the field of data visualization: accessibility and bias. In particular, the rise of generative models playing a role in accessibility is a worrying trend for data visualization. These models are increasingly used to help author visualizations as well as generate descriptions of existing visualizations for people who are blind, low vision, or use assistive technologies such as screen readers. Sighted human-to-human bias has already been established as an area of concern for theory, research, and design in data visualization. But what happens when someone is unable to verify the model output or adequately interrogate algorithmic bias, such as a context where a blind person asks a model to describe a chart for them? In such scenarios, trust from the user is not earned, rather reliance is compelled by the model-to-human relationship. In this work, we explored the dangers of AI-generated descriptions for accessibility, playing a game of telephone between models, observing bias production in model interpretation, and re-interpretation of a data visualization. We unpack ways that model failure in visualization is especially problematic for users with visual impairments, and suggest directions forward for three distinct readers of this piece: technologists who build model-assisted interfaces for end users, users with disabilities leveraging models for their own purposes, and researchers concerned with bias, accessibility, or visualization. \onlineid 0 \vgtccategory Research \vgtcinsertpkg \teaser In a game of telephone, we had models describe a visualization and then a new model instance had to create a visualization based only on the first model’s description. We recursively repeated this 3 times across 2 different models. Models produced a variety of different kinds of bias in the descriptions and subsequent image constructions, some with specifically troublesome ramifications for blind users in particular. Introduction keywords Visualization, accessibility, bias The visualization community has long recognized the importance of addressing human-to-human bias in data representation and interpretation [ 49 ] . Yet as the visualization community increasingly delegates the work of visualization interpretation to generative AI systems, we risk creating new forms of bias that are more subtle, more systematic, and more difficult to detect and correct [ 32 , 11 ] . The rapid integration of generative AI models into accessibility tools presents a profound shift in how people with disabilities interact with data visualizations [ 37 , 21 ] . While these technologies promise to democratize access to visual information [ 35 ] , they simultaneously introduce new forms of bias that threaten to undermine the very accessibility they claim to enhance [ 42 ] . This intersection of generative AI, accessibility, and data visualization demands urgent attention from our research community, not merely as a technical challenge to be solved, but as a fundamental threat to equitable access to information. A doctor reviewing patient mortality rates, a policymaker analyzing crime data, and a financial advisor studying market trends can all depend on data visualizations to make critical decisions. But when a blind or low-vision user encounters the same visualizations, they may have no choice but to rely on a generative model’s interpretation. Suddenly, an AI system becomes the sole mediator between life-altering information and high-stake decisions. Unlike traditional accessibility tools that provide structured, verifiable access to data, or when descriptions are human-authored and author-verified, generative models operate as black boxes, producing descriptions that users cannot independently validate. This creates a dangerous power asymmetry: the model’s interpretation becomes the user’s reality, regardless of its accuracy, completeness, or embedded biases. We refer to this process with asymmetrical power dynamics as “compelled reliance,” where users are provided no reasonable alternatives except to rely on the model output. The current landscape of AI-assisted accessibility tools suffers from a troubling lack of accountability mechanisms [ 33 ] . When a model generates an incorrect or biased description, who bears responsibility? The model developer? The application creator? The institution deploying the technology? This diffusion of responsibility creates an accountability vacuum that leaves users with disabilities particularly vulnerable to harm with no path forward for justice [ 12 ] . To demonstrate the danger of compelled reliance on visualization and chart descriptions created by generative models, we played a game of ‘telephone’ with two popular LLM models: ChatGPT (model 4o) and Claude (Sonnet 4) [ 34 , 2 ] . Telephone is a game where information is passed from one person to the next, and at the end it loops back to the originator of the information. We demonstrate how information lost or manipulated upstream can produce unsolvable problems downstream in the context of visualization design and chart description generation. In our experimentation, we started with a human-generated visualization, then asked for a description, sent the description to a model to create a visualization, used the visualization as a prompt to ask for a new description, and so on. We didn’t experiment with models intending to prove that bias exists (other work has already explored generative model bias, such as [ 23 , 46 , 7 ] ), but rather to expose observations-as-provocations for discussion. We offer this demonstration as a call to action for AI, accessibility, and visualization researchers. We aim to catalyze much-needed conversations about the ethical implications of using generative AI for visual analytics and push our community toward more responsible development, deployment, interrogation, and even non-use of these powerful technologies. 1 Why does machine bias in visualization accessibility matter? People often rely on heuristics and biases to make decisions [ 24 ] , but access to visualizations and visual analytics promises more evidence-based insights and potentially more optimal data-driven decisions. However, even in the most pristine data analytics pipeline, bias is inevitably woven into every stage of authoring visualizations, whether the work is conducted by humans or machines [ 45 , 20 , 36 , 52 , 15 ] . For example, the collection and omission of data is a well-documented source of bias. D’Ignazio and Klein’s work on ‘capta’ demonstrates how what gets counted and what doesn’t can shape our understanding of social phenomena [ 14 ] . In visual analytics, missing data patterns can mislead viewers when not properly encoded [ 45 , 39 ] . The sifting, sorting, and cleaning of data can further introduce additional biases [ 51 ] . When the data is ready to be visualized, design choices, such as selecting a chart type, a color palette, or a textual annotation, can prime viewers toward specific conclusions [ 6 , 27 , 26 ] . Design artifacts to enhance data accessibility, such as generating captions or alt-texts for blind and low-vision users, often well-intentioned (and even necessary for access), can nonetheless emphasize different aspects of the same data, potentially exacerbating algorithmic biases and social biases [ 29 ] . It’s all bias, all the way down. So “are we doomed?” as Michael Correll has remarked multiple past occasions [ 13 ] , or does this present an opportunity for the visualization community to take action against the expanding constellation of biases that generative AI has introduced to

Read the complete text