Bridging the Interpretation Gap in Accessibility Testing: Empathetic and Legal-Aware Bug Report Generation via Large Language Models

Ryoya Koyama, Jialong Li*, Zhiyao Wang, Devi Karolita, Kenji Tei · 2026

Read Bridging the Interpretation Gap in Accessibility Testing: Empathetic and Legal-Aware Bug Report Generation via Large Language Models by Ryoya Koyama, Jialong Li*, Zhiyao Wang, Devi Karolita, Kenji Tei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23828.txt.

Introduction

Koyama and co-authors present HEAR, a system that turns mobile accessibility test output into clear bug reports. Test tools flag interface violations, but their raw logs are hard for managers and designers to act on. HEAR rebuilds the screen context, adds a disability persona matched to each violation, and explains the barrier and legal risk. They test it on real issues from four popular Android apps, with a study of 12 users. Reports felt more empathetic and urgent, with little extra reading burden.

Catalog record

CREATORRyoya Koyama, Jialong Li*, Zhiyao Wang, Devi Karolita, Kenji Tei
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37903
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.23828

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bridging the Interpretation Gap in Accessibility Testing: Empathetic and Legal-Aware Bug Report Generation via Large Language Models * Corresponding Author: [email protected] Ryoya Koyama, Jialong Li*, Zhiyao Wang, Devi Karolita, Kenji Tei Abstract Modern automated accessibility testing tools for mobile applications have significantly improved the detection of interface violations, yet their impact on remediation remains limited. A key reason is that existing tools typically produce low-level, technical outputs that are difficult for non-specialist stakeholders, such as product managers and designers, to interpret in terms of real user harm and compliance risk. In this paper, we present HEAR ( H uman-c E ntered A ccessibility R eporting), a framework that bridges this interpretation gap by transforming raw accessibility bug reports into empathetic, stakeholder-oriented narratives. Given the outputs of the existing accessibility testing tool, HEAR first reconstructs the UI context through semantic slicing and visual grounding, then dynamically injects disability-oriented personas matched to each violation type, and finally performs multi-layer reasoning to explain the physical barrier, functional blockage, and relevant legal or compliance concerns. We evaluate the framework on real-world accessibility issues collected from four popular Android applications and conduct a user study (N=12). The results show that HEAR generates factually grounded reports and substantially improves perceived empathy, urgency, persuasiveness, and awareness of legal risk compared with raw technical logs, while imposing little additional cognitive burden. Reproducibility Package: https://github.com/HaRRrrusame/hear-a11y-llm . I Introduction Mobile applications have become the primary gateway for essential daily activities, ranging from financial banking and healthcare management to social interaction and emergency services. Ensuring equitable access to these digital ecosystems is not merely a legal obligation mandated by regulations such as the Americans with Disabilities Act (ADA) and the European Accessibility Act (EAA), but a fundamental human right. According to the World Health Organization, over 1 billion people—approximately 16% of the global population—live with some form of disability [ 27 ] . Consequently, the software engineering community has studied automated techniques to detect accessibility barriers in mobile apps. In recent years, the landscape of automated accessibility testing has evolved significantly. Industry-standard tools like Google Accessibility Scanner [ 7 ] and Lint-based static analyzers provide developers with immediate feedback on interface compliance. More recently, the research community has introduced dynamic crawling frameworks, such as Groundhog [ 21 ] and Latte [ 20 ] , which simulate user interactions to uncover deep-seated issues that manifest only during runtime execution. These tools have successfully lowered the technical barrier for detection , enabling the identification of thousands of violations across millions of apps. However, a critical issue remains: while the capability to detect accessibility bugs has improved, the rate of remediation (fixing) remains disappointingly low. A key reason for this stagnation is not the absence of data, but the inaccessibility of the data itself to non-technical stakeholders. Current tools typically output low-level, syntax-oriented logs—such as JSON files containing view hierarchies, coordinate bounds, and cryptic error codes (e.g., bounds=[10,10][20,20], expected 48dp ). This technical representation creates a profound ”interpretation gap” within software development teams. Product Managers (PMs) and UI Designers, who typically control the development roadmap and resource allocation, often lack the specialized knowledge to map these technical metrics to actual user experiences. To a PM focused on conversion rates, a log entry stating ”Touch target size is 32dp” may appear as a trivial ”polish” issue with low priority. They fail to perceive the semantic reality: that for a user with Parkinson’s disease, this ”minor” issue effectively blocks a critical payment workflow. Consequently, accessibility fixes are frequently deprioritized in favor of new features, simply because the disability user impact and legal risk of the barriers are lost in interpretation. To this end, this paper proposes HEAR ( H uman-c E ntered A ccessibility R eporting), an automated framework designed to bridge this interpretation gap by transforming cold, code-centric testing results into persuasive, human-centric narratives that evoke stakeholder empathy and highlight urgency. Specifically, HEAR primarily consists of two key steps. First, it dynamically maps accessibility violations to specialized user personas; for instance, a ”small touch target” violation is mapped to the specific lived experience of a user with Parkinson’s disease who struggles with motor precision. Second, the framework enriches these narratives by integrating dual-perspective impact assessments: it simulates the user-centric hurdle to demonstrate how a bug leads to workflow abandonment, while simultaneously providing a regulatory-centric risk analysis based on local accessibility laws. By synthesizing these perspectives into an intuitive, high-level report, our framework empowers stakeholders to recognize the functional severity and legal vulnerability underlying each accessibility bug. To summarize, this paper makes the following contributions: • We identify the challenge of the “interpretation gap” in current accessibility testing workflows, highlighting how the lack of semantic context in technical logs hinders effective remediation by stakeholders. • We introduce HEAR , an automated framework that reconstructs UI context from decoupled logs and screenshots. By dynamically injecting ability-based personas and utilizing Chain-of-Thought (CoT) reasoning, the framework translates technical violations into human-centric narratives of user struggle and legal risk. • We evaluate the reliability of HEAR on 103 bug instances (from four popular Android applications and conduct a user study ( N=12 ) to assess its utility. The remainder of this paper is organized as follows. Section II provides the necessary background. Section III details our proposed framework. Section IV presents the experimental setup and results. Section V discusses related work. Finally, Section VI concludes the paper and outlines future directions. II Background II-A Automated Accessibility Testing In the field of modern mobile accessibility testing, Google’s ecosystem provides the de facto standard for the Android platform. The core of this ecosystem is the Accessibility Testing Framework (ATF), a library that performs static analysis on Android UI components. Tools such as the Google Accessibility Scanner and Android Lint utilize ATF to traverse the View Hierarchy ( T_{xml} ) of a rendered screen, evaluating individual UI elements against a set of heuristic rules derived from the Web Content Accessibility Guidelines (WCAG). These automated tools typically prioritize three primary categories of verification. First, they validate Touch Target Size, ensuring that interactive elements possess a density-independent pixel (dp) size of at least 48\times 48 dp to accommodate users with limited fine motor control. Second, they perform Content Labeling checks, verifying that visual elements, such as an ImageView , possess textual descriptions (e.g., contentDescription ) necessary for screen readers to convey meaning to visually impaired users. Third, they analyze Color Contrast, calculating the ratio between foreground text and background colors to ensure readability for users with low vision. Despite their scalability, the output of these tools is inherently low-level and data-centric. Results are typically serialized as JSON containing geometric bounds and cryptic error codes, lacking the semantic context required for developers to fully understand the user’s struggle. II-B Personas in Software Engineering In the domain of Software Engineering (SE), personas have emerged as a critical technique for requirements engineering and user-centered design [ 10 ] . A persona is a fictional, yet realistic, archetype representing a specific segment of users. In SE workflows, personas serve as a communication tool to facilitate a shared understanding of user needs among diverse stakeholders, including product managers, designers, and developers. They are instrumental in preventing the ”Elastic User” phenomenon,

Read the complete text