A11yShape: AI-Assisted 3-D Modeling for Blind and Low-Vision Programmers Conference: The 27th International ACM SIGACCESS Conference on Computers and Accessibility; October 26–29, 2025; Denver, CO, USA The 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’25), Octob

Zhuohao (Jerry) Zhang, Haichang Li, Chun Meng Yu, Faraz Faruqi, Junan Xie, Gene S-H Kim, Mingming Fan, Angus Forbes, Jacob O. Wobbrock, Anhong Guo, Liang He · 2025

Read A11yShape: AI-Assisted 3-D Modeling for Blind and Low-Vision Programmers Conference: The 27th International ACM SIGACCESS Conference on Computers and Accessibility; October 26–29, 2025; Denver, CO, USA The 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’25), Octob by Zhuohao (Jerry) Zhang, Haichang Li, Chun Meng Yu, Faraz Faruqi, Junan Xie, Gene S-H Kim, Mingming Fan, Angus Forbes, Jacob O. Wobbrock, Anhong Guo, Liang He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03852.txt.

Introduction

Zhuohao Zhang and colleagues built a tool in 2025 to help blind and low-vision programmers make three-dimensional models. Most modeling tools rely on sight, which shuts out users who cannot see the shape as it changes. Their system, A11yShape, works with a code-based model editor, where a shape is written as a program.

Spoken-style descriptions explain the current shape, a version history tracks each change, and a tree view lists the parts in order. A linking feature highlights the same part across code, tree, description, and rendering at once. In a study, four blind programmers learned the tool, then built a dozen models on their own with results they judged met their goals.

Catalog record

CREATORZhuohao (Jerry) Zhang, Haichang Li, Chun Meng Yu, Faraz Faruqi, Junan Xie, Gene S-H Kim, Mingming Fan, Angus Forbes, Jacob O. Wobbrock, Anhong Guo, Liang He
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE95 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-03852

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A11yShape: AI-Assisted 3-D Modeling for Blind and Low-Vision Programmers Conference: The 27th International ACM SIGACCESS Conference on Computers and Accessibility; October 26–29, 2025; Denver, CO, USA The 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’25), October 26–29, 2025, Denver, CO, USA DOI: 10.1145/3663547.3746362 ISBN: 979-8-4007-0676-9/2025/10 CCS: Human-centered computing Accessibility systems and tools CCS: Human-centered computing Accessibility technologies Zhuohao (Jerry) Zhang, Haichang Li, Chun Meng Yu, Faraz Faruqi, Junan Xie, Gene S-H Kim, Mingming Fan, Angus Forbes, Jacob O. Wobbrock, Anhong Guo, Liang He Abstract . Building 3-D models is challenging for blind and low-vision (BLV) users due to the inherent complexity of 3-D models and the lack of support for non-visual interaction in existing tools. To address this issue, we introduce A11yShape , a novel system designed to help BLV users who possess basic programming skills understand, modify, and iterate on 3-D models. A11yShape leverages LLMs and integrates with OpenSCAD, a popular open-source editor that generates 3-D models from code. Key functionalities of A11yShape include accessible descriptions of 3-D models, version control to track changes in models and code, and a hierarchical representation of model components. Most importantly, A11yShape employs a cross-representation highlighting mechanism to synchronize semantic selections across all model representations—code, semantic hierarchy, AI description, and 3-D rendering. We conducted a multi-session user study with four BLV programmers, where, after an initial tutorial session, participants independently completed 12 distinct models across two testing sessions, achieving results that aligned with their own satisfaction. The result demonstrates that participants were able to comprehend provided 3-D models, as well as independently create and modify 3-D models—tasks that were previously impossible without assistance from sighted individuals. Figure 1. With A11yShape, (A) a blind or low-vision (BLV) user can create, interpret, and verify 3-D models through (B) a user interface composed of three parts: Code Editor Panel, AI Assistant Panel, and Model Panel. These panels are linked by a cross-representation highlighting mechanism that connects code, textual descriptions, hierarchical model abstractions, and 3-D visual renderings. The system supports the creation of (C) diverse, customized 3-D models created by BLV users. A schematic overview of a blind or low-vision (BLV) user interacting with A11yShape to create 3-D models. Panel A shows the BLV user. Panel B illustrates the A11yShape interface, which includes three components: Code Editor panel, AI Assistant Panel providing textual descriptions, and Model Panel showing a hierarchical structure and 3-D renderings. A cross-representation highlighting mechanism links elements across these panels to support non-visual navigation in the user interface. Panel C displays examples of 3-D models created by the user. † † cc-license: by 1. Introduction Recent advances in large language models (LLMs) have significantly expanded opportunities for blind and low-vision (BLV) individuals to independently perform creative tasks previously inaccessible without sighted assistance. With the emergence of readily available and cost-effective LLMs, BLV users now have greater potential to independently engage with visually complex tasks like image editing ( Lee et al. 2024 ; Chang et al. 2024 ) or application programming ( Saben et al. 2024 ; Zhang et al. 2025 ) , potentially benefiting the 1.7% of programmers with visual impairments ( Stack Overflow 2022 ) and others previously excluded from programming due to visual barriers ( Mountapmbeme et al. 2022b ; Pandey et al. 2021 ; Albusays et al. 2017 ) . Despite these promising developments, highly intricate visual-spatial workflows such as three-dimensional (3-D) modeling remain under-explored in accessibility research. 3-D modeling challenges even sighted users due to its demands on spatial reasoning, complex visual interfaces, tricky input articulations, and mental visualization requirements. In the meantime, programming interfaces for 3-D modeling may provide a more accessible pathway for BLV users, as text-based code can transform abstract spatial concepts into concrete, rule-based instructions that don’t rely on visuals. For BLV programmers, designers, and students who need to understand and create 3-D models, investigating accessible approaches to 3-D modeling under these new possibilities is essential. We introduce A11yShape , an interactive 3-D modeling system developed through participatory design with a BLV co-author. Unlike traditional approaches that rely on author-curated captions for model descriptions, A11yShape leverages both the underlying model code and rendered images to produce detailed and accurate textual explanations of 3-D models, which have been validated through user studies to achieve high ratings in author-curated evaluation metrics. A11yShape integrates OpenSCAD ( OpenSCAD 2021 ) , a commonly used code-to-model environment, with advanced capabilities of GPT-4o to synthesize complementary textual descriptions. Furthermore, A11yShape extends interactive access through three attributes: (1) a hierarchical representation enabling structured navigation of model components, (2) integrated version control to track iterative model changes, and (3) an interactive verification loop allowing BLV users to directly query and validate spatial attributes or design decisions. Most importantly, A11yShape employs a dynamic cross-representation highlighting mechanism that synchronizes selections across multiple model representations, enabling users to seamlessly navigate among code, semantic hierarchy, AI-generated descriptions, and rendered elements. To investigate how BLV programmers engage with A11yShape, we conducted a multi-session study involving four BLV participants each completing three separate, successive sessions, totaling 12 sessions. We observed that, although with notable flaws like components being misaligned or overlapping in conflict, participants successfully performed previously inaccessible tasks to create three complete 3-D models both by guidance and in free-form over the sessions. After the initial tutorial session, participants independently created 12 distinct models across the testing sessions, with outcomes that met their own satisfaction. To encourage deeper engagement with the system, participants were given ample time, each spending approximately four hours to complete three models. Our findings revealed that the cross-representation highlighting mechanism enabled fluid navigation between different ways of understanding models, and AI-generated descriptions in particular were perceived as compensating for the lack of visual verification. Participants developed distinctive workflows based on varying levels of AI assistance and adopted strategic approaches including: incremental building through AI-verification loops, leveraging semantic hierarchies for error correction, and using real-world metaphors for mental model construction. Despite overall success, certain challenges emerged, including high cognitive load from interpreting textual descriptions, difficulty in understanding spatial relationships, and uncertainty about operation success in the absence of visual or tactile feedback. These findings suggest promising directions for assistive technologies that can empower BLV users to independently engage with inherently visual creative workflows. In summary, this paper makes the following contributions: • A11yShape, the first AI-assisted 3-D modeling system leveraging code-augmented LLM descriptions, hierarchical component navigation, and interactive verification loops. • Empirical insights from an extensive multi-session user study that reveals how BLV users navigate spatial cognition challenges, develop mental models, and employ different strategies to create desired 3-D models without visual feedback. 2. Related Work Our work is built upon prior work in three key areas: the use of LLMs and other AI techniques for generating and interpreting 3-D models, approaches that explore modalities for supporting BLV users in 3-D modeling and understanding, and AI-assisted tools that support creativity tasks for BLV users. 2.1. AI-Driven 3-D Model Generation Recent work has explored the use of generative models and large language models for automatic 3-D content creation in various applications, such as avatar and scene generation ( Li et al. 2023 ) and modifiable 3-D model asset generation ( Sun et al.

Read the complete text