OmniAcc: Personalized Accessibility Assistant Using Generative AI

Siddhant Karki, Ethan Han, Nadim Mahmud, Suman Bhunia, John Femiani, Vaskar Raychoudhury · 2025

Read OmniAcc: Personalized Accessibility Assistant Using Generative AI by Siddhant Karki, Ethan Han, Nadim Mahmud, Suman Bhunia, John Femiani, Vaskar Raychoudhury (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07220.txt.

Introduction

People who use wheelchairs often cannot tell in advance whether a street has ramps or safe crossings. Siddhant Karki and 5 co-authors built OmniAcc, described in a 2025 research paper, to close that gap. OmniAcc is an AI assistant that combines GPT-4 with satellite imagery and OpenStreetMap data. It finds and maps features for wheelchair users, such as ramps and crosswalks.

The system plans personalized routes and gives hands-free navigation with instant spoken answers. It uses zero-shot learning, meaning it spots features without task-specific training examples. A case study on crosswalk detection reached high accuracy in the authors' tests. The authors argue such tools can help mobility-aid users travel with more confidence. They may also help city planners see where accessible infrastructure is missing.

Catalog record

CREATORSiddhant Karki, Ethan Han, Nadim Mahmud, Suman Bhunia, John Femiani, Vaskar Raychoudhury
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-07220

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OmniAcc: Personalized Accessibility Assistant Using Generative AI Siddhant Karki, Ethan Han, Nadim Mahmud, Suman Bhunia, John Femiani, Vaskar Raychoudhury Abstract Individuals with ambulatory disabilities often encounter significant barriers when navigating urban environments due to the lack of accessible information and tools. This paper presents OmniAcc, an AI-powered interactive navigation system that utilizes GPT-4, satellite imagery, and OpenStreetMap data to identify, classify, and map wheelchair-accessible features such as ramps and crosswalks in the built environment. OmniAcc offers personalized route planning, real-time hands-free navigation, and instant query responses regarding physical accessibility. By using zero-shot learning and customized prompts, the system ensures precise detection of accessibility features, while supporting validation through structured workflows. This paper introduces OmniAcc and explores its potential to assist urban planners and mobility-aid users, demonstrated through a case study on crosswalk detection. With a crosswalk detection accuracy of 97.5%, OmniAcc highlights the transformative potential of AI in improving navigation and fostering more inclusive urban spaces. Introduction Traveling across the built environment, especially in unfamiliar surroundings, is challenging for wheelchair users because of unknown barriers [ \citeauthoryearDing et al.2007 , \citeauthoryearVölkel, Kühn, and Weber2008 ] posed by the path as well as the destinations like public buildings. Uneven surfaces such as roads and sidewalks, the presence of stairs and steep slopes, the absence of curb cuts and pedestrian crosswalks, and varied weather conditions thwart successful outings for someone using a wheelchair. Similarly, accessible parking locations and accessible entrance(s) to a building or other public places are not always readily evident, and wheelchair users often need to look around for pointers or ask people, and sometimes there is no one to ask [ \citeauthoryearHammel et al.2015 ] . Our research team performed I-CORP interviews with 53 participants 1 1 1 https://uwm.edu/lubar-entrepreneurship-center/deep-dive-mypath/ including wheelchair users, their family members, accessibility advocates, and related experts who have repeatedly pointed out that the challenges of accessible end-to-end navigation persist despite the increasing focus on built environment accessibility. A college student using a mobility scooter said - “ I’m majoring in Economics…I was originally a Physics major…I wanted to be a Physicist, or an engineer, but the building in which all the Physics classes and labs were, was completely inaccessible and all my creativity could not…fix that and so, unfortunately, I was forced to change majors because of that ”. Due to the absence of sufficient credible prior information available on any common mapping platform, mobility-aid users often need to plan extensively before an outing. They usually browse through maps, street views, publicly available user reviews, forum discussions, and building plans, and/or call the destination facility to inquire about the current accessibility status of a destination facility. In addition, our I-CORP interview participants also felt the need for a navigation system that facilitates human-like query-response interaction in real-time with personalized routing support. This is required to help users with unforeseen routing challenges due to sudden temporary barriers such as roadblocks and diversions, etc. See Figure 1 for a use-case scenario. In this paper, we introduce the novel OmniAcc system built to provide AI-based real-time and interactive navigational assistance primarily to wheeled mobility-aid users. However, such a system requires reliable and extensive accessibility information. We plan to gradually enhance the OmniAcc system with accessibility information tapped from various reliable sources while focusing on saturating small geographical areas. OpenStreetMap (OSM) is a primary data source with user-contributed accessibility features or barriers, such as surface types, inclines, accessible entrances, stairs, ramps, crosswalks with/without pedestrian signals, etc. However, our close inspection reveals that OSM data is often incorrect, outdated, or just incomplete. Manually investigating and incorporating or updating all these path features in the OSM is time-consuming and infeasible for large geographic areas. Scalable detection and update of path features into the OSM are required for the successful operation of the OmniAcc system. In this paper, we specifically consider crosswalk as a path feature and focus on its successful detection from geospatial imagery (high-resolution GeoTIFF images). Once a crosswalk is detected correctly, it will be entered in the OSM and used for the route generation algorithm. Researchers have addressed the problem of crosswalk detection from satellite imagery. Verma et al. [ \citeauthoryearVerma and Ukkusuri2024 ] used deep learning, specifically the YOLO v5 object detection model, to automate crosswalk detection from satellite imagery. Antwi et al. [ \citeauthoryearAntwi et al.2024 ] developed an automated GIS-based framework using bi-temporal aerial imagery and a YOLO-based model to detect and update crosswalk changes in Florida counties. Hosseini et al. [ \citeauthoryearHosseini et al.2023 ] introduced TILE2NET, an open-source tool that uses semantic segmentation to extract sidewalk, crosswalk, and footpath data from high-resolution aerial imagery, enabling scalable and cost-effective pedestrian network generation for urban planning. In addition, Ning et al. [ \citeauthoryearNing et al.2022 ] developed a novel method combining aerial and street view imagery using convolutional neural networks to extract and refine sidewalk networks. Moran [ \citeauthoryearMoran2022 ] utilized satellite imagery to map crosswalk coverage across San Francisco, revealing uneven spatial distribution and disparities between neighborhoods. While these studies effectively detect crosswalks, most rely on large training datasets, which are not always reliable or readily available, limiting their scalability and applicability. Figure 1: The overarching goal of OmniAcc. Example of a conversation between a wheelchair user and the OmniAcc Chatbot. A key enabler of this is the automatic identification of features such as crosswalks. In contrast to the above, OmniAcc incorporates OpenAI’s GPT-4o model [ \citeauthoryearIslam and Moushi2024 ] as a general purpose multimodal model that allows zero-shot crosswalk detection, with labeled data used exclusively for evaluation. While LLMs have proven themselves as powerful low shot and zero shot classifiers, to the best of our knowledge, this is the first such attempt at applying multimodal large language models specifically for zero-shot road feature detection. In particular, we incorporate visual prompting strategies by exploring different overlays or annotations on the imagery in addition to traditional text-based prompt engineering. We chose the GPT-4o model due to its advanced multimodal understanding, which makes it well-suited for comprehending textual and image inputs for detecting wheelchair-accessible features. The use of high resolution orthophoto imagery is also useful for our solution, as it provides precise geospatial metadata and high-resolution detail necessary for identifying accessible features, such as crosswalks. We evaluate crosswalk detection on data collected from Oxford, Ohio, showcasing excellent performance with an accuracy of 97.5%. In this paper, we make the following unique contributions. • We propose OmniAcc, a novel Gen-AI-based real-time and interactive routing and navigation system for wheeled mobility-aid users. • We demonstrate that effective prompting strategies in a multimodal large language model (OpenAI’s GPT-4o model) enable zero-shot learning. We do this by efficiently processing high-resolution aerial imagery to identify and classify crosswalks without learning from labeled data. • We explore and evaluate novel visual prompting strategies, which go beyond text based prompt engineering by incorporating visual overlays to determine which annotations on the imagery are most effective in guiding the zero shot task. • Our extensive experimentation with different images shows that OmniAcc system can detect crosswalks with 97.5% accuracy, this is a very significant result in zero-shot image recognition. Related Work Research on enhancing urban accessibility focuses on three main areas: (1) pedestrian infrastructure detection,

Read the complete text