From Woofs to Words: Towards Intelligent Robotic Guide Dogs with Verbal Communication
Yohei Hayamizu, David DeFazio, Hrudayangam Mehta, Zainab Altaweel, Jacqueline Choe · 2026
Read From Woofs to Words: Towards Intelligent Robotic Guide Dogs with Verbal Communication by Yohei Hayamizu, David DeFazio, Hrudayangam Mehta, Zainab Altaweel, Jacqueline Choe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-12574.txt.
Introduction
Yohei Hayamizu and co-authors give robotic guide dogs a way to speak with their handlers. Such robots help blind users avoid obstacles and find their way. Plain dialogue add-ons fail here, since speech must track a fast-changing scene. Their system uses large language models to voice both route plans and scene details. That lets handler and robot make joint choices as they move. A human study tested wording plans, and a simulation checked route speed and accuracy.
Catalog record
| CREATOR | Yohei Hayamizu, David DeFazio, Hrudayangam Mehta, Zainab Altaweel, Jacqueline Choe |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33355 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.12574
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Woofs to Words: Towards Intelligent Robotic Guide Dogs with Verbal Communication Yohei Hayamizu, David DeFazio, Hrudayangam Mehta, Zainab Altaweel, Jacqueline Choe Abstract Assistive robotics is an important subarea of robotics that focuses on the well-being of people with disabilities. A robotic guide dog is an assistive quadruped robot that helps visually impaired people in obstacle avoidance and navigation. Enabling language capabilities for robotic guide dogs goes beyond naively adding an existing dialog system onto a mobile robot. The novel challenges include grounding language in the dynamically changing environment and improving spatial awareness for the human handler. To address those challenges, we develop a novel dialog system for robotic guide dogs that uses LLMs to verbalize both navigational plans and scenes. The goal is to enable verbal communication for collaborative decision-making within the handler-robot team. In experiments, we conducted a human study to evaluate different verbalization strategies and a simulation study to assess the efficiency and accuracy in navigation tasks. Website — https://sites.google.com/view/woofs-words 1 Introduction Guide dogs play a crucial role in the lives of visually impaired individuals, enhancing their independence, confidence, companionship, and mobility ( 48 ) . Unfortunately, breeding and training guide dogs takes multiple years and can be very costly ( 45 ) , and even then, guide dog training centers have less than a 50% graduation rate ( 34 ) . At the same time, many people with visual impairments need guide dogs but have difficulties caring for them, e.g., walking miles a day and dog allergies. As a result, only a very small portion of the visually impaired use guide dogs, e.g., about 2% in the U.S. ( 34 ) and only about 400 guide dogs for more than 10 million visually impaired people in China ( 47 ) . While biological guide dogs are helpful in leading the visually impaired individuals around obstacles, they have fundamental limitations in understanding and communicating with humans. Even after systematic training, guide dogs can only respond to very short, predefined verbal cues ( 19 ) . Robotic guide dogs have the potential to be valuable navigation aids for visually impaired individuals ( 46 ; 49 ; 14 ; 26 ; 30 ; 18 ; 25 ; 24 ) . To the best of our knowledge, there was no existing research focusing on natural language interactions for robotic guide dogs. Figure 1: A legally blind person walking with our robotic guide dog system during a participant study. The integration of dialog systems into robotic guide dogs introduces unique challenges not present in virtual dialog agents. First, it is important for the dialog system to ground open-vocabulary language to the current environment, to generate relevant and feasible navigation goals, as well as navigational plans for realizing the goals. For instance, when a handler says “ I am thirsty ”, the dialog system should provide options that are available, e.g., by saying “ There is a vending machine that serves water bottles and a water fountain on this floor ”, and identify the handler’s preference after providing additional information, such as “ Navigating to the water fountain requires going through a long corridor and entering a kitchen room. ’’. A second challenge is ensuring the spatial awareness of the visually impaired handler, which is crucial for the handler to make informed decisions about routes and destinations. 1 1 1 A common misunderstanding is that a visually impaired handler completely follows the biological guide dog in navigation. The reality is that they work as a team, where the handler makes decisions about the routes (e.g., using tactile and echolocation sensing), and the guide dog helps avoid obstacles. Figure 2: Our system uses human-robot dialog to define a formal service task. An LLM first determines relevant navigable locations, and a task planner generates multiple action sequences (plans) for each candidate. These plans are summarized for the human via plan verbalization , detailing metrics like navigation cost and door openings. After human selection, the robot executes the chosen plan, providing navigation guidance while providing scene verbalization to describe the surroundings. In this paper, we develop a dialog system that leverages a large language model (LLM) ( 1 ) for dialog management and extracting navigation tasks from open-vocabulary service requests. We integrate task planning into language synthesis to enrich LLM responses with candidate navigation plans , which is referred to as plan verbalization . After the handler-robot team agrees on a navigation task, the robot will take the sequence of actions generated by the planner, e.g., open door X , enter corridor Y , and approach room Z , to fulfill the service request. During navigation, our dialog interface verbalizes scene descriptions to improve the human handler’s spatial awareness and facilitate collaborative decision-making, referred to as scene verbalization . The plan verbalization (before navigation) and scene verbalization (during navigation) capabilities together identify the main novelty of our proposed dialog system for robotic guide dogs. We demonstrate the capabilities of our guide dog robot (see Figure 1 ) through a human study with visually impaired individuals. This study, which involves real-world indoor navigation tasks, allowed us to evaluate the system’s performance and analyze its potential societal contributions. To supplement these real-world findings, we also conducted simulation experiments to quantitatively measure the accuracy of our dialog system in identifying navigation tasks from open-vocabulary language. 2 Related Work In this section, we discuss existing robotic guide dog systems, human-robot dialog systems, and non-robotic navigational aid systems for visually impaired people, highlighting how our system differs in each of these categories. Robotic Guide Dog Systems have been developed to assist visually impaired individuals in navigating, thanks to reduced hardware costs and recent advances in quadruped locomotion control. Systems of this type perform human-robot co-navigation tasks, where a human holds a leash to establish a physical connection with a quadruped robot ( 15 ; 24 ) . Some of these systems only support unidirectional robot-to-human communication via the forces the human feels from the leash ( 46 ; 49 ; 14 ) . In those systems, the robot helps the human avoid obstacles at the motion level, and the human completely follows the robot during navigation, assuming both sides know the goal ahead of time. Other systems allow for bidirectional communication, such that the human can indicate desired navigation directions via buttons ( 26 ) , predefined voice commands ( 30 ; 20 ; 29 ) , or tugs on the leash ( 18 ) during navigation. Compared with those robotic guide dog systems, ours further supports human-robot communication via bi-directional, multi-turn dialog. Robot Dialog Systems have greatly improved the interaction capabilities of robots ( 42 ; 7 ; 22 ) . These systems are optimized for different goals, including mediating the human-robot perception gap ( 9 ) , learning from dialog experiences ( 3 ) , social dialog ( 11 ) , learning navigation actions from dialog histories ( 43 ) , and understanding ambiguous human instructions ( 44 ) . Earlier works have focused on constructing formal navigation actions from natural language instructions ( 12 ) or commands ( 35 ) . While effective for various dialog tasks, none of the above works were designed for the context