Vision-Language System using Open-Source LLMs for Consent and Instruction Gestures in Medical Interpreter Robots DOI: 10.1145/3776734.3794357 ISBN: 979-8-4007-2321-6/2026/03 Conference: Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction; March 16–19, 2026; Edinburgh, Scotland, UK Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI Companion ’26), March 16–19, 2026, Edinburgh, Scotland, UK CCS: Computer systems organization Robotics CCS: Computer systems organization Real-time system architecture CCS: Human-centered computing User models CCS: Human-centered computing Natural language interfaces CCS: Applied computing Health informatics
Thanh-Tung Ngo, Emma Murphy, Robert J. Ross · 2026
Read Vision-Language System using Open-Source LLMs for Consent and Instruction Gestures in Medical Interpreter Robots DOI: 10.1145/3776734.3794357 ISBN: 979-8-4007-2321-6/2026/03 Conference: Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction; March 16–19, 2026; Edinburgh, Scotland, UK Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI Companion ’26), March 16–19, 2026, Edinburgh, Scotland, UK CCS: Computer systems organization Robotics CCS: Computer systems organization Real-time system architecture CCS: Human-centered computing User models CCS: Human-centered computing Natural language interfaces CCS: Applied computing Health informatics by Thanh-Tung Ngo, Emma Murphy, Robert J. Ross (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05751.txt.
Description
Ngo and team present a privacy-keeping framework for medical go-between robots spotting speech acts and making gestures. Talk is vital in healthcare across language walls, where gestures matter. Built on local open-source models, it uses few-shot prompting for intent finding. The authors present a data set of clinic talks tagged for speech acts and gesture clips. Their naming module hits 0.90 score with sharpness and F1-score. The approach improves speed and, in user studies, beats the speech-gesture base on human-likeness while keeping similar fitness.
Catalog record
| CREATOR | Thanh-Tung Ngo, Emma Murphy, Robert J. Ross |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21014 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.05751