Vision-Language System using Open-Source LLMs for Consent and Instruction Gestures in Medical Interpreter Robots DOI: 10.1145/3776734.3794357 ISBN: 979-8-4007-2321-6/2026/03 Conference: Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction; March 16–19, 2026; Edinburgh, Scotland, UK Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI Companion ’26), March 16–19, 2026, Edinburgh, Scotland, UK CCS: Computer systems organization Robotics CCS: Computer systems organization Real-time system architecture CCS: Human-centered computing User models CCS: Human-centered computing Natural language interfaces CCS: Applied computing Health informatics

Thanh-Tung Ngo, Emma Murphy, Robert J. Ross · 2026

Read Vision-Language System using Open-Source LLMs for Consent and Instruction Gestures in Medical Interpreter Robots DOI: 10.1145/3776734.3794357 ISBN: 979-8-4007-2321-6/2026/03 Conference: Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction; March 16–19, 2026; Edinburgh, Scotland, UK Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI Companion ’26), March 16–19, 2026, Edinburgh, Scotland, UK CCS: Computer systems organization Robotics CCS: Computer systems organization Real-time system architecture CCS: Human-centered computing User models CCS: Human-centered computing Natural language interfaces CCS: Applied computing Health informatics by Thanh-Tung Ngo, Emma Murphy, Robert J. Ross (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05751.txt.

Description

Ngo and team present a privacy-keeping framework for medical go-between robots spotting speech acts and making gestures. Talk is vital in healthcare across language walls, where gestures matter. Built on local open-source models, it uses few-shot prompting for intent finding. The authors present a data set of clinic talks tagged for speech acts and gesture clips. Their naming module hits 0.90 score with sharpness and F1-score. The approach improves speed and, in user studies, beats the speech-gesture base on human-likeness while keeping similar fitness.

Catalog record

CREATORThanh-Tung Ngo, Emma Murphy, Robert J. Ross
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21014
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05751

Browse the full library · About the Tower of Babel