ScriptHOI: Learning Scripted State Transitions for Open-Vocabulary Human-Object Interaction Detection

Minh Anh Nguyen, Quang Huy Tran, Bao Ngoc Le, SuiYang Guang, Tuan Kiet Pham, Linh Chi Vo · 2026

Read ScriptHOI: Learning Scripted State Transitions for Open-Vocabulary Human-Object Interaction Detection by Minh Anh Nguyen, Quang Huy Tran, Bao Ngoc Le, SuiYang Guang, Tuan Kiet Pham, Linh Chi Vo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05057.txt.

Description

Minh Anh Nguyen and five co-authors improve open-vocabulary detection of human-object interactions, where the system must recognise action phrases never labelled in training. Vision-language detectors often guess an interaction from objects alone, for instance predicting cutting from a knife and cake being present. Such guesses skip whether hand, contact, and object state truly support the action.

Their ScriptHOI framework writes each phrase as a soft script of state changes, split into slots for body role, contact, shape, affordance, motion, and object state. A visual tokenizer reads the same slots from each detected pair, and a matcher scores coverage and conflict. Those scores calibrate predictions and expose missing visual evidence.

Catalog record

CREATORMinh Anh Nguyen, Quang Huy Tran, Bao Ngoc Le, SuiYang Guang, Tuan Kiet Pham, Linh Chi Vo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.05057

Browse the full library · About the Tower of Babel