Robust Egocentric Visual Attention Prediction Through Language-guided Scene Context-aware Learning

Sungjune Park, Hongda Mao, Qingshuang Chen, Yong Man Ro, Yelin Kim · 2026

Read Robust Egocentric Visual Attention Prediction Through Language-guided Scene Context-aware Learning by Sungjune Park, Hongda Mao, Qingshuang Chen, Yong Man Ro, Yelin Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01818.txt.

Description

Predicting where a person wearing a camera will look next is valuable for assistive technology, but first-person video is cluttered and ambiguous. Sungjune Park, Hongda Mao, Qingshuang Chen, Yong Man Ro, and Yelin Kim lean on scene context, summarized in language, to guide attention prediction.

A context perceiver first turns a language description of the scene into context-aware video features. Training then rewards focus on the true point of interest and penalizes attention to regions unlikely to draw a first-person gaze. The framework was tested on the Ego4D and Aria Everyday Activities datasets. It reached state-of-the-art accuracy. It stayed robust across varied, dynamic scenes. A verbal summary of a scene, it seems, can steady visual attention models.

Catalog record

CREATORSungjune Park, Hongda Mao, Qingshuang Chen, Yong Man Ro, Yelin Kim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-01818

Browse the full library · About the Tower of Babel