Learning Situated Awareness in the Real World

Chuhan Li, Ruilin Han, Joy Hsu, Yongyuan Liang, Rajiv Dhawan, Jiajun Wu, Ming-Hsuan Yang, Xin Eric Wang · 2026

Read Learning Situated Awareness in the Real World by Chuhan Li, Ruilin Han, Joy Hsu, Yongyuan Liang, Rajiv Dhawan, Jiajun Wu, Ming-Hsuan Yang, Xin Eric Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-16682.txt.

Description

Chuhan Li and co-authors introduce SAW-Bench, a benchmark for situated awareness in real-world egocentric video. Situated awareness means relating oneself to the nearby physical world and reasoning about actions in context. Most multimodal benchmarks stress object-to-object spatial links, overlooking observer-centric links tied to viewpoint, pose, and motion.

SAW-Bench holds hundreds of self-recorded videos shot with Ray-Ban Meta smart glasses across varied indoor and outdoor scenes, plus over two thousand human-annotated question-answer pairs. Six awareness tasks probe observer-centric understanding. Even the best model tested, Gemini 3 Flash, trails humans by a wide margin. Models exploit partial geometric cues yet often fail to infer coherent camera geometry, a systematic spatial reasoning gap.

Catalog record

CREATORChuhan Li, Ruilin Han, Joy Hsu, Yongyuan Liang, Rajiv Dhawan, Jiajun Wu, Ming-Hsuan Yang, Xin Eric Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE147 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-16682

Browse the full library · About the Tower of Babel