Grounded 3D-Aware Spatial Vision-Language Modeling

An-Chieh Cheng Yang Fu Yatai Ji Ligeng Zhu Guanqi Zhan Zhuoyang Zhang, [0.4em] Zhaojing Yang Song Han Yao Lu Pavlo Molchanov Vidya Nariyambut Murali Jan Kautz, [0.4em] Xiaolong Wang Hongxu Yin Sifei Liu, [0.4em] UCSD · 2026

Read Grounded 3D-Aware Spatial Vision-Language Modeling by An-Chieh Cheng Yang Fu Yatai Ji Ligeng Zhu Guanqi Zhan Zhuoyang Zhang, [0.4em] Zhaojing Yang Song Han Yao Lu Pavlo Molchanov Vidya Nariyambut Murali Jan Kautz, [0.4em] Xiaolong Wang Hongxu Yin Sifei Liu, [0.4em] UCSD (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30307.txt.

Description

An-Chieh Cheng and 14 co-authors present GR3D, a spatial vision language model with explicit, implicit and monocular grounding. An implicit mechanism identifies entity mentions during generation and inserts region tokens into the text stream, letting the model cite visual evidence. A monocular 3D design predicts bounding boxes from grounded region queries, with intrinsic-aware normalization and geometric supervision. These capabilities let GR3D decompose spatial problems into 2D perception followed by 3D inference. GR3D improves across grounded and non-grounded benchmarks, showing grounding strengthens spatial understanding.

Catalog record

CREATORAn-Chieh Cheng Yang Fu Yatai Ji Ligeng Zhu Guanqi Zhan Zhuoyang Zhang, [0.4em] Zhaojing Yang Song Han Yao Lu Pavlo Molchanov Vidya Nariyambut Murali Jan Kautz, [0.4em] Xiaolong Wang Hongxu Yin Sifei Liu, [0.4em] UCSD
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53672
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30307

Browse the full library · About the Tower of Babel