Allocentric Perceiver: Disentangling Allocentric Reasoning from Egocentric Visual Priors via Frame Instantiation

Hengyi Wang, Ruiqiang Zhang, Chang Liu, Guanjie Wang, Zehua Ma, Han Fang, Weiming Zhang · 2026

Read Allocentric Perceiver: Disentangling Allocentric Reasoning from Egocentric Visual Priors via Frame Instantiation by Hengyi Wang, Ruiqiang Zhang, Chang Liu, Guanjie Wang, Zehua Ma, Han Fang, Weiming Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05789.txt.

Description

Answering where something sits from another object point of view defeats most vision language models, which stay stuck in the camera frame. Hengyi Wang and 6 co-authors introduce Allocentric Perceiver, a training free strategy for such allocentric questions. It first recovers metric three dimensional states from one or more images using off the shelf geometric tools.

The method then builds a query conditioned reference frame aligned with the target object and the intent of the instruction. Reconstructed geometry is transformed into that frame by direct computation, and the backbone model is prompted with structured, geometry grounded descriptions. Mental rotation is thus offloaded from implicit reasoning to explicit math. Across several model families and spatial reasoning benchmarks, allocentric scores rose by about a tenth while camera frame performance stayed strong. The system beat both fine tuned spatial models and leading proprietary models.

Catalog record

CREATORHengyi Wang, Ruiqiang Zhang, Chang Liu, Guanjie Wang, Zehua Ma, Han Fang, Weiming Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05789

Browse the full library · About the Tower of Babel