Direction-aware 3D Large Multimodal Models

Quan Liu Weihao Xuan Junjue Wang Naoto Yokoya Ling Shao Shijian Lu · 2026

Read Direction-aware 3D Large Multimodal Models by Quan Liu Weihao Xuan Junjue Wang Naoto Yokoya Ling Shao Shijian Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19063.txt.

Description

Three-dimensional multimodal models need to know which way they face. Quan Liu, with five co-authors, fixes a basic flaw in 2026. Point-cloud benchmarks offer rich directional questions but omit ego poses. Without poses, directional reasoning is ill-posed.

The authors define a rigorous direction-aware paradigm. Ego poses are identified and added to benchmarks. Point clouds are then transformed to match those poses. Two designs drive the work. PoseRecover recovers poses automatically from RGB-D video extrinsics, using object-frustum intersection and visibility checks. PoseAlign transforms point clouds to align with poses, rather than stuffing poses into text prompts or projection layers. Tests across several backbones show consistent gains. ScanRefer overlap rises by nearly a third. Scan2Cap judge accuracy rises by over a tenth. Instruction tuning alone suffices.

Catalog record

CREATORQuan Liu Weihao Xuan Junjue Wang Naoto Yokoya Ling Shao Shijian Lu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-19063

Browse the full library · About the Tower of Babel