Cross-Modal Coreference Alignment: Enabling Reliable Information Transfer in Omni-LLMs

Hongcheng Liu, Yuhao Wang, Zhe Chen, Pingjie Wang, Zhiyuan Zhu, Yixuan Hou, Yanfeng Wang, Yu Wang Shanghai Jiao Tong University {hongcheng_liu, colane, yuwangsjtu}@sjtu.edu.cn · 2026

Read Cross-Modal Coreference Alignment: Enabling Reliable Information Transfer in Omni-LLMs by Hongcheng Liu, Yuhao Wang, Zhe Chen, Pingjie Wang, Zhiyuan Zhu, Yixuan Hou, Yanfeng Wang, Yu Wang Shanghai Jiao Tong University {hongcheng_liu, colane, yuwangsjtu}@sjtu.edu.cn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05522.txt.

Description

Liu and seven colleagues target a weak spot in omni-modal language models: matching the same referent across sight, sound, and text. They formalize cross-modal coreference, build the nine-task CrossOmni dataset with human-written reasoning traces, and test 13 models that show systematic gaps. Both a training-free prompting fix and a training-based method lift performance and carry over to joint reasoning tasks.

Catalog record

CREATORHongcheng Liu, Yuhao Wang, Zhe Chen, Pingjie Wang, Zhiyuan Zhu, Yixuan Hou, Yanfeng Wang, Yu Wang Shanghai Jiao Tong University {hongcheng_liu, colane, yuwangsjtu}@sjtu.edu.cn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65836
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.05522

Browse the full library · About the Tower of Babel