Beyond Unimodal Shortcuts: MLLMs as Cross-Modal Reasoners for Grounded Named Entity Recognition

Jinlong Ma, Yu Zhang, Xuefeng Bai, Kehai Chen, Yuwei Wang, Zeming Liu, Jun Yu, Min Zhang · 2026

Read Beyond Unimodal Shortcuts: MLLMs as Cross-Modal Reasoners for Grounded Named Entity Recognition by Jinlong Ma, Yu Zhang, Xuefeng Bai, Kehai Chen, Yuwei Wang, Zeming Liu, Jun Yu, Min Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04486.txt.

Description

Jinlong Ma and 7 co-authors tackle grounded named entity recognition in a 2026 paper. The task asks a system to find named things in text, label their type, and point to the matching region in an image.

The authors found that large multimodal models often take shortcuts. A model may lean on the picture alone or on the text alone instead of checking both together. Their method, MCR, injects structured reasoning steps that force cross-checking between the two sources. A second component adjusts the model's reasoning using verifiable constraints. In tests on entity recognition and visual grounding, MCR reduced this bias and beat earlier baselines.

Catalog record

CREATORJinlong Ma, Yu Zhang, Xuefeng Bai, Kehai Chen, Yuwei Wang, Zeming Liu, Jun Yu, Min Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-04486

Browse the full library · About the Tower of Babel