SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenes

Chuhan Wang, Xintong Li, Jennifer Yuntong Zhang, Junda Wu, Chengkai Huang, Lina Yao, Julian McAuley, Jingbo Shang · 2026

Read SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenes by Chuhan Wang, Xintong Li, Jennifer Yuntong Zhang, Junda Wu, Chengkai Huang, Lina Yao, Julian McAuley, Jingbo Shang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05600.txt.

Description

Chuhan Wang and 7 co-authors present SceneAlign, a 2026 approach that trains multimodal models to reason faithfully about complex visual scenes. In crowded scenes, models often hallucinate objects, mislink relations, or skip reasoning steps. Text-only training tricks let models exploit language habits instead of actually looking.

SceneAlign uses scene graphs, structured maps of objects and relations, to create hard negative cases. It perturbs reasoning-critical nodes in four ways that mimic common grounding failures, producing explanations that sound plausible but rest on wrong visual facts. Preference training on those contrastive pairs pushes models toward accurate, step-by-step grounding. Across seven benchmarks, answer accuracy and reasoning faithfulness both raised consistently.

Catalog record

CREATORChuhan Wang, Xintong Li, Jennifer Yuntong Zhang, Junda Wu, Chengkai Huang, Lina Yao, Julian McAuley, Jingbo Shang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-05600

Browse the full library · About the Tower of Babel