SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenes
Chuhan Wang, Xintong Li, Jennifer Yuntong Zhang, Junda Wu, Chengkai Huang, Lina Yao, Julian McAuley, Jingbo Shang · 2026
Read SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenes by Chuhan Wang, Xintong Li, Jennifer Yuntong Zhang, Junda Wu, Chengkai Huang, Lina Yao, Julian McAuley, Jingbo Shang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05600.txt.
Description
Chuhan Wang and 7 co-authors present SceneAlign, a 2026 approach that trains multimodal models to reason faithfully about complex visual scenes. In crowded scenes, models often hallucinate objects, mislink relations, or skip reasoning steps. Text-only training tricks let models exploit language habits instead of actually looking.
SceneAlign uses scene graphs, structured maps of objects and relations, to create hard negative cases. It perturbs reasoning-critical nodes in four ways that mimic common grounding failures, producing explanations that sound plausible but rest on wrong visual facts. Preference training on those contrastive pairs pushes models toward accurate, step-by-step grounding. Across seven benchmarks, answer accuracy and reasoning faithfulness both raised consistently.
Catalog record
| CREATOR | Chuhan Wang, Xintong Li, Jennifer Yuntong Zhang, Junda Wu, Chengkai Huang, Lina Yao, Julian McAuley, Jingbo Shang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05600