PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment

Yantao Li, Qiang Hui, Chenyang Yan, Kanzhi Cheng, Fang Zhao, Chao Tan, Huanling Gao, Jianbing Zhang, Kai Wang, Xinyu Dai, Shiguo Lian · 2026

Read PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment by Yantao Li, Qiang Hui, Chenyang Yan, Kanzhi Cheng, Fang Zhao, Chao Tan, Huanling Gao, Jianbing Zhang, Kai Wang, Xinyu Dai, Shiguo Lian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06652.txt.

Description

Li and colleagues address process hallucinations in multimodal reasoning, where models reach correct answers while misperceiving visual evidence. They propose PaLMR, aligning process and outcomes through a perception-aligned data layer and an optimization layer. The data layer builds reasoning data with pseudo-ground-truths and visual facts; the optimization layer fuses rewards with process-aware scoring for visually faithful chains-of-thought. Experiments on Qwen2.5-VL-7B reduce hallucinations and improve visual reasoning fidelity, reaching top results on HallusionBench while staying strong on MMMU, MathVista, and MathVerse.

Catalog record

CREATORYantao Li, Qiang Hui, Chenyang Yan, Kanzhi Cheng, Fang Zhao, Chao Tan, Huanling Gao, Jianbing Zhang, Kai Wang, Xinyu Dai, Shiguo Lian
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47069
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06652

Browse the full library · About the Tower of Babel