PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment
Yantao Li, Qiang Hui, Chenyang Yan, Kanzhi Cheng, Fang Zhao, Chao Tan, Huanling Gao, Jianbing Zhang, Kai Wang, Xinyu Dai, Shiguo Lian · 2026
Read PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment by Yantao Li, Qiang Hui, Chenyang Yan, Kanzhi Cheng, Fang Zhao, Chao Tan, Huanling Gao, Jianbing Zhang, Kai Wang, Xinyu Dai, Shiguo Lian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06652.txt.
Description
Li and colleagues address process hallucinations in multimodal reasoning, where models reach correct answers while misperceiving visual evidence. They propose PaLMR, aligning process and outcomes through a perception-aligned data layer and an optimization layer. The data layer builds reasoning data with pseudo-ground-truths and visual facts; the optimization layer fuses rewards with process-aware scoring for visually faithful chains-of-thought. Experiments on Qwen2.5-VL-7B reduce hallucinations and improve visual reasoning fidelity, reaching top results on HallusionBench while staying strong on MMMU, MathVista, and MathVerse.
Catalog record
| CREATOR | Yantao Li, Qiang Hui, Chenyang Yan, Kanzhi Cheng, Fang Zhao, Chao Tan, Huanling Gao, Jianbing Zhang, Kai Wang, Xinyu Dai, Shiguo Lian |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47069 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.06652