Re-Align : Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization

Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu · 2025

Read Re-Align : Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization by Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13146.txt.

Description

Shuo Xing and co-authors present, in a 2025 paper, a training method that cuts the fabrications vision-language models produce when image and text disagree. Models that see and read have unlocked wide applications. Their confident inventions remain the standing complaint against them.

Preference training fixed text-only models by ranking answers. Applied bluntly to vision models, it can ignore what the picture actually shows. The authors add retrieval to the preference process, grounding comparisons in fetched evidence. Alignment then rewards answers faithful to the visual input. Fewer cross-modal inconsistencies would make such models safer for medicine, retail, and accessibility tools that describe the world to people who cannot see it.

Catalog record

CREATORShuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-13146

Browse the full library · About the Tower of Babel