From Exploration to Exploitation: A Two-Stage Entropy RLVR Approach for Noise-Tolerant MLLM Training

Donglai Xu, Hongzheng Yang, Yuzhi Zhao, Pingping Zhang, Jinpeng Chen, Wenao Ma, Zhijian Hou, Mengyang Wu, Xiaolei Li, Senkang Hu, Ziyi Guan, Jason Chun Lok Li, Lai Man Po, Independent Researcher The Chinese University of Hong Kong City University of Hong Kong · 2025

Read From Exploration to Exploitation: A Two-Stage Entropy RLVR Approach for Noise-Tolerant MLLM Training by Donglai Xu, Hongzheng Yang, Yuzhi Zhao, Pingping Zhang, Jinpeng Chen, Wenao Ma, Zhijian Hou, Mengyang Wu, Xiaolei Li, Senkang Hu, Ziyi Guan, Jason Chun Lok Li, Lai Man Po, Independent Researcher The Chinese University of Hong Kong City University of Hong Kong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-07738.txt.

Description

Donglai Xu, Hongzheng Yang, and Yuzhi Zhao train multimodal models with rewards that survive noisy labels. Reinforcement learning with verifiable rewards needs trusted labels, yet real annotations err often. Pure entropy tricks can lock onto wrong labels and blunt the ranking signal group-based training depends on.

A two-stage token-level entropy method first encourages exploration, then shifts toward exploitation. The schedule guides learning dynamically instead of fixing one pressure throughout. Noise tolerance improves because early openness resists premature commitment to bad labels. Practical multimodal training data is messy by nature, so methods that expect mess are likelier to survive deployment.

Catalog record

CREATORDonglai Xu, Hongzheng Yang, Yuzhi Zhao, Pingping Zhang, Jinpeng Chen, Wenao Ma, Zhijian Hou, Mengyang Wu, Xiaolei Li, Senkang Hu, Ziyi Guan, Jason Chun Lok Li, Lai Man Po, Independent Researcher The Chinese University of Hong Kong City University of Hong Kong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-07738

Browse the full library · About the Tower of Babel