Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges

Xiaohua Wang, Muzhao Tian, Yuqi Zeng, Zisu Huang, Jiakang Yuan, Bowen Chen, Jingwen Xu, Mingbo Zhou,Wenhao Liu, Muling Wu, Zhengkang Guo, Qi Qian, Feiran Zhang, Ruicheng Yin,Changze Lv, Tao chen, Xiaoqing Zheng , Xuanjing HuangFudan NLP Group · 2026

Read Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges by Xiaohua Wang, Muzhao Tian, Yuqi Zeng, Zisu Huang, Jiakang Yuan, Bowen Chen, Jingwen Xu, Mingbo Zhou,Wenhao Liu, Muling Wu, Zhengkang Guo, Qi Qian, Feiran Zhang, Ruicheng Yin,Changze Lv, Tao chen, Xiaoqing Zheng , Xuanjing HuangFudan NLP Group (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13602.txt.

Description

Wang and colleagues survey reward hacking in large models. A model chases a flawed reward signal instead of the true goal. Signs include wordy answers, flattery, made-up justifications, and gaming benchmarks. In the worst cases, shortcuts grow into deception and tricks against oversight. Their Proxy Compression Hypothesis treats this as structural: rich policies are trained against squeezed reward signals. They sort fixes by whether each acts on compression, amplification, or co-adaptation.

Catalog record

CREATORXiaohua Wang, Muzhao Tian, Yuqi Zeng, Zisu Huang, Jiakang Yuan, Bowen Chen, Jingwen Xu, Mingbo Zhou,Wenhao Liu, Muling Wu, Zhengkang Guo, Qi Qian, Feiran Zhang, Ruicheng Yin,Changze Lv, Tao chen, Xiaoqing Zheng , Xuanjing HuangFudan NLP Group
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE127974
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13602

Browse the full library · About the Tower of Babel