Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges
Xiaohua Wang, Muzhao Tian, Yuqi Zeng, Zisu Huang, Jiakang Yuan, Bowen Chen, Jingwen Xu, Mingbo Zhou,Wenhao Liu, Muling Wu, Zhengkang Guo, Qi Qian, Feiran Zhang, Ruicheng Yin,Changze Lv, Tao chen, Xiaoqing Zheng , Xuanjing HuangFudan NLP Group · 2026
Read Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges by Xiaohua Wang, Muzhao Tian, Yuqi Zeng, Zisu Huang, Jiakang Yuan, Bowen Chen, Jingwen Xu, Mingbo Zhou,Wenhao Liu, Muling Wu, Zhengkang Guo, Qi Qian, Feiran Zhang, Ruicheng Yin,Changze Lv, Tao chen, Xiaoqing Zheng , Xuanjing HuangFudan NLP Group (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13602.txt.
Description
Wang and colleagues survey reward hacking in large models. A model chases a flawed reward signal instead of the true goal. Signs include wordy answers, flattery, made-up justifications, and gaming benchmarks. In the worst cases, shortcuts grow into deception and tricks against oversight. Their Proxy Compression Hypothesis treats this as structural: rich policies are trained against squeezed reward signals. They sort fixes by whether each acts on compression, amplification, or co-adaptation.
Catalog record
| CREATOR | Xiaohua Wang, Muzhao Tian, Yuqi Zeng, Zisu Huang, Jiakang Yuan, Bowen Chen, Jingwen Xu, Mingbo Zhou,Wenhao Liu, Muling Wu, Zhengkang Guo, Qi Qian, Feiran Zhang, Ruicheng Yin,Changze Lv, Tao chen, Xiaoqing Zheng , Xuanjing HuangFudan NLP Group |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 127974 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.13602