Cheems : A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratch

Xueru Wen, Jie Lou, Zichao Li, Yaojie Lu, XingYu, Yuqiu Ji, Guohai Xu, Hongyu Lin, Ben He, Xianpei Han, Le Sun, Debing Zhang · 2025

Read Cheems : A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratch by Xueru Wen, Jie Lou, Zichao Li, Yaojie Lu, XingYu, Yuqiu Ji, Guohai Xu, Hongyu Lin, Ben He, Xianpei Han, Le Sun, Debing Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17173.txt.

Description

Xueru Wen and colleagues built Chinese reward models from scratch in a 2025 guidance paper. Reward models grade language model answers to steer them toward human taste, but nearly all research uses English and machine-made data. For Chinese, that leaves training sets and test sets thin and shaky, so models may learn the wrong lessons about what readers want.

The team offers a fully human-checked test set for Chinese contexts and a large preference dataset labeled by people and machines together. Leading open models scored poorly on the human test, showing real gaps. A model trained on their data topped the test, which argues that careful human supervision still beats piles of synthetic labels.

Catalog record

CREATORXueru Wen, Jie Lou, Zichao Li, Yaojie Lu, XingYu, Yuqiu Ji, Guohai Xu, Hongyu Lin, Ben He, Xianpei Han, Le Sun, Debing Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-17173

Browse the full library · About the Tower of Babel