Measuring Human Preferences in RLHF is a Social Science Problem
Bijean Ghafouri, Eun Cheol Choi, Priyanka Dey, Emilio Ferrara · 2026
Read Measuring Human Preferences in RLHF is a Social Science Problem by Bijean Ghafouri, Eun Cheol Choi, Priyanka Dey, Emilio Ferrara (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03238.txt.
Introduction
Ghafouri and colleagues argue that reinforcement learning from human feedback rests on a shaky premise: annotator responses often are not genuine preferences at all. People may answer without opinions, build preferences on the spot, or read the same question differently, and these flaws bite hardest on value judgments. Filtering inconsistent annotators flipped harm verdicts for about 19 percent of prompts, showing the field often models noise as signal.
Catalog record
| CREATOR | Bijean Ghafouri, Eun Cheol Choi, Priyanka Dey, Emilio Ferrara |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 139347 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.03238