Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework
Xilai Ma, Liye Zhao, Weijun Yao, Haibing Di, Wenya Wang, Jing Li · 2026
Read Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework by Xilai Ma, Liye Zhao, Weijun Yao, Haibing Di, Wenya Wang, Jing Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10043.txt.
Description
Ma and five colleagues personalise language models from simple yes-or-no feedback, aiming at each user's own tastes rather than one average taste. Most methods study one user's history alone and ignore how users differ from each other. Their framework, C-BPO, treats the target user's data as positive signals and other users' data as implicit negative signals, so differences between users come into focus.
A snag follows: shared task knowledge can look like a personal quirk and get wrongly punished. The authors handle it with an objective drawn from positive-unlabelled learning theory, which strips a positive bias from the negative signals. That keeps alignment to true quirks without harming general helpfulness. Tests across personalization tasks and several base models show C-BPO beating baselines time and again.
Catalog record
| CREATOR | Xilai Ma, Liye Zhao, Weijun Yao, Haibing Di, Wenya Wang, Jing Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10043