Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework

Xilai Ma, Liye Zhao, Weijun Yao, Haibing Di, Wenya Wang, Jing Li · 2026

Read Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework by Xilai Ma, Liye Zhao, Weijun Yao, Haibing Di, Wenya Wang, Jing Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10043.txt.

Description

Ma and five colleagues personalise language models from simple yes-or-no feedback, aiming at each user's own tastes rather than one average taste. Most methods study one user's history alone and ignore how users differ from each other. Their framework, C-BPO, treats the target user's data as positive signals and other users' data as implicit negative signals, so differences between users come into focus.

A snag follows: shared task knowledge can look like a personal quirk and get wrongly punished. The authors handle it with an objective drawn from positive-unlabelled learning theory, which strips a positive bias from the negative signals. That keeps alignment to true quirks without harming general helpfulness. Tests across personalization tasks and several base models show C-BPO beating baselines time and again.

Catalog record

CREATORXilai Ma, Liye Zhao, Weijun Yao, Haibing Di, Wenya Wang, Jing Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.10043

Browse the full library · About the Tower of Babel