Optimizing Alignment with Less: Leveraging Data Augmentation for Personalized Evaluation
Javad Seraj · 2024
Read Optimizing Alignment with Less: Leveraging Data Augmentation for Personalized Evaluation by Javad Seraj (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07429.txt.
Description
Javad Seraj and three co-authors align an open language model's judgments with one particular human evaluator, using cleverly chosen data rather than more data. Personalized judging matters because evaluation is subjective, and a model that mimics an average rater may not match your rater. Labeled examples for one person's taste are, by nature, scarce.
Their augmentation technique selects the most effective samples from the limited pool for training. On a mathematical reasoning evaluation task, correlation with the reference judge rose about seven percent over the baseline method. Against the untouched base model, the improvement was about thirty percent. The study shows careful sample selection can beat brute-force fine-tuning in low-data personalization.
Catalog record
| CREATOR | Javad Seraj |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-07429