Optimizing Alignment with Less: Leveraging Data Augmentation for Personalized Evaluation

Javad Seraj · 2024

Read Optimizing Alignment with Less: Leveraging Data Augmentation for Personalized Evaluation by Javad Seraj (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07429.txt.

Description

Javad Seraj and three co-authors align an open language model's judgments with one particular human evaluator, using cleverly chosen data rather than more data. Personalized judging matters because evaluation is subjective, and a model that mimics an average rater may not match your rater. Labeled examples for one person's taste are, by nature, scarce.

Their augmentation technique selects the most effective samples from the limited pool for training. On a mathematical reasoning evaluation task, correlation with the reference judge rose about seven percent over the baseline method. Against the untouched base model, the improvement was about thirty percent. The study shows careful sample selection can beat brute-force fine-tuning in low-data personalization.

Catalog record

CREATORJavad Seraj
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-07429

Browse the full library · About the Tower of Babel