Fairness Aware Reward Optimization

Ching Lam Choi, Vighnesh Subramaniam, Phillip Isola, Antonio Torralba, Stefanie Jegelka · 2026

Read Fairness Aware Reward Optimization by Ching Lam Choi, Vighnesh Subramaniam, Phillip Isola, Antonio Torralba, Stefanie Jegelka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07799.txt.

Description

Reward models trained on human preferences can absorb demographic skews and pass them into aligned language models. Ching Lam Choi and four co-authors offer Faro, a training method that builds fairness constraints directly into reward modeling rather than patching bias before or after.

Faro supports demographic parity, equalized odds, and counterfactual fairness. The paper proves fairness certificates with controllable slack, shows fairness carries from reward to policy under standard fine-tuning, and shows that a genuine accuracy-fairness frontier exists. Across models and benchmarks, Faro cuts bias and harmful outputs while holding or improving quality. Alignment teams gain both theory and a usable in-training method.

Catalog record

CREATORChing Lam Choi, Vighnesh Subramaniam, Phillip Isola, Antonio Torralba, Stefanie Jegelka
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07799

Browse the full library · About the Tower of Babel