Fairness Aware Reward Optimization
Ching Lam Choi, Vighnesh Subramaniam, Phillip Isola, Antonio Torralba, Stefanie Jegelka · 2026
Read Fairness Aware Reward Optimization by Ching Lam Choi, Vighnesh Subramaniam, Phillip Isola, Antonio Torralba, Stefanie Jegelka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07799.txt.
Description
Reward models trained on human preferences can absorb demographic skews and pass them into aligned language models. Ching Lam Choi and four co-authors offer Faro, a training method that builds fairness constraints directly into reward modeling rather than patching bias before or after.
Faro supports demographic parity, equalized odds, and counterfactual fairness. The paper proves fairness certificates with controllable slack, shows fairness carries from reward to policy under standard fine-tuning, and shows that a genuine accuracy-fairness frontier exists. Across models and benchmarks, Faro cuts bias and harmful outputs while holding or improving quality. Alignment teams gain both theory and a usable in-training method.
Catalog record
| CREATOR | Ching Lam Choi, Vighnesh Subramaniam, Phillip Isola, Antonio Torralba, Stefanie Jegelka |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07799