Debiasing Reward Models via Causally Motivated Inference-Time Intervention
Kazutoshi Shinoda, Kosuke Nishida, Kyosuke Nishida · 2026
Read Debiasing Reward Models via Causally Motivated Inference-Time Intervention by Kazutoshi Shinoda, Kosuke Nishida, Kyosuke Nishida (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27495.txt.
Description
Shinoda and two colleagues propose a causal inference-time intervention to debias reward models, which align language models with preferences but react to spurious features such as response length. Their method finds neurons whose activations track bias attributes, then suppresses those signals at the neuron level. On reward-model benchmarks, it reduces sensitivity to spurious features across bias types without trade-offs. When used for preference annotation, 2B and 7B reward models, editing less than 2 percent of neurons, match a 70B reward model on AlpacaEval and MT-Bench.
Catalog record
| CREATOR | Kazutoshi Shinoda, Kosuke Nishida, Kyosuke Nishida |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49949 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.27495