Debiasing Reward Models via Causally Motivated Inference-Time Intervention

Kazutoshi Shinoda, Kosuke Nishida, Kyosuke Nishida · 2026

Read Debiasing Reward Models via Causally Motivated Inference-Time Intervention by Kazutoshi Shinoda, Kosuke Nishida, Kyosuke Nishida (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27495.txt.

Description

Shinoda and two colleagues propose a causal inference-time intervention to debias reward models, which align language models with preferences but react to spurious features such as response length. Their method finds neurons whose activations track bias attributes, then suppresses those signals at the neuron level. On reward-model benchmarks, it reduces sensitivity to spurious features across bias types without trade-offs. When used for preference annotation, 2B and 7B reward models, editing less than 2 percent of neurons, match a 70B reward model on AlpacaEval and MT-Bench.

Catalog record

CREATORKazutoshi Shinoda, Kosuke Nishida, Kyosuke Nishida
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49949
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.27495

Browse the full library · About the Tower of Babel