Modeling Human Beliefs about AI Behavior for Scalable Oversight

Leon Lang [email protected], Patrick Forré [email protected] · 2025

Read Modeling Human Beliefs about AI Behavior for Scalable Oversight by Leon Lang [email protected], Patrick Forré [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-21262.txt.

Introduction

Leon Lang and Patrick Forré wrote this paper in two thousand twenty-five. It argues that oversight of strong AI should model what human judges believe about how the AI acts, not just what the judges say.

When a task outruns the judge, wrong beliefs about the AI can poison feedback and warp the values the AI learns. The authors formalize belief models, study their role in value learning, and mark when doubt cannot be cleared. A looser trick, belief model covering, cuts the need for exact models. They sketch a plan to borrow the inner codes of tuned foundation models as stand-ins for human beliefs. The aim is sound values from flawed judges.

Catalog record

CREATORLeon Lang [email protected], Patrick Forré [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE264 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-21262

Browse the full library · About the Tower of Babel