Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing

Parsa Mirtaheri, Mikhail Belkin · 2026

Read Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing by Parsa Mirtaheri, Mikhail Belkin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17199.txt.

Description

Mirtaheri and Belkin detect motivated reasoning in language models by probing their internal activations. In multiple choice tests with an injected hint, models may shift answers toward the hint and then rationalize the choice without naming the hint. The authors train supervised probes on the model's residual stream. Probes applied before any chain of thought predict motivated reasoning as well as a monitor that reads the full trace. Probes applied after generation do better still.

Catalog record

CREATORParsa Mirtaheri, Mikhail Belkin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44924
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.17199

Browse the full library · About the Tower of Babel