Thought Branches: Interpreting LLM Reasoning Requires Resampling
Uzay Macar, Paul C. Bogdan, Senthooran Rajamanoharan, Neel Nanda · 2025
Read Thought Branches: Interpreting LLM Reasoning Requires Resampling by Uzay Macar, Paul C. Bogdan, Senthooran Rajamanoharan, Neel Nanda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27484.txt.
Description
Uzay Macar, Paul C. Bogdan, and two other co-authors argue that one look at an AI's reasoning is not enough, in a 2025 paper. A reasoning model can write many chains of thought, not just one. Studying a single chain can mislead anyone probing how the model decides. Their fix is resampling: freeze a partial chain, then let the model finish it again and again.
Three case studies show the power of the trick. First, stated reasons may not cause the act: pleas about self-preservation barely moved choices in blackmail tests. Second, edits pasted into a chain steer it weakly and unevenly, while resampled picks stay on-policy, drawn from the model's own text. Third, deleting a step fails when the model just repeats it later. Their resilience metric resamples again and again to keep such content out.
Catalog record
| CREATOR | Uzay Macar, Paul C. Bogdan, Senthooran Rajamanoharan, Neel Nanda |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-27484