Examining Two Hop Reasoning Through Information Content Scaling

David Johnston, Sergio Sanz, Nora Belrose · 2025

Read Examining Two Hop Reasoning Through Information Content Scaling by David Johnston, Sergio Sanz, Nora Belrose (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03490.txt.

Description

David Johnston and co-authors ask how well transformer models handle two-step reasoning in this 2025 paper. A two-step question links two facts, such as naming the boss of someone's mother. Earlier work found that models learn such questions in an uneven way. The team tests how this skill grows as models get bigger.

Their tests suggest that silent two-step answers force a model to learn each fact twice. When the model spells out its steps, that double cost goes away. The team also shows that very small models can get stuck just memorizing answers. The work offers a new way to probe how these models store and link facts.

Catalog record

CREATORDavid Johnston, Sergio Sanz, Nora Belrose
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-03490

Browse the full library · About the Tower of Babel