Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models

Xukai Liu, Ye Liu, Jipeng Zhang, Yanghai Zhang, Kai Zhang, Qi Liu · 2026

Read Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models by Xukai Liu, Ye Liu, Jipeng Zhang, Yanghai Zhang, Kai Zhang, Qi Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03542.txt.

Description

Xukai Liu and five co-authors probe how language models answer questions that need several linked facts, called multi-hop reasoning. A popular idea says the model works out bridge facts first, layer by layer, before reaching the final answer. Testing real multi-hop queries, the team found the opposite often happens: later answers can become readable in earlier layers than the bridge facts themselves.

They call that pattern layer-order inversion, and it grows stronger as questions need more hops. Their explanation is a recall-and-extract picture: shallow layers recall broadly, then deeper attention layers select what matters. The framework also helps explain why step-by-step prompting helps and why some multi-hop answers fail even when single steps look right. Knowing the true order of internal work guides better model design.

Catalog record

CREATORXukai Liu, Ye Liu, Jipeng Zhang, Yanghai Zhang, Kai Zhang, Qi Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-03542

Browse the full library · About the Tower of Babel