Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models
Xukai Liu, Ye Liu, Jipeng Zhang, Yanghai Zhang, Kai Zhang, Qi Liu · 2026
Read Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models by Xukai Liu, Ye Liu, Jipeng Zhang, Yanghai Zhang, Kai Zhang, Qi Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03542.txt.
Description
Xukai Liu and five co-authors probe how language models answer questions that need several linked facts, called multi-hop reasoning. A popular idea says the model works out bridge facts first, layer by layer, before reaching the final answer. Testing real multi-hop queries, the team found the opposite often happens: later answers can become readable in earlier layers than the bridge facts themselves.
They call that pattern layer-order inversion, and it grows stronger as questions need more hops. Their explanation is a recall-and-extract picture: shallow layers recall broadly, then deeper attention layers select what matters. The framework also helps explain why step-by-step prompting helps and why some multi-hop answers fail even when single steps look right. Knowing the true order of internal work guides better model design.
Catalog record
| CREATOR | Xukai Liu, Ye Liu, Jipeng Zhang, Yanghai Zhang, Kai Zhang, Qi Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03542