Internal Chain-of-Thought: Empirical Evidence for Layer - wise Subtask Scheduling in LLMs

Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu · 2025

Read Internal Chain-of-Thought: Empirical Evidence for Layer - wise Subtask Scheduling in LLMs by Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14530.txt.

Description

Language models may plan in layers, doing one subtask per depth band, and Zhipeng Yang and co-authors test that picture in work from twenty twenty-five. Fifteen two-step tasks served as the bench. Masking tricks cut layers from the loop, and a patch test swapped signals across tasks.

Subtasks did sit at split depths, and a lens on hidden states showed them firing in order. A real-world bench showed the same stair pattern. The picture gives engineers a finer lever: steer the exact layer where a subtask runs. It also makes model thought less of a black box.

Catalog record

CREATORZhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-14530

Browse the full library · About the Tower of Babel