Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers

Mohan Tang, Sidu Lu · 2026

Read Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers by Mohan Tang, Sidu Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17993.txt.

Description

Mohan Tang and Sidi Lu argue that a Transformer can follow only a bounded number of steps along any hidden path. That cap limits models on problems solved step by step. Their answer is Turbo Connection, an architecture sending several backward links from upper-layer states of one token into the lower layers of the next token.

Fine-tuning language models with the method lifted accuracy by under one percent to over ten percent on grade-school math, parity, and multi-step arithmetic tests. Dense backward connections mattered, clearly beating sparse variants that pass only a single hidden state or vector. On parity, one fine-tuned small model scored only about fifty-four percent, but the modification carried it to full accuracy without retraining from scratch or elaborate curriculum learning. The results point to path depth as a key factor in reasoning ability, at little latency cost.

Catalog record

CREATORMohan Tang, Sidu Lu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17993

Browse the full library · About the Tower of Babel