A transformer architecture alteration to incentivise externalised reasoning

Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan, [5pt] Cameron Tice, Edward James Young, Puria Radmard, [5pt] MARS · 2026

Read A transformer architecture alteration to incentivise externalised reasoning by Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan, [5pt] Cameron Tice, Edward James Young, Puria Radmard, [5pt] MARS (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21376.txt.

Description

Pavlova and co-authors propose a transformer change that pushes models to reason out loud. They add an early-exit mechanism at intermediate layers. The model learns to exit at shallow layers when the next token needs little computation. After calibration, reinforcement learning rewards the earliest exit that keeps task performance. Early results on small reasoning models show adaptive computation per token. The idea aims to cut the excess internal computation that models could use for hidden planning. Hard tokens would keep the deep computation, while easy ones exit fast.

Catalog record

CREATORElizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan, [5pt] Cameron Tice, Edward James Young, Puria Radmard, [5pt] MARS
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23100
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21376

Browse the full library · About the Tower of Babel