A transformer architecture alteration to incentivise externalised reasoning
Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan, [5pt] Cameron Tice, Edward James Young, Puria Radmard, [5pt] MARS · 2026
Read A transformer architecture alteration to incentivise externalised reasoning by Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan, [5pt] Cameron Tice, Edward James Young, Puria Radmard, [5pt] MARS (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21376.txt.
Description
Pavlova and co-authors propose a transformer change that pushes models to reason out loud. They add an early-exit mechanism at intermediate layers. The model learns to exit at shallow layers when the next token needs little computation. After calibration, reinforcement learning rewards the earliest exit that keeps task performance. Early results on small reasoning models show adaptive computation per token. The idea aims to cut the excess internal computation that models could use for hidden planning. Hard tokens would keep the deep computation, while easy ones exit fast.
Catalog record
| CREATOR | Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan, [5pt] Cameron Tice, Edward James Young, Puria Radmard, [5pt] MARS |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23100 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21376