Hierarchical vs. Flat Iteration in Shared-Weight Transformers Sang-Il Han 1,2 1 Korea University of Technology and Education 2 Biomonster Co.
2026
Read Hierarchical vs. Flat Iteration in Shared-Weight Transformers Sang-Il Han 1,2 1 Korea University of Technology and Education 2 Biomonster Co. (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14442.txt.
Introduction
Han tests whether shared-weight recurrence with hierarchical structure can match independent-layer stacking in a Transformer language model. His HRM-LM swaps independent layers for a recurrent pair at two speeds: a fast module refining at every step and a slow module shrinking every T steps, run with shared parameters. The sharpest, most robust finding comes from a parameter-matched Universal Transformer ablation at 1.2 billion parameters across five runs: a clear empirical gap separates the two approaches. The study frames that gap as the central result.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 115171 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.14442