INCRT: An Incremental Transformer That Determines Its Own Architecture
Giansalvo Cirrincione · 2026
Read INCRT: An Incremental Transformer That Determines Its Own Architecture by Giansalvo Cirrincione (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10703.txt.
Description
Cirrincione introduces INCRT, a transformer that settles its own architecture during training. Starting from one head, it adds heads when needed and prunes redundant ones. Each growth choice follows one geometric quantity from the task's structure, with no hand-tuned schedule. Two theorems back the design: convergence to a minimal sufficient setup and a cap on head count. Tests on SARS-CoV-2 classification and SST-2 sentiment match predictions within 12 percent. Final models match or beat BERT-base with three to seven times fewer parameters and no pretraining.
Catalog record
| CREATOR | Giansalvo Cirrincione |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63223 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.10703