An evolutionary perspective on modes of learning in Transformers
Alexander Y. Ku, Thomas L. Griffiths, Stephanie C.Y. Chan · 2025
Read An evolutionary perspective on modes of learning in Transformers by Alexander Y. Ku, Thomas L. Griffiths, Stephanie C.Y. Chan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-09855.txt.
Description
Alexander Ku, Thomas Griffiths, and Stephanie Chan compare two ways a Transformer model learns, through a lens from biology, in a paper from twenty twenty-five. These models can slowly bake knowledge into their weights, or use the prompt in front of them to adapt at once. Biology makes a like split between slow genetic change and flexible traits within one life.
The authors vary how stable the test setting is and how reliable its cues are. Stable settings favor learning in the weights, often with a sharp switch. Shifting settings with good cues favor learning from context. Shifts between the two modes also appear during training. What drives a shift is how good the mode can get, and how costly it is to acquire.
Catalog record
| CREATOR | Alexander Y. Ku, Thomas L. Griffiths, Stephanie C.Y. Chan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-09855