Meta-Learning Transformers to Improve In-Context Generalization
Lorenzo Braccaioli [email protected], Anna Vettoruzzo [email protected], Prabhant Singh [email protected], Joaquin Vanschoren [email protected], Mohamed-Rafik Bouguelia [email protected], Nicola Conci [email protected] · 2025
Read Meta-Learning Transformers to Improve In-Context Generalization by Lorenzo Braccaioli [email protected], Anna Vettoruzzo [email protected], Prabhant Singh [email protected], Joaquin Vanschoren [email protected], Mohamed-Rafik Bouguelia [email protected], Nicola Conci [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05019.txt.
Description
Lorenzo Braccaioli and co-authors train transformers in 2025 on many small, focused data sets rather than one huge one, to improve in context learning. In context learning lets a model handle a new task from the prompt alone, with no change to its weights.
Big training sets are costly to store, hard to grade, and can hold sensitive data. A curated set of small domain sets, joined through meta learning, matched the big set approach on its home ground. It also generalized better beyond that ground. Tests covered a held out domain, a continual setting where data vanish after a time, and an unsupervised case.
Catalog record
| CREATOR | Lorenzo Braccaioli [email protected], Anna Vettoruzzo [email protected], Prabhant Singh [email protected], Joaquin Vanschoren [email protected], Mohamed-Rafik Bouguelia [email protected], Nicola Conci [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 119 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-05019