Meta-Learning Transformers to Improve In-Context Generalization

Lorenzo Braccaioli [email protected], Anna Vettoruzzo [email protected], Prabhant Singh [email protected], Joaquin Vanschoren [email protected], Mohamed-Rafik Bouguelia [email protected], Nicola Conci [email protected] · 2025

Read Meta-Learning Transformers to Improve In-Context Generalization by Lorenzo Braccaioli [email protected], Anna Vettoruzzo [email protected], Prabhant Singh [email protected], Joaquin Vanschoren [email protected], Mohamed-Rafik Bouguelia [email protected], Nicola Conci [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05019.txt.

Description

Lorenzo Braccaioli and co-authors train transformers in 2025 on many small, focused data sets rather than one huge one, to improve in context learning. In context learning lets a model handle a new task from the prompt alone, with no change to its weights.

Big training sets are costly to store, hard to grade, and can hold sensitive data. A curated set of small domain sets, joined through meta learning, matched the big set approach on its home ground. It also generalized better beyond that ground. Tests covered a held out domain, a continual setting where data vanish after a time, and an unsupervised case.

Catalog record

CREATORLorenzo Braccaioli [email protected], Anna Vettoruzzo [email protected], Prabhant Singh [email protected], Joaquin Vanschoren [email protected], Mohamed-Rafik Bouguelia [email protected], Nicola Conci [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE119 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-05019

Browse the full library · About the Tower of Babel